Cactus

About Cactus

Cactus is an on-device AI toolkit for smartphones, laptops, and edge hardware, with cloud fallback for the cases local compute cannot handle. Speech, vision, and text models deploy through a single toolkit and one API, and a hybrid router decides per request whether to serve on-device or in the cloud based on complexity — a thermostat command runs locally, a complex multi-step operation goes to the cloud, and clear audio transcribes on-device while noisy audio is routed out. The published numbers are 5x cost savings, sub-120ms on-device latency, and under 6% word error rate on transcription. The Cactus Engine underneath is open source and fully auditable, which is a real requirement when the code ships inside a customer's mobile app: quantized models with hardware-specific acceleration tuned for battery-efficient inference, and zero-copy memory mapping for minimal RAM use and near-instant model loading. It runs cross-platform across iOS, Android, and desktop, installs via Homebrew, and the repository has 4.2k+ stars. The team also ships its own models — Needle, a 26M-parameter tool-calling model distilled from Gemini — which signals the product is aimed at agentic on-device function calling rather than just local chat. The team comes out of Y Combinator, Oxford, DeepRender, Salesforce, Google, AWS, and MIT.

Key Features

Cactus Pros & Cons

Tags