A 14MB open-source AI model targets $200 phones and microcontrollers

A 14MB open AI model from Cactus Compute puts tool calling on sub-$200 phones, Raspberry Pi boards, and ESP32-class microcontrollers. The company says the model, Needle 2, runs a full session in about 28MB of RAM. The code is Apache 2.0, and the SAN paper backing the architecture is on arXiv.

The bet is mechanical. Needle 2 treats every function call as a typed schema: bytes that must match a user-supplied grammar, with the empty call standing in for refusal. The same machinery drives document extraction and classifiers. Cactus argues this is why 45M parameters suffice where chat-style agents need billions, since function calling needs no world knowledge, only correct typing.

The hardware list is vendor-reported. Cactus reports 500 tokens/second on a Raspberry Pi 5, 300 to 700 on sub-$200 phones like the Samsung A-Series, 400 to 1,500 on Meta Quest 3S and Apple Vision Pro, and ESP32-S3 chips, where the session fits in 28MB. The install is pip install cactus-needle, and the engine runs offline after a one-time fetch.

The numbers are Cactus's own: no independent benchmark, no peer review of the SAN paper, and "Apple FM" is the company's shorthand, not a standard test. The market-shape framing (four in five edge devices under $200, and 21 billion IoT devices against 1.5 billion PCs) also comes from the vendor post. The release puts the test in plain terms: a real 14MB model on a sub-$200 phone or microcontroller, doing real work without the AI silicon flagship phones carry.