An $80 Computer Now Runs Generative AI, Fully Offline

For the first time, a $80 single-board computer can generate text at roughly twice the speed of human speech, with no cloud, no API key, and no monthly bill. The numbers come from a Google AI Edge walkthrough published August 11, 2026 on running LiteRT and Gemma 4 E2B on a Raspberry Pi 5. The figures are vendor-reported, but the code, model, and sample are public: a developer can clone the repo and run the same pipeline this weekend.

That is the development, and it is bigger than "Google shipped another on-device demo." It is a tier change. Generative AI is no longer one category that requires hyperscaler budgets. There is now an offline, $80 tier, and it trades raw capability for things cloud models cannot give you at once: privacy (the prompt never leaves the device), latency (no round trip), cost (a one-time $60 to $100 board instead of metered inference), and autonomy (the device works when the internet is down or off-limits).

How the convergence happened

Three pieces had to land at the same time, and they just did.

Google's Gemma family now includes a 270M base, a 300M embedding model, a 1B multilingual text model, and the new E-series: Gemma 4 E2B for edge and mobile, with memory-mapped per-layer embeddings, and a stronger E4B sibling for harder reasoning. E2B is small enough to fit on a Pi, and it is still a generative model that follows instructions.

A unified runtime. LiteRT (formerly MediaPipe) now spans CPU and GPU on the Pi 5. The Pi's Broadcom VideoCore VII GPU runs LiteRT through a WebGPU (Vulkan) backend called ML Drift, so the same model file lands on CPU or GPU without a rewrite. LiteRT also handles classical ML, Ultralytics YOLO (including the new YOLO26n object detector), and Moonshine speech recognition.

A single toolchain. LiteRT CLI (pip install litert-cli) folds convert, quantize, benchmark, and inference into one command. A "LiteRT CLI skill" plus a set of advanced skills also work inside Google Antigravity and other AI coding agents, so an LLM agent can drive the edge pipeline directly.

What the demo actually does

The Reachy Mini voice robot from Pollen Robotics runs a concurrent dual pipeline on a single Pi 5. YOLO handles object detection on the GPU, Moonshine ASR transcribes speech on the CPU, Gemma 4 E2B reasons on the CPU, and a streaming TTS layer answers out loud. No request leaves the board.

The numbers Google reports, on this specific path and reproducible from the public sample: 99 tokens per second on prefill, 9 tokens per second on decode, 1432 MB peak memory, and roughly 300 words per minute end to end. The model averages about 4.2 characters per token, which works out to roughly 27 characters a second, about double normal conversational speech.

Those numbers are vendor-reported for the specific Gemma 4 E2B + LiteRT-LM + Pi 5 + Reachy Mini voice path. No independent third-party benchmark of this exact stack exists yet, so the cleanest summary is "anchorable to the post, reproducible from the sample," not "lab-confirmed."

What it does not buy you

The Pi 5 itself is a modest machine. Its quad-core ARM Cortex-A76 CPU delivers about 153.6 GFLOPS in FP32 and tops out near 2.0 TOPS in INT8, and the VideoCore VII GPU runs around 76.8 GFLOPS in FP32 and 0.24 TOPS in INT8. A Pi 5 is not a data center. E2B is a small model: coherent, useful, and small enough to fit, not a substitute for GPT-class cloud reasoning on hard problems.

Hardware acceleration is also still a promise. LiteRT with Gemma is "coming soon" to Hailo-10H accelerators and to the Raspberry Pi AI HAT+ and AI HAT+ 2, which would let the same model offload through the same LiteRT workflows. No firm date, so any production deployment today runs on the Pi's CPU and integrated GPU.

Independent adoption data is also thin: no public count of LiteRT downloads, litert-community Hugging Face pulls, or developer projects in the wild. The honest summary is that a developer can plausibly ship a generative product in a weekend, not that developers are already shipping at scale.

Why now

Until this year, "AI on a Pi" meant a toy that handled a few tokens per second on a stripped-down model. The combination of a capable E-series model, a runtime that spans CPU and GPU, a CLI that compresses the toolchain, and a $80 board that runs a full speech pipeline locally, all available today, all open, is the change. The cost-per-query is now zero. The privacy posture is now local by default. The latency is now bounded by the board.

The next test is the ecosystem. The August post put the code, the model, the CLI, and the sample pipeline in public view, and named Hailo-10H integration as a near-term item. When that integration lands, the same E2B model will run faster on the same board, and a single $80 stack will move closer to a developer default for local AI. Until then, the claim is honest: it works on a Pi 5 today, and it is about to get cheaper to run.