AIBRIEF
A Google memory trick that stashes 25M of 28.9M parameters in a flash lookup table (reading only ~450 bytes per token into RAM) puts a 28.9M parameter language model on an $8 ESP32 S3, roughly 110 times the previous on chip LLM record (~260K params).
An ESP32-S3 microcontroller that costs about $8 now runs a 28.9-million-parameter language model end-to-end at roughly 9.5 tokens per second. That is about 110 times the size of the previous on-chip LLM, which topped out near 260K parameters. The result is a short-story generator, not a general assistant.
The slvDev/esp32-ai project uses Google's Per-Layer Embeddings from Gemma 3n/4: 25 million of the parameters live in a flash lookup table, and only about 450 bytes are read per token into the chip's 512KB of SRAM. The 1.64MB output head sits in 8MB of PSRAM, leaving 5.2MB free after allocation. Measured bandwidth caps the run at roughly 58 tokens per second, so the model is PSRAM-bandwidth-bound, not compute-bound.
Trained on TinyStories, the model writes short, simple stories only. It does not answer questions, follow instructions, write code, or carry world knowledge. The ablation table reports PLE perplexity 11.41 versus 12.58 for the baseline (+0.098 nats) on the same training set.
Reception on Hacker News treated the build as a memory-architecture demonstration. The benchmarks and on-device profile are self-reported by the project author; no independent replication appears in the source set.