AINEWS
Gemma is Google's open weight model family.
The ESP32-S3 is a $10 hobbyist microcontroller with 512KB of fast RAM, less than a single uncompressed phone photo holds. A developer just ran a 28.9-million-parameter language model on one, roughly 111 times past the prior bar for that chip class, by leaning on a technique called Per-Layer Embeddings from Google's open-weight Gemma family.
The project, ESP32-AI, is a one-person GitHub release by Slava, a Ukrainian developer publishing under the handle slvDev. The parameter count is the headline. The architectural lever that made it possible is the actual story: a technique called Per-Layer Embeddings, documented in the Gemma 3n developer guide and the Gemma 4 technical report, designed to fit large language models onto phones and laptops with constrained memory. The constraint it solves is the same one the ESP32-S3 has.
The Gemma 3n developer guide describes Per-Layer Embeddings as a technique that splits the embedding table across memory tiers and only materializes the rows needed for any one token. The mechanism is designed to push model weights out of fast memory; the technique is now being picked up by hobbyists on hardware two orders of magnitude cheaper than the devices it was originally designed for.
Running an LLM on a chip with this little fast memory is normally impossible because every weight has to be reachable in microseconds. The Gemma trick recognizes that the embedding table, the model's giant lookup table that turns a token (one chunk of text, often a word or word-piece) into a vector, is most of the weight budget but only a few rows are needed for any one token. So the table can sit in slow memory.
In the ESP32 build, that means the 25-million-parameter embedding table lives in 16MB of flash storage, and the chip only pulls a few rows per step. The weights that have to stay in fast memory fit in the board's 512KB of SRAM plus 8MB of PSRAM. Quantization to 4 bits, shrinking each model number from 8 bits to 4, brings the total file down to 14.9 MB, small enough to flash onto the board.
The trick is not free. Each token now costs an extra flash-memory access, which is orders of magnitude slower than hitting SRAM, so generation crawls compared with a desktop LLM. Aggressive 4-bit quantization also costs accuracy; the model's outputs are far below what a 7B or 70B-parameter model can produce. The results table in the repo is honest about the narrow set of useful tasks at this parameter count. Slava's own framing is a working demo, not a ChatGPT replacement.
The prior record on this chip class, cited by Slava on X and attributed to "Mr. Dave Bennett," was around 260,000 parameters. 28.9 million is roughly 111 times that bar, the first working demonstration of a meaningful LLM on sub-$10 hardware. The Tom's Hardware writeup surfaced the project, but the architectural credit belongs to Gemma's Per-Layer Embeddings and the quantization work in the repo.
A $10 board that runs without a network connection keeps inference local. Audio or sensor input never leaves the device. For always-listening or privacy-sensitive hardware, that has been the blocker, not the parameter count.
Voice-controlled appliances, sensor hubs, and battery-powered nodes that used to need a cloud round-trip can now keep the inference on-device. Generation will be slow and the model narrow, but for a class of small, always-on tasks that does not matter. The hardware line for running an LLM at all just moved from a server rack to a hobbyist chip.