An engineer got Google's open-source 26B AI to run on a 2013 server with no GPU

A repurposed HP StoreVirtual storage box from 2013, with no GPU and about $300 of used parts, is now running Google's open-source Gemma 4 26B-A4B at roughly five tokens per second, about reading speed. The result, posted by engineer Ryan Findley on his NeoMindLabs blog on June 8, 2026, is not a benchmark win. It is a worked example of what an old server can do once the inference engine knows how to talk to it.

The dual Xeon E5-2690 v2 chips inside the box are Ivy Bridge silicon from 2013, Intel's server line, with DDR3 memory and no AVX2 or FMA3 instructions, the SIMD vector extensions that most modern inference code assumes. The hardware is older than most modern AI tutorials. The box has no discrete GPU. Total spend: roughly $300.

Gemma 4 is Google's family of open-weights large language models, and the 26B-A4B variant is a 26-billion-parameter mixture-of-experts design, a setup that activates only a fraction of its parameters per token, so it can be large on paper and lean at runtime. Findley runs it at Q8_0 quantization, a scheme that stores weights in 8-bit precision to fit the model in the box's memory budget.

ikawrakow/ik_llama.cpp PR #2138, titled "fix build and MoE inference on CPUs without AVX2 (AVX1-only)," is the patch that landed pre-Haswell Intel support in the ik_llama.cpp fork, a single-maintainer CPU-focused variant of the popular llama.cpp inference engine. Upstream llama.cpp has long assumed AVX2 or newer. This patch backports the missing kernels so the same binary can run on a 2013 chip. Without that change, the rest of the stack would not even compile on the box.

On top of the patch, the fork layers four CPU-specific optimizations. The fork's CPU-aware MoE routing keeps the active experts in cache. Speculative decoding, drafting several tokens with a small model and verifying them with the large one, cuts the per-token cost. Flash attention, the memory-efficient attention kernel used on GPUs, is ported to CPU. Runtime weight repacking reshapes the quantized weights to match the chip's vector units. Together they push decode to about 5.2 tokens per second and prompt evaluation to roughly 16 tokens per second on the Ivy Bridge pair.

A commenter on the related Hacker News thread reported Prism Bonsai 27B, a ternary (2-bit) quantized model, running at 44-plus tokens per second on an M4 Max laptop, about an order of magnitude faster than Findley's 5.2 tokens per second. That comparison is community commentary, not a vendor benchmark. The 13-year-old server is far slower, and the gap is the point: the software path matters as much as the silicon.

Findley, in the post and the thread, defines being "good with AI" as the ability to point a modern model at a problem nobody packaged for you, run it on hardware it was never tuned for, and tell whether the answer is correct. The subscription is the easy version of that skill; the harder version is knowing the model well enough to bypass it. A 2013 box running Gemma 4 at reading speed is a fluency story, not a deployment one.

If PR #2138 is merged into the main llama.cpp tree, pre-Haswell support becomes a default capability rather than a fork, and the next "old box runs new AI" post will not need its own patch.