AINEWS
Kimi K3, an open source 2.8 trillion parameter model, streams through a single consumer GPU by paying in time and disk instead of shrinking the model, and the tradeoff defines who it's for.
AirLLM v3.1.0, released in July 2026, claims it can run Kimi K3, a 2.8-trillion-parameter open-source model, end to end on 3.72GB of VRAM. The measurement was taken on a single RTX 6000 Ada, a workstation card with 48GB of memory, using only a fraction of it. The release pushes the project's matrix further than any previous version: 70-billion-parameter dense models on 4GB, Llama 3.1 405B on 8GB, DeepSeek-V3 at 671 billion on roughly 12GB, and now 2.8 trillion on under 4GB.
The "fit on 4GB" framing is technically accurate and conceptually misleading. AirLLM does not shrink the model. It streams it. For dense models, where every parameter is used for every token, the project reads model layers off disk in sequence and holds only one layer's weights in VRAM at a time. For Mixture-of-Experts models, where only a fraction of the parameters activate for any given token, the project takes the idea further: it streams individual experts, loading just the experts a token actually routes to.
Kimi K3 is a sparse MoE model. Its active parameter count per token is a small fraction of the headline 2.8T, the same way DeepSeek-V3's 671B total becomes 37B activated per token. AirLLM's per-expert streaming loads only those active experts, so the 4GB of VRAM is not holding a 2.8-trillion-parameter model. It is holding the slice of one that any single token needs, while the rest of the 1.56TB checkpoint sits on disk waiting its turn.
The cost is time. The project's own measurement puts Kimi K3 at roughly five minutes per token on the RTX 6000 Ada. The model is disk-bound, not memory-bound. A consumer reading "5 minutes per token" will recognize this is not a production configuration. It is a feasibility demonstration for independent developers, small research teams, and edge experiments that cannot justify data-center hardware.
The dependency stack is the other cost, and the project's own README and release notes are direct about it. Kimi K3 support requires compressed-tensors, flash-attention, a CUDA 12 PyTorch build (no prebuilt flash-attn wheel exists for CUDA 13 yet), and transformers pinned to 4.56.x because K3's remote code does not load on the 5.x line. The package is on PyPI and installs with a single pip command, but the runtime prerequisites are a wall the README does not pretend is small.
The approach trades throughput for access. A developer with a single 16GB or 24GB consumer GPU can load models that previously could not fit, accepting that the first token will take minutes and that steady-state throughput will be limited by disk and, for MoE models, by system RAM bandwidth. The Hacker News thread on the project captures the split. Some users run local LLMs as a hobby and treat the slow path as a feature. Others, including practitioners testing whether MoE viability on PCs depends on Threadripper-class memory bandwidth, report numbers like 80-120 tokens per second on an RTX 3080 with Gemma 4 and faster throughput with Qwen 3.5, figures that suggest dense models at 70B-class scale can run at conversational speed on a single mid-range card.
AirLLM defines a feasibility tier, not a production tier. The tool fits the developer who already owns a consumer GPU, accepts minutes-per-token latency, and is willing to wrestle with a specific dependency stack. Batch inference, multi-user serving, and any workload where latency and cost-per-token matter still need a data-center GPU with the memory to hold the model whole.
The team has not announced a v3.2 target. The current release is the one carrying the 2.8T claim. Per-expert streaming of a sparse model, disk-bound throughput, and a 1.56TB checkpoint are the constraints that will define the next one too.