A Small Lab Built an AI That Only Reads Pre-1931 Text. It Cost $807.

A small lab spent about $807 over three months teaching a 2.82-billion-parameter language model to write like it is still 1930. The model, Bart, sits well below the size of frontier systems, and the team built it as a controlled probe into what an LLM can learn from a strictly pre-modern corpus.

Bart, formally Bartholomew III, is a transformer announced on r/MachineLearning and detailed in Unbounded Labs' engineering blog, with the training corpus, evaluation suite, fine-tuning dataset, and model weights all released publicly. The cutoff is deliberate: every text published after 1930 is excluded, which means no twentieth-century science, no modern history, and none of the slang, idioms, or vocabulary that shaped the last ninety-five years of English.

To build the corpus, the team started with Harvard's Institutional Books Library, a 242-billion-token digital archive, and ran it through a cleaning pass that cut it down to 23 billion usable tokens. They then expanded the source pool to land at 20.1 billion tokens of pre-1931 English for the final training mix. Bart was trained from scratch on that corpus over a five-day run on a single Nvidia H100 GPU, holding 60 percent Model FLOPs Utilization, the share of the chip's theoretical compute that the training loop actually keeps busy. That efficiency figure is the more interesting engineering number than the parameter count, because it means the small team was not leaving performance on the table when it ran on one box.

The cost line, around $807, is the one that travels. The lab says it covers direct compute, storage, and bandwidth for the three months of experiments, not engineering time, and it is a floor rather than a true unit-economics number. Even so, it puts a price tag on what a disciplined small team can do with off-the-shelf hardware and a tightly scoped dataset.

Because no benchmark suite existed for vintage-only models, the team built one: Vintage CORE, a set of twenty tests purpose-built for language models that have never seen modern text. Alongside the suite, the team released 416,000 graded supervised fine-tuning question-and-answer pairs grounded in the same pre-1930 corpus, so other researchers can adapt the model without leaving the vintage world. The weights live on Hugging Face as jbduran/bart-sft, and a public chat demo lets anyone prod the model in a browser, with a content caveat that Bart reproduces the assumptions and biases of the period it was trained on.

Two prior projects sit in the same neighborhood. Talkie, a 13-billion-parameter vintage model, was trained on 260 billion tokens of pre-1931 English, and GPT-1900, a 3.3-billion-parameter model, was confined to pre-1900 text. The Unbounded Labs team claims Bart beats both on Vintage CORE at its scale, though the comparison rests on a benchmark the team itself built and no independent third-party reproduction has yet been published. Self-reported numbers in a first-party release deserve the usual discount, and any "best at scale" framing should be read as the team's own measurement until outside labs repeat the run.

The constraint changes what an LLM can learn, and the team is using vintage training to test how much. A model cut off from the twentieth century cannot know about computers, satellites, antibiotics, or modern English idiom, and the lab treats that absence as the variable. If a small system can still reason about the writing it has seen, then the gap between the model and the world shrinks when the corpus is treated as a deliberate choice rather than an ever-growing pile. If it cannot, that failure mode is itself a signal about how much of modern LLM competence comes from the sheer volume of recent text rather than from the underlying training procedure.

The lab is asking for compute grants, funding, and mentors to run a larger model, and points to a ten-hour autonomous research session on a single H100, which the team says produced one hundred experiments and twenty-six improvements, as proof that the approach can compound. The cheapest, smallest version of the experiment is now in everyone else's hands to test, and the team's next vintage run sits behind that funding ask.