AIBRIEF
torchwright — a hobbyist built compiler — writes the exact internal weights of a small transformer shaped like Microsoft's Phi 3 language model and emits a standard model file that loads in Hugging Face, the popular open AI model hub.
What can a transformer express before any training happens? A hobbyist builder going by physicsrob has published a small open-source project called torchwright that tries to answer by construction.
Write a Python computation graph, and the compiler emits the exact weights of a vanilla Phi-3-shaped transformer that runs it. The output is a stock Hugging Face checkpoint that loads with no custom code. There is no gradient descent anywhere; the weights are written, not learned.
The repo ships twelve runnable demos. One is a binary-increment circuit with carry propagation; another is a single-digit adder. These are small, deterministic algorithms, the sort a neural network would normally have to learn from examples.
The lineage runs through RASP and Tracr, which mapped custom languages onto transformer sublayers years ago, and the builder's own post credits them. The differentiator is twofold: ordinary Python goes in, and the output drops into vanilla HuggingFace with no shim.
torchwright is a probe. The framing stays narrow: separate what a transformer architecture can express from what it can learn. The answer covers tiny circuits, not trained behavior.