Supervised Fine-Tuning Copies Secrets Verbatim. Reinforcement Learning Doesn't.

Supervised fine-tuning copies secrets into a model's weights, verbatim. Reinforcement learning on verifiable rewards writes nothing of the kind. A new measurement paper, posted to arXiv last week, puts that contrast in bits and traces it back to design choices a team can actually change.

The paper, "How Many Bits Can an Adapter Write?" by Kaizhen Tan, extends a 2026 compression-based memorization test (Morris et al. 2026b) to the dominant way large models are specialized today: low-rank adapters, or LoRA. LoRA freezes the base model and trains a few small matrices on top; the result is a tuning file a fraction of the size of a full fine-tune. The question Tan asks is what those small files actually remember.

The instrument borrows the Morris et al. trick: feed both a reference model and a tuned model uniformly random token sequences, and measure how many bits the arithmetic coder saves with the tuned version. On random data, every saved bit is a memorized bit. Tan reports three numbers per adapter. Unintended memorization mem_U is how much the tuned model shaves off the reference's codelength on random text. Parameter-write cost CL(Δθ) is the codelength of the trainable weights themselves, treating the adapter as a small file to be compressed. Behavior-write bits W(D) is how far tuning moved the base model's likelihoods on a canonical answer set.

The headline number: LoRA adapters store roughly two bits per trainable parameter, well short of the ~3.6 bits per parameter Morris et al. found for full fine-tuning. Tan confirms this by fine-tuning the same base under the same budget. A rank-16 adapter on a 7-billion-parameter Qwen2.5 base carries on the order of 4×10⁷ trainable weights; at the full-model rate that would be roughly 20 MB of training-text equivalent, but real adapters come in far below that line.

Where the parameters sit matters more than how many there are. Moving the same budget from attention layers into the multilayer perceptron roughly doubles capacity. Halving the storage precision barely changes what the adapter reads back. On a random, structureless frozen base, the same adapter collapses to about 29% memorization, against roughly 98% on a pretrained base. The base model is doing most of the storage work; the adapter is steering it.

That gap is where the design choices live. Tan's bit-level audit links the written bits directly to canary extraction and membership-inference leakage on identical checkpoints. On realistic Qwen2.5 fine-tunes, leakage rises with the bits the adapter actually writes, not with the parameters it nominally carries. Parameter count is the wrong number to audit.

At matched downstream accuracy, an adapter trained with reinforcement learning from verifiable rewards via GRPO writes nothing the estimator can distinguish from zero. Private secrets that supervised fine-tuning copies down verbatim, an RLVR adapter never records. The result is the first bit-level SFT-vs-RLVR comparison on identical adapter checkpoints, and it lands at the moment adapter-based specialization has become the production default and safety teams are starting to ask for memorization budgets rather than parameter counts.

The paper is a v1 preprint with no peer review yet, and the comparisons sit on Qwen2.5-family bases. Generalizing to other base families or to RL algorithms beyond GRPO is the author's own future work, not a present claim. Within that scope, the lesson is designable: a team that wants a smaller memorization budget picks the training regime first, then the layer placement, and only then the parameter count.