A $400 GPU Tested OpenAI's Recipe for Installing a Character Trait. It Moved 2.4 Points.

One researcher, one $400 Nvidia graphics card, 200 training steps, and 20 distinct prompts moved an AI's "traditionalism" trait by 2.4 points on a 100-point scale. The paper being reproduced treats anything under roughly +15 as too small to count as a successful "install." The reproduction did not falsify the paper's claim. It measured the compute gap between the lab that produced the work and the smallest test an outsider can run.

The result is documented in a single Reddit thread in r/MachineLearning. The reproducer's own diagnosis, after ruling out four failure modes, is that the install most likely failed for methodological reasons: too few prompts, a noisy judge model, and roughly five orders of magnitude less compute than the paper used.

"Persistently beneficial" is the paper's term. In plain language, the goal is to teach a model to hold a particular character trait, in this case a measured, conservative style of answer (OCEAN low-Openness, in the standard personality framework the paper uses), even when a user is trying to talk it out of that trait. The paper's own threshold for declaring the install adequate is roughly a +15-point move on the trait rubric, surviving that adversarial pressure.

The base model is Qwen2.5-7B-Instruct, a 7-billion-parameter open-weights model. The training method is GRPO, a reinforcement-learning algorithm that ranks a group of generated answers against each other rather than against an absolute reward, run with a small set of trainable weights on top of the frozen model, a parameter-efficient technique abbreviated LoRA. The hardware is a single RTX 3090, a consumer-grade Nvidia graphics card that retails for around $400 used. The judge model that scores the answers is gpt-4.1-mini, a closed-source third party, not the paper's original setup.

The trait being trained is OCEAN low-Openness, chosen because the 7B model starts at 57/100 on the rubric with a wide spread, which gives the training loop measurable headroom. The reward is 85% quality and 15% coherence, with a hard gate against degenerate, looping, or refusal output. Twenty-five percent of the prompts are trait prompts; the other 75% are general chat prompts from the no_robots dataset.

After 200 steps, the trait moved from 57.0 to 59.4, a +2.4-point shift with a 95% confidence interval of [+0.2, +4.8]. The reproducer's own internal bar for the install to be "adequate" for testing persistence was about +15. They were not there.

The reproducer then walked through four failure modes and ruled each one out. Coherence held at 76 after training. Answer length was exactly the same as the base model, so the model was not collapsing into a shorter-output reward hack. Repetition was 0%, and refusals were 0%. The model scored 58.9 on the 20 training prompts (each seen 10 times) and 59.4 on held-out prompts, so it never memorized the training set, which means it could not have failed to generalize from it. The judge separates the six sampled answers per prompt by about 18 points on average, and only 25% of GRPO groups showed degenerate reward spread, so the gradient was not dead. An independent upstream evaluation, whose name the thread does not give, gives +3.4, and the reproducer's own generated prompts give +2.8, both close enough to the main result to rule out a question-writing artifact.

One real confound was found and fixed mid-run. A completion-length cap was truncating 30% to 70% of samples and zeroing their reward, which meant about 90% of what looked like early "learning" was the model learning to be shorter. After the cap was removed, the trait signal was still flat. The +2.4 result is post-fix.

A paper author, contacted by the reproducer and paraphrased in the thread, offered two concrete reasons the install might not have taken at this scale. Twenty distinct trait prompts is too few; the paper's full setup uses a much larger prompt set. Per-example prescriptive rubrics, a separate scoring rubric written for each prompt rather than one shared rubric, appear to matter for the install step. Both are method-level fixes the reproducer has not yet tried.

The OpenAI alignment blog on the work and Karan Singhal's notes on the same paper describe the result under the paper's own compute regime, and neither claims a $400 GPU at 200 steps is the right test. The next experiment, the reproducer says, is more prompts, per-example rubrics, a different judge model, and, if the compute ever becomes available outside the lab, the paper's full setup. A frontier safety claim that can be checked only by the lab that produced it is a claim the field cannot audit, and a $400 card with 200 steps is the current floor of who gets to try.