AINEWS
Princeton and Chicago researchers found that newer AI reasoning models developed their own biases during simulated hiring rounds, not from training data.
In a 40-round simulated hiring game adapted from a psychology study on human stereotyping, OpenAI's newest reasoning model, o3, sorted job candidates into job niches 1.83 on a 2.0 segregation scale while human players managed 0.84. The bias did not come from training data. It emerged from the model's own experience making hiring decisions.
That mechanism is the finding from a new Princeton University and University of Chicago preprint, reported by MIT Technology Review. It puts a new kind of pressure on every job seeker facing an LLM résumé screen, every HR team buying these tools, and every regulator writing AI-employment rules. The standard audit checks whether an AI absorbed bias from its training corpus. This study suggests the bias forms after deployment, in the act of choosing.
The researchers put ChatGPT, Claude, and Gemini into a mayor's-consultant role. Each model hired for 20 jobs spanning doctors, lawyers, child-care aides, and janitors, evaluating candidates from four fictional ethnic groups the researchers named Tufa, Aima, Reku, and Weki as stand-ins for demographic categories, so no real group was named. Every candidate was equally likely to succeed; quality was held constant across groups. Over 40 rounds, the models learned to segregate.
Within a few rounds, the pattern set in and held. After one Aima candidate failed as a doctor, the model steered subsequent Aimas toward janitor roles it had already classified as lower in warmth and competence. Human players in the original psychology study did the same thing, just less. The newer, more capable reasoning models, the class of AI that thinks step-by-step before answering, did it more, not less.
OpenAI's o3 hit 1.83 on the segregation scale, near the maximum of 2.0. DeepSeek's R1 ran close behind. The earlier ChatGPT and Claude versions showed milder but still measurable segregation. The relationship ran the wrong way: more capable reasoning models segregated harder, not softer. As labs ship still more powerful models, and as those models keep more user memory between sessions, the surface area for adaptive stereotyping grows. That growth comes just as enterprises are wiring LLM screeners into applicant-tracking systems.
The researchers call the failure mode "feedback-loop bias." Standard fairness audits check whether a model reproduces stereotypes from its training data. They do not check whether a model invents new ones by tracking who it hired and who quit. An employer that has done a vendor's standard bias audit may still be running a tool that, given a few hundred simulated hires, will sort résumés into niches the audit never looked for.
The next round of AI hiring tools will be agentic: systems that remember earlier conversations, follow up with candidates, and tune their screening criteria over weeks or months. The same memory that makes those tools useful on a candidate's fifth interaction is what gives them more ammunition for stereotyping, per MIT Technology Review's account of the paper. The team's framing: the more memory a model retains, the more raw material it has to build novel biases from outcomes.
For a job seeker, the practical test is whether the prospective employer's screening AI has been audited for feedback-loop bias, not just for training-data bias. The question is uncomfortable to ask in an interview, and it is the right one. For an HR or procurement team, the test is whether the vendor's audit methodology, not just the audit certificate, covers adaptive exploration. For a regulator, the test is whether AI-employment rules written around training-data discrimination still cover discrimination the model invents on its own.
The Princeton-Chicago team plans to extend the work to live applicant-tracking systems in partnership with employers who will share anonymized hiring data. The preprint is not yet peer reviewed. The mechanism the game revealed, that capable models can build their own stereotypes from outcomes the same way human decision-makers do, will be easier to argue against once a real applicant-tracking system shows it. Until the live data lands, the next interview question for any employer running LLM screening is the same one a hiring audit would ask: has the model been checked for stereotypes it builds on its own, or only for the ones it inherited?