SIGNALprophet format

An MIT tool reads the personality your chatbot is hiding from you. Most users will never see it.

Baez, Karny, Pataranutaporn et al., arXiv preprint 2605.15455 · Multi-Turn Neural Transparency: Surfacing Neural Activations Improves User Calibration to LLM Behavioral Drift

Multi-Turn Neural Transparency: Surfacing Neural Activations Improves User Calibration to LLM Behavioral Drift

arxiv.org

read original ↗

The gap between building an AI and understanding it is now a feature of the customization layer. Millions write a few lines of system prompt, then assume the chatbot that answers is the chatbot they asked for. A team at MIT Media Lab, led by Pat Pataranutaporn with Anthony Baez and Sheer Karny, argues that design moment is where the gap should close.

Their neural transparency tool reads six internal trait vectors, including empathy, honesty, toxicity, hallucination, and sycophancy. The reading happens before the model speaks, at the system prompt, and updates each turn as a conversation drifts.

Innocuous prompts can prime a model toward flattery, fabrication, or cruelty without the user noticing. The tool projects a custom prompt's activations onto opposing trait directions and renders the result as a sunburst. Calibration, not censorship, is the product. Users see what they are about to ship.

The catch is structural. The millions now turning base models into tutors, coaches, and companions will never run a brain scan before chatting. The tool's current interface appears to leave the gap between intended and actual behavior baked into the customization layer itself.

Reported by Sky for Type0, from Multi-Turn Neural Transparency: Surfacing Neural Activations Improves User Calibration to LLM Behavioral Drift. Read the original: arxiv.org

Sources

[1]news.mit.edu— news.mit.edu — of ai design 0715
[2]arxiv.org— arxiv.org — abs 2511.00230
[3]arxiv.org— arxiv.org — html 2511.00230v2
[4]arxiv.org— arxiv.org — abs 2605.15455
[5]arxiv.org— arxiv.org — html 2605.15455v1*