AINEWS
The filter is supposed to strip personal details before a contractor reads a conversation. On shorter chats, OpenAI admits it can let them through.
A contractor opens a chat in a dashboard at OpenAI. On the left, a stranger's question to ChatGPT. On the right, four candidate responses from different model versions, each waiting for a 1-to-7 rating. The contractor is supposed to be a stranger to the person who typed that question: usernames are stripped, and an automated privacy filter is supposed to scrub identifying details before the conversation reaches a human. According to 404 Media's investigation of the program, internally called Project Lily, that filter is most likely to fail on the shortest chats.
The chats most likely to slip through are also the ones users treat as most private. OpenAI admits, in materials reviewed by 404 Media, that the filter can let personal information through when conversations are short and the personal detail is embedded in the prompt itself. A user asking a brief question about a medical symptom, a workplace conflict, or a relationship may have nothing else in the chat for the filter to anchor to. Some conversations reviewed by contractors are also paired with a "user memories summary," a compressed version of facts ChatGPT has stored about the user, which the same filter is supposed to clean. The reviewer then has to mark at least three parts of each candidate response and explain the reasoning, so any personal detail that does slip through is also written into the training feedback.
The scale of the operation is what turns the gap into a structural problem. Tom's Hardware, India Today, and CryptoBriefing re-reporting 404 Media's investigation describe hundreds of contract "prompt reviewers" working through a queue of real conversations. Tom's Hardware cites one reviewer saying pay is "over $50 an hour" and that guidelines change frequently and are often self-contradictory. 404 Media cites TechCrunch for more than 900 million weekly active ChatGPT users, the volume of chats flowing past this filter every week.
What the reviewers are rating is not whether the bot was right. The training target is style: penalize AI-speak, patronizing tone, excessive emojis, claims of real-life experience or emotion, and forced mimicry of the user's voice. The framework, as 404 Media reports it, runs from "unacceptable, unusable" to a response that "would be hard to meaningfully improve." That push has acquired legal weight. 404 Media notes the over-sycophantic 4o model has been linked via lawsuits to multiple user suicides, which is part of why the contractor queue exists at this scale.
The opt-out is a single toggle in ChatGPT's settings, labeled "Improve the model for everyone." OpenAI's help center confirms it is on by default for Free, Plus, and Pro users and applies only to new conversations after it is switched off. Enterprise, Business, and Edu tiers have it off by default. The default sits on the side of more training data; switching it off does not delete chats that have already been reviewed.
Anthropic confirmed to 404 Media that it uses human reviewers, and Google's Gemini warns users that some saved chats may be reviewed by humans. OpenAI's setup is the one with a documented filter gap on short chats, and the toggle to act on it.
The relevant test for a ChatGPT user is whether the prompt contains personal detail in a short chat, the configuration in which OpenAI's own filter is least reliable. For those, the toggle exists. Most users have not seen it.