Every week, roughly 2 million people send ChatGPT a message about health insurance. Six hundred thousand more ask it about medical symptoms from places the CDC defines as hospital deserts, meaning communities more than 30 minutes from the nearest general hospital. Seven out of ten health messages arrive after clinics have closed. These numbers, which OpenAI's Chengpeng Mou shared via Simon Willison's blog on April 5th, are drawn from anonymized U.S. ChatGPT data. They are also, in a quiet and rarely acknowledged way, training data.
The healthcare system is broken in well-documented ways. More than 40 million Americans turn to ChatGPT with health questions every day, according to OpenAI's January 2026 report, which was covered by FierceHealthcare, The Verge, and Axios. More than 230 million people globally ask ChatGPT health-related questions each week, The Verge reported. Of ChatGPT's more than 800 million regular users, one in four submits a healthcare prompt weekly. That is 200 million people or more, every week, asking a commercial system for medical guidance.
What those people are doing, whether they know it or not, is generating the highest-quality medical training signal that exists for the next version of the same system. A question about insurance coded in precise language, asked by someone who has researched their symptoms and formulated a specific query, is worth more to a model than a random sample. The people using ChatGPT as a substitute for a doctor are the same people producing the data that makes the doctor substitute better.
This is the data-labor inversion, and it deserves to be named plainly.
The standard AI narrative goes something like this: artificial intelligence serves humanity, answering questions, automating drudgery, extending capability. That story is true as far as it goes. What it omits is the direction of the arrow in reverse. Every symptom check, every insurance appeal drafted in ChatGPT, every late-night question about whether chest pain is serious, all of it flows into the training pipeline. OpenAI has said that conversations within its dedicated ChatGPT Health product are not used to train foundation models by default, The Verge confirmed. But ChatGPT itself is not ChatGPT Health. And the broader consent apparatus, the terms of service that users click through and the opt-out mechanisms that exist in some jurisdictions, is not the same as meaningful informed consent about how medical conversations improve a commercial product.
HIPAA, the U.S. healthcare privacy law, does not apply to consumer AI platforms. OpenAI's head of health, Nate Gross, said so directly in a Verge interview last month. The law that governs how your hospital handles your records has no jurisdiction over what happens when you paste your symptoms into a chat box. OpenAI faces multiple lawsuits from people whose loved ones were harmed after using ChatGPT, including at least one case documented in the American College of Physicians' journals in August 2025, where a man was hospitalized for weeks after following ChatGPT's advice to replace dietary salt with sodium bromide.
The consent question is the load-bearing issue for anyone building on top of this dynamic. OpenAI has positioned its healthcare push as humanitarian infrastructure: ChatGPT Health Connect, announced in January, links to medical records and offers a dedicated health experience. OpenAI worked with more than 260 physicians providing feedback on model outputs more than 600,000 times across 30 areas of focus, The Verge reported. The company is investing in the appearance of medical-grade responsibility.
But the underlying data relationship has not changed. Patients are doing labor, formulating questions, generating examples of medical language, surfacing edge cases in how people describe symptoms, and that labor is making the product better without compensation or, in any meaningful sense, consent. The opt-out mechanism exists, but it requires users to find and exercise a right they almost certainly do not know they have.
For founders and engineers, the uncomfortable implication is that the most transformative AI healthcare product of the decade may be built substantially on the unpaid cognitive work of people who had no other option. The 600,000 weekly messages from hospital deserts are not a user base. They are a training set. The 40 million daily health questions are not a distribution channel. They are a data flywheel.
OpenAI can defuse this framing if it chooses. Demonstrating genuine anonymization and building opt-out mechanisms that are as accessible as the product itself would go a long way. The ChatGPT Health product's explicit exclusion of conversations from foundation model training is a start. But the broader dynamic, across the non-Health product, remains. Until that changes, the healthcare AI revolution is running on something it has not named.
