She got four bottles of the wrong drug. Her pharmacy won't name the AI calling its patients.

Kathy Callaghan's phone rang at her home in Vermont. An AI voice on the other end muttered the name of a medication, slurred and barely intelligible. She could not tell which drug the system was asking about. She approved the refill anyway, the way she had many times before. Four bottles of a medication she takes only twice a week arrived on her counter, and she had not asked for any of them.

Callaghan is one of nearly a dozen Kinney Drugs customers across Vermont and New York who told VtDigger that the pharmacy chain's new AI refill assistant is failing in ways that go beyond awkward phone calls. Some say the system has lost track of longtime accounts, ordered wrong dosages, or failed to notify them when a prescription is ready. Others describe long delays between a doctor's order and the pharmacy filling it. A few have stopped going to Kinney Drugs at all. Others say they cannot, because no other pharmacy serves their rural town.

Kinney Drugs, a regional chain with stores across Vermont and upstate New York, rolled out the AI assistant earlier this summer to "improve communication and efficiency" around prescription refills. The company has not publicly identified the vendor behind the system or described how it handles customer data, calls, or the handoff to a human when something goes wrong.

The pattern matches what the ACLU and other privacy advocates have warned about as AI moves into healthcare at scale. "There's a delay in laws that are coming onto the books, so we've not seen a detailed picture of how companies are going to comply or how states are going to enforce the laws," said Cody Venzke, a senior staff attorney with the American Civil Liberties Union. Venzke added that healthcare AI is outpacing both state and federal privacy rules.

Vermont's main response so far is a data privacy bill, S.71, that Gov. Phil Scott signed on June 16. The bill is meant to give Vermonters new rights over how companies collect and use their personal data, including health data handled by AI systems. It does not take effect for two years. Privacy advocates argue the bill is already too weak and that, even at full strength, it would not have caught the kinds of failures customers describe at Kinney Drugs, because the bill targets data collection and consent, not the operational accuracy of an AI that places a phone call and asks a patient to approve a prescription.

The result is a two-year window in which a pharmacy AI can place calls and act on patient answers, with no clear rule for what customers should be told about the system, what they have to consent to, or who is liable when the system orders the wrong drug.

Callaghan still gets the calls. The AI voice still mutters drug names she cannot parse. She has stopped approving the refills and is asking her doctor to move her prescriptions elsewhere. For now, she has nowhere else in her town to take them.