Google Open-Sources a Compiler for AI on Encrypted Data

For most of the last decade, the deal between users and AI services has been a one-way trade. Either the data stayed protected from a breach, or it stayed useful to the model that recommended the next video, ranked the next result, or summarized the next document. Running both sides of that bargain at once meant trusting the operator with the plaintext, then hoping the operator did not lose it.

Google's new open-source compiler, HEIR (Homomorphic Encryption Intermediate Representation), doesn't change that bargain. What it changes is who can build the alternative. Homomorphic encryption lets a server compute directly on encrypted data and return an encrypted result that only the client can decrypt. For years, getting a model to run that way efficiently required a team of cryptographers to hand-tune every operation. HEIR, released in August 2025 and documented on heir.dev, is meant to make that tuning automatic across different homomorphic-encryption schemes, so an engineer who knows Python and a model specification can ship a private-inference demo without a PhD on staff.

The mechanism matters because the underlying idea is older than most of the AI stack it now wants to plug into. Fully homomorphic encryption (FHE) has been a research goal since 2009. The catch has never been feasibility; it has been throughput. Multiplying two ciphertexts costs orders of magnitude more than multiplying two unencrypted numbers, and a typical neural-network layer does that multiplication millions of times. The Google blog post on Private Computing puts the trade-off on a cost curve that is finally bending: HEIR's compiler pipeline removes the manual per-scheme translation step, which had been the wall between "it works on a researcher's laptop" and "it runs in someone's product."

The arXiv preprint 2508.11095, "A Universal Compiler for Homomorphic Encryption," and the NDSS paper "HEIR: A Unified Representation for Cross-Scheme Compilation of Fully Homomorphic Computation" describe a single intermediate representation that targets multiple FHE backends, including BFV, BGV, CKKS, and TFHE. That is the part engineers used to do by hand. HEIR also lands inside Google's Private Computing Toolkit, alongside differential privacy, private set membership, and private information retrieval, which positions the release as one piece of a layered privacy stack rather than a standalone privacy product.

The honest counterweight sits in the Hacker News thread on the release, where the working assumption is that FHE remains "traditionally horrifically slow" in absolute terms, and that HEIR is most likely to enable small, stripped-down models, such as a content-recommender over a few thousand encrypted features, rather than frontier-scale LLMs. The compiler removes the cryptographer; it does not remove the arithmetic. Anyone trying to run a frontier-scale model on ciphertext through HEIR today will still wait longer than the heat death of a customer session.

Homomorphic encryption is no longer a category that exists only on a research whiteboard. It is a category that exists in a publicly auditable compiler and a getting-started guide that any team can fork. The user-data trade-off is no longer a question of whether private AI is possible. It is a question of how many milliseconds of latency, how many dollars of compute, and how many cryptography hires a given team is willing to absorb. The cost is moving, and the direction is down, but the cost is still real.

The watch item from here is whether the model zoo grows. HEIR's examples directory ships with a small set of hand-picked demos; what tells the next chapter is whether independent contributors add more, and whether the latency numbers in the arXiv paper's benchmarks start to land in production traces rather than only in conference tables. The compiler is live. The cost story is just starting.