AINEWS
A Sharper 'Forget' for Image Generators
A new training free operator, CARE (Covariance Aware Retained subspace Erasure), erases a single concept from a text to image model while leaving related styles and subjects intact.
A new training free operator, CARE (Covariance Aware Retained subspace Erasure), erases a single concept from a text to image model while leaving related styles and subjects intact.
When image generators forget a concept, they often forget too much. A new method called CARE (Covariance-Aware Retained-subspace Erasure) takes on the "collateral damage" problem in text-to-image diffusion editing, the family of models behind Stable Image, Midjourney, and DALL·E. Rather than retraining, CARE computes a single closed-form direction that strips a target concept while leaving the model's broader visual knowledge intact. The paper was posted to arXiv this month with code released on GitHub.
Diffusion models store visual concepts as directions in the cross-attention value space, the part of the model that ties prompt words to image features. To remove a celebrity likeness, a copyrighted style, or an explicit category, prior value-space methods computed one direction associated with the target and subtracted it. The target went away. Neighboring styles and unrelated subjects often went with it, because the single direction also carried structure the operator was meant to keep.
The deeper issue, the authors argue, is that one direction conflates two things at once: what makes the target distinctive, and what it shares with concepts the operator should preserve. CARE separates them by anchoring an explicit "retained subspace," a bank of value vectors drawn from the concepts meant to survive the edit. A shrinkage parameter γ controls how aggressively the operator whitens the erasure direction relative to that subspace. When γ is large, CARE reduces to the plain value-space erasure that came before it. At finite γ, it projects out components of the target direction that align with retained concepts and keeps only the components that distinguish the target.
The math is unglamorous but cheap. A single offline Woodbury inversion on an M-by-M matrix (where M is the number of retained anchor vectors, far smaller than the model's hidden dimension) produces the corrected direction. The paper reports about 1.2 seconds of offline preprocessing and roughly 0.7% per-image generation overhead after that. Inference stays in the normal diffusion loop; the operator runs once per concept.
At generation time, the edit is a gated rank-one subtraction on the model's value activations. The operator scales the corrected direction by the cosine similarity between the target and the prompt's value vector, then subtracts that term only where the target and the prompt actually align. Tokens unrelated to the target pass through untouched, which is what cuts the collateral damage.
According to the authors' SCOREBOARD ablations, CARE preserves non-target concepts more cleanly than ESD, Saliency-based methods, MACE, and ReFACT-style baselines, while keeping erasure rates competitive. The summary sits on Takara's paper digest, with the full table set in the arXiv PDF.
Two caveats travel with the result. The evaluation is the authors' own; concept-erasure benchmarks vary across the field, and no independent replication surfaced with this preprint. A sharper operator is not a policy answer either. Concept erasure remains an imperfect control surface for deepfakes, copyright, and NSFW filtering. A method that removes a style cleanly does not settle what should be removed, or who decides.
The code is the most testable part of the release. The GitHub repository ships a flowchart that walks the pipeline from prompt text and target concept through the Woodbury solve to the gated rank-one subtraction, and the arXiv paper lays out the operator in full. The watch items for practitioners: whether the closed-form direction transfers to diffusion backbones beyond the tested set, whether the retained-subspace construction generalizes when "what to keep" is chosen by a deployer rather than a researcher, and how the shrinkage parameter γ holds up across concepts the model was not benchmarked on.