AINEWS
A Chinese lab's new daily dilemma value alignment benchmark, D2VBench, grades eight major LLMs on loyalty, honesty, and family versus work tradeoffs. It also surfaces what daily life value alignment still misses.
What if the real test of an AI's ethics is not the trolley problem, but the dinner where saying yes to a friend means saying no to your kid?
That is the kind of question a new benchmark from China's TJUNLP lab is built to ask. D2VBench, released this month as an arXiv preprint, replaces the high-minded thought experiments that have long dominated AI ethics scoring with 10,000 everyday scenarios drawn from the tradeoffs people actually negotiate: honesty against loyalty, family against work, the individual against the collective.
Value alignment is whether an AI's choices match human values when those values conflict. It is one of the hardest things to measure in language models, because human ethics are not a list of rules but a stack of competing pulls. D2VBench is the latest attempt to make that mess testable.
The benchmark is grounded in 158 manually annotated value concepts, from filial duty to personal autonomy to professional integrity. Researchers then used a multi-stage process of LLM-assisted generation and human review to expand those concepts into the 10,000 instances. Each instance is paired with a hybrid evaluation: multiple-choice questions that score pattern recognition, plus open-ended questions that score how the model reasons out loud when more than one value is on the line. Eight mainstream LLMs were run through the full set, and the authors report the benchmark separates them across both categories and value dimensions, with claimed high reliability and robustness.
D2VBench is also a critique of what came before. The paper argues that prior AI ethics benchmarks have covered daily value conflicts too thinly and have leaned on simplistic evaluation formalisms: single-answer multiple choice, ranking tasks, or short-fill prompts that reward a particular answer key. By contrast, the hybrid MCQ-plus-open-ended structure is designed to catch the difference between a model that picks the right answer and a model that can articulate why, and why not, when values collide.
A few limits are worth flagging. The benchmark was built by a single research group using LLM-assisted generation with human review, which means the value taxonomy and scenario selection reflect one lab's cultural and definitional choices. The 158 concepts and the 10,000 instances are an editorial act as much as a measurement, and the eight models tested are not identified in the public summary. The full paper HTML is the place to read the actual list, prompts, and scoring rubric, and the preprint is also indexed on HuggingFace Papers for discovery. The dataset and code are publicly released on GitHub, so independent teams can rerun the evaluation and stress-test the claims. The paper is a preprint and has not been peer-reviewed, and the source basis for this analysis is a TLDR summary of the original, so any stronger claim about reliability or model differences still needs the primary text behind it.
The next AI ethics scores will be graded on whether a model can reason through a dinner invitation that costs your kid a recital, not just whether it can pass the trolley test. That is a useful step forward for the field, and an open invitation to argue about what belongs in the answer key.