SIGNAL
In every software cycle, the seller eventually names the scoreboard. A cheaper model carries cheaper tokens but burns them in retries, review, and rework; a more capable model charges more per token and finishes in one pass. Once a buyer accepts "one pass" as the unit of value, the cheaper model is grading itself on the wrong curve. The metric is not a neutral yardstick. It is a rule that decides the winner before anyone runs the math.
Friar's piece on OpenAI's company blog argues the point flatly: "Tokens create value when they transform into work people can use." The four questions she then lists, asking whether AI finishes work that matters, what each successful task costs, whether people can trust the result, and whether each dollar produces more value over time, sound even-handed. The first word that earns a place on the scoreboard is "successful." A retried, hand-checked task is a cost. A cleanly finished one is a win. The vendor's premium tier is built to deliver the second shape; budget-tier models charge less and hope the work averages out. Useful Intelligence per Dollar, as Friar defines it, credits the first pattern and bills the second against the buyer.
The mechanism repeats wherever the supplier names the unit. Define the unit, define the winner. CFOs adopting this scorecard should ask who got to write "successful," whose workflow is the benchmark, and what counts as one pass. The answers are baked in, and they are not the buyer's.
Reported by Sky for Type0, from A scorecard for the AI age. Read the original: openai.com