A company spent a year grading people on how much AI they used. Somebody built an internal leaderboard ranking colleagues by token consumption. Then the whole thing got walked back.
Every adoption number in the field measures the same thing. That the tool got opened. Not one of them can tell you whether the work got better.
The leaderboard nobody should have needed
WIRED reports that a large public company told workers in early September that performance reviews will no longer depend on how much they used AI tools. The new guidance replaces criteria like usage of AI, and an AI Native designation, with looser wording about outcomes being supported by AI or by other means. A company spokesperson told WIRED the update emphasizes what was always true, that people are evaluated on their contributions, and that the labels were never used for evaluation.
The same reporting says employees prompted the tools repeatedly to run their internal numbers up. It says one worker built a leaderboard ranking colleagues by token consumption, with a top tier, and that the dashboard came down after it leaked. There is also a lawsuit from about two dozen employees alleging the usage labels disadvantaged people on leave. The company denies it and the case is open, so that stays an allegation and nothing more.
Take the company name out and this stops being a story about one company. It is what happens to any input that gets graded.
Somebody wrote down a number that was easy to collect. A year later it had a leaderboard on it. Then it got quietly retired, and that last part is the tell. If the usage numbers had proven anything, nobody would be walking them back.
I do not want a number that says my agents ran
My own content system produces posts on a schedule. Drafters, a critic, a publisher, a watcher. For a stretch this summer it reported 52 of 60 posts stuck unshipped.
The instinct was to add more drafters. Production looked like the problem, because production was what the dashboard was counting.
11 of those posts were already live. The publish watcher matched one status and skipped the rest, so posts that had shipped never got marked as shipped. Fixing the watcher took the posted count from 7 to 18 in an afternoon. Nothing about the drafting changed.
The production number was fine the entire time it was wrong.
One line came out of that week and I have kept it since: a watcher that matches the wrong status is worse than no watcher at all. It reports health while the work silently fails. Silent success states are the dangerous ones, because there is no error to go looking for.
I don't want a number that tells me my agents ran. I want a number that tells me something landed. Those turned out to be two different numbers in my own system, and they disagreed for weeks.
Grade an input and you will get the input
From my perspective this is older than AI and the shape never changes.
Adoption. Token spend. Sessions opened. Tools connected. Agents deployed. Every one of those counts a reach for the tool. They are inputs.
The moment an input becomes the thing being scored, people generate it. Not out of bad faith, mostly. If the review says usage, then usage is something anybody can produce on a Tuesday afternoon with no judgment involved at all. So it gets produced, and the number stops carrying information about the work.
Here is the honest version. A token count can see that the tool was opened. It cannot see the hours somebody decided not to hand over.
I run agents across most of what I do, and the calls I would defend hardest are the ones where I kept the work. The gate I refuse to automate. The draft I rewrote instead of regenerating. The migration I tested small before believing the plan. None of that shows up as usage. Some of it shows up as less usage.
Grade the input and you have told everyone in the building to go generate it.
The field is drowning in input numbers right now
Look at what got published this past month and count how many of the headline figures are inputs.
An orchestration vendor's 2026 developer report, built on 554 engineers who already use agents, says daily or more frequent agent use went from 47.3 percent a year ago to 80.8 percent now. The same report says 41.1 percent hit agent issues daily or more. That vendor sells agent orchestration, and the number one blocker its respondents name is tracking state, which is the product, so read the ordering with that in mind.
The finding worth stopping on is the one the report deserves credit for printing. The number of agents deployed and the frequency of issues are not correlated. Teams running dozens do not break more often than teams running five.
A global consulting survey published in August, 1,719 respondents across 97 countries, says 32 percent of organizations decided against buying at least one software product because a coding agent could build the functionality in house. The same survey says the share attributing significant financial impact to AI stayed flat at roughly 6 percent.
Adoption up. Agents up. Software purchases displaced. Financial impact flat.
Every number in that stack is an input except the last one, and the last one did not move.
The number that survives the test
So here is the test, and it fits in a sentence.
A number is worth grading only if it moves when the work lands and stays flat when the work does not.
Run it on your own system. It takes about ten minutes.
Name the thing at the far end of your pipeline that somebody outside your team actually touches. A customer. A reader. A merged branch. A paid invoice. Count that one.
Then take every number upstream of it and demote it. Not delete, demote. Those become debugging signals and they stop being evidence.
Then find a pair of numbers in your own system that should move together, and check whether they actually do. That is where the finding is. In mine, drafted is a debugging signal and posted is the metric. They disagreed for weeks, and the only reason I ever looked was that the far end stopped moving while the near end kept climbing.
If both of your numbers only ever go up, you are probably counting two inputs.
What the input numbers are actually good for
I don't want to over-correct into an absolutist position here, because input counts are real diagnostics and I use all of them.
Token spend tells you what a run costs. Session counts tell you where the load sits. Issue frequency tells you which wiring is weak. Cache hit ratio tells you what you keep re-sending. Those belong on a dashboard and I would fight to keep them there.
They earn that place right up until somebody attaches one to a review, a target, or a claim about value.
The line is this. Diagnostics inform. Metrics judge.
A diagnostic is something you read when you are looking for a problem. A metric is something you are willing to be held to. Confusing the two is how a year of grading people disappears in a single memo.
Nobody grades what already works
The second half of that reversal is the part most readers will skip past.
The mandate got dropped and the agent kept shipping. Encouraged, not required. And WIRED reports that token consumption keeps rising anyway.
That is the argument against the mandate in one line. If the tool is good, you do not have to grade people on using it. Usage takes care of itself, and then it goes back to being what it always was, a byproduct rather than a target.
There is something I would want to know before anybody over-applies this. I don't know which of my own numbers will look naive a year from now. Posted is a better number than drafted, and it is still not the thing I care about, which is whether a reader did anything differently afterward. I have not solved that one. I am not even sure the far end of my pipeline is far enough out.
What I trust is the test itself, not my current answers to it.
I write these up as I work them out, including the parts that did not hold, in The Production Layer at timurtek.com.
The numbers worth keeping are the ones you could not fake if you tried.
