The New AI Vanity Metric Nobody Is Talking About


One of the things I find fascinating about technology is that while the technology changes, human behavior rarely does.

Every generation of business leaders eventually falls into the same trap.

We become obsessed with measuring the thing that is easiest to count rather than the thing that actually creates value.

A few decades ago, organizations measured productivity by hours worked. Then we measured meetings attended. Then we measured emails sent. Then we measured activity in CRM systems. Today, some organizations are starting to measure AI success through token consumption. And I think we are watching the same mistake unfold all over again.

Over the past year, “token maxxing” became a surprisingly common phenomenon inside AI-forward organizations. Leaders wanted proof that employees were embracing AI. Teams wanted evidence that their investments were paying off. Employees wanted to demonstrate that they were participating in the AI revolution.

The result was predictable.

Token usage became the metric.

The assumption seemed logical. More tokens meant more AI usage. More AI usage meant more adoption. More adoption meant more innovation.

Unfortunately, that logic breaks down very quickly.

Because activity is not the same thing as value.

Stories began emerging of organizations tracking token consumption, creating internal leaderboards, and celebrating employees who generated enormous amounts of AI activity. In one reported example, an engineer consumed over 281 billion tokens in a single month. Across the industry, teams began building increasingly complex agent workflows, running lengthy reasoning chains, and generating vast amounts of AI activity.

The problem was not the technology.

The problem was the behavior the metric encouraged.

Whenever people are measured on a specific number, they naturally optimize for that number. Some employees began creating AI tasks that produced little meaningful business value. Others ran workflows that looked impressive from an AI utilization perspective but delivered questionable outcomes. In some cases, organizations unintentionally created an environment where looking busy with AI became more important than creating business impact with AI.

None of this should surprise us.

We have seen this movie before.

When sales organizations measure call volume, people maximize calls. When support organizations measure handle time, people shorten conversations. When engineering organizations measure lines of code, developers write more code.

Every metric creates a behavior.

The challenge for leaders is making sure the behavior aligns with the outcome they actually want.

What makes this particularly important in AI is that the economics are changing rapidly.

During the first wave of generative AI adoption, many organizations benefited from heavily subsidized usage. Flat-rate subscriptions masked the true cost of the underlying compute. Teams could experiment freely without thinking too much about the economics behind every interaction.

That environment is beginning to change.

Token costs have fallen dramatically. Models are becoming cheaper and more capable every year. Yet at the same time, overall consumption is exploding.

Why?

Because modern AI systems do far more than answer questions.

They reason. They search. They use tools. They read documents. They call APIs. They execute workflows. They collaborate with other agents.

Every one of those activities consumes tokens.

And increasingly, many of those tokens are invisible to the end user.

A simple request may trigger thousands or even millions of reasoning tokens behind the scenes. Long-running agents can repeatedly process the same information. Poorly designed workflows can continuously reread context, re-execute actions, or invoke tools unnecessarily.

The result is that organizations can accumulate substantial AI costs without realizing it.

One reported company reportedly generated an AI bill approaching $500 million in a single month because usage controls and governance mechanisms were not in place.

That number is shocking.

But it also points to a much larger lesson.

The issue is not that AI is expensive.

The issue is that unmanaged consumption rarely produces proportional value.

This is where I believe the conversation needs to shift.

The future of enterprise AI is not about token consumption.

It is about token efficiency.

Just as organizations measure revenue per employee, profit per customer, or cost per acquisition, they will increasingly need to think about value per token.

The question leaders should be asking is not:

“How much AI are we using?”

The better question is:

“What are we getting for every dollar we spend?”

For example:

  • Did customer onboarding become faster?
  • Did implementation timelines decrease?
  • Did support resolution improve?
  • Did proposal quality increase?
  • Did employees create more capacity for strategic work?
  • Did customers receive a measurably better experience?

Those are business outcomes.

Those are the metrics that matter.

Once organizations begin thinking this way, the entire conversation changes.

Model selection changes. Workflow design changes. Governance changes. Architecture decisions change.

Instead of automatically selecting the most powerful model available, organizations begin matching the right model to the right problem. A lightweight model may be perfectly capable of classifying tickets, extracting information from documents, or generating summaries. More advanced models can then be reserved for complex reasoning, strategic analysis, creative work, or high-value decision support.

In many ways, this is no different than how we think about people.

You would not assign every task in your organization to your most senior executive.

You match capability to complexity.

AI requires the same discipline.

What I believe we are witnessing right now is the beginning of the next phase of enterprise AI maturity.

The first phase was proving the technology worked. The second phase was driving adoption. The third phase will be driven by economics.

Leaders will increasingly be asked to demonstrate not how much AI they are consuming, but how much business value they are creating from that consumption.

And the organizations that figure this out first will have a significant advantage.

Because as AI becomes embedded into every workflow, every decision, and every business process, the winners will not be the companies generating the most tokens.

They will be the companies generating the most value from every token.

That is a very different game.

And it is one that many organizations have not started playing yet.