If you only have a minute, here is what you need to know.
- The first two pieces in this series said the AI bill is real and that capping it is the wrong move. The question I kept getting back was simpler: so what exactly would you do? This is the answer, in build order.
- Put one gateway in front of every model call before you do anything else. It is the single move that makes attribution, routing, plan-swapping, and soft budgets possible. Without it, you are guessing.
- Define the unit of value before rollout, not after. Cost per merged pull request, per shipped feature, per closed ticket. One number per team, agreed up front.
- Fund an efficiency function with a slice of what you save. A rules file, progressive context loading, and a routing layer pay back inside a week and compound after that.
- Move power users off metered pricing and onto flat plans. Same behavior, sharply lower effective cost.
- This is a 90-day build, not a policy memo. Visibility first, measurement second, efficiency and routing third, culture throughout.
In The Token Tax I argued the waste is real and comes from untrained developers. In Tokenomics: Why a Spend Cap Is the Most Expensive Way to Save Money I argued the reflex to cap that spend costs you more than it saves. Both pieces were about what is true and what is wrong.
The most common reply I got was a fair one. "Fine. I believe you. Now tell me exactly what you would build."
So here it is. Not principles. Not a menu of levers to pick from. The actual system, in the order I would stand it up, if you handed me the AI bill on Monday and told me to govern it without rationing the team into the shadows.
Step zero: one gateway, or none of this works
Every move that follows depends on one thing being true: every model call in your organization passes through a single point you control.
Today it almost certainly does not. Developers hit the Anthropic API directly, the OpenAI API directly, a Copilot endpoint, three different SDKs, and two personal accounts nobody told you about. You have no chokepoint, so you have no data, so every decision after this is a guess.
Before you measure anything, route everything through one gateway. Self-hosted, that is LiteLLM or Helicone sitting as a proxy in front of every provider. In a Microsoft shop, that is Azure API Management doing the same job, which I walked through in The Trust Boundary Moved and Yes, You Can Run Claude on Azure. The technology matters less than the principle.
One gateway gives you four things at once: a log of every call, attribution metadata on every call, a place to route models, and a place to swap pricing plans.
Nothing else in this system works until that single chokepoint exists.
The gateway is also where you tag every request as it passes through: which developer, which project, which task type. That metadata is the difference between "we spent $85,000" and "we spent $85,000, here is exactly who and on what." You add it once, at the proxy, and every downstream report inherits it for free.
The hard part: not everything will route. Some tools will not point at your proxy. A few IDE plugins, some Copilot surfaces, and the occasional SDK hard-code their own endpoints, and you cannot force them through a custom gateway.
Do not let that stall the project. Close the gap from the other side: hand out model access through organization-level provider keys instead of personal ones, and use network egress rules to block direct calls to the provider domains that try to skip the gateway.
You will not hit 100% coverage in week one. Get the 80% you can route now, and treat the rest as a list to close, not a reason to wait.
Step one: decide what a dollar is supposed to buy
Most teams turn on cost tracking first, then go hunting for something to compare the number against. Do it the other way around. Decide what the money is supposed to produce before you look at what it costs.
Sit down with each team and agree on one result that matters to them. For a product team, that might be the cost of each pull request that actually ships. For a support engineering group, the cost of each ticket they close. For a platform team, the cost of each feature they deliver.
Pick the result first and write it down before the dashboards go live. A target everyone agreed to up front carries weight. A number you work out after the bill arrives does not.
A rising bill is not the problem. A rising cost per result is. If a team's bill goes up but the cost of each shipped pull request goes down, that team is getting more done for less, which is exactly what you want. A spend cap cannot tell that team apart from one that is simply wasting money.
The hard part: connecting the spend to the result. Dividing tokens by merged pull requests only works if something ties the two together, and that is what the tags from step zero are for. Stamp each call with the branch or work-item the developer is on, carry that through the gateway log, and the division becomes arithmetic.
Skip it and cost per result stays a slogan. You end up with a pile of token spend, a pile of shipped work, and no line connecting them.
This is not a new idea. The FinOps Foundation, the group that taught enterprises how to manage their cloud bills, has made the same discipline official for AI. Its main measure, Time to Achieve Business Value, compares what the work costs with AI against what it would cost with people doing it by hand. That comparison is the whole game. I went deeper on measuring results in Measuring Agentic AI Impact.
Step two: spend some of the savings on making each dollar go further
This is the step that turns the whole effort from cost-cutting into something better. Once the gateway shows you where the money is going, take part of what good governance saves and put it back into getting more output from every dollar you spend. Do this as an ongoing job that someone owns, not a one-time cleanup.
The savings here are large, not small. A few examples from my own testing:
- Show the model only the tools it needs. Instead of handing it the full description of every tool it might use, show it just what the task in front of it requires. In my benchmarks this approach, called progressive disclosure, cut token use by 85 to 100 times and made the model more accurate at the same time.
- Give the model a rules file. A short file that explains how your team works costs about 3,000 tokens to include and saves 15,000 to 20,000 per session, because the model stops relearning your conventions from scratch every time you start. I covered this in The Harness Is the Multiplier.
- Pack structured data more tightly. A format called TOON hands tables and records to the model in a more compact form. On uniform, table-shaped data it cut the tokens that data uses by up to 85% in my tests, with the savings shrinking on nested or irregular data.
- Let the model call tools through code. Instead of cramming every tool definition into the prompt, the model writes code that calls the tools it needs. The approach I benchmarked here clears out the bloated context that drives a big share of the bill.
A cap lowers the ceiling on what your team can do. Efficiency raises the floor on what each dollar produces.
None of these are research projects. Each is a few days of work against a problem you can measure before and after. Pick your most expensive project, ship one of them, and watch the cost per result drop. The first win pays for the second.
Step three: route models and right-size plans at the gateway
Now use the chokepoint you built for the second thing it is good at.
Not every task needs your most expensive model. Route the simple, high-volume work, classification, formatting, short completions, to a cheap fast model by default, and reserve frontier models for the reasoning that actually needs them. Done at the gateway, this is a policy change, not a code change, and it applies to every team at once.
The hard part: deciding what counts as simple, automatically, on every call. You have three ways to do it:
- Route on the task-type tag from step zero. Cleanest, but only as good as your tagging.
- Route by agent or role. Your code-review agent gets the frontier model, your commit-message agent gets the cheap one. Coarse, but reliable.
- Run a small, cheap model first to classify the request and pick the tier. Flexible, but it adds a call and can misroute.
Start with role-based routing. It is the one that does not fall apart when a tag goes missing.
Then fix the pricing mismatch. A predictable, heavy daily user is almost always cheaper on a flat-rate subscription than on per-token billing. I build production enterprise software on a $200 flat subscription for exactly this reason.
Find your power users in the attribution data from step zero and move them off metered API access onto flat plans. Same behavior, sharply lower effective cost, zero retraining.
Step four: budgets that alert, accountability that coaches
Only now do budgets enter the picture, and they never become walls.
Set a soft budget per team and alert the lead at 70 and 90 percent. The alert triggers a conversation, not a block. You never cut off a developer mid-task, because that is the exact moment they open a personal account and paste your code into it.
A 2025 Microsoft survey of UK employees found that among those using unapproved AI, 28% said their company gave them no approved option. Restriction does not curb shadow AI. It causes it. And the bill you pushed off the books comes back worse: IBM put the shadow-AI breach premium at about $670,000.
Pair the soft budget with showback. Show each team its own consumption and its own cost per result, and let the numbers create the pressure.
When an outlier appears, treat it as a coaching signal. Maybe they are your most productive engineer pointing AI at the hardest problems. Maybe they never learned structured prompting and need an afternoon of training. The attribution data tells you which, and the response is help, not a violation notice.
The 90-day build
You do not roll this out as a policy. You stand it up as a system, in sequence.
Days 1 to 14: the chokepoint. Stand up the gateway. Route every team through it. Turn on logging and per-call attribution. By the end of week two you can see spend by developer, by project, by task type. You will find your top 5% of users drive 40 to 50% of spend, and now you know whether they are your best engineers or your most wasteful ones.
Days 15 to 45: the metric and the first lever. Agree the result that matters with each team and publish the cost per result. Pick your highest-cost project and ship one efficiency lever against it. Measure before and after. This is where the system starts paying for itself.
Days 46 to 90: routing, plans, and culture. Turn on model routing at the gateway. Move power users to flat plans. Switch on soft budgets with alerts at 70 and 90 percent. Start the showback cadence. By day 90 you are not capping spend. You are governing it, and the number that matters, cost per result, is trending down while output trends up.
What this actually buys you
A cap makes the scary line on the chart go flat by making your team do more work by hand. This system makes the line meaningful by tying every dollar to an outcome, then makes each dollar go further.
Here is the size of the prize. Between routing simple work to cheaper models, moving power users to flat plans, and shipping one efficiency lever, I would expect a team to cut its cost per result by a third to a half inside the first 90 days, while shipping more, not less. That is an estimate from doing this, not a benchmark, and your mix will move the number. The direction is not in doubt. The spend becomes legible first, then it bends.
A spend cap freezes one number and blinds you to the rest. This system makes every dollar traceable to a result, then drives that cost down quarter after quarter.
The fully loaded cost of a US developer is about $83 an hour. An hour of heavy AI coding runs $10 to $30 in tokens, and the cost of the same AI capability has fallen about 50x in a year. You are governing the cheapest, fastest-falling input in your business. Govern it like an investment, not a leak.
The teams that win the next phase will not be the ones that spent the least on tokens. They will be the ones that built the system to get the most out of every one they spent. The cap takes an afternoon. This takes 90 days. One of them is still working a year from now.
Matthew Kruczek is Managing Director at EY, leading Microsoft domain initiatives within Digital Engineering. Connect with Matthew on LinkedIn to discuss token economics, AI cost governance, and developer enablement for your organization.
References
- U.S. Bureau of Labor Statistics. "Software Developers, Occupational Outlook Handbook." Median annual wage $133,080 (May 2024). bls.gov
- Microsoft. UK workplace AI survey (Censuswide), October 2025. 28% of unapproved-AI users cite no approved alternative. ukstories.microsoft.com
- IBM. "Cost of a Data Breach Report 2025." Shadow-AI breach premium about $670,000. ibm.com
- FinOps Foundation. "FinOps for AI Overview." Time to Achieve Business Value KPI; cost tied to business outcome. finops.org
- Epoch AI. "LLM inference prices have fallen rapidly but unequally across tasks." Median 50x/year decline. epoch.ai
- Kruczek, M. "The Token Tax: Why Untrained Developers Are Your Most Expensive AI Problem." matthewkruczek.ai
- Kruczek, M. "Tokenomics: Why a Spend Cap Is the Most Expensive Way to Save Money." matthewkruczek.ai
