Every ISV bolting AI features onto their product is walking toward the same surprise. The monthly cloud bill, a number they thought they understood after fifteen years of FinOps maturity, is about to change shape. Not size. Shape.
An EC2 instance costs the same whether it serves one request or a thousand. That is capacity-shaped cost, and every FinOps practice in the industry was built around it: reserve capacity, rightsize it, tag it, watch it. LLM inference costs nothing like that. Every call is priced differently, driven by variables no finance team has ever tracked - prompt length, context accumulation, output verbosity, model tier, cache hit rate, retry count. And in agentic systems, one more variable that changes everything: the number of reasoning loops an agent takes before it decides it is done. Token cost is behaviour-shaped. It can grow 10x in an afternoon without a single infrastructure change, because behaviour changed, not capacity.
Here is the uncomfortable fact. The major clouds do not yet provide FinOps systems that extract token usage at the granularity this problem demands. The components exist - billing exports, anomaly detectors, model-level metering - but as standalone layers, not as a system that answers a business question. Traditional FinOps tooling, including the newest AI-assisted entrants, still serves the old requirement: core component usage against the account. It has no business view of AI consumption, and it is almost blind to the cost dynamics of agentic systems.
Traditional FinOps watches the bill arrive. Agentic FinOps decides, in real time, whether the bill deserves to exist.
— Sethunath U N, Chief Consultant & Advisor, Mavis DxEven the Newest Tools Are Watching the Wrong Layer
Take the most current example. AWS recently previewed its FinOps Agent - a Bedrock-powered agent that investigates cost anomalies, answers cost questions in natural language, schedules reports, and consolidates optimisation recommendations into Jira tickets. It is a genuinely useful product. It is also a precise illustration of the gap.
Look at what it reads: Cost Explorer, Cost Anomaly Detection, Cost Optimization Hub, Compute Optimizer, CloudTrail. Every one of those sources sits downstream of the bill. The agent can tell you, eloquently, what you spent on compute and storage and where an anomaly landed. It cannot tell you what a resolved customer query cost, which tenant's AI usage is eroding margin, or that an agent's retry loop started accelerating an hour ago. It does not meter LLM traffic at the call level, carries no concept of cost per business transaction, and its anomaly signal fires after the spend has already billed. The smartest FinOps tool on the market today is an intelligent reader of yesterday's bill.
- Spend by account, service, and tag - after billing lands
- Anomalies detected once the spike is already on the invoice
- Rightsizing recommendations for compute and storage capacity
- No visibility into tokens, agent loops, or per-call behaviour
- Cost per business transaction - per resolved query, per booking, per document
- Leading indicators: token drift, retry loops, cache decay, model-tier creep
- Every cost movement attributable to a tenant, an agent, and a release
- Levers that act in real time: routing, throttling, per-agent budgets
Closing that gap is not a dashboard project. The FinOps system can only consume what engineering emits, and most engineering teams today emit nothing usable. So before any dashboard, three disciplines. Each one is decided at design time. None of them can be retrofitted cheaply.
Three Disciplines Before a Single Dashboard
Picture three agents in one product: a conversational assistant whose cost grows with dialogue length, a batch document extractor whose cost grows with volume, and a multi-step orchestrator whose cost grows with reasoning loops and tool calls. Three completely different burn patterns. The discipline that tames all three is the same: no agent talks to a model directly. Every call routes through a central LLM gateway, and the gateway emits one structured usage event per call - request ID, tenant, feature, agent, step number, model, prompt version, input tokens, output tokens, cached tokens.
If instrumentation is left voluntary, coverage decays exactly where the new, unmodelled spend appears. The gateway makes metering an enforced property of the architecture, not a favour from each team. The OpenTelemetry GenAI semantic conventions give you a ready-made event schema; adopt them early rather than inventing your own.
A call that reaches a model without attribution fields is a call the gateway rejects.
Token costs do not drift the way infrastructure costs drift. They jump - and they jump on releases. A prompt edit, a model swap, a new tool added to an agent graph. So prompts, model versions, and agent definitions must be versioned artefacts that pass through CI/CD like code, and the pipeline must include a gate most teams have never built: a token profile test. Run a fixed set of representative tasks against the changed prompt or agent, count tokens per task, fail the build if the delta against baseline exceeds a threshold.
The second half of the discipline is the version stamp. Because the prompt and model version ride inside every usage event from the gateway, a cost spike on Tuesday afternoon traces to the release that shipped Tuesday morning. Without the stamp, you are correlating by eyeball.
Nothing ships without a measured tokens-per-task delta, and every deployed version is visible in the cost data.
This is where AI PaaS platforms complicate the picture, and where I learned the lesson hands-on building an agent on Amazon Bedrock AgentCore. A single agent task does not produce one line on the bill. It fans out across four billing dimensions with four different units: Runtime bills per-second of CPU and memory per session, Gateway bills per tool invocation, Memory bills for storage and retrieval, and the model calls bill through Bedrock in tokens. None of them individually tells you what the task cost.
The only way to reassemble the truth is a join: the granular billing feed (AWS CUR, ideally normalised to the FinOps Foundation's FOCUS specification) joined with your gateway telemetry on a shared identifier. Which means the pre-requisite is decided at design time - the agent must propagate its request and session IDs into every runtime session, every tool call, every model invocation. Retrofit this after six months of production traffic and that history is unrecoverable.
One correlation ID flows from the business transaction through every billing dimension it touches.
Here is the runtime picture the first discipline produces - three very different agents, one enforced metering path:
The release discipline turns the CI/CD pipeline into the place where cost surprises are caught before production sees them:
And the reconciliation discipline is what makes an AgentCore bill legible - four billing dimensions rejoined into one business number:
The Registry Is the Control Plane
The three disciplines answer what you spent and why. There is a question that comes before both: who is allowed to spend, on whose behalf, under what limits. That question lives in the agent registry.
Neither AWS nor GCP ships a standalone product called an agent registry for FinOps. What they ship are the ingredients. On AWS, an agent deployed to AgentCore Runtime is a first-class resource - it has an ARN, versions, and a workload identity, which means it can carry cost allocation tags and appears in the billing data as an attributable resource. On GCP, a deployed Vertex AI Agent Engine agent is a managed resource carrying labels, and labels flow straight into billing export, with the Gemini Enterprise governance layer sitting above. The registry as a FinOps control plane is a discipline you assemble from these parts, and it earns its keep four ways.
It Closes the Denominator
Your gateway meters every call that passes through it - but cannot see the agent that never did. Spend with no matching registry entry is your shadow-agent detector. This is EC2 tag sprawl replayed at the agent layer, except agents multiply faster.
It Holds Ownership and Declared Cost
Owner, cost centre, and - the part most teams miss - a declared expected cost per task. Once declared, FinOps becomes variance analysis: actual against declared, per agent, per version. Drift is visible while absolute spend is still small.
It Anchors Policy the Gateway Enforces
Model allow-lists, tier routing rules, per-agent token budgets, throttling thresholds - attached to the registry entry, read by the gateway at call time. A cost decision becomes a configuration change, not an engineering release.
It Kills Zombie Agents
Cross deprecated registry entries against usage events and you find agents still consuming while producing no business transactions. Worse than idle VMs - an abandoned agent with a retry loop keeps billing.
From Tokens to Business Units: A Worked Example
Decision makers should never see the word "tokens." They should see cost per resolved ticket, cost per booking, cost per document processed - and for an ISV, gross margin per customer. The translation is not a modelling exercise. It is a declaration on the registry entry plus one aggregation. Here is what the declaration looks like for a simple flight status check agent, the kind every airline SaaS platform is building right now:
For every request ID where the completion event fired, sum all cost records sharing that ID - token charges, tool invocation, runtime seconds - and divide by completion events. That single number, cost per status check resolved, is what climbs the stairs to the decision maker. Everything below it exists to explain that number when it moves. And when the discipline holds, the resulting view tells a story, not a spreadsheet:
Read it top to bottom. The headline is in business units: cost per resolved check, 8% above declaration. The composition card explains where each fraction of a cent goes. The trend card does the release discipline's work - the step change in week four lines up with a specific version, so the conversation becomes "what did that release change," not "why is the bill up." And the tenant table carries a column that will eventually appear in ISV board decks, though almost nobody produces it today: AI cost as a percentage of ARR, per customer. One tenant in this view runs 45% above declared unit cost because its complex queries escalate to the expensive model at three times the fleet average. That is a pricing conversation, found by a FinOps system.
The figures are illustrative and model price cards will keep moving. The structure is what lasts: declared against actual, composition, variance by tenant. A CxO reads this view in one glance, with no clarifying questions. That is the test.
Early Warnings Come From Behaviour, Not Bills
Traditional cost alerting is lagging by construction - the signal is the bill, and the bill arrives after the burn. With tokens, spend can multiply within hours, so the warning system must watch behaviour, the thing that predicts cost before it lands.
Behavioural Drift
Tokens-per-request trending up (prompt bloat, context accumulation), output length drift after a change, cache hit rate quietly dropping, agent loop and retry counts rising. A runaway agent is a fork bomb that bills per iteration.
Spend Velocity
Burn rate per hour against budget, per tenant and per feature. Anomaly detection on tokens per minute, not dollars per day. Watch p95 and p99, not averages - token consumption is heavy-tailed, and one tenant can dominate the bill.
Release Correlation
Canary every new prompt or model version against a token budget the way you canary a deployment against error rates. A canary burning 40% more per task is caught before rollout, not on the invoice.
Circuit Breakers
Per-tenant and per-agent budget caps that throttle or degrade to a cheaper model rather than fail silently into a six-figure surprise. Cloud-native quotas are account-grained; the agent-grained enforcement is yours to build.
The Unit of Accountability Has Changed
Step back and the whole argument compresses into one shift. Classic FinOps made the account, and later the tag, the unit of accountability - and built two decades of maturity around it. Agentic systems break that unit, because the thing that spends is no longer a resource you provisioned. It is a behaviour you deployed.
In classic FinOps the unit of accountability was the account or the tag. In agentic FinOps, it is the registered agent. If it is not in the registry, it does not get tokens.
— Sethunath U N, Chief Consultant & Advisor, Mavis DxNone of this requires waiting for the cloud providers to ship the finished system. The parts exist today: a gateway you enforce, a pipeline gate you write, an ID you propagate, a registry you assemble from resources and tags the platforms already give you. What it requires is the same thing FinOps always required - engineering discipline installed before the bill teaches it to you. The ISVs that build this foundation in the next twelve months will price their AI features from knowledge. The rest will price from hope, and reconcile from pain.
Is your AI feature priced from knowledge, or from hope?
If your product is adding AI features and nobody can yet say what a single AI-served transaction costs per customer, that answer is buildable - and the foundation is cheaper to install now than to retrofit after the first bill shock. Thirty minutes will tell you how far your current FinOps setup is from an agentic-ready one.
Outcome-based · No-commit · Monthly engagement
You continue only when you see real value. That is our promise.
Start the Conversation →