The Missing Layer in Enterprise AI: AI Financial Control

Dylan Patel, founder of SemiAnalysis, whose account of his firm's AI spending illustrates the shift from AI experimentation to a material operating expense.

In April 2026, Dylan Patel, co-founder of the research firm SemiAnalysis, described his own company's AI bill on the Invest Like the Best podcast. A year earlier the firm spent tens of thousands of dollars on AI tools. By the time of the interview it was running at roughly seven million dollars a year, which he characterized as about 28 percent of a payroll of around twenty-five million. Patel said he expected the token bill to exceed payroll before the year was out.

These are his figures, self-reported and unaudited. But the mechanics he described are worth more than the headline number. The spend did not come from a procurement decision. It came from usage. Patel described a staff member spending a few thousand dollars in model tokens to build an application that maps the material layers in semiconductor die images, saying a former Intel engineer told him comparable work had previously occupied a full team. He said an analyst wired Claude into economic data APIs and generated a 2,000-task benchmark, work he estimated would otherwise have taken 200 economists a year. None of it was budgeted. It accumulated, task by task, until it became one of the largest line items in the business.

That is the shape of the problem this piece is about. Enterprise AI is turning software from a fixed, licensed cost into a variable, consumption-driven one, and it is doing so faster than most organizations have built the instruments to measure it. The result is a financial-control problem that traditional cloud cost management was not designed to solve.

The SemiAnalysis story is unusual only in its candor. The direction is broad. McKinsey's State of AI in 2025 found that 88 percent of respondents said their organizations regularly used AI in at least one business function, up from 78 percent in 2024, while nearly two-thirds had not yet begun scaling AI across the enterprise. Writer's 2026 enterprise survey reported that 59 percent of companies now invest at least a million dollars a year in AI technology, while 29 percent report significant returns. Gartner forecast worldwide AI spending of nearly 1.5 trillion dollars in 2025.

The gap between adoption and returns is where money leaks, and some of it is invisible by design. IBM's 2025 Cost of a Data Breach report found that organizations with high levels of shadow AI had average breach costs 670,000 dollars higher than organizations with low or no shadow AI, and that 20 percent of surveyed organizations experienced a breach linked to shadow AI. Cyberhaven put a finer point on the behavior beneath the number: sensitive data's share of employee submissions to AI chatbots rose from 10.7 percent to 27.4 percent between March 2023 and March 2024, a 156 percent increase. Spend and risk are both accumulating in places the finance function cannot easily see.

Cloud FinOps exists because cloud spending had the same problem fifteen years ago, and the practice matured into an established organizational discipline and software category. Commercial market-research firms now estimate the broadly defined cloud-FinOps market at roughly 15 billion dollars in 2025; Polaris projects 39.56 billion by 2034 and Precedence projects 41.89 billion by 2035, both at roughly 11 percent annual growth. These are third-party forecasts with differing category definitions, not a single measured figure, but the direction is clear. Cloud FinOps works because cloud resources are discrete and stable. A virtual machine runs for a number of hours. Storage holds a number of gigabytes. You can tag it, allocate it to a cost center, and forecast it.

AI breaks each of those assumptions. The unit of billing is the token, and for API-based model services billing depends on token usage, so cost scales with request volume and composition rather than with how long a server runs. Workloads are bursty and experimental, often sharing inference endpoints across teams. And the price of the underlying hardware is far less forgiving: GPU-instance pricing varies materially by accelerator, region and commitment, and a multi-GPU training node can cost orders of magnitude more per hour than a general-purpose virtual machine. The old playbook of tag it, allocate it, report it assumes stable, attributable resources. AI spending is harder to attribute precisely when applications share endpoints and API accounts.

AI infrastructure is increasingly built around rack-scale GPU systems — here, NVIDIA GB200 NVL72 — rather than conventional general-purpose compute.

The distinction is not incremental. Traditional FinOps answers a simple question: how much did we spend on AWS last month. The question AI raises is harder. How much did this particular customer-facing agent cost, which models did it call, how many tokens did each step consume, was the request compliant, and what business outcome did it produce. Answering that requires instrumentation the cloud era never needed.

It helps to place the problem in the wider infrastructure it belongs to. Compute Notes uses the following analytical framework, which reads the AI stack as five layers. It is our lens, not an industry-standard taxonomy.

The AI Economic Stack — a Compute Notes framework. The industry has built Layers 1–3; Layer 4, the economics of intelligence, remains fragmented.

Layer 1, Compute. The GPUs, accelerators and data centers. This is where most capital and most attention have gone.

Layer 2, Inference. The models, tokens and APIs that turn compute into usable intelligence.

Layer 3, Orchestration. The agents, workflows and routing logic that chain model calls into work.

Layer 4, Economics. The layer that meters what was consumed, controls what may be consumed, allocates cost to an owner, and optimizes for the cheapest path to the same outcome. Governance and reconciliation cut across this layer rather than sitting beside it.

Layer 5, Business outcome. Revenue, productivity and automation, the reason any of it is funded.

The industry has built extraordinary capability across Layers 1 through 3. Capital has poured into compute, model quality has compounded, and orchestration frameworks have made agents genuinely useful. Layer 4 is the one that remains fragmented. There is no widely adopted system that sits across an organization's AI usage and answers, with authority, what intelligence cost, who consumed it, and whether the same result could have been reached for less.

It is worth being precise, because the language in this space has become loose. AI FinOps, narrowly, is the extension of cloud FinOps into AI infrastructure: measure, allocate and optimize AI spend after the fact. A financial control plane is broader. It measures, controls, allocates, optimizes and reconciles AI consumption at runtime, in the request path, before the money is spent rather than only after.

Concretely, that means four capabilities working together. Meter: token- and step-level accounting for every model call, tagged to a product, team or experiment. Control: runtime policy, such as which models are permitted, ceilings on tokens per request, or automatic routing to a cheaper model, enforced as the request happens. Allocate: attribution of cost to an owner, so an agent's bill can be charged back like any other resource. Optimize: the standing question of whether the same outcome could have been produced with a smaller model, a cached response or a shorter prompt. Governance and vendor reconciliation wrap around these as cross-cutting functions rather than defining the category on their own. In practice, most tools today do one or two of these well, and combining all four across multiple providers in a single place remains rare.

No single company owns this. What exists instead is four established categories, each reaching toward the same job from a different starting point.

Cloud FinOps platforms own the finance relationship. CloudZero and Finout publicly describe ingesting and allocating AI-provider spend across services such as OpenAI, Anthropic and AWS Bedrock, attributing it to teams, features and customers. Others in the category, such as Vantage and nOps, are extending toward AI spend, and established enterprise vendors such as Apptio, now part of IBM, and Flexera own broad IT-financial-management footprints whose token-level capabilities are still developing. Their strength is the finance function. Their structural limitation is that they generally sit beside the request path rather than inside it, so they report and allocate rather than enforce in real time.

LLM observability tools own measurement. Langfuse, LangSmith and Datadog's LLM module document token and cost tracking at the generation, trace or request level, and others such as Helicone and Arize offer related telemetry. The capabilities documented here focus on measurement rather than gateway-style spend enforcement: they can tell you what a call cost, but are not primarily built to block it before it runs.

An AI gateway sits between enterprise applications and model providers, creating a potential control point for routing, usage and policy.

AI gateways own the request path, which makes them the most control-plane-like of the four. LiteLLM, Kong's AI Gateway and Cloudflare's AI Gateway proxy or route model requests, placing policy and telemetry in the request path, and Portkey, TrueFoundry and OpenRouter occupy the same layer. Because they sit inline, some can issue virtual keys with budgets, enforce rate and usage limits, route or fall back between models, and cut off spend at a hard dollar limit. LiteLLM documents virtual keys and budgets checked before routing; Portkey documents cost- and token-based budgets that expire a key; Cloudflare documents spend limits that block further requests and can trigger fallback routing. Gateways are structurally positioned to combine request-path metering, routing and enforcement, though the breadth of allocation and finance features varies by product.

AI governance and security platforms are adjacent rather than central. Cisco's AI Defense, built partly on its acquisition of Robust Intelligence, along with vendors such as Credo AI and Cranium, handle policy, model validation, risk and audit artifacts. They intersect the control plane at the level of what is allowed, but cost allocation is not their principal role, and it would be a mistake to file them as financial-control companies.

Hyperscalers provide strong provider-native billing and governance for their own AI services. AWS, for example, can attribute Amazon Bedrock inference cost by application, team or workload through application inference profiles surfaced in its billing tools, and Azure and Google Cloud offer their own native cost and governance tooling. What they do not primarily offer is a neutral view across a competitor's tokens; independent platforms position themselves as the option for multi-provider visibility, which is precisely the opening they occupy.

The useful question is not who wins but which position is structurally strongest. A control plane wants several things at once: a place in the request path, direct access to token telemetry, the ability to enforce policy rather than merely observe, a relationship with finance and procurement, cross-provider neutrality, and switching costs once embedded. On those tests the gateway layer looks advantaged today, because it sits inline and can act. But it is early. Gateways generally lack the finance relationship the FinOps incumbents already own, and an incumbent that acquired a gateway would combine the two. The honest read is convergence in intent, fragmentation in structure, with no basis yet to declare a winner.

As systems become agentic, the economic unit moves up from GPU capacity toward tokens, tasks and outcomes.

Underneath the tooling question is a shift in what is being counted. Cost analysis in the cloud era was about capacity: how much compute you reserved and how well you used it. As systems become agentic, the unit of economic analysis moves up the stack. It is no longer just GPU-hours. It is tokens, then the cost of an agent completing a task, then the business value of that task. GPU economics gives way to token economics, which gives way to agent economics, which resolves into business economics.

This is why the control problem is not a passing artifact of high prices. Model pricing has fallen for some services and model generations even as enterprise usage expands, and the accounting is moving toward the cost of an outcome rather than the cost of a machine. An organization that can see, per task, what intelligence cost and what it produced can direct its spending toward the work that pays for itself. One that cannot will spend more each quarter with progressively less idea of what it is buying.

Four forces make this a present problem rather than a future one. Agents turn AI from something that answers questions into something that executes multi-step work, which multiplies both the number of billable calls and the difficulty of attributing them. Multi-model adoption means enterprises now draw from several providers at once, so no single vendor's dashboard sees the whole bill. For deployed API-based AI applications, inference can become a substantial and usage-variable cost as adoption scales. And regulation is beginning to pull governance and finance toward the same ledger: for high-risk systems, the EU AI Act requires automatic event logging, while sectoral and organizational requirements may create further traceability obligations. Gartner projects that by 2030, 40 percent of enterprises will experience a security incident tied to unauthorized AI use, a forecast that reads less like a warning about outside attackers and more like a description of ungoverned internal adoption.

The case for a financial control plane rests on a structural claim, not a market-size table. The claim is that a new layer of enterprise software tends to form wherever spending becomes large, variable and hard to attribute, and that AI now meets all three conditions. Cloud FinOps became a real market on the same logic. Whether AI financial control becomes a category of comparable scale depends on whether the convergence resolves upward into a system of record or stays split across the four silos above.

We would resist a precise total addressable market figure here. The intellectually honest version is a hypothesis, not a forecast: take enterprise AI spend, estimate the share that will require centralized economic control, apply a plausible software take-rate, and you have a revenue pool. Each of those inputs is currently too uncertain to multiply into a defensible number, and manufacturing one would be exactly the false precision this analysis is meant to avoid. What can be said with more confidence is where the value concentrates. The layer that ends up owning token telemetry, enforcement and the finance relationship at once has the makings of a system of record, and system-of-record positions are where durable software value and acquisition interest tend to accrue.

The signals worth tracking are specific. Whether the gateway players move upward into allocation, chargeback and finance-grade reporting, or stay developer tools. Whether a FinOps incumbent acquires a gateway to combine the request path with the finance relationship. Whether any hyperscaler offers genuinely cross-provider cost visibility, which would cut against its own lock-in incentives and so would be telling if it happened. Whether an interoperable standard emerges for token metering and model-version provenance, without which cross-vendor attribution stays manual. And whether enterprises begin to appoint an owner for AI economics at all, the way cloud eventually earned a FinOps function. The companies to watch are the ones assembling more than one of meter, control, allocate and optimize rather than perfecting a single verb.

AI infrastructure has spent this cycle optimizing intelligence. The next phase will optimize the economics of intelligence. Model pricing will keep shifting, agents will keep getting more autonomous, and enterprises will keep running more inference across more providers. Each of those trends makes one capability more valuable, not less: knowing exactly what intelligence costs, who consumed it, what it produced, and whether the same result could have been reached more efficiently.

The company that ends up owning that control layer may never train a competitive model. It may not need to. Owning the economics of the stack is a different position from owning the intelligence in it, and on current evidence it is the one that is still open.

Colossus. The Supply and Demand of Tokens. Invest Like the Best, Episode 469. April 23, 2026. https://colossus.com/episode/supply-demand-of-tokens/

McKinsey & Company. The State of AI in 2025: Agents, innovation, and transformation. 2025. https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai

Writer. Key findings from our 2026 AI adoption survey. April 14, 2026. https://writer.com/blog/ai-adoption-survey-2026/

Gartner. Gartner Says Worldwide AI Spending Will Total $1.5 Trillion in 2025. September 17, 2025. https://www.gartner.com/en/newsroom/press-releases/2025-09-17-gartner-says-worldwide-ai-spending-will-total-1-point-5-trillion-in-2025

Gartner. Gartner Identifies Critical GenAI Blind Spots That CIOs Must Urgently Address (40% of enterprises to experience shadow AI security incidents by 2030). November 19, 2025. https://www.gartner.com/en/newsroom/press-releases/2025-11-19-gartner-identifies-critical-genai-blind-spots-that-cios-must-urgently-address0

IBM. Cost of a Data Breach Report 2025. 2025. https://www.ibm.com/think/x-force/2025-cost-of-a-data-breach-navigating-ai

Cyberhaven. AI Adoption and Risk Report. 2024. https://www.cyberhaven.com/

Precedence Research. Cloud FinOps Market. 2026. https://www.precedenceresearch.com/cloud-finops-market

Polaris Market Research. Cloud FinOps Market Size Worth USD 39.56 Billion by 2034. December 12, 2024. https://www.polarismarketresearch.com/press-releases/cloud-finops-market

Amazon Web Services. Application inference profiles. Amazon Bedrock User Guide. https://docs.aws.amazon.com/bedrock/latest/userguide/cost-mgmt-application-inference-profiles.html

Amazon Web Services. Amazon EC2 On-Demand Instance Pricing. https://aws.amazon.com/ec2/pricing/on-demand/

Amazon Web Services. Amazon EC2 P5 Instances. https://aws.amazon.com/ec2/instance-types/p5/

CloudZero. AI ROI: Stop Tracking What You Spend, Start Proving What It Returns. July 21, 2026. https://www.cloudzero.com/blog/are-ai-costs-worth-it/

Finout. Finout vs. AWS Cost Explorer: A Full Comparison. August 2, 2026. https://www.finout.io/blog/finout-vs-aws-cost-explorer-a-full-comparison

Langfuse. Improved Cost Tracking. December 20, 2024. https://langfuse.com/changelog/2024-12-20-improved-cost-tracking

LangChain. Cost Tracking. LangSmith Documentation. https://docs.langchain.com/langsmith/cost-tracking

Datadog. Cost. LLM Observability Documentation. https://docs.datadoghq.com/llm_observability/monitoring/cost/

LiteLLM. Virtual Keys. Documentation. https://docs.litellm.ai/docs/proxy/virtual_keys

LiteLLM. Life of a Request. Documentation. https://docs.litellm.ai/docs/proxy/architecture

Portkey. Budget Limits. Documentation. October 9, 2025. https://docs.portkey.ai/docs/product/ai-gateway/virtual-keys/budget-limits

TrueFoundry. Enterprise AI Gateway. https://www.truefoundry.com/ai-gateway

Kong. AI Gateway. Documentation. https://developer.konghq.com/ai-gateway/

Cloudflare. Spend limits. AI Gateway Documentation. June 18, 2026. https://developers.cloudflare.com/ai-gateway/features/spend-limits/

OpenRouter. API Credit & Rate Limits. Documentation. https://openrouter.ai/docs/api_reference/limits

Cisco. Cisco AI Defense. https://www.cisco.com/site/us/en/products/security/ai-defense/index.html

UK Government. Credo AI: Responsible AI Governance Platform. September 19, 2023. https://www.gov.uk/ai-assurance-techniques/credo-ai-responsible-ai-governance-platform

Cranium. Secure & Govern Enterprise AI. https://cranium.ai/

European Union. Regulation (EU) 2024/1689: Artificial Intelligence Act, Article 19 — Automatically Generated Logs. https://artificialintelligenceact.eu/article/19/

National Institute of Standards and Technology. AI Risk Management Framework. https://www.nist.gov/itl/ai-risk-management-framework

Carol Chen

Founder, Compute Notes

Builder of AI-native businesses and investor in AI infrastructure

LinkedIn: https://www.linkedin.com/in/carol-c-76461498/

Website: https://www.computenotes.co/

Get a free AI spend teardown → https://watt.computenotes.co/

#AIInfrastructure #FinOps #EnterpriseAI #AIEconomics #Compute