← Agent Frameworks 🕐 23 min read
Agent Frameworks

FinOps Foundation AI Working Group, FOCUS Roadmap, and Enterprise AI Cost Standards (2026)

Executive Summary · March 2026

The week of June 8–10, 2026 was the most consequential week in enterprise AI cost governance to date.

See also (wiki): focus-standard-ai-cost · ai-cost-chargeback-showback

Research date: June 17, 2026
Audience: FinOps practitioners, platform engineers, CFOs at Fortune 500 companies running production LLM workloads
Signal confidence: High — primary sources from FinOps Foundation, Linux Foundation press releases, and FinOps X 2026 proceedings (June 8–10, San Diego)

The week of June 8–10, 2026 was the most consequential week in enterprise AI cost governance to date. At FinOps X 2026 in San Diego, the Linux Foundation announced the intent to form the Tokenomics Foundation, ratified FOCUS 1.4, previewed the FOCUS 1.5 AI roadmap, and launched two new certifications (AI Value, Technology Value). This note synthesizes the full landscape: what the FinOps Foundation’s AI Working Group has published, what Tokenomics Foundation actually is, what FOCUS 1.4 delivered and 1.5 will add, how the enterprise maturity model applies to AI specifically, the organizational mechanics of chargeback for shared models, the tooling options beyond AWS-native, and the unit economics KPIs practitioners are instrumenting today.

1. FinOps Foundation AI & ML Working Group

Status and Structure

The FinOps Foundation’s FinOps for AI Working Group is a community-driven group within the Foundation’s open practitioner network. Membership is open to any FinOps Foundation member (practitioner tier is free). The WG operates through GitHub Issues and async Slack channels, with output published to finops.org/wg/. Unlike chartered task forces with fixed rosters, the WG is contributor-model: published papers list contributing practitioners but there is no fixed membership count.

The WG accelerated output in late 2024–2026 as generative AI spend moved from experiment to production for most enterprise members. The framework received a substantial 2026 refresh extending its scope from pure cloud spend to include AI/ML, SaaS, licensing, and broader technology categories — the first structural change to the FinOps Framework since 2022.

Published Deliverables (2024–2026, in chronological order)

Publication Date Core Content
FinOps for AI Overview 2024 (ongoing) Scope, user stories gathered from members, foundational taxonomy. Available at finops.org/wg/finops-for-ai-overview/
How to Forecast AI Services Costs in Cloud 2024–2025 Demand-based vs. capacity-based forecasting approaches for managed AI services; traffic shape modeling for OpenAI, Bedrock, Vertex
Cost Estimation of AI Workloads 2025 Pre-deployment cost modeling for LLM workloads; token budget sizing; GPU reservation vs. on-demand tradeoffs
Driving Cost Efficiency into AI Deep Learning Pipelines with FinOps 2025 Training cost governance, checkpoint frequency, gradient accumulation economics. Available as PDF at finops.org/assets/
Effect of Optimization on AI Forecasting 2025 How optimization actions (prompt compression, caching, model tiering) distort cost forecasts; recalibration methodology
GenAI FinOps: How Token Pricing Really Works August 2025 Dissects per-token billing across providers; surfaces context window creep as the primary hidden cost driver — conversation history compounds token volume with each turn; cached token pricing on Anthropic vs. OpenAI vs. Bedrock compared
Scaling Kubernetes for AI/ML Workloads with FinOps October 2025 GPU cluster cost governance on Kubernetes; autoscaling vs. static allocation economics; KEDA + Karpenter patterns for inference; pod-level cost attribution with OpenCost
GenAI Capacity Models: AWS, GCP, Azure February 2026 Traffic shape analysis for provisioned throughput (PTU on Azure, Provisioned Concurrency on Bedrock); spillover to on-demand and its cost cliff; committed vs. burst spend modeling
Optimizing GenAI Usage: A FinOps Perspective on Cost, Performance, and Efficiency 2026 Builds on the forecasting paper; introduces token efficiency as a first-class optimization metric alongside cost; model tiering decision framework
Unlocking AI Business Value with FinOps 2026 Scope, forecasting, optimization, and unit economics tied to business outcomes; the foundation’s closest document to an AI FinOps playbook

What the WG Says About LLM Token Cost Allocation vs. Traditional Compute

The WG is explicit that token-based AI billing breaks the core assumptions of infrastructure FinOps:

  1. No persistent resource to tag. Traditional chargeback relies on tagging VMs, buckets, or databases — assets that exist continuously. An LLM API call is a transaction, not an asset. There is no EC2 instance to tag with cost-center: finance.

  2. Cost is a function of input content, not just request count. A single call with a 32K-token context costs ~32× more than a 1K-token call, but both appear as one API request in logs. Request-count quotas and alerts miss this entirely.

  3. Context window creep multiplies cost non-linearly. Maintaining conversation history means token volume grows quadratically with conversation length. The WG quantifies this in the August 2025 paper — a 10-turn conversation with 500-token responses and full history retention costs ~5× the naïve per-turn estimate.

  4. Model version drift changes cost baselines. GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro have significantly different per-token prices. A team switching models mid-quarter invalidates prior forecasts without a model-normalized cost metric.

WG-recommended allocation dimensions: Five dimensions cover ~95% of enterprise chargeback use cases:

  • team — owning engineering group
  • project — initiative or product
  • environment — dev / staging / prod
  • model-name — specific LLM variant (e.g., gpt-4o, claude-3-5-sonnet-20241022)
  • cost-center — finance mapping for P&L assignment

These dimensions must be applied at the API gateway or SDK wrapper layer, not at the cloud billing layer, because cloud invoices aggregate across models and lack request-level metadata. The WG recommends instrumenting a middleware layer (internal proxy, LangSmith, Helicone, or similar) that logs {team, project, model, input_tokens, output_tokens, cached_tokens, latency_ms} per request before billing data is reconciled.

2. Tokenomics Foundation

Formation

The Linux Foundation announced the intent to launch the Tokenomics Foundation on June 3, 2026, two days before FinOps X 2026 opened. The formal announcement at FinOps X (June 8) included founding member commitments and initial working group structure. The website is at tokeneconomics.com.

Founding Members

Accenture, Booking.com, Flexera, Google Cloud, IBM, JPMorganChase, KPMG, Microsoft, Oracle, Salesforce, SAP, ServiceNow. Cast AI was named a Founding Premier Board Member (the highest membership tier, carrying board representation).

The member list is significant: it combines token consumers (JPMorganChase, Booking.com, SAP, ServiceNow as enterprise buyers), hyperscalers (Google Cloud, Microsoft, Oracle), consulting (Accenture, KPMG), platform (Flexera, Cast AI), and SaaS (Salesforce). This is the same constituency pattern that made the Cloud Native Computing Foundation effective — buyers and suppliers in the same governance body.

Mission

The Tokenomics Foundation addresses a governance vacuum: enterprises are scaling generative and agentic AI workloads into production, token spend is rising exponentially, and the financial discipline to manage it has not kept pace.

The Foundation’s stated scope covers three pillars:

  1. Token production — token factory effectiveness; how efficiently GPU capital is converted into tokens (tokens per watt, tokens per dollar of infrastructure)
  2. Token consumption — FinOps for AI; how enterprises track, allocate, and optimize token spend
  3. Token monetization — AI value; tying token consumption to business outcomes and revenue

Relationship to FinOps Foundation

The Tokenomics Foundation operates in close partnership with the FinOps Foundation rather than competing with it. The relationship is complementary:

  • FinOps Foundation: practitioner community, framework, working groups, certification (now including AI Value and Technology Value)
  • Tokenomics Foundation: specification body, standards funding (including FOCUS extension into token billing), benchmark publishing, annual conference (Tokenomicon, planned for June 2027)

The Tokenomics Foundation will fund expansion of the FOCUS specification into token-based spending models. This is the critical structural commitment: FOCUS development has historically been volunteer-driven; the Tokenomics Foundation provides sustained funding for AI-specific FOCUS work.

Tokenomicon Conference

The Linux Foundation announced Tokenomicon — a new conference focused on AI cost economics, positioned as a reimagining of FinOps X. Planned for June 2027, it retains FinOps X as a component while expanding scope to token producers (GPU manufacturers, neoclouds, inference providers) and AI value quantification. The name is a deliberate signal: this is the conference of record for everyone who touches the AI cost stack, not just cloud FinOps practitioners.

How Tokenomics Foundation Differs from FOCUS for AI

FOCUS is a billing normalization specification — it defines column names, data types, and semantics for cost and usage exports so practitioners can write provider-agnostic tooling. It answers: “what should an invoice row look like?”

Tokenomics Foundation is an economics standards body — it defines benchmarks, best practices, and value frameworks for token-based AI. It answers: “what does good look like?” and “how do you measure token ROI?”

They are complementary: FOCUS gives you normalized data; Tokenomics Foundation gives you the frameworks to analyze that data. An analogy: GAAP defines how to record revenue; a management accounting body defines how to do cost-benefit analysis.

3. FOCUS 1.4 and the FOCUS 1.5 AI Roadmap

FOCUS 1.4 (Released June 4, 2026)

The FOCUS Steering Committee ratified FOCUS 1.4 on June 4, 2026, announced at FinOps X. Key additions:

  • +2 datasets (Price Sheet dataset; Recommendations dataset)
  • +47 columns across datasets — zero incompatible changes for existing implementations
  • Invoice reconciliation support — the Price Sheet dataset enables practitioners to tie billing line items back to published list prices, closing a long-standing audit gap
  • Commitment details — enhanced columns for reserved capacity, savings plans, and committed-use discounts across providers

AI-relevant additions in FOCUS 1.4:

  • x_InputTokens, x_OutputTokens, x_CachedTokens — provider-extension columns (the x_ prefix denotes non-standard, provider-specific fields) that surface token splits. As of 1.4, these are extension columns, not ratified standard columns — meaning providers can populate them but tooling cannot assume their presence.
  • ConsumedQuantity and ConsumedUnit refinements that improve normalization of token-based billing vs. compute-hour billing

FOCUS 1.5 Roadmap

FOCUS 1.5 is explicitly scoped to surface AI model identity and token consumption in the Cost and Usage dataset as ratified (non-extension) columns. Based on public communications from the FOCUS Steering Committee at FinOps X 2026:

Planned ratified columns for 1.5:

  • InputTokenCount — tokens consumed in the prompt/context window
  • OutputTokenCount — tokens generated in the response
  • CachedTokenCount — tokens served from provider-side prompt cache (Anthropic, OpenAI) at reduced price
  • ModelId — canonical model identifier (e.g., anthropic.claude-3-5-sonnet-20241022-v2:0 on Bedrock)
  • ModelVersion — version string separate from model family for provider normalization
  • InferenceType — enumerated: on-demand | provisioned | batch | distillation | fine-tuned
  • CapacityReservationId — links rows to PTU/Provisioned Concurrency commitments for commitment utilization tracking

Price Sheet dataset extensions for 1.5:

  • Worked examples for token-based billing (pay-per-token) and generation-based billing (pay-per-request with tiered token bands)
  • Unit price normalization to $/1M tokens across provider billing models that currently use different units ($M tokens, K tokens, per-character)

Expected timeline: The Steering Committee has not published a ratification date for 1.5. Based on FOCUS release cadence (1.0 → 1.0.1 → 1.1 → 1.2 → 1.4 over roughly 24 months) and the stated funding from the Tokenomics Foundation, a 2027 ratification is the most likely scenario, with a draft specification for public comment in late 2026.

Practitioner Workarounds Before FOCUS 1.5

Enterprises running production AI workloads today cannot wait for FOCUS 1.5. Current patterns:

1. Provider-native exports + custom ETL Pull x_InputTokens / x_OutputTokens from AWS Cost and Usage Report (available for Bedrock since late 2024), OpenAI usage export API (/v1/usage), and Azure OpenAI metering export. Build a dbt model that normalizes these to a common schema: {date, team, model_id, input_tokens, output_tokens, cached_tokens, cost_usd}.

2. API gateway instrumentation Deploy a thin proxy (LiteLLM, Helicone, Portkey, or a home-built middleware) in front of all LLM API calls. The proxy logs token counts at the request level with your internal attribution metadata. This is the only way to get team-level token attribution — cloud billing exports do not carry internal cost-center metadata.

3. FOCUS 1.4 extension columns For providers that have adopted FOCUS 1.4 exports, consume the x_InputTokens / x_OutputTokens columns with explicit null-handling. Write tooling that works against the extension column when present and falls back to ConsumedQuantity / list price math when absent.

4. Semantic layer normalization Use a FinOps platform (Finout, Vantage, CloudZero) that already normalizes multi-provider AI billing. This trades schema control for speed to value — acceptable for showback; requires augmentation with gateway logs for accurate chargeback.

4. Enterprise AI FinOps Maturity Model

The FinOps Foundation Crawl / Walk / Run Framework Applied to AI

The FinOps Foundation’s standard Crawl/Walk/Run maturity model applies to AI with the following AI-specific capability additions. The 2026 framework refresh explicitly incorporates AI capabilities into the maturity progression.

Crawl — Visibility (Months 1–3)

Goal: ingest billing data from all AI providers; produce a basic spend dashboard.

What “Crawl” looks like for AI:

  • Pull billing data from OpenAI Usage API, AWS CUR (Bedrock), Azure OpenAI metering, GCP Vertex AI billing
  • Tag resources with environment (dev/staging/prod) at minimum — prevents dev experimentation costs from obscuring production budget
  • Set anomaly alerts on weekly token spend by provider (±30% threshold is a common starting point)
  • Identify top-5 model/team combinations by spend — this rarely requires complex tooling, often just a spreadsheet pivot on the usage export

Common Crawl failures: Teams skip the billing normalization step and look only at cloud invoices, missing SaaS AI spend (Salesforce Einstein, ServiceNow AI, Workday AI features) which often equals or exceeds direct API spend in large enterprises.

Walk — Accountability (Months 4–9)

Goal: allocate 100% of AI spend to a cost owner; establish regular business reviews; begin optimization.

What “Walk” looks like for AI:

  • Deploy API gateway instrumentation (proxy layer) for all internal LLM calls; log {team, project, model, tokens, cost} per request
  • Implement showback reporting — weekly email or dashboard to team leads showing their AI spend and token consumption
  • Establish model tiering policy: define which use cases require frontier models (GPT-4o, Claude 3.5 Sonnet) vs. can use mid-tier (GPT-4o-mini, Claude 3 Haiku) vs. can use open-source (Llama 3.1, Mistral)
  • Set token budgets by team/project; wire alerts when 70% consumed
  • First-pass cost-per-task measurement for 2–3 high-spend use cases (e.g., “cost per summarization”, “cost per customer email draft”)

Walk milestone gate: 80%+ of AI spend allocated to a named cost owner. The FinOps Foundation defines this as the threshold for moving from Walk to Run.

Run — Unit Economics (Months 10+)

Goal: tie AI cost to business outcome; automate optimization; influence technology selection decisions.

What “Run” looks like for AI (the practice in 2026):

  • Unit economics per business outcome, not per token: cost per successful customer resolution, cost per contract reviewed, cost per sales email that received a response
  • Automated model routing: route requests to the cheapest model that meets latency and quality SLA — typically implemented via LiteLLM router or custom gateway with quality gate sampling
  • Prompt efficiency governance: quarterly prompt audit to identify prompts with >2K average input tokens that haven’t been reviewed for compression
  • Chargeback (not just showback) for P&L-owning business units running production AI features
  • AI spend included in technology selection decisions: when evaluating a new AI vendor, total cost of ownership (licensing + token consumption + engineering integration) is modeled before contract

What “Run” does NOT look like:

  • Optimizing for cost per token in isolation. A team that switches from Claude 3.5 Sonnet to Claude Haiku to halve token cost but doubles retry rate (due to quality degradation) has increased their real cost-per-outcome.
  • Dashboards that no one acts on. Run-stage AI FinOps produces automated actions: request routing, budget enforcement, model deprecation alerts.

Leading Enterprise Patterns

JPMorganChase (Tokenomics Foundation founding member): JPM has publicly discussed a cost-per-analyst-productivity framework where AI costs are measured against analyst output metrics, not against token counts. The firm treats AI spend as a direct operational cost tied to revenue-generating activity, similar to data vendor spend. Token cost is a leading indicator; analyst output is the lagging KPI.

Microsoft (internal practice, publicly discussed): Microsoft’s internal FinOps team measures GitHub Copilot effectiveness as “accepted suggestions per developer per week” vs. “Copilot license + token cost per developer per week.” The unit economics metric is cost per accepted code suggestion, with a target benchmark that the team has declined to publish but described as “single-digit cents per accepted suggestion at scale.”

Booking.com (Tokenomics Foundation founding member): Booking.com has discussed measuring AI cost as a percentage of customer service interaction cost. The unit is AI assist cost per resolved customer contact — this includes token cost, latency SLA infrastructure, and the human escalation rate (since AI-unresolved contacts require human agents at much higher cost).

5. Chargeback vs. Showback for AI

FinOps Foundation Guidance

The FinOps Foundation is explicit: the choice between chargeback and showback depends on organizational accounting policy, not FinOps maturity level. Showback is always required. Chargeback is conditional.

From the Foundation’s Invoicing & Chargeback capability page:

“Showback is always required in any FinOps practice, but chargeback is dependent on organizational accounting policies.”

This is more conservative than the prevailing vendor narrative (which positions chargeback as “mature” and showback as “immature”). The Foundation’s position: showback with strong accountability mechanisms achieves most of the cost discipline goals that chargeback achieves, without the organizational friction of P&L adjustments.

When to Do True Chargeback for AI

Chargeback is warranted for AI when:

  1. The business unit’s AI spend is material enough to affect their P&L in a meaningful way (typical enterprise threshold: >$500K/year per BU)
  2. The business unit has budget authority — i.e., they can actually make trade-off decisions (buy more capacity vs. optimize) when they see their bill
  3. The AI product generates revenue or directly attributable cost savings — i.e., there is a clear return to measure against the charge
  4. Organizational accounting policy already chargebacks other technology spend (if IT does not chargeback SaaS licenses, adding AI chargeback creates an inconsistent precedent)

Handling Shared Model Costs

The hardest AI chargeback problem is shared foundation model infrastructure: one deployed model (e.g., a fine-tuned Llama 3.1 on a dedicated GPU cluster, or an Azure PTU reservation) serving multiple business units.

The FinOps Foundation and practitioner community have converged on two allocation methods:

Token-weighted allocation BU_i share = (BU_i input+output tokens) / (total input+output tokens across all BUs)

This is the most common method. It is fair for API-based spend. It slightly underfavors high-complexity workloads that use smaller token counts but more expensive models.

Request-weighted allocation BU_i share = (BU_i request count) / (total request count)

Used when BUs have very different token-per-request profiles and token-weighted would unfairly penalize verbose use cases (e.g., document analysis vs. classification).

Fixed allocation for committed capacity For provisioned capacity (PTU commitments, dedicated GPU clusters), a common pattern is:

  1. Allocate committed capacity cost proportionally to the BUs that requested the commitment
  2. Allocate variable/spillover cost by token weight
  3. Report committed utilization rate per BU — a BU that underutilizes its committed allocation is shown the “wasted committed cost”

Organizational Change Management Patterns

Pattern 1: Showback-first, chargeback-later Deploy showback for 2–3 quarters before switching to chargeback. Teams develop an intuition for their spend before the P&L hit lands. This dramatically reduces “surprise and outrage” responses that derail AI programs.

Pattern 2: Shadow billing Run chargeback calculations in parallel with showback for one quarter — teams see what their bill “would have been” before it’s real. Surfaces objections, incorrect allocations, and data quality issues before they become finance disputes.

Pattern 3: Internal pricing floors Some enterprises set an internal “AI budget” per team or per employee that is pre-allocated (not tracked to zero), then chargeback only overages. This reduces friction for normal AI use while creating accountability for runaway spend.

6. AI FinOps Tooling Landscape

Taxonomy

Three layers of tooling serve different needs:

  1. FinOps platforms with AI modules — incumbent cloud cost tools that have added AI provider integrations (Apptio Cloudability, CloudHealth, CloudZero)
  2. AI-native FinOps platforms — tools built primarily for multi-provider AI cost management (Finout, Vantage)
  3. AI gateway/observability tools — tools that sit in the request path, capturing token data at call time (Helicone, LangSmith, Portkey, LiteLLM); not strictly FinOps platforms but provide the telemetry that FinOps platforms lack

Platform Profiles

Apptio Cloudability (IBM)

AI coverage: Amazon SageMaker, Amazon Bedrock, Azure AI Foundry, third-party SaaS AI (OpenAI, Anthropic via SaaS billing connectors). Unified visibility spans managed services, self-managed IaaS, and SaaS.
Strengths: Enterprise-grade chargeback and showback with finance-team UX; audit-ready P&L allocation reports; strong integration with IBM Turbonomic for GPU rightsizing recommendations.
Weaknesses: Limited token-level granularity — tracks spend by service and model family, not by request or team. Dependent on cloud billing exports, which aggregate tokens. Requires custom ETL for token-level attribution.
Best fit: Large enterprises with an established Cloudability implementation wanting to add AI cost visibility without changing platforms.

CloudHealth (Broadcom/VMware)

AI coverage: AWS, Azure, GCP native AI services via billing integration. Publishes practical guidance on OpenAI/Anthropic billing normalization and attribution patterns.
Strengths: Strong multi-cloud normalized billing; good documentation on AI cost observability patterns for FinOps practitioners.
Weaknesses: Less purpose-built for AI than newer entrants; AI features are bolt-ons to a cloud-first architecture.
Best fit: Organizations standardized on CloudHealth wanting incremental AI cost visibility.

CloudZero

AI coverage: OpenAI, Anthropic, AWS AI services, Google Vertex; unit cost mapping against engineering telemetry.
Strengths: CostIntelligence platform maps AI spend to engineering metrics (deploys, PRs, customers) — closer to unit economics than pure cost tracking; strong for SaaS companies measuring cost-per-customer.
Best fit: Product companies building AI features where cost-per-customer or cost-per-feature is the primary KPI.

Finout

AI coverage: OpenAI, Anthropic, AWS SageMaker + Bedrock, GCP Vertex AI, Azure AI Foundry, Databricks, Snowflake — all ingested into MegaBill alongside cloud and Kubernetes spend.
Key feature: Virtual Tags — allocation rules using any available metadata (API keys, namespaces, service names, custom dimensions), applied retroactively without code changes. When org structure changes or a new AI provider is added, logic updates in minutes.
Token tracking: Token-level allocation with anomaly detection. No additional cost for AI features within existing Finout subscription.
FinOps X 2026: Finout was an exhibitor and published a same-day recap, indicating strong community presence.
Best fit: Enterprises wanting a single pane of glass across cloud + SaaS + AI with minimal engineering overhead; strong for chargeback-ready reporting.

Vantage

AI coverage: Native OpenAI, Anthropic, Databricks, Anyscale token-level ingest. 25+ SaaS integrations.
Key differentiator: MCP server — lets engineers query AI spend from inside AI coding assistants (e.g., “what did our Claude spend look like last week?”). The only major FinOps platform with MCP support and Terraform provider support as of June 2026.
Strengths: Self-serve, mid-market accessible; strong token-level cost views with per-model, per-team, per-customer cost allocation via virtual tagging; unit costs and anomaly detection.
Best fit: Mid-market and growth-stage companies; engineering-led FinOps teams; organizations using AI coding assistants for internal FinOps workflows.

Spot.io (NetApp)

AI coverage: GPU cost optimization for cloud-hosted training and inference workloads via Spot instances and preemptible VMs. Spot Ocean ML extends Kubernetes cost optimization to AI/ML workloads.
Strengths: Significant (40–70%) GPU cost reduction for interruption-tolerant training workloads; automated bin-packing for mixed GPU fleets.
Weaknesses: Not a token-cost tracking platform — focused on infrastructure layer, not API billing layer. Does not integrate with OpenAI/Anthropic API spend.
Best fit: Teams running self-hosted model training or large-scale inference on cloud GPU fleets, not API-first teams.

The Missing Layer: API Gateway Observability

None of the FinOps platforms above capture request-level data — they all depend on billing exports, which aggregate across requests. For true team-level attribution (the Walk/Run maturity requirement), enterprises must instrument a proxy layer:

  • Helicone — open-source and hosted; logs every LLM request with cost, tokens, latency, metadata; integrates with OpenAI, Anthropic, Azure, Bedrock, Gemini; exports to Postgres/S3 for FinOps platform ingestion
  • LiteLLM — open-source proxy; model routing + cost tracking; supports 100+ providers; exports spend logs per team/project/key
  • Portkey — similar to LiteLLM with stronger enterprise access control; per-virtual-key budgets and spend alerts
  • LangSmith (LangChain) — observability focused; cost tracking is secondary; better for debugging than for FinOps

The recommended architecture: Helicone or LiteLLM as the request gateway (captures token-level attribution), feeding into Finout or Vantage (produces FinOps-grade reports and chargeback allocations). Neither layer alone is sufficient.

7. KPIs and Unit Economics

The Foundation’s Unit Economics capability page and the AI working group papers converge on a tiered KPI framework:

Layer 1: Cost Visibility KPIs (Crawl stage — always required)

KPI Formula Typical update frequency
Cost per 1M input tokens total_input_cost / (input_tokens / 1,000,000) Weekly by provider and model
Cost per 1M output tokens total_output_cost / (output_tokens / 1,000,000) Weekly by provider and model
Cache hit rate cached_tokens / (input_tokens + cached_tokens) Daily — cache hit rate above 40% meaningfully reduces costs on Anthropic and OpenAI
Token spend by team sum(cost) GROUP BY team, week Weekly
Unallocated AI spend % untagged_spend / total_ai_spend Weekly — target <5% at Walk maturity

Layer 2: Efficiency KPIs (Walk stage)

KPI Formula Notes
Input:Output token ratio input_tokens / output_tokens per use case High ratios (>10:1) indicate context verbosity; typically optimizable through compression or structured inputs
Average context length mean(input_tokens) per use case Baseline for compression potential
Model tier utilization % of spend on frontier vs. mid-tier vs. open-source Target distribution depends on use case mix; typical enterprise goal is <40% on frontier models
Retry rate by model retry_requests / total_requests High retry rate (>5%) on a cheaper model signals quality degradation that inflates real cost-per-outcome
Cost per request total_cost / request_count by use case Baseline for regression testing after prompt or model changes

Layer 3: Unit Economics KPIs (Run stage)

KPI Formula Benchmarks
Cost per successful task completion total_ai_cost / accepted_outputs (where accepted = passes quality gate) Varies widely; customer service: $0.01–$0.08 per resolved contact; code review: $0.05–$0.30 per review accepted by engineer
Cost per active user per month monthly_ai_cost / MAU Copilot-equivalent tools: $5–$15/user/month at enterprise scale, including infrastructure
Cost per business outcome ai_cost / outcome_count (e.g., per deal assisted, per contract reviewed, per ticket resolved) Must be defined per use case; no universal benchmark
Token efficiency ratio accepted_output_value_tokens / total_input_tokens (normalized by use case) Higher is better; measures how much valuable output is generated per unit of input investment
AI cost as % of service delivery cost ai_cost / (ai_cost + human_cost + infra_cost) for a given workflow Useful for ROI framing in CFO conversations

Instrumentation Approach

To compute Layer 3 KPIs, two data streams must be joined:

  1. Token telemetry — from API gateway (Helicone/LiteLLM): {request_id, team, model, input_tokens, output_tokens, cost, timestamp}
  2. Outcome telemetry — from the application: {request_id, outcome_type, accepted: bool, quality_score}

The join key is request_id, which must propagate from the LLM call through to the application’s outcome logging. This is the engineering work that most teams skip — they instrument cost but not outcome — making Layer 3 KPIs impossible to compute retroactively.

Recommended implementation: emit a structured log event on every LLM interaction that includes both the gateway-captured token data and the application-level outcome. Route to a data warehouse (Snowflake, BigQuery, Redshift). Build a dbt model that computes the Layer 3 KPIs as weekly aggregates.

Published Cost Benchmarks (2026)

  • GPT-4-class inference: $0.30–$0.75 per million tokens (input+output blended) as of early 2026, down from ~$20/M in late 2022 — a 280× reduction in ~3.5 years
  • Claude 3.5 Sonnet: ~$3/M input, $15/M output (direct API) — significantly higher than mid-tier; justified for tasks requiring deep reasoning or long documents
  • Llama 3.1 70B (self-hosted on cloud GPU): $0.08–$0.15/M tokens at full GPU utilization on H100s; cost competitive with mid-tier APIs when GPU utilization exceeds 60%
  • Inference accounts for 80–90% of total enterprise AI compute consumption; training is a decreasing share of total spend for most enterprises (not training labs)
  • Enterprise AI spend: $37 billion globally in 2025 (up from $13.8B in 2024); inference is the dominant component

Practitioner Decision Tree

“Do I need a new platform or can I start with what I have?”

  • If you have <$100K/month AI spend: use provider native dashboards + a spreadsheet pivot. Do not buy a FinOps platform for AI yet.
  • If you have $100K–$1M/month: Vantage or Finout provide strong ROI. Deploy a gateway proxy (Helicone or LiteLLM) before or in parallel.
  • If you have >$1M/month and need P&L chargeback: Finout MegaBill or Apptio Cloudability depending on whether you need AI-native flexibility or enterprise finance-grade audit trails.
  • If you have significant self-hosted GPU workloads: add Spot.io Ocean ML for infrastructure optimization alongside a token-tracking platform for API spend.

“What is the minimum viable AI FinOps stack?”

  1. API proxy with {team, project, model, tokens, cost} logging — this is non-negotiable; everything else depends on it
  2. Weekly cost report by team — can be a Slack message generated from a SQL query against the proxy logs
  3. Anomaly alert on weekly spend ±40% — single alert rule, any cloud native monitoring tool
  4. Quarterly review of model tier distribution vs. use case requirements

Everything above this minimum is optimization. Build the minimum before buying enterprise tooling.

Outstanding Gaps and Watch Items

  1. FOCUS 1.5 draft — no public draft as of June 17, 2026. Watch focus.finops.org and the FOCUS GitHub repository (github.com/FinOps-Open-Cost-and-Usage-Spec/FOCUS_Spec) for the AI columns RFC.

  2. Tokenomics Foundation working group charters — the technical roadmap and initial WG structures were announced at FinOps X but detailed charters have not been published. Watch tokeneconomics.com for Q3 2026 releases.

  3. Agentic AI cost accounting — none of the current frameworks adequately handle multi-step agentic workflows where a single “task” may invoke dozens of LLM calls, tool calls, and sub-agents. The FinOps Foundation’s Finout published a four-step allocation framework for AI agents in 2026, but this is not yet Foundation-level guidance. Revenium joined the Foundation in 2026 specifically focused on agentic AI spend governance — a signal that this gap is recognized.

  4. Tokenomicon 2027 — the first dedicated AI cost economics conference. Expect specification announcements, Tokenomics Foundation WG outputs, and potential FOCUS 1.5 preview.

Research synthesized from FinOps Foundation working group publications (finops.org/wg/), Linux Foundation press releases, FinOps X 2026 keynote proceedings, and independent analysis from Cast AI, Finout, Vantage, and Amnic. All claims tied to named sources above.