← Agent Frameworks 🕐 17 min read
Agent Frameworks

AI Cost Attribution for Multi-Tenant SaaS Platforms (2026)

Why This Is Now a First-Order Problem · March 2026

In 2024 the median SaaS company added AI features under the assumption that inference costs would shrink into the noise. They have not.

In 2024 the median SaaS company added AI features under the assumption that inference costs would shrink into the noise. They have not. ICONIQ’s January 2026 snapshot reports average AI product gross margin at 52%, up from 41% in 2024 but still 25–30 points below the prior-decade SaaS ceiling. Bessemer’s State of AI 2025 placed LLM-native company gross margins around 65%. The 2025 State of AI Cost Governance survey found 84% of companies report AI delivery costs cutting product gross margins by more than 6 percentage points; over a quarter see hits of 16 points or more.

Inference now represents roughly 23% of revenue for pure AI-first companies — that is, for every $1M in AI product revenue, ~$230K walks out as inference cost. The arithmetic is brutal: adding $15 in direct variable cost to an $80/month seat drops gross margin from 80% to ~65% immediately, with no path back unless costs are attributed, tracked, and either absorbed into pricing or passed through.

The companies managing this well share one property: they treat AI token consumption as a first-class financial object with per-tenant attribution from day one — not as a lumped infrastructure line on the P&L.

1. Architecture Patterns: Per-Tenant Isolation vs. Shared Model Gateway

The Spectrum

Per-tenant inference isolation — each tenant routes through a dedicated container, API key, or virtual node — is the gold standard when:

  • You have fewer than ~200 enterprise tenants and each pays $10K+/year
  • SLAs require hard blast radius guarantees (one tenant’s runaway agent cannot degrade another’s latency)
  • Compliance requires verifiable data segregation at the inference layer (HIPAA, FedRAMP)

Cost: ~3–5× higher infrastructure overhead than shared. Cold start latency is a real problem for tenants with infrequent usage.

Shared model gateway with attribution — all tenants share an LLM gateway, with request metadata carrying tenant identity for post-hoc cost allocation — is the default for:

  • PLG products with hundreds to tens of thousands of tenants
  • B2B products where tenants pay $500–$5K/year and full isolation doesn’t pencil
  • Early-stage products that need operational simplicity before they have margin to optimize

The gateway approach shifts cost accuracy from infrastructure topology to metadata discipline. Every request must carry a tenant identifier in headers or request body; the gateway fans that identifier out to billing logs, rate-limit buckets, and cost ledger writes.

Decision Matrix by Tenant Count

Tenant Count Recommended Pattern Attribution Mechanism Notes
1–50 Per-tenant isolation or dedicated VMs Native provider billing by key Cost is manageable; isolation simplifies compliance
50–500 Shared gateway + virtual key per tenant LiteLLM virtual keys / LLM proxy tags Balance between isolation and density
500–5,000 Shared gateway + tag-based attribution Gateway metadata → SpendLogs / Prometheus Isolation at rate-limit bucket, not infra level
5,000+ Shared gateway + async ledger from streaming logs ClickHouse or TimescaleDB aggregation Real-time attribution becomes a dedicated subsystem

The Blast Radius Problem

At shared gateways with no per-tenant rate limits, one tenant’s agentic loop can consume 50× their expected tokens in a single incident, which either spikes your provider bill before anyone notices or degrades other tenants if you’re behind a rate-limited organizational key.

The correct model keys the rate-limit bucket by (tenant_id, model, workload_tier) so that throttling a runaway tenant is surgical. A single runaway incident without enforcement typically costs $2,000–$8,000 by the time someone notices. With three-layer rate limiting in place, the same incident costs $20–$100.

2. Tenant Cost Metering: Request Count vs. Token Count vs. Compute Time

Why Request Count Fails

Request count is seductive because it’s easy to instrument. It fails because the variance in cost per request is 100–1,000× for frontier models. A simple chat reply at 50 input / 100 output tokens costs $0.0003 on GPT-4o. A repo-wide code review at 15,000 input / 3,000 output tokens costs $0.045 — 150× more. Billing tenants per request either overcharges low-usage tenants or bleeds money on high-usage ones.

Use request count only as a secondary dimension for rate limiting, not for cost attribution or pricing.

Token Count Is the Correct Unit — with Caveats

Token count maps directly to provider billing and is the industry standard for LLM-native products (GitHub Copilot as of June 1, 2026; Cursor; Windsurf; all Anthropic API products). Three separate token dimensions matter:

  • Input tokens: typically 3–5× cheaper than output on most frontier models; must be tracked separately
  • Output tokens: the dominant cost driver for generation-heavy features (summarization, code generation)
  • Cached tokens: 50–90% discount depending on provider; critical to track separately to report accurate margin on cache-heavy tenants

Token counts must be captured from provider API responses, not estimated. All major providers return usage.prompt_tokens, usage.completion_tokens, and (where applicable) usage.prompt_tokens_details.cached_tokens in the response body.

Compute Time: When It Applies

Compute time matters for self-hosted inference (vLLM, Ollama, SGLang) where you are paying for GPU time, not tokens. In that model, cost = (GPU-seconds consumed) × (GPU cost per second). Token count is still useful for normalization and benchmarking, but the billing atom is time-on-hardware. For cloud API products, compute time is hidden inside the provider’s token pricing and is not a useful attribution unit.

Recommendation

  • Cloud API tenants: attribute by (input_tokens, output_tokens, cached_tokens, model) per request
  • Self-hosted inference tenants: attribute by (wall_clock_ms, model, gpu_tier) per request, with token count as a parallel metric for unit economics analysis
  • Free tier tenants: attribute by token count for accounting; bill by count only if/when they exceed the free pool

3. LiteLLM Multi-Tenant Configuration

LiteLLM’s proxy server is the most widely deployed open-source LLM gateway for multi-tenant SaaS. Its attribution model follows: Organization → Team → Project → Virtual Key.

Virtual Keys per Tenant

Each tenant gets a virtual key. The key carries metadata that is stamped on every LiteLLM_SpendLogs row the key generates.

# litellm_config.yaml
litellm_settings:
  success_callback: ["langfuse"]

general_settings:
  master_key: "sk-master-XXXX"
  database_url: "postgresql://..."

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY

# Provision a virtual key for tenant acme-corp
import litellm
from litellm.proxy.proxy_client import ProxyClient

client = ProxyClient(base_url="http://localhost:4000", api_key="sk-master-XXXX")

key = client.generate_key(
    team_id="team-acme-corp",
    key_alias="acme-corp-prod",
    metadata={"tenant_id": "acme-corp", "plan": "enterprise"},
    budget_duration="1mo",
    max_budget=500.00,         # hard monthly cap, $500
    soft_budget=400.00,        # alert threshold
    tpm_limit=200_000,         # tokens-per-minute rate limit
    rpm_limit=500,             # requests-per-minute rate limit
    tags=["tenant:acme-corp", "env:prod"],
)
# Returns: {"key": "sk-acme-XXXX", "team_id": "team-acme-corp", ...}

Tag-Based Attribution (Enterprise Feature)

Tags on requests enable cross-cutting attribution — by feature, by user within a tenant, by environment — independent of the key hierarchy.

# At request time, attach tags in metadata
response = openai_client.chat.completions.create(
    model="gpt-4o",
    messages=[...],
    extra_body={
        "metadata": {
            "tags": ["tenant:acme-corp", "feature:code-review", "user:u-882"],
            "user_id": "u-882",
        }
    }
)

Tag-level budgets can be set independently, so feature:code-review can have its own monthly cap that fires alerts before it blows the team’s total.

SpendLogs Query Patterns

LiteLLM_SpendLogs is a PostgreSQL table written by the proxy on every completed request. Key columns:

Column Type Notes
request_id uuid Idempotency key
api_key varchar The virtual key that issued the request
user varchar user_id passed in metadata
team_id varchar Team / tenant identifier
model varchar e.g. gpt-4o, claude-3-7-sonnet-20250219
prompt_tokens int Input token count from provider response
completion_tokens int Output token count
cache_read_input_tokens int Cached tokens (where provider reports them)
spend float Dollar cost computed from token × model price
startTime timestamp Request start
endTime timestamp Request end
metadata jsonb Arbitrary tags, user IDs, feature names

Per-tenant monthly spend rollup:

SELECT
    team_id,
    DATE_TRUNC('month', "startTime") AS month,
    SUM(prompt_tokens)            AS total_input_tokens,
    SUM(completion_tokens)        AS total_output_tokens,
    SUM(cache_read_input_tokens)  AS total_cached_tokens,
    SUM(spend)                    AS total_spend_usd,
    COUNT(*)                      AS request_count
FROM "LiteLLM_SpendLogs"
WHERE "startTime" >= DATE_TRUNC('month', NOW())
GROUP BY 1, 2
ORDER BY total_spend_usd DESC;

Per-feature spend within a tenant:

SELECT
    metadata->>'tags' AS tags,
    SUM(spend)        AS spend_usd
FROM "LiteLLM_SpendLogs"
WHERE team_id = 'team-acme-corp'
  AND "startTime" >= NOW() - INTERVAL '30 days'
  AND metadata->>'tags' LIKE '%feature:%'
GROUP BY 1
ORDER BY 2 DESC;

Budget alert query (used by monitoring job):

SELECT
    team_id,
    SUM(spend)                AS mtd_spend,
    t.max_budget,
    SUM(spend) / t.max_budget AS pct_consumed
FROM "LiteLLM_SpendLogs" sl
JOIN "LiteLLM_TeamTable" t USING (team_id)
WHERE sl."startTime" >= DATE_TRUNC('month', NOW())
GROUP BY 1, t.max_budget
HAVING SUM(spend) / t.max_budget > 0.80  -- alert at 80%
ORDER BY pct_consumed DESC;

LiteLLM v1.83.10 (released 2026) added multi-window budgets: a key or team can carry simultaneous daily and monthly budget periods, each with its own reset cadence and alert thresholds at 50% / 80% / 95%.

4. AWS Bedrock Application Inference Profiles (AIPs) for CUR Attribution

For products built on Amazon Bedrock, Application Inference Profiles provide native AWS-level per-tenant cost attribution that flows directly into the Cost and Usage Report (CUR 2.0) without a sidecar billing system.

How AIPs Work

An AIP is a wrapper around a foundation model ID. You create the profile, attach cost allocation tags, and pass the AIP’s ARN as the modelId in your InvokeModel or Converse calls. Bedrock stamps every request’s billing record with the profile’s tags.

import boto3

bedrock = boto3.client("bedrock", region_name="us-east-1")

# Create one AIP per tenant (or per tenant + workload)
response = bedrock.create_inference_profile(
    inferenceProfileName="tenant-acme-corp-prod",
    description="Inference profile for tenant acme-corp, prod workload",
    modelSource={
        "copyFrom": "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-7-sonnet-20250219-v1:0"
    },
    tags=[
        {"key": "tenant_id",  "value": "acme-corp"},
        {"key": "env",        "value": "prod"},
        {"key": "cost_center","value": "cc-4421"},
    ]
)
aip_arn = response["inferenceProfileArn"]
# arn:aws:bedrock:us-east-1:123456789:application-inference-profile/tenant-acme-corp-prod

# At inference time, use the AIP ARN instead of the model ID
runtime = boto3.client("bedrock-runtime", region_name="us-east-1")

result = runtime.converse(
    modelId=aip_arn,   # <-- AIP ARN, not the model ARN
    messages=[{"role": "user", "content": [{"text": "Summarize this document..."}]}]
)

CUR Usage Type Suffixes

In CUR 2.0, AIPs produce distinct usage type strings:

  • Base model: USE1-INPUT_TOKEN_COUNT:anthropic.claude-3-7-sonnet-20250219-v1:0
  • AIP-attributed: USE1-INPUT_TOKEN_COUNT:tenant-acme-corp-prod

This means AWS Athena queries over your CUR bucket can group spend by the AIP name — effectively by tenant — without any additional attribution infrastructure.

-- Athena query over CUR 2.0 S3 export
SELECT
    resource_tags_user_tenant_id        AS tenant_id,
    line_item_usage_type                AS usage_type,
    SUM(line_item_unblended_cost)       AS cost_usd,
    SUM(line_item_usage_amount)         AS token_units
FROM cur_database.cur_table
WHERE product_service_code = 'AmazonBedrock'
  AND line_item_line_item_type = 'Usage'
  AND DATE_FORMAT(line_item_usage_start_date, '%Y-%m') = '2026-06'
GROUP BY 1, 2
ORDER BY cost_usd DESC;

Operational Constraints

  • Cost allocation tags must be activated in the AWS Billing console before they appear in CUR. Allow 24 hours post-activation for tag population to begin.
  • AIPs are region-scoped; cross-region inference requires cross-region inference profiles (CRIPs), which add a 10–20% overhead to token cost.
  • AIP cost data in CUR is daily-grain aggregated, not per-request. For real-time per-request attribution, pair AIPs with a gateway-layer SpendLogs table; use CUR for reconciliation.
  • As of April 2026, Bedrock also supports IAM-principal-based cost allocation, so IAM role per tenant is an alternative to AIPs for shops that already have role-per-tenant IAM architecture.

5. Database Schema for Tenant Cost Ledger

A standalone tenant cost ledger, decoupled from the LLM gateway’s operational tables, is the right architecture at scale. The gateway writes high-frequency operational rows; the ledger aggregates them into billing-grade records.

Core Tables

-- Tenants master table
CREATE TABLE tenants (
    tenant_id        UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    external_id      TEXT UNIQUE NOT NULL,    -- your app's tenant slug, e.g. "acme-corp"
    name             TEXT NOT NULL,
    plan             TEXT NOT NULL,           -- free | starter | growth | enterprise
    created_at       TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

-- Raw token events (written by gateway integration, high volume)
CREATE TABLE token_events (
    event_id         UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    tenant_id        UUID NOT NULL REFERENCES tenants(tenant_id),
    user_id          TEXT,                    -- optional: end-user within tenant
    feature_slug     TEXT NOT NULL,           -- e.g. "code-review", "summarize", "chat"
    model            TEXT NOT NULL,           -- e.g. "gpt-4o", "claude-3-7-sonnet"
    provider         TEXT NOT NULL,           -- openai | anthropic | bedrock | gemini
    input_tokens     INT NOT NULL,
    output_tokens    INT NOT NULL,
    cached_tokens    INT NOT NULL DEFAULT 0,
    cost_usd         NUMERIC(12,8) NOT NULL,  -- computed at write time from model price table
    request_id       TEXT,                    -- provider request ID for dedup
    occurred_at      TIMESTAMPTZ NOT NULL DEFAULT NOW()
) PARTITION BY RANGE (occurred_at);

-- Partition by month for cost-effective retention
CREATE TABLE token_events_2026_06
    PARTITION OF token_events
    FOR VALUES FROM ('2026-06-01') TO ('2026-07-01');

-- Indexes for common query patterns
CREATE INDEX idx_token_events_tenant_occurred
    ON token_events (tenant_id, occurred_at DESC);

CREATE INDEX idx_token_events_feature
    ON token_events (tenant_id, feature_slug, occurred_at DESC);

CREATE INDEX idx_token_events_model
    ON token_events (model, occurred_at DESC);

-- Hourly aggregates (materialized by background job, used for dashboards)
CREATE TABLE tenant_cost_hourly (
    tenant_id        UUID NOT NULL REFERENCES tenants(tenant_id),
    hour             TIMESTAMPTZ NOT NULL,    -- truncated to hour
    feature_slug     TEXT NOT NULL,
    model            TEXT NOT NULL,
    input_tokens     BIGINT NOT NULL DEFAULT 0,
    output_tokens    BIGINT NOT NULL DEFAULT 0,
    cached_tokens    BIGINT NOT NULL DEFAULT 0,
    cost_usd         NUMERIC(12,6) NOT NULL DEFAULT 0,
    request_count    INT NOT NULL DEFAULT 0,
    PRIMARY KEY (tenant_id, hour, feature_slug, model)
);

-- Monthly billing snapshots (used for invoice generation)
CREATE TABLE tenant_billing_periods (
    id               UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    tenant_id        UUID NOT NULL REFERENCES tenants(tenant_id),
    period_start     DATE NOT NULL,
    period_end       DATE NOT NULL,
    input_tokens     BIGINT NOT NULL,
    output_tokens    BIGINT NOT NULL,
    cached_tokens    BIGINT NOT NULL,
    raw_cost_usd     NUMERIC(12,4) NOT NULL,  -- actual provider cost
    billed_amount    NUMERIC(12,4) NOT NULL,  -- after markup / floor / overage
    stripe_invoice_id TEXT,
    status           TEXT NOT NULL DEFAULT 'pending',  -- pending | invoiced | paid | void
    created_at       TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    UNIQUE (tenant_id, period_start)
);

-- Model price table (updated when providers change rates)
CREATE TABLE model_prices (
    model            TEXT NOT NULL,
    provider         TEXT NOT NULL,
    input_cost_per_mtok  NUMERIC(10,6) NOT NULL,  -- cost per million input tokens
    output_cost_per_mtok NUMERIC(10,6) NOT NULL,
    cache_read_cost_per_mtok NUMERIC(10,6) NOT NULL DEFAULT 0,
    effective_from   TIMESTAMPTZ NOT NULL,
    PRIMARY KEY (model, effective_from)
);

Real-Time Dashboard Query

-- Current month spend by tenant, with budget utilization
WITH mtd AS (
    SELECT
        tenant_id,
        SUM(cost_usd)       AS cost_usd,
        SUM(input_tokens)   AS input_tokens,
        SUM(output_tokens)  AS output_tokens
    FROM tenant_cost_hourly
    WHERE hour >= DATE_TRUNC('month', NOW())
    GROUP BY tenant_id
)
SELECT
    t.external_id,
    t.plan,
    COALESCE(m.cost_usd, 0)       AS mtd_cost_usd,
    COALESCE(m.input_tokens, 0)   AS mtd_input_tokens,
    COALESCE(m.output_tokens, 0)  AS mtd_output_tokens
FROM tenants t
LEFT JOIN mtd m USING (tenant_id)
ORDER BY mtd_cost_usd DESC;

TimescaleDB / ClickHouse at Scale

For products with >5,000 active tenants or >1M requests/day, raw token_events writes will saturate a standard PostgreSQL instance. Two paths:

  • TimescaleDB: drop-in PostgreSQL extension; hypertables handle partitioning automatically; continuous aggregates replace manual hourly rollup jobs
  • ClickHouse: orders of magnitude faster for analytical queries; write token_events directly from the gateway via the ClickHouse HTTP API; use ReplacingMergeTree for idempotent inserts with the provider request_id as the dedup key

6. Pricing Model Design

Three models dominate the market. They are not mutually exclusive — the industry is converging on hybrid architectures that use elements of all three.

Token Passthrough

What it is: You bill the tenant at or near your wholesale token cost. No markup. Revenue comes from the subscription platform fee or seat license; inference is a managed cost.

Who uses it: GitHub Copilot Enterprise (post-June 2026 transition), Cursor Agent tier (via direct model passthrough mode), enterprise data platforms where the LLM is a commodity feature.

Economics: Only viable if the seat fee is large enough to subsidize variance. At $39/seat, GitHub discovered that heavy agentic users could generate $700+/month of inference — 18× the subscription. The June 2026 pivot to token billing was the result.

When to choose it: When your product’s value is the workflow, not the intelligence. When tenants are sophisticated buyers who understand token costs. When you need enterprise procurement to pass through AI cost line items to their own cost centers.

Margin-on-Inference

What it is: You buy tokens at wholesale from the provider and sell at a marked-up rate to tenants. The margin covers your gateway infrastructure, reliability layer, and gross margin on AI.

Typical markup: 15–50% above wholesale. At 30% markup on $0.015/1K input tokens = $0.0195/1K billed to tenant. Transparent to enterprise buyers who check your published rate card against provider pricing.

Who uses it: LLM API resellers, vertical copilot builders, workflow automation platforms where inference is a direct product component. Also the default for any company using LiteLLM’s virtual keys to intermediate model access.

Economics: A 30% gross margin on inference is not great standalone, but it converts AI cost from a variable drag on software margin into a positive-margin line item. The aggregated infrastructure (prompt caching, routing to cheaper models for simpler tasks) can bring blended provider cost down 20–40%, turning the margin-on-inference line into 40–70% GM.

Flat Fee with Overages

What it is: Tenants pay a flat monthly fee that includes a token pool. Usage beyond the pool triggers per-token overages.

Who uses it: PLG products (Notion AI, Intercom Fin, Grammarly Business), B2B tools where predictable pricing wins deals, any product where the marketing story requires “included AI” with no surprise bills for typical users.

Token pool sizing: The pool must be sized so that the median tenant is at ≤80% utilization and the 95th percentile tenant is near but not over the pool. If p95 is consistently 150% of the pool, you are effectively giving away inference to your most active users — which is fine if those are your best expansion leads, and bad if those are free-tier users.

Overage rate: Typically 2–3× the blended provider cost per token to ensure overages are margin-positive and to create a natural incentive for tenants to upgrade to a higher pool tier rather than accumulate overage charges.

Pricing Model Gross Margin on AI Revenue Predictability Enterprise Procurement Fit PLG Fit
Token passthrough Negative to breakeven Low High Low
Margin on inference 20–50% Medium Medium Medium
Flat fee + overages Variable; -10% to +60% depending on pool sizing High High High
Outcomes-based Can be 70%+ with efficient models Low Medium Low

7. Cost Floor Enforcement

Minimum Invoice Thresholds

Setting a minimum invoice amount — typically $5–$25 — prevents billing runs from generating invoices whose Stripe processing fee ($0.30 + 2.9%) exceeds the invoice value. Implement as a pre-billing check:

MIN_INVOICE_USD = 10.00

def compute_invoice(tenant_id: str, period: BillingPeriod) -> Invoice | None:
    usage = get_period_usage(tenant_id, period)
    billed = apply_pricing(usage, tenant.plan)

    if billed.total_usd < MIN_INVOICE_USD:
        # Roll over to next period rather than charging
        carry_forward_credit(tenant_id, billed.total_usd)
        return None

    return Invoice(tenant_id=tenant_id, amount_usd=billed.total_usd, ...)

Carried-forward amounts appear as a credit on the next invoice. This avoids the UX problem of charging $0.47 on a credit card while keeping the ledger accurate.

Free Tier Token Pools

Free tiers need hard limits, not soft budgets. A soft budget fires an alert but continues serving; a hard limit returns HTTP 429 after the pool is exhausted.

FREE_TIER_MONTHLY_POOL = {
    "input_tokens":  50_000,
    "output_tokens": 25_000,
}

def check_free_tier_limit(tenant_id: str) -> LimitStatus:
    mtd = get_mtd_usage(tenant_id)
    return LimitStatus(
        input_remaining  = FREE_TIER_MONTHLY_POOL["input_tokens"]  - mtd.input_tokens,
        output_remaining = FREE_TIER_MONTHLY_POOL["output_tokens"] - mtd.output_tokens,
        hard_limited     = (mtd.input_tokens  >= FREE_TIER_MONTHLY_POOL["input_tokens"] or
                            mtd.output_tokens >= FREE_TIER_MONTHLY_POOL["output_tokens"])
    )

Return a 429 with a Retry-After header set to the pool reset date and a body indicating upgrade path. Free-tier tenants who hit the limit consistently are your highest-intent upgrade leads.

Burst Allowances

To avoid punishing legitimate burst usage (a user runs a one-time large batch job), implement a burst allowance separate from the monthly pool:

BURST_ALLOWANCES = {
    "starter": {"tokens": 20_000, "window_minutes": 60},
    "growth":  {"tokens": 100_000, "window_minutes": 60},
}

The burst window refills on a rolling 60-minute basis using a token-bucket algorithm. Burst tokens do not count against the monthly pool — they are strictly a rate limit, not a usage cap. This pattern prevents both the runaway-agent problem and the false positive of throttling a legitimate large job.

8. Chargeback to Tenants: Stripe Integration

Usage Record Architecture

Stripe’s native metered billing (plus the January 2026 Metronome acquisition for enterprise contract scenarios) handles the invoice generation side. The SaaS platform’s job is to push accurate usage records to Stripe at the right cadence.

import stripe

stripe.api_key = "sk_live_..."

def report_tenant_usage(tenant_id: str, period_end: datetime):
    """
    Called by billing job at end of billing period.
    Pushes token consumption to Stripe for metered invoice generation.
    """
    usage = get_period_usage(tenant_id, period_end)
    tenant = get_tenant(tenant_id)

    # Stripe subscription item ID for this tenant's metered component
    # Provisioned when tenant upgrades; stored on tenant record
    si_id = tenant.stripe_metered_subscription_item_id

    # Report overage tokens (usage above included pool)
    overage_tokens = max(0, usage.total_output_tokens - tenant.plan.included_output_tokens)

    if overage_tokens > 0:
        stripe.billing.MeterEvent.create(
            event_name="ai_output_tokens",
            payload={
                "stripe_customer_id": tenant.stripe_customer_id,
                "value": str(overage_tokens),
            },
            timestamp=int(period_end.timestamp()),
        )

As of Stripe’s March 2026 AI token billing preview, Stripe can also auto-detect model pricing and apply markup automatically — but the waitlist is slow. The pattern above (push raw token counts, price on your side or via Stripe price IDs) works without the preview feature.

Invoice Line Item Design

Each AI invoice should have distinct line items that map to the tenant’s cost ledger:

# Build an invoice with multiple AI line items
invoice = stripe.Invoice.create(
    customer=tenant.stripe_customer_id,
    auto_advance=False,
    description=f"AI usage — {period_label}",
    metadata={"tenant_id": tenant_id, "billing_period": period_label}
)

# Line item 1: Included plan tokens (already covered by subscription)
# Not billed; shown as $0 informational line
stripe.InvoiceItem.create(
    customer=tenant.stripe_customer_id,
    invoice=invoice.id,
    description=f"AI tokens included in {tenant.plan.name} plan",
    quantity=tenant.plan.included_tokens,
    unit_amount=0,
    currency="usd",
)

# Line item 2: Overage output tokens
if usage.overage_output_tokens > 0:
    stripe.InvoiceItem.create(
        customer=tenant.stripe_customer_id,
        invoice=invoice.id,
        description="AI output tokens (overage)",
        quantity=usage.overage_output_tokens,
        unit_amount=int(OVERAGE_RATE_PER_TOKEN_USD * 100_000),  # in cents
        currency="usd",
        metadata={"feature_breakdown": json.dumps(usage.by_feature)}
    )

Stripe caps subscriptions at 20 line items. For tenants with >20 billing dimensions (feature-level breakdown), compute the multi-dimensional detail on your side and push a single consolidated overage line to Stripe with the detail in the invoice description or a hosted usage portal.

Metronome for Enterprise Contracts

For enterprise tenants with negotiated token pools, commit-and-consume structures, or annual contracts with monthly draw-downs, Stripe + Metronome (post-acquisition) handles the contract layer. Metronome supports:

  • Multi-dimensional pricing catalogs (different rates for different features/models)
  • Commit reconciliation (minimum annual commits against actual usage)
  • Custom invoice schedules that differ from Stripe’s billing cycle

9. AI Cost as a Signal for Product Decisions

Per-tenant cost attribution produces a second-order dataset that most product teams ignore: token cost per feature, which directly reveals which product features are economically viable at the current pricing tier.

Token Cost per Feature

Track (feature_slug, model, avg_cost_per_invocation) and join it against feature adoption and retention signals:

-- Cost per feature invocation vs. usage frequency
WITH feature_economics AS (
    SELECT
        feature_slug,
        COUNT(*)                        AS invocations_30d,
        AVG(cost_usd)                   AS avg_cost_per_call,
        SUM(cost_usd)                   AS total_cost_30d,
        COUNT(DISTINCT tenant_id)       AS tenants_using
    FROM token_events
    WHERE occurred_at >= NOW() - INTERVAL '30 days'
    GROUP BY feature_slug
)
SELECT
    feature_slug,
    invocations_30d,
    ROUND(avg_cost_per_call::numeric, 5)   AS avg_cost_usd,
    total_cost_30d,
    tenants_using,
    -- Revenue attribution requires join to your billing/plan table
    total_cost_30d / NULLIF(invocations_30d, 0) AS cost_per_call_usd
FROM feature_economics
ORDER BY total_cost_30d DESC;

Common findings from running this query:

  1. The long-tail problem: 2–3 features often account for 60–80% of total inference cost. These are candidates for model downgrades (GPT-4o → GPT-4o-mini for simpler tasks) or prompt compression.
  2. The zombie feature: A feature with high per-call cost, low adoption, and few tenants using it. Kill it or put it behind a higher plan tier.
  3. The surprise cost center: A feature the team considered “lightweight” (e.g., auto-complete suggestions fired on every keystroke) turns out to be 40% of inference costs due to call volume.

Model Routing for Cost Reduction

Once you have cost-per-feature data, apply model routing: route requests to cheaper models for features where output quality is less critical.

FEATURE_MODEL_MAP = {
    "code-review":       "claude-3-7-sonnet-20250219-v1:0",  # quality-critical
    "summarize-short":   "claude-haiku-4-20250514-v1:0",      # cost-optimized
    "chat-suggestion":   "gpt-4o-mini",                       # high-volume, low cost
    "rag-answer":        "gpt-4o",                             # accuracy-critical
}

A well-tuned routing table commonly cuts blended inference cost by 30–50% without measurable quality degradation on cost-optimized routes, based on reported patterns from LiteLLM’s enterprise deployments.

10. Tenant Cost Anomaly Detection

Budget Alert Tiers

Set alerts at three thresholds per tenant, not one:

Threshold Action Channel
50% of monthly budget consumed Informational log; no action Internal metrics
80% of budget consumed Alert to tenant admin (in-app + email) Notify tenant
95% of budget consumed Alert to tenant admin + your ops team Escalate
100% (hard limit) Throttle at gateway; 429 to tenant Enforce

LiteLLM v1.83.10 supports all four tiers natively on virtual keys and team budgets.

Runaway Usage Detection

Runaway usage is distinct from budget exhaustion — it is abnormal token velocity for a specific tenant, even if they have budget headroom. The canonical signal is a Z-score spike on hourly token consumption:

from scipy import stats
import numpy as np

def detect_runaway(tenant_id: str, window_hours: int = 168) -> bool:
    """
    Returns True if the current hour's token consumption is a statistical
    outlier vs. the trailing 7-day window for this tenant.
    """
    hourly = get_hourly_token_counts(tenant_id, hours=window_hours)
    if len(hourly) < 24:
        return False  # not enough history

    historical = np.array(hourly[:-1])  # exclude current hour
    current    = hourly[-1]

    z_score = (current - historical.mean()) / (historical.std() + 1e-9)
    return z_score > 4.0  # 4-sigma threshold

Concrete runaway patterns to alert on:

  1. Agent loop: Token consumption doubles every 5–10 minutes with no gaps (no human think-time). Signature: very high output tokens → fed back as input tokens in the next call.
  2. Context explosion: Input tokens grow linearly across a session as the full conversation history is stuffed on every call. Signature: input tokens per call increase monotonically within a session.
  3. Prompt injection escalation: Unusual model output tokens (very long completions) following short input prompts. Signature: output/input token ratio exceeds 10×.

Enforcement Response Tiers

async def enforce_anomaly(tenant_id: str, anomaly_type: str):
    match anomaly_type:
        case "agent_loop":
            # Immediate hard throttle — refund cost, block session
            await set_tenant_rate_limit(tenant_id, rpm=0, duration_minutes=5)
            await notify_ops(f"Agent loop detected for {tenant_id}")
        case "context_explosion":
            # Soft intervention — inject system message warning
            await set_context_truncation_hint(tenant_id, max_context_tokens=8192)
        case "budget_approaching":
            # No enforcement; notify tenant
            await send_budget_alert(tenant_id, threshold_pct=80)

11. Real-World AI COGS Benchmarks by Product Category

These figures synthesize data from Bessemer State of AI 2025, ICONIQ January 2026, BVP Atlas monetization playbook, and The SaaS CFO editorial coverage. Ranges reflect variance across company size, model mix, and pricing sophistication.

AI COGS as % of Revenue by Product Category

Product Category AI COGS as % of Revenue Typical Gross Margin Notes
Coding assistant / Copilot 25–45% 45–65% High output token intensity; agentic features 3–5× more expensive than autocomplete
Document summarization 10–20% 70–82% Large input, small output; prompt caching recovers 40–60% of cost on repeat docs
Conversational AI / chatbot 15–30% 60–75% Highly variable; free-tier abuse is the primary risk factor
AI search / RAG 8–15% 72–85% Input-heavy; smaller models viable for embedding and retrieval layers
AI writing assistant 18–28% 65–75% Moderate output intensity; quality requirements resist model downgrade
AI analytics / BI copilot 12–22% 68–80% SQL generation is cheap; natural language explanation of results is expensive
Voice AI / transcription 5–12% 78–88% Whisper-class transcription is now commodity-priced; downstream LLM analysis varies
Agentic / multi-step workflows 35–60% 35–55% Multiple model calls per task; most exposed to runaway-agent cost spikes

Key Industry Benchmarks

  • ICONIQ January 2026: Average AI product gross margin 52%; up from 41% (2024) and 45% (2025). Trajectory is positive but still well below traditional SaaS norms.
  • Bessemer State of AI 2025: LLM-native company gross margins ~65%, vs. 80–90% ceiling for prior-decade cloud SaaS.
  • Token price deflation: LLM API prices dropped ~80% between early 2025 and early 2026. GPT-4o input fell from $5.00 to $2.50/MTok; o4 Mini input at $0.55/MTok. This is partially offsetting volume growth.
  • GitHub Copilot case study: Post-June 2026 token billing transition, heavy agentic users reported 10–50× cost increases vs. prior flat rate ($39/month → $400–$800/month equivalent). This is the clearest public signal that flat-rate AI pricing at scale is structurally untenable.
  • Inference cost floor: Even with aggressive caching, routing, and model mix optimization, frontier-model SaaS products are unlikely to get blended AI COGS below 8–12% of revenue at current pricing tiers. Products targeting >75% gross margin must either use smaller models, charge usage-based overages, or build differentiation above the inference layer.

Margin Recovery Levers (Ranked by Impact)

  1. Prompt caching: 50–90% cost reduction on repeat/similar prompts; zero quality impact. Highest ROI of any optimization.
  2. Model routing: Route simpler tasks to smaller models (Haiku, GPT-4o-mini). Typical 30–50% blended cost reduction.
  3. Context length discipline: Enforce max context windows per feature. Each 2× reduction in average context length halves input token cost.
  4. Token-aware pricing tier: Move heavy users to usage-based tiers instead of absorbing their cost in flat plans.
  5. Batch inference for async features: Bedrock Batch API and OpenAI Batch API offer 50% discount for non-real-time use cases (document processing, nightly analytics summaries).

Summary: The Implementation Stack

A production-grade AI cost attribution system for multi-tenant SaaS has four layers:

  1. Gateway layer — LiteLLM proxy with virtual key per tenant, tag-based metadata, SpendLogs table; or AWS Bedrock AIPs for Bedrock-native stacks
  2. Ledger layer — token_events table (PostgreSQL + TimescaleDB, or ClickHouse at scale) with hourly rollups into tenant_cost_hourly
  3. Billing layer — Stripe metered billing for usage records; Metronome for enterprise contracts with commits and multi-dimensional pricing
  4. Signal layer — Cost-per-feature queries driving model routing decisions, pricing tier adjustments, and feature kill/keep decisions

The companies that get this right treat the signal layer as a product function, not an engineering afterthought. Token cost per feature, joined to feature adoption and retention, is the clearest available signal for which AI features are worth their margin.

Sources