← Agent Frameworks 🕐 14 min read
Agent Frameworks

Bedrock Cost Forecasting and Capacity Planning (2026)

Executive Summary · March 2026

LLM workloads break the core assumptions of every FinOps forecasting model built before 2023.

LLM workloads break the core assumptions of every FinOps forecasting model built before 2023. Cost is not proportional to seats, instances, or hours — it is proportional to tokens, and tokens vary unpredictably with prompt design, model version, agentic amplification, and week-over-week adoption curves. Real teams report spending 1.5–2× their initial Bedrock estimate in the first 12 months. This note builds a forecasting stack from first principles: bottom-up token modeling, CloudWatch percentile-based capacity decisions, Reserved tier break-even arithmetic, AWS Budgets ML alerting, and the QBR artifact set a CFO will actually act on.

1. Why LLM Cost Forecasting Is Hard

1.1 Non-Deterministic Output Lengths

Unlike EC2 (billed by hour) or S3 (billed by GB), Bedrock charges output tokens separately at 3–5× the input rate. Output length is non-deterministic: the same prompt produces 120 tokens on one call and 840 on the next, depending on model temperature, sampler settings, and context window fill. Average output tokens across an application can drift ±40% week-over-week purely from prompt changes, not usage changes.

Claude Sonnet 4.5/4.6 pricing as of mid-2026:

  • Input: $3.00 / million tokens
  • Output: $15.00 / million tokens

A workload averaging 500 output tokens/request has 5× the per-call cost of one averaging 100 tokens at the same input volume. Forecasters must track output:input ratio as a first-class signal.

1.2 Rapid and Non-Linear Price Changes

Model prices have dropped roughly 90% since GPT-4’s launch in 2023:

  • GPT-4 (March 2023): $30/M input tokens
  • GPT-4o (May 2024): $5/M input tokens (83% drop in 15 months)
  • Budget frontier (2026): $0.05–$0.15/M input tokens for capable models

Gartner (March 2026) forecasts inference costs for 1-trillion-parameter models will fall >90% by 2030 vs. 2025. The implication: a 3-year Bedrock cost model that locks in today’s per-token rates will overstate year-3 inference cost by 60–80%. Build price-drop assumptions into every scenario.

1.3 Usage Growth Curves Are Convex, Not Linear

Enterprise AI adoption follows an S-curve, but the early exponential phase is steep. New AI products routinely see 2–4× monthly active user growth in the first six months of deployment. Average prompt token length grew nearly 4× from early 2024 to 2026, driven by longer system prompts, multi-turn context, and RAG retrieval injection.

1.4 Agentic Amplification

A single user-visible agent step typically triggers 4–8 internal model calls (planner, tool router, result synthesizer, validator). Agentic models consume 5–30× more tokens per user task than a single-turn chatbot. A workload that looks like 1M tokens/day at the API surface is actually 5–20M tokens/day in Bedrock CUR. Forecasters who instrument only the outer API call miss 80–95% of actual spend.

1.5 Hidden Cost Multipliers (30–80% on Top of Inference)

Charge Typical Overhead
Knowledge Base / OpenSearch Serverless $345/month minimum for RAG
Data transfer (cross-region, egress) 20–35% of inference cost
Fine-tuning storage + training hours Variable; $0.0008/token for training
Guardrails, model evaluation, Bedrock Agents 5–15% adder
Retries and exponential backoff 2–8% waste on throttled workloads

FinOps teams that scope only inference tokens systematically underforecast by 30–80%.

2. AWS Cost Explorer Forecasting for Bedrock

2.1 How Cost Explorer Generates Forecasts

AWS Cost Explorer uses ML models trained on up to 36 months of Cost and Usage Report (CUR) data. As of November 2025, CE extended forecast horizons from 12 to 18 months and added AI-powered anomaly detection. The underlying algorithm is approximately a linear trend + seasonality decomposition, equivalent to a Holt-Winters model applied to tagged cost dimensions.

CE requires approximately 5 weeks (35 days) of billing history before generating a forecast for a new service or tag combination.

2.2 Critical Limitations for AI Workloads

Limitation Impact
Linear extrapolation on exponential growth CE will underforecast by 50–300% for workloads in the early S-curve acceleration phase
No awareness of model price changes A forthcoming price drop is invisible to CE; forecasts assume current rates persist
No token-level decomposition CE shows $ spent, not tokens × price. Cannot separate volume growth from price compression
12-month historical cap (legacy) Seasonality and annual cycles invisible on accounts <12 months old
Granularity limited to service tags Without Bedrock Projects cost attribution tags, all models pool together

Practical rule: Trust CE forecasts only for workloads with ≥3 months of stable, non-accelerating history. For growth-phase workloads, always override CE with a bottom-up model.

2.3 Tagging Strategy for CE Accuracy

Enable Bedrock Projects (GA 2025) to get per-project inference cost attribution in CUR. Tag every Bedrock API call with:

  • project — maps to application or business unit
  • env — prod / staging / dev
  • model-family — claude-4 / nova-pro / llama-4

Without these tags, CE cannot isolate the signal from the noise when forecasting individual workloads.

3. Bottom-Up Forecasting Model

The canonical formula for monthly Bedrock inference cost:

Monthly Cost = Σ_workloads [
  (requests_per_day × avg_input_tokens × input_price_per_token)
  + (requests_per_day × avg_output_tokens × output_price_per_token)
] × days_in_month × growth_factor × amplification_factor

3.1 Python Reference Implementation

from dataclasses import dataclass
from typing import List

@dataclass
class WorkloadForecast:
    name: str
    requests_per_day: float        # current daily call volume
    avg_input_tokens: float        # mean input tokens per call
    avg_output_tokens: float       # mean output tokens per call
    input_price_per_mtok: float    # $/million input tokens
    output_price_per_mtok: float   # $/million output tokens
    monthly_growth_rate: float     # e.g. 0.10 = 10% MoM growth
    agentic_amplification: float   # 1.0 for single-turn; 4–8 for agents


def forecast_monthly_cost(
    workload: WorkloadForecast,
    month_offset: int = 0,         # 0 = current month, 1 = next, etc.
    price_decay_annual: float = 0.30,  # 30% annual price reduction assumption
) -> dict:
    # Volume growth
    requests = workload.requests_per_day * (1 + workload.monthly_growth_rate) ** month_offset
    effective_input = workload.avg_input_tokens * workload.agentic_amplification
    effective_output = workload.avg_output_tokens * workload.agentic_amplification

    # Price decay applied proportionally
    monthly_decay = (1 - price_decay_annual) ** (month_offset / 12)
    input_price = workload.input_price_per_mtok * monthly_decay
    output_price = workload.output_price_per_mtok * monthly_decay

    days = 30
    input_cost = requests * effective_input * input_price * days / 1_000_000
    output_cost = requests * effective_output * output_price * days / 1_000_000

    return {
        "month": month_offset,
        "requests_per_day": round(requests, 1),
        "input_cost_usd": round(input_cost, 2),
        "output_cost_usd": round(output_cost, 2),
        "total_inference_cost_usd": round(input_cost + output_cost, 2),
        "effective_input_price_per_mtok": round(input_price, 4),
    }


def forecast_range(workload: WorkloadForecast, months: int = 12) -> List[dict]:
    return [forecast_monthly_cost(workload, m) for m in range(months)]


# Example: Claude Sonnet 4.5 document summarization pipeline
summarizer = WorkloadForecast(
    name="doc-summarizer-prod",
    requests_per_day=5_000,
    avg_input_tokens=3_200,
    avg_output_tokens=450,
    input_price_per_mtok=3.00,
    output_price_per_mtok=15.00,
    monthly_growth_rate=0.08,      # 8% MoM — post-launch acceleration
    agentic_amplification=1.0,     # single-turn, no agent overhead
)

for row in forecast_range(summarizer, months=6):
    print(row)

3.2 SQL for Athena (CUR-Based Actuals)

Use this query to extract token volume actuals from CUR for calibrating the bottom-up model:

SELECT
    DATE_TRUNC('week', line_item_usage_start_date) AS week,
    resource_tags_user_project                      AS project,
    line_item_usage_type,
    SUM(line_item_unblended_cost)                   AS cost_usd,
    -- Bedrock CUR splits InputTokens and OutputTokens in usage_amount
    SUM(CASE WHEN line_item_usage_type LIKE '%InputTokens%'
             THEN line_item_usage_amount ELSE 0 END) AS input_tokens,
    SUM(CASE WHEN line_item_usage_type LIKE '%OutputTokens%'
             THEN line_item_usage_amount ELSE 0 END) AS output_tokens,
    COUNT(*)                                         AS record_count
FROM
    aws_cost_and_usage_report
WHERE
    line_item_product_code = 'AmazonBedrock'
    AND line_item_usage_start_date >= DATE_ADD('month', -3, CURRENT_DATE)
GROUP BY 1, 2, 3
ORDER BY 1 DESC, cost_usd DESC;

Calibrate the avg_input_tokens and avg_output_tokens fields in the Python model from this weekly actuals table. Refit monthly.

4. Token Volume Growth Modeling

4.1 Observed Enterprise Growth Rates

Deployment Phase Typical Weekly Growth Notes
Pilot / proof-of-concept +5–15% WoW Small user base; high variance
Early production (0–3 months) +3–8% WoW Word-of-mouth expansion
Mid-scale (3–9 months) +1–3% WoW Org-wide rollout flattening
Mature (9+ months) +0–1% WoW Saturation; growth from new use cases only

Compound these: 5% WoW for 12 weeks = 80% volume increase. 3% WoW for 26 weeks = 114% increase.

4.2 S-Curve Modeling in Python

import numpy as np

def logistic_token_forecast(
    baseline_daily_tokens: float,
    carrying_capacity: float,       # maximum sustainable tokens/day
    growth_rate: float,             # logistic k parameter; ~0.15–0.30 for fast enterprise
    weeks: int = 52,
    inflection_week: int = 20,      # week at which growth is fastest
) -> np.ndarray:
    """
    Returns array of projected daily tokens for each week.
    Uses standard logistic: L / (1 + e^(-k(t - t0)))
    Anchored so week 0 = baseline_daily_tokens.
    """
    t = np.arange(weeks)
    raw = carrying_capacity / (1 + np.exp(-growth_rate * (t - inflection_week)))
    # Scale so raw[0] matches baseline
    scale = baseline_daily_tokens / raw[0]
    return raw * scale

4.3 Key Calibration Signals

  • Prompt length drift: Monitor avg_input_tokens weekly. If it grows >5% WoW, investigate: new RAG injection, longer system prompts, or context window expansion. This is a separate multiplier from user count growth.
  • Output:input ratio: Track weekly. A rising ratio signals more verbose model responses (e.g., chain-of-thought enabled, or migration to a reasoning model). A step-change here doubles output cost overnight.
  • Agentic creep: As developers add tool calls and multi-agent workflows, the amplification factor grows. Track total_tokens / user_visible_requests as the amplification metric.

5. Capacity Planning for Provisioned Throughput

5.1 How Provisioned Throughput Works

Provisioned Throughput (PT) allocates dedicated Model Units (MUs). Each MU provides a defined TPM (tokens per minute) budget for a specific model. PT pricing is hourly, billed 24/7 regardless of utilization.

PT is appropriate when:

  • On-demand quotas are insufficient for peak traffic
  • Latency SLAs require eliminating throttle-induced retries
  • Workload is predictable enough that idle PT waste < throttle-retry waste

5.2 Right-Sizing from CloudWatch Percentile Data

AWS CloudWatch emits EstimatedTPMQuotaUsage per model per minute at no additional cost (GA March 2026). This metric accounts for burndown multipliers (e.g., Claude Sonnet 4 output tokens burn at 5× vs. input) and cache write tokens.

Formula:

EstimatedTPMQuotaUsage = InputTokenCount
                       + CacheWriteInputTokens
                       + (OutputTokenCount × burndown_rate)

Right-sizing workflow:

  1. Collect 4 weeks of EstimatedTPMQuotaUsage at 1-minute resolution.
  2. Compute P50, P90, P95, P99 across all minutes.
  3. Set PT commitment at P90 of peak-hour TPM (typically the worst business-hours window).
  4. Allow on-demand overflow to absorb P90–P99 burst — this is significantly cheaper than buying PT to cover the top 10%.
import boto3
import numpy as np
from datetime import datetime, timedelta

cw = boto3.client('cloudwatch', region_name='us-east-1')

def get_tpm_percentiles(model_id: str, days: int = 28) -> dict:
    end = datetime.utcnow()
    start = end - timedelta(days=days)

    response = cw.get_metric_statistics(
        Namespace='AWS/Bedrock',
        MetricName='EstimatedTPMQuotaUsage',
        Dimensions=[{'Name': 'ModelId', 'Value': model_id}],
        StartTime=start,
        EndTime=end,
        Period=60,
        Statistics=['Maximum'],
    )

    values = [dp['Maximum'] for dp in response['Datapoints'] if dp['Maximum'] > 0]
    if not values:
        return {}

    arr = np.array(values)
    return {
        'p50':  np.percentile(arr, 50),
        'p90':  np.percentile(arr, 90),
        'p95':  np.percentile(arr, 95),
        'p99':  np.percentile(arr, 99),
        'max':  arr.max(),
        'recommended_pt_tpm': np.percentile(arr, 90),  # right-size target
    }

5.3 Decision Rule

P90 TPM P99 / P90 Ratio Recommendation
< 50K Any On-demand only; PT not cost-justified
50K–200K < 1.5× Buy PT at P90; let burst absorb the tail
50K–200K ≥ 1.5× Investigate bursty access pattern before committing
> 200K < 1.3× PT at P90; batch workloads eligible for 50% batch discount instead
> 200K ≥ 1.5× PT at P80; add retry with exponential backoff to smooth the tail

6. Reserved Service Tier Capacity Planning

6.1 What the Reserved Tier Is

AWS launched the Bedrock Reserved Service Tier in November 2025. Unlike Provisioned Throughput (which allocates a static MU count), the Reserved Tier is a commitment to a minimum TPM volume with overflow to Standard on-demand. Reserved customers pay a fixed monthly per-TPM rate. Overflow above the reserved level bills at Standard on-demand rates.

Minimum commitment: 100,000 TPM (1-month or 3-month terms available).

6.2 Break-Even Analysis

The Reserved Tier saves money only if utilization is high enough that the committed hourly spend is cheaper than equivalent on-demand. Define:

R  = reserved TPM purchased
r  = reserved price per 1K TPM-month   (e.g., $X from AWS pricing page)
P  = on-demand price per million tokens (varies by model)
U  = average hourly utilization of reserved capacity (fraction 0–1)

Monthly cost of Reserved Tier:

Cost_reserved = R / 1000 × r

Monthly cost of equivalent on-demand (at same average utilization):

tokens_per_month = R × U × 60 × 720   # TPM × utilization × minutes/hour × hours/month
Cost_on_demand = tokens_per_month / 1_000_000 × P

Break-even utilization:

U_breakeven = (R / 1000 × r) / (R × 60 × 720 / 1_000_000 × P)
            = (r × 1_000_000) / (1000 × 60 × 720 × P)
            = r × 1_000_000 / (43_200_000 × P)

def reserved_tier_breakeven(
    reserved_price_per_ktpm_month: float,  # $ per 1K TPM per month
    on_demand_price_per_mtok: float,       # $ per million tokens on-demand
) -> float:
    """Returns the utilization fraction at which Reserved = On-Demand cost."""
    minutes_per_month = 60 * 720  # 43,200
    breakeven = (reserved_price_per_ktpm_month * 1_000_000) / (
        1_000 * minutes_per_month * on_demand_price_per_mtok
    )
    return round(breakeven, 4)


# Illustrative: if Reserved = $2.00/1K TPM-month, On-Demand = $15.00/M output tokens
be = reserved_tier_breakeven(2.00, 15.00)
print(f"Break-even utilization: {be:.1%}")
# Interpretation: if average hourly utilization of reserved capacity exceeds this
# fraction, Reserved Tier is cheaper. Below it, on-demand wins.

6.3 Forecasting Whether 100K TPM Minimum Is Met

Before committing, confirm the workload actually reaches 100K TPM during normal business hours. Use the CloudWatch P50 from peak-hour analysis:

def will_meet_reserved_minimum(
    p50_peak_tpm: float,
    minimum_tpm: float = 100_000,
    safety_margin: float = 0.80,  # require P50 to be 80% of minimum
) -> dict:
    threshold = minimum_tpm * safety_margin
    meets = p50_peak_tpm >= threshold
    return {
        'p50_peak_tpm': p50_peak_tpm,
        'required_p50': threshold,
        'commit_recommended': meets,
        'utilization_at_p50': round(p50_peak_tpm / minimum_tpm, 2),
    }

If P50 peak-hour TPM is below 80K, the workload will spend much of the reserved commitment at sub-breakeven utilization. Wait until P50 exceeds 80K before committing.

7. FinOps Foundation AI Forecasting Methodology

The FinOps Foundation’s FinOps for AI Working Group has published three sequential papers relevant to Bedrock cost forecasting (available at finops.org/wg/):

  1. Cost Estimation of AI Workloads — baseline methodology targeting >90% expenditure accuracy; covers the bottom-up token model, agentic amplification factors, and adjacent infrastructure charges.

  2. How to Forecast AI — examples and techniques for the Quantify Business Value domain; introduces unit cost metrics (cost per inference, cost per successful outcome) and S-curve volume adjustment.

  3. Effect of Optimization on AI Forecasting (2025) — addresses how prompt compression, caching, and model downgrades break historical cost trends. Key finding: optimization events create step-change discontinuities that CE linear models cannot anticipate. The paper recommends “forecast segments” — splitting the time series at each major optimization event and fitting separate trend lines.

Key FinOps Foundation forecasting principles for AI:

  • Use unit economics (cost per outcome) as the primary trend metric, not raw spend
  • Track three separate growth signals: user count, tokens-per-user, price-per-token
  • Acceptable variance: ±20–25% at Crawl, ±15% at Walk, ±10–12% at Run maturity
  • 90%+ monthly forecast accuracy is the FinOps Run target; AI workloads in acceleration phase are held to Crawl standards (±25%) until 3+ months of stable history

8. AWS Budgets Forecasting and Alert Architecture

8.1 How AWS Budgets Forecasting Works

AWS Budgets uses ML to project end-of-month spend from mid-month actuals. It requires 35 days of billing history before enabling forecast alerts. The forecast model is refreshed daily, making it more responsive to recent spend changes than Cost Explorer’s longer-horizon model.

Budgets supports two alert types:

  • Actual threshold alerts — fire once per period when cumulative spend crosses a dollar or percentage threshold
  • Forecast alerts — fire when the ML projection estimates end-of-month spend will exceed a threshold, even if current spend has not yet hit it

8.2 Three-Tier Budget Architecture for Bedrock

Tier 1: Per-project budget (tagged by Bedrock Projects)
  - Alert at 70% actual: "heads up" notification to dev team
  - Alert at 90% actual: escalation to eng lead
  - Alert at 110% forecast: pre-emptive warning before overrun

Tier 2: Per-environment budget (prod vs. dev/staging)
  - dev/staging budget separate from prod to prevent dev experiments
    from inflating prod forecast signals
  - Alert at 100% actual for dev: enforce hard ceiling via IAM action

Tier 3: Organization-wide Bedrock budget
  - Alert at 80% forecast: CFO notification (SNS → email)
  - Budget Action at 95% actual: attach SCP to limit non-prod accounts

8.3 Lambda Action for Slack and Ticket Integration

import json, boto3, os

def lambda_handler(event, context):
    """Triggered by SNS from AWS Budgets threshold alert."""
    message = json.loads(event['Records'][0]['Sns']['Message'])

    budget_name   = message.get('budgetName', 'unknown')
    alert_type    = message.get('notificationType', 'ACTUAL')
    threshold_pct = message.get('thresholdExceeded', '?')

    slack_payload = {
        "text": f":money_with_wings: *Bedrock Budget Alert*",
        "blocks": [{
            "type": "section",
            "text": {
                "type": "mrkdwn",
                "text": (
                    f"*Budget:* `{budget_name}`\n"
                    f"*Type:* {alert_type}\n"
                    f"*Threshold:* {threshold_pct}% exceeded\n"
                    f"<https://console.aws.amazon.com/cost-management/home#/budgets|View in Cost Management>"
                )
            }
        }]
    }

    import urllib.request
    req = urllib.request.Request(
        os.environ['SLACK_WEBHOOK_URL'],
        data=json.dumps(slack_payload).encode(),
        headers={'Content-Type': 'application/json'},
        method='POST'
    )
    urllib.request.urlopen(req)
    return {'statusCode': 200}

9. Scenario Planning for Model Price Changes

9.1 Building Price-Drop Scenarios

Every multi-month Bedrock forecast should include three price scenarios:

Scenario Annual Price Decay Justification
Conservative 15% AWS absorbs price reductions slowly; enterprise contracts
Base 30% Historical frontier model deflation rate 2023–2026
Aggressive 50–60% Model commoditization accelerates; open-source parity

Gartner (March 2026): inference costs for 1T-parameter models will fall >90% by 2030, implying ~40% annualized compression over 5 years.

def price_scenario_comparison(
    monthly_base_cost: float,
    months: int = 24,
    scenarios: dict = None,
) -> dict:
    if scenarios is None:
        scenarios = {
            'conservative': 0.15,
            'base':         0.30,
            'aggressive':   0.55,
        }
    results = {}
    for name, annual_decay in scenarios.items():
        monthly_decay = (1 - annual_decay) ** (1/12)
        cumulative = sum(
            monthly_base_cost * monthly_decay ** m for m in range(months)
        )
        results[name] = {
            'annual_decay_pct': annual_decay * 100,
            'month_24_price_factor': round(monthly_decay ** months, 3),
            'cumulative_24_month_cost': round(cumulative, 0),
        }
    return results

9.2 Switching Cost Analysis

When a new cheaper model (e.g., Nova Pro → Nova Lite) ships at 60% lower cost, the decision is not purely arithmetic. Factor in:

Cost Typical Range
Prompt re-tuning engineering hours 40–120 hours per major use case
Regression testing token suite $50–$500 in Bedrock test inference
Quality validation / human review 20–60 hours
Rollout, monitoring, rollback plan 10–20 hours

Rule of thumb: switching break-even occurs within 2–4 months for workloads spending >$5K/month on inference. Below $1K/month, the engineering cost rarely pays back in year 1.

9.3 Hedging with Cross-Model Architecture

Architect applications to swap model IDs without code changes (parameter-driven model selection). This eliminates re-deployment cost and allows same-day price arbitrage when AWS drops a new model. The switching cost collapses from weeks of engineering to a single config update.

10. Forecast Accuracy Tracking

10.1 MAPE Targets for AI Spend

MAPE = mean(|actual - forecast| / actual) × 100

FinOps Maturity MAPE Target Context
Crawl (0–6 months AI FinOps) ≤ 25% Acceptable for growth-phase workloads
Walk (6–18 months) ≤ 15% Requires 3+ months of stable history per workload
Run (18+ months) ≤ 10–12% Mature, predictable workloads only

AI workloads are held to Crawl MAPE standards for the first 3 months post-launch, regardless of overall FinOps program maturity. Attempting to hold AI forecasts to 5% MAPE (appropriate for compute/storage) will create false failures and erode CFO trust in the forecasting function.

10.2 Monthly Forecast Audit Cadence

def compute_mape(actuals: list, forecasts: list) -> float:
    assert len(actuals) == len(forecasts)
    errors = [abs(a - f) / a for a, f in zip(actuals, forecasts) if a > 0]
    return round(sum(errors) / len(errors) * 100, 1)


# Track rolling 3-month MAPE by workload
monthly_actuals   = [12_400, 14_100, 16_800, 19_200]
monthly_forecasts = [11_000, 13_500, 15_500, 21_000]

mape = compute_mape(monthly_actuals[1:], monthly_forecasts[:-1])  # 1-month-ahead MAPE
print(f"1-month-ahead MAPE: {mape}%")

10.3 Why 20–30% MAPE Is Acceptable for AI vs. 5% for Compute

Traditional compute (EC2, RDS) has deterministic billing: hours × rate. Variance comes only from resource count changes, which FinOps teams control. For LLM workloads:

  • Output token counts are stochastic and user-prompt-dependent
  • Model price can change without notice
  • Agentic amplification varies with prompt effectiveness
  • Adoption growth curves have genuine uncertainty

The analogy is forecasting SaaS variable usage vs. forecasting electricity consumption from fixed hardware. A 20% variance on a well-modeled AI workload represents better signal than a 5% variance on a FinOps report that excluded agentic overhead.

11. QBR Artifacts for AI Cost

11.1 The Four Metrics a CFO Actually Acts On

  1. Cost per outcome — cost per successful completion (e.g., cost per document summarized, per support ticket resolved). This normalizes for both volume growth and price changes. Trend should be down-and-to-the-right.

  2. Inference efficiency ratio — billable inference tokens / total tokens consumed (including retries, cache writes, dropped calls). Target >85%. Below 70% indicates retry storms or bad caching.

  3. Unit cost vs. benchmark — your cost per 1K output tokens vs. the model’s published list price. Delta = waste from over-provisioned PT, sub-optimal prompts, or excessive max_tokens.

  4. Forecast vs. actual variance — monthly MAPE, trended. Shows the CFO whether the team’s models are getting more reliable or drifting. A declining MAPE is the proof of FinOps program maturity.

11.2 QBR Slide Structure

Slide 1: Executive Summary
  - Total AI spend this quarter: $X (±Y% vs. forecast)
  - Cost per [primary outcome] trend: [$A → $B → $C over 3 quarters]
  - Largest variance driver: [one sentence explanation]

Slide 2: Spend Decomposition
  - Waterfall: inference / knowledge base / training / governance / data transfer
  - Model-by-model breakdown (Bedrock Projects tags)
  - Environment split: prod / dev / staging

Slide 3: Volume vs. Price Decomposition
  - How much of the cost change was volume growth vs. price change?
  - Formula: Δcost = Δvolume × old_price + old_volume × Δprice + Δvolume × Δprice
  - This separates "we're getting more value" from "the model got cheaper"

Slide 4: Capacity Decisions
  - PT utilization by model: are reserved commitments earning out?
  - Reserved Tier utilization vs. break-even threshold
  - Recommendations: expand, contract, or hold

Slide 5: Next Quarter Forecast (3 scenarios)
  - Conservative / base / aggressive with explicit assumptions
  - Price scenario used (% annual decay assumption)
  - Growth rate used and justification

Slide 6: Cost Reduction Opportunities
  - Prompt compression potential (estimated token savings)
  - Batch API candidates (50% discount for non-latency-sensitive jobs)
  - Model downgrade candidates (quality/cost tradeoff analysis)

12. Multi-Year AI Cost Modeling (TCO)

12.1 Full Program Cost Stack

A complete enterprise AI TCO model has six layers, not one:

Layer Year 1 Share Year 3 Share (Estimated) Notes
Inference (Bedrock) 40–50% 35–45% Grows with usage; offset by price deflation
License / seat costs 15–20% 10–15% Microsoft 365 Copilot, GitHub Copilot, etc.
Infrastructure 10–15% 8–12% Vector DBs, knowledge bases, data pipelines
Training / fine-tuning 5–10% 3–8% Front-loaded; amortizes over model lifetime
Implementation & integration 20–25% 5–8% Heavy in Y1; drops to maintenance in Y3
Governance & compliance 3–8% 8–15% Grows as regulatory requirements mature

Forrester (2025) found M365 Copilot total implementation cost of $11.3M vs. $5.8M license cost — a 1.95× multiplier. Apply a similar 1.5–2× TCO multiplier on top of Bedrock inference-only estimates for enterprise programs.

12.2 Three-Year TCO Model

def three_year_tco(
    year1_inference_cost: float,
    year1_license_cost: float,
    year1_implementation_cost: float,
    volume_growth_annual: float = 0.60,   # 60% YoY inference volume growth
    price_decay_annual: float = 0.30,     # 30% annual price decline
    governance_growth_annual: float = 0.20,
) -> dict:
    results = {}
    for year in [1, 2, 3]:
        y = year - 1  # 0-indexed
        inference = year1_inference_cost * (
            (1 + volume_growth_annual) ** y * (1 - price_decay_annual) ** y
        )
        license     = year1_license_cost * 1.05 ** y           # 5% annual increase
        impl        = year1_implementation_cost * 0.25 ** y    # drops 75% after Y1
        governance  = year1_inference_cost * 0.05 * (1 + governance_growth_annual) ** y

        total = inference + license + impl + governance
        results[f'year_{year}'] = {
            'inference':   round(inference),
            'license':     round(license),
            'implementation': round(impl),
            'governance':  round(governance),
            'total':       round(total),
        }

    cumulative = sum(v['total'] for v in results.values())
    results['3yr_cumulative'] = round(cumulative)
    return results


# Example: $500K/year inference baseline
tco = three_year_tco(
    year1_inference_cost=500_000,
    year1_license_cost=200_000,
    year1_implementation_cost=400_000,
)
for k, v in tco.items():
    print(k, v)

12.3 The Price-Deflation / Volume-Growth Offset

The defining dynamic of AI TCO over a 3-year horizon: volume growth (60–150% annually) and price deflation (30–55% annually) run in opposite directions. In most enterprise scenarios:

  • Year 1: Volume growth wins — spend accelerates
  • Year 2: Near parity — rough plateau
  • Year 3: Price deflation begins to dominate for mature workloads; cost-per-outcome declines materially

This means AI TCO models that project flat or linear growth are almost always wrong in both directions: they overstate year-3 inference costs (missing price deflation) and understate year-1 costs (missing agentic amplification and adjacent charges).

12.4 CFO Framing: Unit Economics, Not Total Spend

The winning framing for multi-year AI investment proposals to CFOs:

“In year 1, we will spend $X to process Y outcomes. By year 3, the same $X will process 3–4× as many outcomes — a structural cost-per-unit decline. The investment is in building the pipeline. The economics improve automatically as models get cheaper and volume grows into existing infrastructure.”

This reframes AI spend from an uncontrolled variable cost into a depreciating unit cost — which is how CFOs evaluate any technology investment.

Implementation Checklist

  • [ ] Enable Bedrock Projects tagging for all workloads (required for per-project CUR attribution)
  • [ ] Deploy CloudWatch dashboard for EstimatedTPMQuotaUsage P50/P90/P99 per model
  • [ ] Build bottom-up forecast spreadsheet or Python model; refresh monthly from Athena CUR query
  • [ ] Set AWS Budgets at three tiers (project / environment / org) with 70/90/110% alert thresholds
  • [ ] Calibrate agentic amplification factor per workload (total tokens / user-visible requests)
  • [ ] Build price-scenario model with conservative / base / aggressive price decay assumptions
  • [ ] Track MAPE monthly; benchmark against FinOps maturity level targets
  • [ ] Adopt parameter-driven model ID selection to enable zero-cost model switching
  • [ ] Evaluate Reserved Tier only when CloudWatch P50 peak-hour TPM exceeds 80K
  • [ ] Include 1.5–2× TCO multiplier in all CFO-level investment proposals

Sources