Bedrock Runtime Token Budget Enforcement (2026)

Cost overruns on LLM inference follow a predictable pattern: a team ships an agent, the agent hits a runaway loop or an unexpectedly large document, and CUR shows the damage three days later. The gap between inference and billing visibility is the core problem. This note maps every enforcement layer available before that gap closes — from LiteLLM virtual-key budgets through AWS Service Quotas to IAM-level kill switches.

---

LiteLLM’s proxy server provides the most granular runtime budget controls available for Bedrock without custom code. Budgets attach to three entities: teams, keys (virtual API keys), and users.

Team and Key Budgets

# litellm_config.yaml
litellm_settings:
  default_team_settings:
    max_budget: 500.00          # USD, per team per budget_duration
    budget_duration: monthly    # options: daily | weekly | monthly | total

team_settings:
  - team_id: "engineering-prod"
    max_budget: 1000.00
    budget_duration: monthly
    soft_budget: 800.00         # triggers warning callback at 80%
    model_aliases:
      bedrock/anthropic.claude-sonnet-4-5: "claude-sonnet-prod"

  - team_id: "engineering-dev"
    max_budget: 100.00
    budget_duration: monthly
    model_aliases:
      bedrock/anthropic.claude-haiku-3-5: "claude-dev"    # cheaper model in dev

max_budget is in USD. LiteLLM tracks spend against this value using the model’s per-token cost, which it maintains in an internal pricing table. When spend exceeds max_budget, subsequent requests return HTTP 429 with body {"error": {"code": 429, "message": "Budget exceeded for team engineering-prod"}}.

Budget Duration Semantics

budget_duration	Reset behavior
daily	Resets at midnight UTC
weekly	Resets Monday 00:00 UTC
monthly	Resets 1st of month 00:00 UTC
total	Never resets; lifetime cap

total is the right choice for project-scoped agents that should burn exactly one fixed allocation and stop.

Soft vs Hard Limits

LiteLLM distinguishes soft_budget (warning, no enforcement) from max_budget (hard stop). At soft_budget, LiteLLM fires the budget_alerts callback. You can wire this to Slack or PagerDuty:

# callbacks in litellm_config.yaml
litellm_settings:
  callbacks: ["slack", "datadog"]
  alerting: ["slack"]
  alerting_threshold: 0.80     # fires at 80% of max_budget

At max_budget, the proxy returns 429 before forwarding the request to Bedrock — no tokens are consumed, no partial billing occurs.

Model Fallback on Budget Exhaustion

LiteLLM supports routing to a cheaper fallback model when a budget is nearly exhausted, rather than a hard 429:

litellm_settings:
  budget_routing:
    - condition: budget_percent_used >= 90
      team_id: "engineering-prod"
      fallback_model: "bedrock/anthropic.claude-haiku-3-5"

This implements graceful degradation: users continue working on Haiku instead of hitting a wall, while compute costs drop to roughly 1/10th of Sonnet pricing.

---

2. Amazon Bedrock Guardrails — maxTokens

Bedrock Guardrails provides a maxTokens constraint that operates inside the Bedrock service layer, independent of the model call’s max_tokens parameter. These are different mechanisms with different purposes.

maxTokens in Guardrails vs max_tokens on the Model Call

Parameter	Layer	What it controls	Enforcement
max_tokens on invoke_model	Model inference	Maximum output token count	Model stops generating at this count
maxTokens in Guardrails	Guardrails policy	Total input+output tokens subject to guardrail evaluation	Guardrail rejects requests exceeding this threshold before inference

The Guardrails maxTokens is primarily a safety/abuse control — it prevents extremely large prompts that would be expensive to evaluate against the guardrail policies themselves. It is not a cost budget in the accounting sense. Using it for cost governance is a secondary use that works but requires careful sizing.

Creating a Guardrail with Token Limits via AWS CLI

aws bedrock create-guardrail \
  --name "prod-token-ceiling" \
  --description "Hard token ceiling for production workloads" \
  --content-policy-config '{"filtersConfig": []}' \
  --contextual-grounding-policy-config '{"filtersConfig": []}' \
  --word-policy-config '{"managedWordListsConfig": []}' \
  --sensitive-information-policy-config '{"piiEntitiesConfig": []}' \
  --topic-policy-config '{"topicsConfig": []}' \
  --blocked-input-messaging "Request exceeds token budget. Please reduce input length." \
  --blocked-outputs-messaging "Response truncated — token budget exhausted." \
  --region us-east-1

The maxTokens parameter is set per-content-type within the guardrail configuration; check the current Bedrock SDK schema as this field is under active development in 2026.

Attaching a Guardrail to an Inference Call

import boto3

bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")

response = bedrock.invoke_model(
    modelId="anthropic.claude-sonnet-4-5-20241022-v2:0",
    guardrailIdentifier="prod-token-ceiling",
    guardrailVersion="1",
    body=json.dumps({
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 4096,
        "messages": [{"role": "user", "content": prompt}]
    })
)

If the guardrail fires, Bedrock returns a response with stopReason: "guardrail_intervened" and the blocked messaging string. The model was not invoked; no inference tokens were billed for input that never reached the model.

---

3. AWS Service Quotas — TPM/RPM as Hard Ceiling

Service Quotas are the ultimate enforcement ceiling. Every Bedrock model has per-region TPM (tokens per minute) and RPM (requests per minute) quotas. These enforce at the AWS account level, not the application level, but they are the backstop that prevents runaway spend from escaping even if application-level controls fail.

Viewing Current Quotas

# List all Bedrock quotas in a region
aws service-quotas list-service-quotas \
  --service-code bedrock \
  --region us-east-1 \
  --query "Quotas[*].{Name:QuotaName,Value:Value,Adjustable:Adjustable}" \
  --output table

# Get a specific quota (Claude Sonnet input TPM)
aws service-quotas get-service-quota \
  --service-code bedrock \
  --quota-code L-XXXXXXXX \
  --region us-east-1

Quota codes vary by model and region. Use the console or list-service-quotas to discover them; there is no stable cross-region mapping.

Requesting a Quota Increase

aws service-quotas request-service-quota-increase \
  --service-code bedrock \
  --quota-code L-XXXXXXXX \
  --desired-value 500000 \
  --region us-east-1

Increases for Bedrock TPM quotas typically resolve in 3–5 business days for accounts in good standing. For time-sensitive production launches, open a support case concurrently.

Monitoring EstimatedTPMQuotaUsage

Bedrock publishes a CloudWatch metric EstimatedTPMQuotaUsage under the AWS/Bedrock namespace. This is the key signal for proactive quota management:

aws cloudwatch get-metric-statistics \
  --namespace AWS/Bedrock \
  --metric-name EstimatedTPMQuotaUsage \
  --dimensions Name=ModelId,Value=anthropic.claude-sonnet-4-5-20241022-v2:0 \
  --start-time 2026-06-18T00:00:00Z \
  --end-time 2026-06-18T01:00:00Z \
  --period 60 \
  --statistics Maximum \
  --region us-east-1

Set a CloudWatch alarm at 80% of quota to get ahead of throttling. Bedrock throttle responses are HTTP 429 ThrottlingException, which clients must handle with exponential backoff.

---

4. Proactive AI Cost Management — Step Functions Pattern

The AWS ML Blog (2025) describes a Step Functions-based real-time rate limiter that enforces per-cost-center token budgets using CloudWatch custom metrics. The pattern avoids polling CUR (which has multi-day latency) by tracking token consumption in near-real-time.

Architecture

Request → API Gateway → Lambda (pre-check) → Step Functions → Bedrock
                              ↓
                    CloudWatch PutMetricData
                    (Namespace: BudgetEnforcement, Metric: TokensConsumed)
                    (Dimensions: CostCenter, Environment)

Lambda Pre-Check (simplified)

import boto3
import json

cloudwatch = boto3.client("cloudwatch")
bedrock = boto3.client("bedrock-runtime")

BUDGET_TOKENS_PER_HOUR = {
    "engineering": 500_000,
    "analytics": 200_000,
    "support": 100_000,
}

def lambda_handler(event, context):
    cost_center = event["cost_center"]
    prompt = event["prompt"]
    model_id = event["model_id"]

    # Check current consumption
    metrics = cloudwatch.get_metric_statistics(
        Namespace="BudgetEnforcement",
        MetricName="TokensConsumed",
        Dimensions=[{"Name": "CostCenter", "Value": cost_center}],
        StartTime=datetime.utcnow() - timedelta(hours=1),
        EndTime=datetime.utcnow(),
        Period=3600,
        Statistics=["Sum"],
    )

    consumed = metrics["Datapoints"][0]["Sum"] if metrics["Datapoints"] else 0
    budget = BUDGET_TOKENS_PER_HOUR.get(cost_center, 50_000)

    if consumed >= budget:
        return {"statusCode": 429, "body": "Hourly token budget exhausted"}

    # Call Bedrock
    response = bedrock.invoke_model(...)
    tokens_used = response["usage"]["input_tokens"] + response["usage"]["output_tokens"]

    # Record consumption
    cloudwatch.put_metric_data(
        Namespace="BudgetEnforcement",
        MetricData=[{
            "MetricName": "TokensConsumed",
            "Dimensions": [{"Name": "CostCenter", "Value": cost_center}],
            "Value": tokens_used,
            "Unit": "Count",
        }],
    )

    return {"statusCode": 200, "body": response}

Step Functions wraps this to handle retries, timeouts, and the concurrency edge case where two simultaneous requests both pass the pre-check before either records consumption. For high-concurrency workloads, replace the CloudWatch check with a DynamoDB atomic counter using conditional writes.

---

5. LiteLLM enable_pre_call_check

LiteLLM’s enable_pre_call_check validates the request against all budget and rate-limit rules before forwarding to Bedrock. Without this flag, LiteLLM may begin a streaming request and discover a budget violation only after tokens have started flowing.

litellm_settings:
  enable_pre_call_check: true

With this enabled:

1. LiteLLM estimates the cost of the request (input tokens × model rate)
2. Checks if estimated cost + current spend > max_budget
3. Returns 429 immediately if over limit — Bedrock is never called
4. No partial billing: rejected requests generate zero Bedrock charges

This is the single most important LiteLLM configuration for cost governance. It converts budget enforcement from reactive (measure after billing) to preventive (block before billing).

---

6. Application-Level Pre-Flight Token Counting

For teams not running LiteLLM, application-level pre-flight counting using the Anthropic count_tokens API prevents over-budget requests before they hit Bedrock.

import anthropic
import boto3

client = anthropic.Anthropic()  # for token counting only
bedrock = boto3.client("bedrock-runtime")

def count_tokens_preflight(messages: list, model: str = "claude-sonnet-4-5-20241022") -> int:
    """Count tokens without inference cost."""
    response = client.messages.count_tokens(
        model=model,
        messages=messages,
    )
    return response.input_tokens

def budget_aware_invoke(messages: list, session_id: str, budget_store: dict) -> dict:
    model = "anthropic.claude-sonnet-4-5-20241022-v2:0"
    
    # Pre-flight count
    estimated_input = count_tokens_preflight(messages)
    estimated_total = estimated_input + 4096  # worst-case output
    
    remaining = budget_store[session_id]["remaining_tokens"]
    
    if estimated_total > remaining:
        raise BudgetExceededError(
            f"Request needs ~{estimated_total} tokens; {remaining} remaining in session budget"
        )
    
    # Invoke
    response = bedrock.invoke_model(
        modelId=model,
        body=json.dumps({
            "anthropic_version": "bedrock-2023-05-31",
            "max_tokens": 4096,
            "messages": messages,
        })
    )
    
    body = json.loads(response["body"].read())
    actual_used = body["usage"]["input_tokens"] + body["usage"]["output_tokens"]
    
    # Deduct actual (not estimated)
    budget_store[session_id]["remaining_tokens"] -= actual_used
    budget_store[session_id]["consumed_tokens"] += actual_used
    
    return body

The count_tokens API call is priced but negligible (~0.001x inference cost). For high-volume applications, cache token counts for identical prompts.

---

7. Budget Alert Tiers

A three-tier alert structure covers the operational range without alarm fatigue:

Tier	Threshold	Action
Soft warning	80% of budget consumed	Slack alert to team; log to CloudWatch; continue serving
Hard warning	95% of budget consumed	Page on-call; switch to cheaper fallback model
Hard stop	100% of budget consumed	Return 429; queue request for next budget period if applicable
Overage buffer	110% (10% grace)	LiteLLM soft_budget_overage — allows in-flight requests to complete

The 10% overage buffer matters for streaming responses and multi-step agent calls that cross the budget boundary mid-execution. Without it, you get partial responses returned to users, which are worse than clean errors.

litellm_settings:
  max_budget: 1000.00
  soft_budget: 800.00
  budget_overage: 100.00      # allows up to $1100 before hard stop

---

8. Multi-Turn Conversation Budget Tracking

Single-request budgets miss the cost accumulation pattern of conversation: each turn re-sends the full prior context. A 20-turn conversation with 1K tokens per turn sends 20K tokens on turn 20 alone — 10x the apparent size.

Session-Level Budget Tracking

from dataclasses import dataclass, field
from typing import Optional

@dataclass
class ConversationBudget:
    session_id: str
    total_token_budget: int          # session lifetime limit
    per_request_token_limit: int     # single request ceiling
    consumed_input: int = 0
    consumed_output: int = 0
    turn_count: int = 0
    
    @property
    def total_consumed(self) -> int:
        return self.consumed_input + self.consumed_output
    
    @property
    def remaining(self) -> int:
        return self.total_token_budget - self.total_consumed
    
    @property
    def budget_percent(self) -> float:
        return self.total_consumed / self.total_token_budget
    
    def check_and_record(self, input_tokens: int, output_tokens: int) -> None:
        if input_tokens > self.per_request_token_limit:
            raise RequestTooLargeError(f"Request {input_tokens} tokens exceeds per-request limit {self.per_request_token_limit}")
        if self.total_consumed + input_tokens + output_tokens > self.total_token_budget:
            raise SessionBudgetExceededError(f"Session {self.session_id} budget exhausted")
        self.consumed_input += input_tokens
        self.consumed_output += output_tokens
        self.turn_count += 1

Store ConversationBudget in Redis or DynamoDB keyed by session_id with a TTL matching the session timeout. For multi-tenant SaaS, nest session budgets within per-customer monthly budgets.

Context Window Growth Warning

At 70% of session budget, emit a user-facing warning: “This conversation is using significant context. Starting a new conversation will give you a fresh token budget.” This is both a UX improvement and a cost control.

---

9. Agent Loop Runaway Detection — Circuit Breakers

Agent loops (tool call → model → tool call → …) are the primary runaway cost vector. An agent asked to “research all competitors” can chain 50+ tool calls before human review.

Circuit Breaker Pattern

class AgentCircuitBreaker:
    def __init__(
        self,
        max_turns: int = 20,
        max_tokens_per_session: int = 100_000,
        max_consecutive_tool_calls: int = 10,
        max_identical_tool_calls: int = 3,
    ):
        self.max_turns = max_turns
        self.max_tokens = max_tokens_per_session
        self.max_consecutive = max_consecutive_tool_calls
        self.max_identical = max_identical_tool_calls
        
        self._turn_count = 0
        self._tokens_consumed = 0
        self._consecutive_tool_calls = 0
        self._tool_call_hashes: dict[str, int] = {}
    
    def check(self, turn_tokens: int, tool_call: Optional[dict] = None) -> None:
        self._turn_count += 1
        self._tokens_consumed += turn_tokens
        
        if self._turn_count > self.max_turns:
            raise CircuitBreakerTripped("max_turns exceeded")
        if self._tokens_consumed > self.max_tokens:
            raise CircuitBreakerTripped("token budget exceeded")
        
        if tool_call:
            self._consecutive_tool_calls += 1
            call_hash = hash(json.dumps(tool_call, sort_keys=True))
            self._tool_call_hashes[call_hash] = self._tool_call_hashes.get(call_hash, 0) + 1
            
            if self._consecutive_tool_calls > self.max_consecutive:
                raise CircuitBreakerTripped("too many consecutive tool calls — possible loop")
            if self._tool_call_hashes[call_hash] > self.max_identical:
                raise CircuitBreakerTripped(f"tool call repeated {self.max_identical}+ times — loop detected")
        else:
            self._consecutive_tool_calls = 0  # model text response resets consecutive count

Identical tool call detection (same function name + same arguments hash) is the strongest loop signal. A legitimate agent rarely needs to call the same tool with identical arguments more than twice.

---

10. Emergency Budget Controls — Kill Switches

AWS Budget Actions (IAM Policy Attachment)

AWS Budget Actions can automatically attach a restrictive IAM policy when a cost threshold is crossed. This is the last-resort kill switch — it operates at the AWS account level, not the application level.

# 1. Create a deny-all Bedrock policy
aws iam create-policy \
  --policy-name BedrockEmergencyStop \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Deny",
      "Action": ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream"],
      "Resource": "*"
    }]
  }'

# 2. Create the budget with an action to attach that policy
aws budgets create-budget-action \
  --account-id 123456789012 \
  --budget-name "bedrock-monthly-ceiling" \
  --notification-type ACTUAL \
  --action-type IAM_POLICY \
  --action-threshold '{"ActionThresholdValue": 5000.0, "ActionThresholdType": "ABSOLUTE_VALUE"}' \
  --definition '{
    "IamActionDefinition": {
      "PolicyArn": "arn:aws:iam::123456789012:policy/BedrockEmergencyStop",
      "Roles": ["arn:aws:iam::123456789012:role/BedrockAppRole"]
    }
  }' \
  --approval-model AUTOMATIC \
  --execution-role-arn "arn:aws:iam::123456789012:role/BudgetActionsRole"

When the monthly Bedrock spend hits $5,000, AWS automatically attaches BedrockEmergencyStop to BedrockAppRole. All subsequent calls fail with AccessDeniedException. A human must manually detach the policy to restore access — there is no auto-resume.

Application-Level Feature Flag Kill Switch

For faster response than IAM policy propagation (which can take 60–90 seconds), maintain a feature flag in Parameter Store:

import boto3
import functools
import time

ssm = boto3.client("ssm")
_cache = {"value": True, "ts": 0}

def bedrock_enabled() -> bool:
    now = time.time()
    if now - _cache["ts"] > 30:  # refresh every 30 seconds
        try:
            r = ssm.get_parameter(Name="/stateofai/bedrock/enabled")
            _cache["value"] = r["Parameter"]["Value"] == "true"
            _cache["ts"] = now
        except Exception:
            pass  # keep last known value on SSM failure
    return _cache["value"]

# Emergency disable: aws ssm put-parameter --name /stateofai/bedrock/enabled --value false --overwrite

This propagates in under 30 seconds — fast enough to stop a runaway agent mid-session.

---

11. Multi-Tenant SaaS Budget Isolation

For SaaS platforms where each customer has their own token allocation, LiteLLM virtual keys are the correct abstraction.

Per-Customer Virtual Keys

import requests

LITELLM_PROXY = "https://litellm.internal"
LITELLM_MASTER_KEY = os.environ["LITELLM_MASTER_KEY"]

def provision_customer(customer_id: str, monthly_budget_usd: float) -> str:
    """Create a virtual key scoped to one customer with monthly budget."""
    response = requests.post(
        f"{LITELLM_PROXY}/key/generate",
        headers={"Authorization": f"Bearer {LITELLM_MASTER_KEY}"},
        json={
            "team_id": f"customer-{customer_id}",
            "max_budget": monthly_budget_usd,
            "budget_duration": "monthly",
            "metadata": {"customer_id": customer_id, "plan": "growth"},
            "models": ["bedrock/anthropic.claude-sonnet-4-5", "bedrock/anthropic.claude-haiku-3-5"],
            "soft_budget": monthly_budget_usd * 0.80,
        },
    )
    return response.json()["key"]

Overage Billing vs Hard Cutoff

Two enforcement strategies for SaaS:

Hard cutoff (prepaid credits model): Set max_budget to the customer’s credit balance. When 429 fires, prompt the user to purchase more credits. Good for developer tools where customers expect metered billing.

Overage billing (subscription model): Set max_budget to 110% of the plan limit. When exceeded, record the overage in your billing system and charge at next invoice. Use LiteLLM webhooks to trigger the billing record:

litellm_settings:
  callbacks: ["custom_callback_api"]

custom_callback_api:
  url: "https://api.yourapp.com/litellm/webhook"
  headers:
    Authorization: "Bearer your-webhook-secret"

The webhook fires on budget_exceeded events with the team ID, total spend, and overage amount.

Spend Retrieval for Customer Invoicing

# Get spend per team for current month
curl -X GET "https://litellm.internal/team/list" \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" | \
  jq '.[] | {team_id, spend, max_budget, budget_reset_at}'

---

12. Testing Budget Enforcement in Staging

Verifying that limits fire correctly requires triggering enforcement without real Bedrock calls or real budget consumption.

LiteLLM Mock Mode

# staging litellm_config.yaml
litellm_settings:
  mock_response: "This is a mock response for testing"
  mock_response_tokens: 500    # simulates 500 output tokens per call
  enable_pre_call_check: true
  
team_settings:
  - team_id: "test-budget-enforcement"
    max_budget: 0.10           # $0.10 limit — exhausted after ~20 mock calls
    budget_duration: total

With mock_response set, LiteLLM returns the stub response and records token costs using the model’s pricing table, but never calls Bedrock. You can exhaust the $0.10 budget in seconds and verify 429 behavior.

Unit Testing Circuit Breakers

def test_circuit_breaker_identical_calls():
    cb = AgentCircuitBreaker(max_identical_tool_calls=3)
    tool_call = {"name": "web_search", "input": {"query": "competitor pricing"}}
    
    cb.check(100, tool_call)
    cb.check(100, tool_call)
    cb.check(100, tool_call)
    
    with pytest.raises(CircuitBreakerTripped, match="loop detected"):
        cb.check(100, tool_call)

def test_circuit_breaker_token_limit():
    cb = AgentCircuitBreaker(max_tokens_per_session=1000)
    cb.check(400)
    cb.check(400)
    
    with pytest.raises(CircuitBreakerTripped, match="token budget exceeded"):
        cb.check(300)  # 400+400+300 = 1100 > 1000

Integration Test Against LiteLLM Staging

def test_budget_429_fires():
    """Verify hard stop returns 429 and not a degraded response."""
    # Exhaust the test team's budget
    client = openai.OpenAI(
        base_url="https://litellm-staging.internal/v1",
        api_key=TEST_TEAM_KEY,
    )
    
    for _ in range(25):  # exhaust $0.10 budget
        try:
            client.chat.completions.create(
                model="claude-sonnet-prod",
                messages=[{"role": "user", "content": "Hello"}]
            )
        except openai.RateLimitError:
            break
    
    # Next call must return 429
    with pytest.raises(openai.RateLimitError) as exc_info:
        client.chat.completions.create(
            model="claude-sonnet-prod",
            messages=[{"role": "user", "content": "Should be blocked"}]
        )
    
    assert "Budget exceeded" in str(exc_info.value)

Reset the test team’s budget between runs:

curl -X POST "https://litellm-staging.internal/team/reset_budget" \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -d '{"team_id": "test-budget-enforcement"}'

---

Enforcement Layer Summary

Layer	Granularity	Latency	Best for
LiteLLM max_budget + enable_pre_call_check	Team / key / user	<1ms (in-process)	Application-level cost governance
Bedrock Guardrails maxTokens	Per request	~10ms (service-side)	Input size abuse prevention
Step Functions / CloudWatch custom metrics	Cost center / environment	~100ms (async check)	Cross-application budget aggregation
AWS Service Quotas	Account / region	~60s (quota enforcement)	Hard ceiling preventing account-level blowout
AWS Budget Actions (IAM)	Account / role	~60–90s (policy propagation)	Emergency kill switch, post-incident containment
SSM Parameter Store feature flag	Application	~30s (cache TTL)	Fastest application-level emergency stop

Deploy them in order of granularity: LiteLLM first (catches 99% of cases), Service Quota alarms second (catches infrastructure-level runaway), Budget Actions last (break-glass only). Running all layers simultaneously adds defense in depth without meaningful latency overhead on the hot path.