LiteLLM Virtual Key and Budget Governance — Enterprise Patterns (2026)

LiteLLM’s proxy gateway implements a layered governance model that addresses the primary challenge of enterprise AI deployments: giving teams autonomy to consume LLM APIs while preventing any single team, application, or end-user from crowding out others or incurring runaway spend. The control plane is entirely REST-driven, backed by PostgreSQL, and optionally fronted by Redis for high-throughput enforcement.

This note covers the complete governance stack: how budgets cascade through four organizational levels, how keys are provisioned and expired, how hard and soft enforcement differ operationally, and what the auth-cache TTL tradeoff means for cost control fidelity under load.

---

1. Four-Level Budget Hierarchy: Organization → Team → Key → User

LiteLLM enforces budgets at every layer of a four-level hierarchy. Requests are checked against all applicable layers on every call; a block at any single layer rejects the request.

Organization (enterprise, ceiling for all child teams)
└── Team (department or product group, aggregate of all member keys)
    └── Virtual Key (application or service account credential)
        └── User (per-end-user attribution within a key's request body)

Layer semantics

Organization is an enterprise-tier feature. It provides tenant isolation for large deployments where separate business units (e.g., Finance, Engineering, Marketing) need independently enforced spending envelopes. An org admin can create teams, assign budgets, and restrict model access within their org boundary — but cannot see or affect other organizations. The proxy admin retains cross-org visibility.

Team is available in the open source tier. A team represents a product squad or cost center. All virtual keys belonging to a team contribute to the team’s aggregate spend. Teams support:

• max_budget: hard dollar ceiling
• soft_budget: advisory threshold that triggers email alerts without blocking
• budget_duration: rolling reset window
• tpm_limit / rpm_limit: rate caps applied to all team keys combined
• model_max_budget: per-model dollar sub-limits within the team

Virtual Key is the token a service presents with each API call. Keys inherit team constraints and can impose additional restrictions on top of them — keys can only be more restrictive than the team they belong to, never more permissive. A key can be associated with both a user (user_id) and a team simultaneously, enabling individual accountability within shared team budgets.

User splits into two distinct concepts that operators frequently conflate:

• Internal user: the LiteLLM-registered developer or service account who owns the key. Tracked via /user/new and /user/info.
• End user: the downstream customer or session identity passed as the user parameter in a /chat/completions request body. Tracked in LiteLLM_SpendLogs.end_user. Budget enforcement at this level requires max_end_user_budget configured in general_settings.

Budget cascade rules

1. Organization budget sets an absolute ceiling. Teams cannot be created or updated with budgets that would exceed the org limit.
2. Team budget is an aggregate: spend by any key in the team counts against the team total.
3. Key budget is additive to team tracking: spend on a key increments both LiteLLM_VerificationTokenTable.spend (the key row) and LiteLLM_TeamTable.spend (the team row) atomically.
4. If any layer — key, team, or org — is exhausted, the request is rejected with HTTP 429 and a BudgetExceededError payload naming the level that blocked it.

Cascade configuration example

# config.yaml — organization and team defaults
general_settings:
  master_key: "os.environ/LITELLM_MASTER_KEY"
  database_url: "os.environ/DATABASE_URL"

litellm_settings:
  # Default parameters applied to every /key/generate call
  # unless the caller explicitly overrides them
  default_key_generate_params:
    max_budget: 5.00          # $5 per key by default
    budget_duration: "30d"    # resets monthly
    models: ["gpt-4o", "claude-3-5-sonnet"]
    team_id: "os.environ/DEFAULT_TEAM_ID"

  # Hard ceilings — no /key/generate request may exceed these
  upperbound_key_generate_params:
    max_budget: 100.00        # no key may have > $100/period budget
    budget_duration: "30d"    # no key may have a reset window > 30 days
    duration: "365d"          # key expiry cannot exceed 1 year
    tpm_limit: 500000         # max 500K TPM per key
    rpm_limit: 2000           # max 2K RPM per key
    max_parallel_requests: 200

# Create org (enterprise endpoint)
curl -X POST http://localhost:4000/organization/new \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "organization_alias": "finance-dept",
    "max_budget": 10000.00,
    "budget_duration": "30d",
    "models": ["gpt-4o", "claude-3-5-sonnet", "bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0"]
  }'

# Create team within org
curl -X POST http://localhost:4000/team/new \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "team_alias": "quant-research",
    "organization_id": "org-finance-dept",
    "max_budget": 2000.00,
    "soft_budget": 1500.00,
    "budget_duration": "30d",
    "tpm_limit": 2000000,
    "rpm_limit": 5000,
    "models": ["gpt-4o", "claude-3-5-sonnet"],
    "metadata": {"alert_emails": "quant-lead@corp.com,finops@corp.com"}
  }'

---

2. Virtual Key Lifecycle: Creation, Rotation, Revocation, Audit

Creation

Keys are created via POST /key/generate. The proxy generates a secrets.token_urlsafe() string, hashes it with SHA-256, and stores the hash as the primary key in LiteLLM_VerificationTokenTable. Only the plaintext is returned to the caller — it is never stored or recoverable from the database.

curl -X POST http://localhost:4000/key/generate \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "user_id": "svc-quant-backtester",
    "team_id": "quant-research",
    "key_alias": "backtester-prod-v2",
    "models": ["gpt-4o"],
    "max_budget": 50.00,
    "budget_duration": "30d",
    "tpm_limit": 200000,
    "rpm_limit": 500,
    "duration": "365d",
    "tags": ["team:quant", "env:prod", "app:backtester"],
    "metadata": {
      "created_by": "ci-cd-pipeline",
      "jira_ticket": "QUANT-441",
      "rotation_policy": "90d"
    }
  }'

Custom key strings: if allow_user_auth is enabled in proxy config, callers may supply a key field to use their own string. Disabled by default — almost never appropriate in enterprise deployments.

Custom validation hook: implement custom_generate_key_fn to enforce business rules before key creation (e.g., reject keys not tied to an approved team ID):

# custom_key_fn.py
from litellm.proxy._types import GenerateKeyRequest

async def custom_generate_key_fn(data: GenerateKeyRequest) -> dict:
    approved_teams = {"quant-research", "ml-platform", "data-science"}
    if data.team_id not in approved_teams:
        return {
            "decision": False,
            "message": f"Team '{data.team_id}' is not in the approved team list. "
                       f"Submit a request to finops@corp.com to onboard your team."
        }
    return {"decision": True}

litellm_settings:
  custom_key_generate: custom_key_fn.custom_generate_key_fn

Rotation

Two rotation modes exist: manual and automated.

Manual rotation (available in all tiers): POST /key/{key}/regenerate. Creates a new token while preserving all existing metadata, spend tracking, and rate limits. The old token is invalidated immediately unless a grace period is specified.

curl -X POST http://localhost:4000/key/sk-abc123/regenerate \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "grace_period": "48h",
    "metadata": {"rotated_by": "ci-pipeline", "rotation_reason": "quarterly-cycle"}
  }'

Automated rotation (enterprise): configure auto_rotate: true on key creation. The proxy background scheduler handles regeneration on the specified interval. Grace periods allow zero-downtime cutover: both old and new tokens remain valid during the overlap window.

# Enable auto-rotation on an existing key
curl -X POST http://localhost:4000/key/update \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "key": "sk-abc123",
    "auto_rotate": true,
    "rotation_interval": "90d"
  }'

# Environment-level rotation settings
export LITELLM_KEY_ROTATION_ENABLED=true
export LITELLM_KEY_ROTATION_CHECK_INTERVAL_SECONDS=3600
export LITELLM_KEY_ROTATION_GRACE_PERIOD=48h

Revocation and blocking

LiteLLM uses soft delete for key revocation. DELETE /key/delete moves the record to a separate deleted table rather than removing it, preserving the audit trail. Blocking is non-destructive and reversible:

# Immediate block (key still exists, requests rejected)
curl -X POST http://localhost:4000/key/block \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"key": "sk-abc123"}'

# Restore
curl -X POST http://localhost:4000/key/unblock \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"key": "sk-abc123"}'

# Hard delete (soft delete — preserves audit trail)
curl -X DELETE http://localhost:4000/key/delete \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"keys": ["sk-abc123", "sk-def456"]}'

CI/CD key provisioning pattern

The preferred pattern for ephemeral service keys in CI/CD pipelines is to provision at deploy time, scope tightly, and expire automatically:

# provision_service_key.py — runs in CI/CD pipeline on each deploy
import httpx
import os
import json

LITELLM_BASE_URL = os.environ["LITELLM_BASE_URL"]
LITELLM_MASTER_KEY = os.environ["LITELLM_MASTER_KEY"]

def provision_key(
    service_name: str,
    team_id: str,
    environment: str,
    max_budget_usd: float = 10.0,
    duration_days: int = 7,
) -> str:
    """Provision a short-lived service key for a CI/CD deployment."""
    payload = {
        "key_alias": f"{service_name}-{environment}-{os.environ.get('GIT_SHA', 'unknown')[:8]}",
        "team_id": team_id,
        "user_id": f"ci-{service_name}",
        "max_budget": max_budget_usd,
        "budget_duration": f"{duration_days}d",
        "duration": f"{duration_days}d",   # hard expiry
        "models": ["gpt-4o-mini"],         # cheapest model only for CI
        "tpm_limit": 50000,
        "rpm_limit": 100,
        "tags": [f"env:{environment}", f"service:{service_name}", "source:ci-cd"],
        "metadata": {
            "git_sha": os.environ.get("GIT_SHA", "unknown"),
            "pipeline_run": os.environ.get("CI_PIPELINE_ID", "unknown"),
            "auto_provisioned": True,
        },
    }

    resp = httpx.post(
        f"{LITELLM_BASE_URL}/key/generate",
        headers={"Authorization": f"Bearer {LITELLM_MASTER_KEY}"},
        json=payload,
        timeout=30,
    )
    resp.raise_for_status()
    return resp.json()["key"]


if __name__ == "__main__":
    key = provision_key(
        service_name="integration-tests",
        team_id="ml-platform",
        environment="ci",
    )
    # Write to a temp file consumed by the test runner; never commit
    with open("/tmp/litellm_ci_key", "w") as f:
        f.write(key)
    print("Provisioned CI key — expires in 7 days")

---

3. Budget Types: Hard vs Soft

Hard budget (max_budget)

The default enforcement mode. When a key, user, team, or org reaches its max_budget, LiteLLM rejects further requests with:

{
  "error": {
    "message": "Budget has been exceeded! Current cost: 50.023, Max budget: 50.0",
    "type": "budget_exceeded",
    "code": "429"
  }
}

Hard budgets are evaluated in auth_checks.py at request authentication time. The check is: current_spend >= max_budget → reject.

When to use: production service accounts, any key issued to automated pipelines, team-level ceilings for cost center accountability.

Soft budget (soft_budget)

An advisory threshold that triggers email notifications without blocking requests. Configured as a dollar value below the hard max_budget. When current team spend crosses soft_budget, the proxy:

1. Checks whether alerting emails are configured in the team’s metadata
2. Deduplicates: sends at most one alert per 24-hour window per team
3. Delivers via the configured email integration (SendGrid, Resend, or SMTP)

# Configure soft budget via API
curl -X POST http://localhost:4000/team/update \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "team_id": "quant-research",
    "soft_budget": 1500.00,
    "max_budget": 2000.00,
    "metadata": {
      "alert_emails": "quant-lead@corp.com,finops@corp.com"
    }
  }'

Soft budget alerts require no proxy restart. The evaluation fires on each request after soft_budget is set.

When to use: early-warning signal for teams doing exploratory work where interruption is costly. Also useful for end-of-month reporting workflows where finance needs a 75%-spend notification before hitting the hard ceiling.

Temporary budget increases

For planned high-traffic events (model evaluations, batch jobs), rather than raising max_budget permanently:

curl -X POST http://localhost:4000/key/update \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "key": "sk-abc123",
    "temp_budget_increase": 200.00,
    "temp_budget_expiry": "3d"
  }'

The temporary increase expires automatically. The original max_budget is restored at expiry without operator intervention.

Decision tree: hard vs soft

Is request interruption acceptable if budget is hit?
├── Yes → Hard budget only (max_budget)
│         Use when: production services, automated pipelines,
│                   regulated cost centers
└── No  → Soft budget for early warning + hard budget as ceiling
          Use when: research teams, exploratory prototyping,
                    shared sandbox environments

Does the team have an operations contact who can act on alerts?
├── Yes → Soft budget = 75-80% of hard budget ceiling
└── No  → Hard budget only (soft alerts without an owner are noise)

---

4. Budget Time Windows: Periods, Reset Behavior, Carry-Forward

Supported budget_duration values

Value	Resets at
"30s", "1m", "1h"	After the interval from key creation
"24h" or "1d"	Midnight UTC daily
"7d"	Sunday midnight UTC weekly
"30d" or "1mo"	First of the month, midnight UTC
"1y"	Annual, January 1 midnight UTC
null (omitted)	Never resets — lifetime budget

The ResetBudgetJob background task scans all keys, users, teams, and org-level budget entries for expired windows. Default scan interval is every 10 minutes (configurable via proxy_budget_rescheduler_min_time / proxy_budget_rescheduler_max_time).

After a database reset, the job invalidates corresponding Redis and in-memory cache counters immediately, ensuring the reset takes effect within the next request cycle.

Carry-forward semantics

LiteLLM does not carry forward unused budget to the next period. Budgets reset to zero at the start of each window. There is no rollover mechanism. If a team spends $800 of a $2000 monthly budget, the remaining $1200 does not accrue — the new period starts at $0 spend against the full $2000 ceiling.

This is intentional for enterprise cost predictability: carry-forward creates compounding variability in monthly cost projections. If your use case requires rollover (e.g., a team that needs to bank budget for quarterly model evaluations), model it at the organizational level by adjusting the team’s max_budget programmatically at period boundaries, not via the LiteLLM budget mechanism.

Multi-window budgets

A single key can carry concurrent budget constraints at different granularities. This is useful for keeping daily variance low while also enforcing a monthly ceiling:

# Not directly supported via a single API call —
# model via key-level daily + team-level monthly
curl -X POST http://localhost:4000/key/generate \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "key_alias": "analytics-daily-capped",
    "team_id": "data-science",
    "max_budget": 20.00,
    "budget_duration": "1d",
    "tpm_limit": 100000,
    "models": ["gpt-4o-mini"]
  }'
# Team itself carries the monthly ceiling:
# team max_budget: 300.00, budget_duration: "30d"

---

5. RPM/TPM Rate Limits: Per-Model vs Global, Bedrock Interaction

Global vs per-model limits

Global limits apply to all traffic through a key or team regardless of which model is called:

{
  "tpm_limit": 500000,
  "rpm_limit": 2000
}

Per-model limits (model_max_budget, model_rpm_limit, model_tpm_limit) constrain traffic to a specific model group within the key or team. This is the primary mechanism for protecting expensive model quotas without affecting cheaper model throughput:

curl -X POST http://localhost:4000/key/generate \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "key_alias": "mixed-tier-service",
    "team_id": "ml-platform",
    "max_budget": 500.00,
    "budget_duration": "30d",
    "tpm_limit": 2000000,       # global TPM ceiling
    "rpm_limit": 5000,           # global RPM ceiling
    "model_max_budget": {
      "gpt-4o": 200.00,          # hard sub-limit on expensive model
      "claude-3-5-sonnet": 150.00
    },
    "model_rpm_limit": {
      "gpt-4o": 100,             # protect expensive quota
      "gpt-4o-mini": 2000        # generous limit on cheap model
    },
    "model_tpm_limit": {
      "gpt-4o": 50000,
      "gpt-4o-mini": 500000
    }
  }'

Limit resolution order

When a request arrives, LiteLLM resolves applicable limits in this precedence order:

1. Key model_rpm_limit / model_tpm_limit for the specific model
2. Key model_max_budget for the specific model
3. Key global rpm_limit / tpm_limit
4. Team model_rpm_limit / model_tpm_limit
5. Team global rpm_limit / tpm_limit
6. Org-level limits

Keys can only add restrictions, not relax team or org-level constraints. If the team has gpt-4o capped at 100 RPM, a key in that team cannot set model_rpm_limit["gpt-4o"]: 500.

AWS Bedrock quota interaction

Bedrock service quotas are enforced independently by AWS (tokens per minute per region per model). LiteLLM’s TPM counters and Bedrock’s quota system both fire, with no coordination between them.

Key operational consideration: Bedrock excludes cached tokens from its TPM quota calculation. LiteLLM historically included cached tokens in its own TPM tracking, causing LiteLLM’s counter to read higher than Bedrock’s. This was corrected in a patch that subtracts cache_read_input_tokens from the TPM counter for Bedrock routes. Verify this fix is present before relying on TPM limits for Bedrock budget control in any LiteLLM version prior to late 2025.

Recommended pattern for Bedrock deployments: set LiteLLM TPM limits at 70-80% of the Bedrock service quota to provide a safety buffer against the counting discrepancy and transient quota bursts from other AWS consumers.

# Provider-level budget routing for Bedrock
litellm_settings:
  provider_budget_config:
    bedrock:
      max_budget: 5000.00
      budget_duration: "1d"
    anthropic:
      max_budget: 2000.00
      budget_duration: "1d"

When a provider exceeds its budget window, LiteLLM automatically routes subsequent requests to alternative providers in the model group — functioning as cost-aware failover.

---

6. upperbound_key_generate_params: Preventing Unlimited Self-Provisioning

This is the primary governance control for self-serve key flows. Without it, any authenticated user with internal_user role can call /key/generate and create keys with arbitrary budgets, durations, and rate limits.

upperbound_key_generate_params sets hard maximums that the _enforce_upperbound_key_params() function checks before any key is written to the database. Requests exceeding these bounds are rejected with a validation error — they never reach the key creation logic.

litellm_settings:
  upperbound_key_generate_params:
    max_budget: 50.00           # No self-provisioned key may exceed $50/period
    budget_duration: "30d"      # Cannot set budget window > 30 days
    duration: "90d"             # Key expiry cannot exceed 90 days
    max_parallel_requests: 100  # Hard concurrency cap
    tpm_limit: 200000           # 200K TPM ceiling
    rpm_limit: 500              # 500 RPM ceiling

As of the v1.83.x release series, upperbound_key_generate_params enforcement was extended to the /key/regenerate endpoint, closing a previous bypass: prior to this fix, key rotation could be used to reset budget parameters to values exceeding the configured bounds.

Pairing with key_generation_settings

upperbound_key_generate_params controls the ceiling on what can be provisioned. key_generation_settings controls who can provision and what fields are mandatory:

litellm_settings:
  key_generation_settings:
    team_key_generation:
      allowed_team_member_roles: ["admin"]   # Only team admins can create team keys
      required_params: ["tags", "key_alias"] # Mandate attribution metadata
    personal_key_generation:
      allowed_user_roles: ["proxy_admin", "internal_user"]
      required_params: ["key_alias"]         # Every personal key must be named

This combination enforces that:

• Self-provisioned keys always have a human-readable alias (required by key_generation_settings)
• No key can be created with budget or duration exceeding policy limits (enforced by upperbound_key_generate_params)
• Team keys require admin approval (controlled by allowed_team_member_roles)

Governance decision tree for self-serve flows

Who is provisioning the key?
├── Proxy admin (CI/CD, FinOps)
│   → No upperbound restriction; full parameter control
│   → Must still respect org-level ceilings
│
├── Team admin (self-serve for team)
│   → upperbound_key_generate_params enforced
│   → key_generation_settings.team_key_generation.allowed_team_member_roles: ["admin"]
│   → required_params: ["tags", "key_alias", "user_id"]
│
└── Internal user (developer self-serve)
    → upperbound_key_generate_params enforced
    → key_generation_settings.personal_key_generation enforced
    → Set default_key_generate_params to apply conservative defaults
    → Consider: allowed_user_roles: ["proxy_admin"] to disable entirely
                (route all key provisioning through a service desk or IDP workflow)

---

7. fail_closed_budget_enforcement and Auth Cache TTL

This is the most operationally consequential security decision in a LiteLLM deployment.

How auth caching works

On each request, LiteLLM must authenticate the virtual key and check whether any budget at the key, team, or org level is exhausted. Without caching, every request triggers a PostgreSQL read — acceptable at low volume but a performance bottleneck for high-throughput services.

LiteLLM caches the UserAPIKeyAuth object (the resolved key + team + org context) in Redis (or in-memory for single-node deployments) for a configurable TTL. Subsequent requests reuse the cached object, skipping the database read.

The tradeoff: a key that has been revoked, blocked, or budget-exceeded will continue to be authenticated from cache until the TTL expires. In a deployment with a 60-second TTL, a revoked key remains active for up to 60 seconds after revocation.

fail_closed_budget_enforcement

When this setting is enabled, every request that involves a budgeted entity — key, user, team, or org — validates against the database, bypassing the cache for the budget check. This ensures that budget exhaustion takes effect immediately rather than within the cache TTL window.

The cost: one additional database read per request on the budget-check path. At 10K RPM this is roughly 10K additional PostgreSQL queries per minute. With connection pooling and read replicas this is manageable; without them it can saturate the database.

general_settings:
  fail_closed_budget_enforcement: true
  # Also ensure DB connection pool is sized appropriately:
  database_connection_pool_limit: 100   # per-worker connections
  database_connection_timeout: 60

Known cache staleness issues (v1.82-v1.84 era)

Several GitHub issues document cache-related budget bypass patterns in the v1.82-v1.84 release range:

1. update_cache resurrection (issue #25553): After /key/update changes max_budget, the async cache write that fires after the next successful request can overwrite the newly invalidated cache entry with the old max_budget value. This creates a window where budget increases or decreases are silently ignored under load. Status: patched in v1.84.

2. Cross-end-user budget leak (issue #29142): When multiple end-users share a single virtual key and max_end_user_budget is configured, the cached UserAPIKeyAuth object retains the end_user_max_budget from whichever end-user populated the cache first. Subsequent requests from different end-users check against the wrong budget ceiling until TTL expires. Status: active as of mid-2026 — do not rely on max_end_user_budget alone for hard enforcement in multi-tenant applications.

3. Tag-budget enforcement skip (issue #27480): When budget tags are passed via x-litellm-tags header (rather than request body metadata), tag-based spend routing silently skips enforcement. Status: patched; verify your LiteLLM version before using header-based tag budgets in production.

Recommended configuration for high-security deployments

general_settings:
  fail_closed_budget_enforcement: true
  # Redis is required for consistent multi-pod enforcement
  redis_host: os.environ/REDIS_HOST
  redis_port: 6379
  redis_password: os.environ/REDIS_PASSWORD

litellm_settings:
  # Disable end-user budget reliance until issue #29142 is confirmed fixed
  # Use per-key budgets with user_id tagging for attribution instead
  default_key_generate_params:
    max_budget: 10.00
    budget_duration: "30d"

---

8. Key Tags for Cost Attribution

Tags are the primary mechanism for FinOps chargebacks, departmental reporting, and infrastructure cost allocation in LiteLLM. They flow from the request through LiteLLM_SpendLogs.request_tags and are available as Prometheus label dimensions.

Four tag injection paths

Path	Precedence	Use Case
Key-level tags (set at /key/generate)	Applied to all requests using that key	Permanent attribution: team, env, app
Request body metadata.tags	Per-request override	Dynamic context: session, feature, experiment
x-litellm-tags HTTP header	Per-request, client-set	Thin-client integrations without metadata control
Team-level tags	Applied to all keys in the team	Org-wide attribution baseline

Tags from multiple paths merge (not override). A request using a key tagged ["team:quant", "env:prod"] and sending metadata: {"tags": ["experiment:gpt4o-eval"]} in the body will have all three tags in SpendLogs.

Tag schema conventions for enterprise attribution

# Key generation with structured attribution tags
curl -X POST http://localhost:4000/key/generate \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "key_alias": "trading-signal-gen-prod",
    "team_id": "quant-research",
    "tags": [
      "team:quant-research",
      "env:prod",
      "app:trading-signal-generator",
      "cost-center:quant-1042",
      "owner:alice@corp.com"
    ],
    "max_budget": 100.00,
    "budget_duration": "30d"
  }'

# Runtime tagging for per-request attribution
import openai

client = openai.OpenAI(
    api_key="sk-litellm-key",
    base_url="http://localhost:4000/v1"
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Analyze this earnings report..."}],
    extra_body={
        "metadata": {
            "tags": ["feature:earnings-analysis", "session:user-XYZ-2026061801"],
            "spend_logs_metadata": {
                "report_id": "AAPL-Q2-2026",
                "user_facing_feature": "earnings-assistant"
            }
        }
    }
)

SpendLogs schema fields for attribution queries

The LiteLLM_SpendLogs table captures the following fields relevant to cost attribution:

Field	Type	Description
api_key	string	SHA-256 hash of the virtual key
user	string	Internal user ID (key owner)
team_id	string	Team the key belongs to
end_user	string	Value from user param in request body
model_group	string	Requested model alias
model	string	Actual model resolved after routing
spend	float	USD cost of the request
prompt_tokens	int	Input tokens
completion_tokens	int	Output tokens
total_tokens	int	Combined token count
request_tags	JSON array	Merged tags from all injection paths
metadata	JSON	Full request metadata including spend_logs_metadata
startTime	timestamp	Request initiation
endTime	timestamp	Response receipt
cache_hit	boolean	Whether response was served from cache
cache_key	string	Semantic cache key if applicable

Spend report queries via API

# Group spend by team (30-day window)
curl "http://localhost:4000/global/spend/report?group_by=team" \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY"

# Group spend by end-user customer
curl "http://localhost:4000/global/spend/report?group_by=customer" \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY"

# Spend for a specific key
curl "http://localhost:4000/key/info?key=sk-abc123" \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY"

---

9. End-User Tracking: user Parameter for Sub-Key Attribution

The user field in a /chat/completions request body maps to end_user in SpendLogs. This enables attribution of spend within a shared service key to individual downstream users or sessions — critical for SaaS products running behind a single LiteLLM team key.

# SaaS application: pass the downstream customer ID
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[...],
    user="customer-id-98312"   # maps to SpendLogs.end_user
)

Per-end-user budget enforcement

Configure max_end_user_budget in general_settings to impose spending caps per customer:

general_settings:
  max_end_user_budget: 1.00   # $1 per end-user per budget window

Critical caveat: as of mid-2026, the cross-end-user budget leak (issue #29142) makes this enforcement unreliable in high-concurrency deployments. Until the fix is confirmed shipped and tested, the safer pattern is to issue separate virtual keys per customer tier rather than relying on max_end_user_budget alone:

# Safer: provision one key per customer at onboarding
def provision_customer_key(customer_id: str, tier: str) -> str:
    budget_by_tier = {"free": 1.0, "pro": 25.0, "enterprise": 500.0}
    payload = {
        "key_alias": f"customer-{customer_id}",
        "user_id": customer_id,
        "team_id": f"customers-{tier}",
        "max_budget": budget_by_tier[tier],
        "budget_duration": "30d",
        "tags": [f"customer:{customer_id}", f"tier:{tier}"],
    }
    # ... POST /key/generate

End-user attribution in Prometheus

The litellm_spend_metric counter includes end_user as a label dimension, enabling per-customer cost queries directly in Grafana:

# Top 10 end-users by spend (last 7 days)
topk(10,
  sum by (end_user) (
    increase(litellm_spend_metric[7d])
  )
)

# Flag end-users approaching $0.90 of $1 budget
sum by (end_user) (litellm_spend_metric) > 0.9

---

10. Bulk Key Operations: Programmatic Provisioning at Scale

LiteLLM does not provide a native bulk-create endpoint as of mid-2026. Bulk provisioning is achieved via parallel REST calls against /key/generate, with retry and rate-limit handling in the provisioning script.

Bulk provisioning pattern (onboarding a new team)

# bulk_provision.py — onboard an entire team from a CSV roster
import asyncio
import httpx
import csv
from dataclasses import dataclass
from typing import List

LITELLM_BASE_URL = "http://localhost:4000"
MASTER_KEY = "sk-master"


@dataclass
class TeamMember:
    user_id: str
    email: str
    role: str          # "admin" | "user"
    model_tier: str    # "standard" | "premium"


async def provision_member_key(
    client: httpx.AsyncClient,
    member: TeamMember,
    team_id: str,
) -> dict:
    model_map = {
        "standard": ["gpt-4o-mini", "claude-3-haiku"],
        "premium": ["gpt-4o", "claude-3-5-sonnet"],
    }
    budget_map = {"standard": 10.0, "premium": 100.0}

    payload = {
        "key_alias": f"{member.user_id}-default",
        "user_id": member.user_id,
        "team_id": team_id,
        "models": model_map[member.model_tier],
        "max_budget": budget_map[member.model_tier],
        "budget_duration": "30d",
        "tags": [
            f"user:{member.user_id}",
            f"tier:{member.model_tier}",
            f"team:{team_id}",
        ],
        "metadata": {"email": member.email, "provisioned_by": "bulk-onboarding"},
    }
    resp = await client.post(
        f"{LITELLM_BASE_URL}/key/generate",
        headers={"Authorization": f"Bearer {MASTER_KEY}"},
        json=payload,
        timeout=30,
    )
    resp.raise_for_status()
    return {"user_id": member.user_id, "key": resp.json()["key"]}


async def bulk_provision(team_id: str, roster_csv: str) -> List[dict]:
    members = []
    with open(roster_csv) as f:
        for row in csv.DictReader(f):
            members.append(TeamMember(**row))

    # Also add each member to the team
    async with httpx.AsyncClient() as client:
        # Step 1: add all members to team
        await client.post(
            f"{LITELLM_BASE_URL}/team/member_add",
            headers={"Authorization": f"Bearer {MASTER_KEY}"},
            json={
                "team_id": team_id,
                "member": [
                    {"user_id": m.user_id, "role": m.role} for m in members
                ],
            },
        )

        # Step 2: provision keys concurrently (max 10 in flight)
        semaphore = asyncio.Semaphore(10)

        async def bounded_provision(member):
            async with semaphore:
                return await provision_member_key(client, member, team_id)

        results = await asyncio.gather(
            *[bounded_provision(m) for m in members],
            return_exceptions=True,
        )

    successes = [r for r in results if isinstance(r, dict)]
    failures = [r for r in results if isinstance(r, Exception)]
    print(f"Provisioned {len(successes)} keys, {len(failures)} failures")
    return successes


if __name__ == "__main__":
    asyncio.run(bulk_provision("team-new-quant", "roster.csv"))

Bulk revocation (offboarding)

async def revoke_team_keys(team_id: str, dry_run: bool = True):
    """List and revoke all active keys for a team."""
    async with httpx.AsyncClient() as client:
        # Fetch all keys for the team
        resp = await client.get(
            f"{LITELLM_BASE_URL}/key/list",
            headers={"Authorization": f"Bearer {MASTER_KEY}"},
            params={"team_id": team_id},
        )
        keys = resp.json()["keys"]
        key_ids = [k["token"] for k in keys]

        if dry_run:
            print(f"DRY RUN: would revoke {len(key_ids)} keys for team {team_id}")
            return

        # Soft-delete in batches of 25
        for i in range(0, len(key_ids), 25):
            batch = key_ids[i:i + 25]
            await client.delete(
                f"{LITELLM_BASE_URL}/key/delete",
                headers={"Authorization": f"Bearer {MASTER_KEY}"},
                json={"keys": batch},
            )
        print(f"Revoked {len(key_ids)} keys for team {team_id}")

---

11. Budget Alert Webhooks: Slack, Email, PagerDuty

Slack and webhook alerting

general_settings:
  alerting: ["slack"]
  alerting_threshold: 300        # seconds before an alert fires for slow calls
  spend_report_frequency: "1d"   # daily spend digest

litellm_settings:
  alert_types:
    - budget_alerts              # key/user/team budget events
    - spend_reports              # periodic digest
    - llm_exceptions             # model errors
    - llm_too_slow               # latency breaches
    # Omit to suppress:
    # - management_endpoint_alerts  (key/team CRUD, off by default)

export SLACK_WEBHOOK_URL="https://hooks.slack.com/services/T00/B00/xxx"

Channel routing by alert type

Different alert types can route to separate Slack channels — useful for separating budget alerts (finance channel) from latency alerts (SRE channel):

general_settings:
  alerting: ["slack"]
  alert_to_webhook_url:
    budget_alerts: "https://hooks.slack.com/services/.../finance-channel"
    llm_too_slow: "https://hooks.slack.com/services/.../sre-channel"
    llm_exceptions: "https://hooks.slack.com/services/.../sre-channel"
    spend_reports: "https://hooks.slack.com/services/.../finance-channel"

Alert deduplication

budget_alert_ttl: 86400 (default) prevents the same budget breach from firing an alert more than once per 24-hour window per entity. Reduce this for higher-frequency alerting in development environments.

general_settings:
  alerting: ["slack"]
  alerting_args:
    budget_alert_ttl: 3600    # re-alert every hour if budget remains exceeded (dev/staging)
    outage_alert_ttl: 60      # model outage re-alert window

Email alerts (team soft budgets)

Team-level soft budget alerts use a separate email path from Slack webhooks. Configure one of:

# Option A: SendGrid
environment_variables:
  SENDGRID_API_KEY: os.environ/SENDGRID_API_KEY
  EMAIL_FROM: "litellm-alerts@corp.com"

# Option B: SMTP
general_settings:
  smtp_host: "smtp.corp.com"
  smtp_port: 587
  smtp_username: os.environ/SMTP_USER
  smtp_password: os.environ/SMTP_PASSWORD
  smtp_sender_email: "litellm-alerts@corp.com"

Then set soft_budget and metadata.alert_emails on each team that needs email notifications (see Section 3 above).

PagerDuty integration

LiteLLM does not have a native PagerDuty integration as of mid-2026. The recommended pattern is to route budget_alerts to a Slack channel that has a PagerDuty Slack app listening, or to use a webhook relay (AWS Lambda, Cloudflare Worker) that translates LiteLLM Slack webhook payloads into PagerDuty Events API calls:

# pagerduty_relay.py — Lambda/Worker that receives LiteLLM Slack webhooks
import json
import httpx

PAGERDUTY_ROUTING_KEY = "R_XXXXXXXX"

def handler(event, context):
    body = json.loads(event["body"])
    text = body.get("text", "")

    # Only page on hard budget breach, not soft
    if "Budget Exceeded" not in text:
        return {"statusCode": 200}

    payload = {
        "routing_key": PAGERDUTY_ROUTING_KEY,
        "event_action": "trigger",
        "payload": {
            "summary": f"LiteLLM Budget Exceeded: {text[:200]}",
            "severity": "error",
            "source": "litellm-proxy",
        },
    }
    httpx.post("https://events.pagerduty.com/v2/enqueue", json=payload)
    return {"statusCode": 200}

---

12. Audit Logging: Compliance and Cost Audit Fields

Key management event hooks

LiteLLM fires key_management_event_hooks.py on all key lifecycle events. These hooks emit structured log records that can be forwarded to any logging backend (Datadog, Splunk, CloudWatch, Elasticsearch):

Event	Trigger
key_created	/key/generate success
key_updated	/key/update success
key_deleted	/key/delete success
key_blocked	/key/block success
key_rotated	/key/regenerate success
team_created	/team/new success
team_updated	/team/update success
user_created	/user/new success

Note: management endpoint alerts are off by default in the Slack alerting configuration. Enable them for compliance logging:

general_settings:
  alerting: ["slack"]
  alert_types:
    - budget_alerts
    - management_endpoint_alerts   # key/team/user CRUD events

SpendLogs for compliance audit

LiteLLM_SpendLogs is the canonical audit table. Key compliance fields:

-- Query: all requests above $1 in the last 30 days, with key and user attribution
SELECT
    sl.startTime,
    sl.api_key,           -- hashed key ID
    vt.key_alias,         -- human-readable key name (join to VerificationTokenTable)
    sl.user,              -- key owner
    sl.team_id,
    sl.end_user,          -- downstream customer
    sl.model,
    sl.spend,
    sl.prompt_tokens,
    sl.completion_tokens,
    sl.request_tags,
    sl.metadata->>'spend_logs_metadata' AS custom_metadata
FROM "LiteLLM_SpendLogs" sl
LEFT JOIN "LiteLLM_VerificationTokenTable" vt ON sl.api_key = vt.token
WHERE sl.startTime > NOW() - INTERVAL '30 days'
  AND sl.spend > 1.0
ORDER BY sl.spend DESC;

Prometheus metrics for real-time compliance dashboards

The following metrics carry full attribution label sets and are suitable for compliance dashboards:

# Total spend by team (for cost center chargeback)
sum by (team, team_alias) (litellm_spend_metric)

# Remaining budget percentage by team
(
  litellm_remaining_team_budget_metric
  / litellm_team_max_budget_metric
) * 100

# Keys within 10% of their budget ceiling (pre-alert monitoring)
(
  litellm_remaining_api_key_budget_metric
  / litellm_api_key_max_budget_metric
) < 0.10

# Request volume by model and team (for quota auditing)
sum by (model, team) (rate(litellm_total_tokens_metric[1h]))

Disabling spend log verbosity

For high-volume deployments where SpendLogs table growth is a concern, disable_spend_logs: true stops database writes but Prometheus metrics continue. This is acceptable for operational monitoring but breaks point-in-time audit capability — do not use in regulated environments without an alternative log sink.

litellm_settings:
  disable_spend_logs: false   # Keep enabled for compliance
  # MAX_STRING_LENGTH_PROMPT_IN_DB is enforced automatically
  # to prevent large prompt payloads from bloating the table

---

Governance Architecture Summary

┌─────────────────────────────────────────────────────────────────┐
│  Proxy Admin (full control, cross-org visibility)               │
│  Config: master_key, upperbound_key_generate_params,            │
│          key_generation_settings, fail_closed_budget_enforcement│
└────────────────────────┬────────────────────────────────────────┘
                         │
              ┌──────────▼──────────┐
              │   Organization      │  (Enterprise tier)
              │   max_budget        │  Budget ceiling for org
              │   allowed models    │  Model access list
              └──────────┬──────────┘
                         │  1:N
              ┌──────────▼──────────┐
              │   Team              │  (Open source tier)
              │   max_budget        │  Aggregate spend ceiling
              │   soft_budget       │  Alert threshold (email)
              │   budget_duration   │  Reset window
              │   tpm_limit         │  Aggregate TPM across all keys
              │   rpm_limit         │  Aggregate RPM across all keys
              │   model_max_budget  │  Per-model sub-limits
              └──────────┬──────────┘
                         │  1:N
              ┌──────────▼──────────┐
              │   Virtual Key       │  Auth token for applications
              │   max_budget        │  Key-level ceiling (≤ team)
              │   budget_duration   │  Can be shorter than team window
              │   tpm_limit         │  Additional key-level constraint
              │   model_rpm_limit   │  Per-model RPM cap
              │   tags              │  Cost attribution labels
              │   duration          │  Hard expiry
              │   auto_rotate       │  Scheduled rotation
              └──────────┬──────────┘
                         │  via `user` param in request body
              ┌──────────▼──────────┐
              │   End User          │  Downstream customer/session
              │   end_user field    │  In SpendLogs.end_user
              │   max_end_user_budget│ Per-customer spend cap
              │                     │  (⚠ cache bug — see §7)
              └─────────────────────┘

Enforcement stack (evaluated per request, first block wins):
  1. Virtual key: expired? blocked? budget exceeded? rpm/tpm exceeded?
  2. Team: aggregate budget exceeded? rpm/tpm exceeded?
  3. Organization: budget exceeded?
  4. End-user: max_end_user_budget exceeded? (if configured)

---

Version Notes and Known Issues (as of June 2026)

Issue	Affected Versions	Status	Workaround
Budget enforcement bypass in v1.82.3 (#26672)	v1.82.3	Patched v1.82.4+	Upgrade; enable fail_closed_budget_enforcement
update_cache resurrects stale max_budget (#25553)	v1.81-v1.83	Patched v1.84	Enable fail_closed_budget_enforcement
Cross-end-user budget leak (#29142)	v1.84+ (active)	Open	Use per-customer virtual keys instead of max_end_user_budget
Tag-budget enforcement skipped on header path (#27480)	v1.80-v1.82	Patched	Pass tags in request body metadata, not headers
Bedrock cached token overcounting in TPM	Pre-late 2025	Patched	Verify version; set TPM limit at 70% of Bedrock quota
upperbound_key_generate_params bypass via /key/regenerate	Pre-v1.83.14	Patched v1.83.14	Upgrade to v1.83.14+

---

Sources

• Virtual Keys — LiteLLM Docs
• Multi-Tenant Architecture — LiteLLM Docs
• Budgets, Rate Limits — LiteLLM Docs
• Team Budgets — LiteLLM Docs
• Team Soft Budget Alerts — LiteLLM Docs
• Alerting / Webhooks — LiteLLM Docs
• Spend Tracking — LiteLLM Docs
• Prometheus Metrics — LiteLLM Docs
• Provider Budget Routing — LiteLLM Docs
• Internal User Self-Serve — LiteLLM Docs
• API Key Management — DeepWiki
• Budget and Spend Tracking — DeepWiki
• Team Management — DeepWiki
• Bug: Budget enforcement bypassed v1.82.3 (#26672)
• Bug: Cross-end-user budget leak (#29142)
• Bug: Tag-budget enforcement skipped on header path (#27480)
• Bug: update_cache resurrects stale key objects (#25553)
• v1.84.0 Release Notes — multi-pod budget accuracy