← Agent Frameworks 🕐 6 min read
Agent Frameworks

Bedrock FinOps Dashboards — QuickSight, Grafana, CloudWatch (2026)

No single tool gives both billing accuracy and near-real-time team attribution for LiteLLM→Bedrock. The recommended architecture uses two tiers:

No single tool gives both billing accuracy and near-real-time team attribution for LiteLLM→Bedrock. The recommended architecture uses two tiers:

Tier 1 — Near real-time (<5 min lag): LiteLLM Prometheus + Grafana

  • Deploy LiteLLM with callbacks: ["prometheus"] in litellm_settings
  • Build Grafana dashboard from litellm_spend_metric, litellm_total_tokens_metric, budget gauges
  • Verify Bedrock Prometheus bug (#17415) is fixed before relying on it (see §5)

Tier 2 — Billing-accurate (24–48h lag): CUDOS v5.8.1 + QuickSight

  • Deploy CUDOS with CUR 2.0 + IAM principal attribution enabled
  • Configure LiteLLM IAM roles with Team/Project tags; activate tags in Billing console
  • CUDOS v5.8.0+ Group By “IAM Principal Tags” breaks Bedrock spend by team in QuickSight

1. CUDOS — Best Pre-Built Bedrock QuickSight Dashboard

Repo: aws-solutions-library-samples/cloud-intelligence-dashboards-framework Deployment: CUR 2.0 → S3 → Glue crawler → Athena → QuickSight SPICE (daily refresh). Deploy via cid-cmd CLI, ~15–30 min from payer account. Demo: https://cid.workshops.aws.dev/demo?dashboard=cudos

Bedrock feature history

Version What was added
v5.0.0 Bedrock section: spend by Account, Region, Pricing Model, UsageType Group; Daily Cost per Resource (Model)
v5.3 UsageType Group breakout: Image Generation / Input Tokens / Output Tokens / Model Units / Model Customization
v5.5 Guardrails added to UsageType Group
v5.6 “Bedrock Model Unit Cost”; “Bedrock Cost per Legal Entity”; Model Name calculated field
v5.8.0 “Cost per 1000 Tokens by Usage Type Group” with Group By: Account / Region / Pricing Model / Usage Type Group / Model / IAM Principal Tags
v5.8.1 Shifted to cost per million tokens; Top 5 models by cost; Spend per Resource (7 months); Batch Inference; Bedrock AgentCore

v5.8.0 IAM Principal Tags group-by is the key feature for LiteLLM multi-team attribution — it reads iamPrincipal/Team and iamPrincipal/Project columns directly.

CUR columns CUDOS uses for Bedrock

  • line_item_usage_type — encodes region + model + token direction + tier + routing (e.g. USE1-Claude4.6Sonnet-input-tokens, USE1-Claude4.6Sonnet-cache-read-input-token-count)
  • line_item_unblended_cost
  • line_item_iam_principal — full IAM ARN (CUR 2.0 only, requires opt-in in Data Exports console)
  • iamPrincipal/{key} — IAM principal tag columns (24h activation delay, not retroactive)
  • resourceTags/{key} — cost allocation tags from activated resource/session tags

CUDOS vs CID vs KPI

CUDOS is the right dashboard for Bedrock token FinOps. CID and KPI focus on RI/SP optimization and modernization KPIs — neither has dedicated Bedrock token visuals.


2. AWS-Native QuickSight Options

AWS Cost and Usage Dashboard (native QuickSight feature): Service-level spend visibility only. No dedicated AI/ML token breakdown. Not a substitute for CUDOS.

Proactive AI Cost Management (AWS ML Blog, 2-part series): Step Functions-based real-time rate limiting that enforces token budgets per cost center. Creates a CloudWatch dashboard with custom metrics: dimensions Model, ModelId, CostCenter, Application, Environment. Runtime enforcement + near-real-time token burn visibility — complements CUDOS’s billing view.


3. Grafana Options for Bedrock

Community dashboards

Grafana dashboard 20537 — “Amazon Bedrock”

  • Data source: CloudWatch (AWS/Bedrock namespace)
  • Panels: InputTokenCount, OutputTokenCount, InvocationLatency, Invocations, errors
  • Dimension: ModelId only — no team/project attribution
  • No cost panels. Operational latency/volume monitoring, not FinOps.

aws-samples/sample-quota-dashboard-for-amazon-bedrock

  • CDK stack: CloudWatch dashboard showing real-time TPM/RPM quota utilization vs Service Quota limits per model
  • Cost: ~$5.73/month (CloudWatch custom metrics)
  • Use case: quota headroom monitoring, not billing

awslabs/generative-ai-cdk-constructs — BedrockMetricsDashboard

  • CloudWatch construct with optional cost widget: Input Token Cost, Output Token Cost, Total Token Cost (calculated from static pricing config — requires manual price updates)
  • Dimension: ModelId only

COAST (aws-samples/coast-grafana-cost-intelligence-dashboards)

  • Architecture: CUR 2.0 → Athena → Amazon Managed Grafana via Athena plugin
  • Existing dashboards: EC2, Athena, Auto Scaling, EKS. No Bedrock dashboard. Useful as scaffold for a custom Bedrock Athena→Grafana build.

Building a custom Bedrock cost Grafana dashboard

The COAST architecture (CUR→Athena→Managed Grafana) is the right stack. No pre-built template exists for Bedrock FinOps in Grafana — it requires custom Athena SQL against the CUR Glue table. The Grafana Athena plugin executes arbitrary SQL, so any CUR query works.


4. CloudWatch Bedrock Metrics — Complete Reference

Namespace: AWS/Bedrock Primary dimension: ModelId

Metric Unit Description
Invocations SampleCount Successful calls (Converse, ConverseStream, InvokeModel, InvokeModelWithResponseStream)
InvocationLatency Milliseconds End-to-end latency (first byte to last token)
InvocationClientErrors SampleCount 4xx errors
InvocationServerErrors SampleCount 5xx errors
InvocationThrottles SampleCount Throttled requests
InputTokenCount SampleCount Prompt tokens
OutputTokenCount SampleCount Response tokens
CacheReadInputTokens SampleCount Prompt cache reads (billed at reduced rate)
CacheWriteInputTokens SampleCount Tokens written to prompt cache
TimeToFirstToken Milliseconds Time to first token for streaming APIs
EstimatedTPMQuotaUsage SampleCount Approximate TPM quota consumption (input tokens + max_tokens reservation)
OutputImageCount SampleCount Images generated (image models; extra dimensions: ImageSize, BucketedStepSize)
LegacyModelInvocations SampleCount Calls to deprecated/legacy models

Latency:

  • TimeToFirstToken and EstimatedTPMQuotaUsage: updated every minute
  • Other runtime metrics: available within minutes, but resource usage data “may be delayed up to 60 minutes” per AWS docs
  • CUR data: 24–48 hour lag

Cost: Native Bedrock runtime metrics emit automatically at no charge. Custom metrics (per-team token counts via CloudWatch Logs metric filters) cost $0.30/metric/month.

Real-time cost estimation: No native “cost” metric in CloudWatch. Estimate by pulling InputTokenCount + OutputTokenCount per ModelId, applying hardcoded per-model pricing in Grafana formula or CloudWatch Metric Math. The awslabs CDK construct does this with an optional pricing config block.

No native team/project dimension in CloudWatch. To add team attribution: use CloudWatch Logs metric filters on Bedrock invocation logs (which carry requestMetadata and identity.arn) and emit custom metrics with Team or Project dimensions.


5. LiteLLM Prometheus Metrics — Full Reference

Enable: Add callbacks: ["prometheus"] to litellm_settings in config.yaml. Expose /metrics at port 4000. For multi-worker: set PROMETHEUS_MULTIPROC_DIR env var.

Spend and token metrics

Metric Labels
litellm_spend_metric end_user, hashed_api_key, api_key_alias, model, team, team_alias, user
litellm_total_tokens_metric same
litellm_input_tokens_metric same
litellm_output_tokens_metric same
litellm_input_cached_tokens_metric same (prompt cache reads)
litellm_input_cache_creation_tokens_metric same (cache writes)
litellm_output_reasoning_tokens_metric same (O1/O3 reasoning)
litellm_output_audio_tokens_metric same

Budget metrics

Metric Labels
litellm_team_max_budget_metric team, team_alias
litellm_remaining_team_budget_metric team, team_alias
litellm_team_budget_remaining_hours_metric team, team_alias
litellm_api_key_max_budget_metric hashed_api_key, api_key_alias
litellm_remaining_api_key_budget_metric hashed_api_key, api_key_alias
litellm_remaining_api_key_requests_for_model per model RPM limits per key
litellm_remaining_api_key_tokens_for_model per model TPM limits per key

Key Grafana queries

Per-team spend (24h): sum(increase(litellm_spend_metric_total[24h])) by (team_alias)

Remaining team budget gauge: litellm_remaining_team_budget_metric

Token burn by model: sum(increase(litellm_total_tokens_metric_total[1h])) by (model)

Grafana dashboard 24055 note: This dashboard uses Azure Monitor / Application Insights as its data source — not Prometheus, not CloudWatch. It is for Azure-deployed LiteLLM only. Build a new Prometheus-based dashboard for AWS deployments; 24055 is not reusable.

Critical bug: Bedrock Prometheus reporting broken

GitHub issue #17415 (December 2025): When LiteLLM routes through Bedrock, litellm_params is None in the Prometheus callback. The callback crashes silently — litellm_proxy_total_requests_metric does not increment for Bedrock requests. This means Prometheus-based Grafana dashboards will show zero or understated counts for Bedrock-routed traffic. Verify this is fixed in your version before relying on Prometheus metrics for Bedrock.


6. Four-Token-Type Reconciliation

Bedrock charges four distinct token types, each with a different unit price and separate CUR line item:

  1. Input tokens
  2. Output tokens
  3. Cache read tokens (~10% of input price — significantly cheaper)
  4. Cache write tokens (more expensive than input)

Summing only input + output is the most common reconciliation error. For workloads using Anthropic Claude with prompt caching enabled, the true cost will diverge materially from input+output-only sums. All four token types must be included in any cost calculation or dashboard.


Sources: CUDOS changelog (GitHub), AWS CUR Bedrock documentation, AWS ML Blog Proactive AI Cost Management, CloudWatch Bedrock runtime metrics docs, LiteLLM Prometheus docs, Grafana dashboard 20537 and 24055, awslabs generative-ai-cdk-constructs, aws-samples coast and quota-dashboard, LiteLLM issue #17415.