← Agent Frameworks 🕐 12 min read
Agent Frameworks

Bedrock Token Counting and Pre-Flight Cost Estimation (2026)

Summary · March 2026

Amazon Bedrock provides a native `CountTokens` API that returns the exact token count a model would bill, at zero cost, before the inference call is made.

Amazon Bedrock provides a native CountTokens API that returns the exact token count a model would bill, at zero cost, before the inference call is made. Combined with CloudWatch’s four runtime metrics (InputTokenCount, OutputTokenCount, CacheReadInputTokens, CacheWriteInputTokens) and invocation logs, teams can implement pre-flight budget gates, per-request cost attribution, and multi-turn context window management without relying on approximate tokenizers. The main constraint is that output tokens are non-deterministic and must be estimated, not counted; quota management and cost estimation both require a separate strategy for the output side.

1. CountTokens API

What it does

CountTokens returns inputTokens — the exact number of tokens the specified model would process if the same payload were submitted to InvokeModel or Converse. The endpoint is free; no charge is incurred for calling it.

Token counting is model-specific. Different foundation models use different tokenization strategies, so the count is only valid for the model specified in modelId. Passing the same prompt to anthropic.claude-3-5-haiku-20241022-v1:0 and a Mistral model will return different counts.

Supported models (as of June 2026)

  • Anthropic Claude 3.5 Haiku (anthropic.claude-3-5-haiku-20241022-v1:0)
  • Anthropic Claude 3.5 Sonnet v1 and v2
  • Anthropic Claude 3.7 Sonnet
  • Anthropic Claude Opus 4
  • Anthropic Claude Sonnet 4 (anthropic.claude-sonnet-4-20250514-v1:0)

Models offered exclusively via cross-Region inference (CRIS) on bedrock-runtime do not support CountTokens on the standard runtime endpoint. For those, use the bedrock-mantle endpoint (see Section 1.4).

HTTP shape

POST /model/{modelId}/count-tokens HTTP/1.1
Content-Type: application/json

{
  "input": { ... }  // Union: "invokeModel" or "converse"
}

Response:

{
  "inputTokens": 39
}

IAM requirement

The caller needs bedrock:CountTokens. Scope it alongside bedrock:InvokeModel on the same model ARN:

{
  "Effect": "Allow",
  "Action": ["bedrock:CountTokens", "bedrock:InvokeModel"],
  "Resource": "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-sonnet-4-20250514-v1:0"
}

Two input formats

InvokeModel format — wraps the raw JSON body as a string:

import boto3
import json

bedrock = boto3.client("bedrock-runtime")

body = json.dumps({
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens": 500,
    "messages": [
        {"role": "user", "content": "What is the capital of France?"}
    ]
})

response = bedrock.count_tokens(
    modelId="anthropic.claude-3-5-haiku-20241022-v1:0",
    input={"invokeModel": {"body": body}}
)

print(response["inputTokens"])  # 14

Converse format — passes structured messages and system prompt directly:

input_payload = {
    "messages": [
        {
            "role": "user",
            "content": [{"text": "What is the capital of France?"}]
        },
        {
            "role": "assistant",
            "content": [{"text": "The capital of France is Paris."}]
        },
        {
            "role": "user",
            "content": [{"text": "What is its population?"}]
        }
    ],
    "system": [{"text": "You are an expert in geography."}]
}

response = bedrock.count_tokens(
    modelId="anthropic.claude-3-5-haiku-20241022-v1:0",
    input={"converse": input_payload}
)

print(response["inputTokens"])  # 39

The Converse format is preferred for multi-turn agents because it mirrors exactly what Converse and ConverseStream consume, including system prompt tokens and prior turn tokens. All visible message content — user turns, prior assistant turns, and system text — is counted. Tool definitions passed in toolConfig and tool result blocks in toolUse content are also tokenized (see Section 5).

Latency overhead

CountTokens is a lightweight synchronous call, but it adds one network round-trip before every inference call. In latency-sensitive paths, gate on token count only when the budget risk is material (e.g., when input exceeds a threshold where overage is likely). For batch pipelines latency is not a constraint; count every record.

Error codes

HTTP Code Meaning
400 ValidationException Payload does not match model’s expected schema; tool feature not supported by model
403 AccessDeniedException Missing bedrock:CountTokens permission
404 ResourceNotFoundException Model ID not found or not available in Region
429 ThrottlingException Account quota exceeded
503 ServiceUnavailableException Endpoint temporarily unavailable; retry with backoff

bedrock-mantle endpoint for CRIS models

Some newer Claude models launch CRIS-only and lack a region-specific bedrock-runtime endpoint for CountTokens. Use the bedrock-mantle endpoint instead:

POST https://bedrock-mantle.{region}.api.aws/anthropic/v1/messages/count_tokens

Authentication: SigV4 with service name bedrock-mantle, or an API key in x-api-key. The boto3 count_tokens method does not target this endpoint — send a raw HTTP POST.

curl -X POST https://bedrock-mantle.us-east-1.api.aws/anthropic/v1/messages/count_tokens \
  -H "x-api-key: $BEDROCK_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic.claude-opus-4-7",
    "messages": [{"role": "user", "content": "How many tokens is this prompt?"}]
  }'

Response: {"input_tokens": N} (note Anthropic shape, not Bedrock shape).

IAM action for mantle: bedrock-mantle:CountTokens, scoped to a Project ARN of the form arn:aws:bedrock-mantle:{region}:{account}:project/{name}.

2. Pre-Flight Budget Validation Pattern

The core pattern

MAX_INPUT_BUDGET = 50_000   # tokens
MAX_COST_CENTS   = 10       # USD cents per call ceiling

MODEL_ID = "anthropic.claude-sonnet-4-20250514-v1:0"

# Input token price: $3.00 per million (on-demand, June 2026)
INPUT_PRICE_PER_TOKEN  = 3.00 / 1_000_000
# Output token estimate multiplier (heuristic; see Section 3)
OUTPUT_PRICE_PER_TOKEN = 15.00 / 1_000_000
AVG_OUTPUT_RATIO       = 0.25  # estimated output / input ratio for this workload

def preflight_check(payload: dict, model_id: str = MODEL_ID) -> dict:
    """Count tokens, estimate cost, return go/no-go decision."""
    response = bedrock.count_tokens(
        modelId=model_id,
        input={"converse": payload}
    )
    input_tokens = response["inputTokens"]

    if input_tokens > MAX_INPUT_BUDGET:
        return {
            "proceed": False,
            "reason": f"input_tokens={input_tokens} exceeds budget={MAX_INPUT_BUDGET}",
            "input_tokens": input_tokens
        }

    # Estimate output tokens from historical ratio
    est_output_tokens = int(input_tokens * AVG_OUTPUT_RATIO)
    est_cost_usd = (
        input_tokens * INPUT_PRICE_PER_TOKEN +
        est_output_tokens * OUTPUT_PRICE_PER_TOKEN
    )
    est_cost_cents = est_cost_usd * 100

    if est_cost_cents > MAX_COST_CENTS:
        return {
            "proceed": False,
            "reason": f"estimated_cost={est_cost_cents:.2f}¢ exceeds ceiling={MAX_COST_CENTS}¢",
            "input_tokens": input_tokens,
            "est_output_tokens": est_output_tokens
        }

    return {
        "proceed": True,
        "input_tokens": input_tokens,
        "est_output_tokens": est_output_tokens,
        "est_cost_cents": round(est_cost_cents, 3)
    }

Where this pattern fits in an agent loop

build_payload()
  → preflight_check()         ← CountTokens API call
      ↓ proceed=False?        → log rejection, return error to caller
      ↓ proceed=True?
  → converse() / invoke_model()
      ↓ response
  → extract_usage()           ← actual token counts from response
  → log_to_cloudwatch()       ← custom metric or invocation log
  → reconcile_estimate_vs_actual()

max_tokens calibration

max_tokens is deducted from your TPM quota at request start, then reconciled to actual output at completion. Overestimating max_tokens blocks quota for other requests unnecessarily. Use historical OutputTokenCount data from CloudWatch to set max_tokens to the 95th-percentile output for each prompt category rather than the model’s maximum context window:

At request start, quota deducted = InputTokenCount + CacheWriteInputTokens + max_tokens
At request end,   quota charged  = InputTokenCount + CacheWriteInputTokens + (OutputTokenCount × burndown_rate)

For Claude 3.7 Sonnet and later models with a 5× output burndown rate, setting max_tokens = 32000 on a prompt that typically generates 1000 tokens reserves 5× more quota than necessary per call.

3. Estimation vs. Actual: The Output Token Problem

Why output tokens cannot be counted pre-flight

CountTokens returns only inputTokens. Output token count is determined by the model at inference time — it is non-deterministic, affected by temperature, top_p, sampling, and the model’s internal state. No pre-flight count is possible.

Approaches to budget systems that include output

Option A — Historical ratio. Track outputTokens / inputTokens per prompt template from invocation logs. Use the 95th-percentile ratio as the budget estimator. This is cheap and works well for structured prompt patterns (classification, extraction) where output length is bounded.

Option B — Hard cap via max_tokens. Set max_tokens to your output budget. The model will truncate at that boundary. This creates predictable cost ceilings at the cost of potentially incomplete responses. Works for summarization, classification, and structured output tasks.

Option C — Two-pass pre-flight. For high-value requests where output length matters, run the request first with max_tokens=1 and temperature=0 to get a token-cost read on the input, then proceed only if input cost is within budget. This does not solve the output estimation problem but isolates the input risk.

Option D — Running budget tracker. For agent loops that accumulate output across turns, track the running total of outputTokens from prior turns returned in usage.outputTokens. Pre-flight check the remaining budget before each new turn rather than trying to predict total output upfront.

Divergence characteristics

Typical pre-flight estimate vs. actual divergence:

  • Input tokens: zero divergence (CountTokens is exact)
  • Output tokens: ±30–200% depending on prompt type; structured outputs (JSON, code) are more predictable than open-ended generation
  • Cache tokens: only known at inference time; CountTokens does not predict cache hits

4. Four Token Types in Bedrock Billing

Definitions

Token type CloudWatch metric Quota contribution Billing
Input tokens (non-cached) InputTokenCount Yes At input rate
Output tokens OutputTokenCount Yes (× burndown rate for Claude 3.7+) At output rate
Cache-read input tokens CacheReadInputTokens No At cache-read rate (≈0.10× input)
Cache-write input tokens CacheWriteInputTokens Yes At cache-write rate (1.25× for 5-min TTL; 2.0× for 1-hour TTL)

Quota formula

At request start:

Reserved = InputTokenCount + CacheWriteInputTokens + max_tokens

At request end (final charge):

Charged = InputTokenCount + CacheWriteInputTokens + (OutputTokenCount × burndown_rate)

CacheReadInputTokens never contribute to quota. The burndown rate is 5× for Claude 3.7 Sonnet and later; 1× for all other models.

What CountTokens covers

CountTokens counts the tokens that will be sent for processing — this maps to the sum of what would become InputTokenCount and CacheWriteInputTokens if caching is configured. It does not predict CacheReadInputTokens because whether a cache hit occurs depends on runtime state (is the cached block still warm?) that is unknowable pre-flight.

Cache pricing structure (as of June 2026)

TTL Write multiplier Read multiplier
5 minutes 1.25× input rate 0.10× input rate
1 hour 2.00× input rate 0.10× input rate

The 1-hour TTL cache became generally available in January 2026. The break-even for 1-hour TTL vs. re-sending tokens is reached when the same block is read more than ~20× before expiry. For workloads with long, stable system prompts (e.g., a 10K-token RAG context loaded once per session), 1-hour TTL substantially reduces per-turn input cost.

5. Tool Use Token Overhead

Tool definitions and tool results inflate the token count beyond the visible message content. This is a frequent source of budget underestimation in agentic workflows.

What adds tokens

Tool schema (toolConfig): Each tool definition — name, description, input schema — is serialized and counted as input tokens every call. A verbose tool description (long field descriptions, many properties) can add hundreds to thousands of tokens per tool, per turn. This overhead is present whether or not the tool is invoked.

Tool result content blocks: When the model calls a tool, the caller sends back a toolResult content block in the next user turn. The tool result — often a JSON payload from an external API — is counted as input tokens for that turn. Large API responses (e.g., a full calendar JSON with 50 events) can add 2000+ tokens to the next turn’s input count.

Thinking tokens (Claude 3.7+ extended thinking): Extended thinking tokens are billed as output tokens at the output rate. They appear in OutputTokenCount in the response usage.

Counting tool-inclusive payloads with CountTokens

Pass the full toolConfig alongside messages when calling CountTokens in the Converse format. This is the only way to get an accurate pre-flight count for tool-enabled requests:

tool_config = {
    "tools": [
        {
            "toolSpec": {
                "name": "get_account_balance",
                "description": "Returns the current balance for a given account ID.",
                "inputSchema": {
                    "json": {
                        "type": "object",
                        "properties": {
                            "account_id": {
                                "type": "string",
                                "description": "The unique identifier for the account."
                            }
                        },
                        "required": ["account_id"]
                    }
                }
            }
        }
    ]
}

payload_with_tools = {
    "messages": conversation_history,
    "system": [{"text": system_prompt}],
    "toolConfig": tool_config
}

response = bedrock.count_tokens(
    modelId=MODEL_ID,
    input={"converse": payload_with_tools}
)

Optimization

For agents with many tools but sparse tool use per turn, consider dynamic tool injection: include only the tools relevant to the current step rather than the full tool catalog. This can reduce per-turn input tokens by 30–60% in workflows with >10 tools.

6. Multi-Turn Conversation Token Accumulation

In a Converse-based agent loop, the full conversation history is re-sent every turn. Input tokens grow monotonically until the history is pruned or summarized.

Tracking running total

class ConversationTokenTracker:
    def __init__(self, model_id: str, context_window: int = 200_000):
        self.model_id = model_id
        self.context_window = context_window
        self.history = []
        self.system = []
        self.cumulative_input_tokens = 0
        self.cumulative_output_tokens = 0

    def count_current_tokens(self) -> int:
        """Count tokens for the current full conversation payload."""
        payload = {"messages": self.history, "system": self.system}
        response = bedrock.count_tokens(
            modelId=self.model_id,
            input={"converse": payload}
        )
        return response["inputTokens"]

    def add_turn(self, user_text: str) -> dict:
        self.history.append({
            "role": "user",
            "content": [{"text": user_text}]
        })

        # Pre-flight: check context window headroom
        current_tokens = self.count_current_tokens()
        remaining = self.context_window - current_tokens
        if remaining < 2000:
            self._prune_history()
            current_tokens = self.count_current_tokens()

        # Run inference
        result = bedrock.converse(
            modelId=self.model_id,
            messages=self.history,
            system=self.system
        )

        # Record actuals from usage block
        usage = result["usage"]
        self.cumulative_input_tokens  += usage["inputTokens"]
        self.cumulative_output_tokens += usage["outputTokens"]

        # Append assistant response
        assistant_text = result["output"]["message"]["content"][0]["text"]
        self.history.append({
            "role": "assistant",
            "content": [{"text": assistant_text}]
        })

        return {
            "response": assistant_text,
            "turn_input_tokens": usage["inputTokens"],
            "turn_output_tokens": usage["outputTokens"],
            "cumulative_input": self.cumulative_input_tokens,
            "cumulative_output": self.cumulative_output_tokens,
        }

    def _prune_history(self):
        """Keep only the last N turns when approaching context limit."""
        if len(self.history) > 4:
            self.history = self.history[-4:]  # retain last 2 complete exchanges

Context window cost modeling

For a long-running session with a fixed system prompt of S tokens and a history that grows by A tokens per assistant turn and U tokens per user turn:

Input tokens at turn N ≈ S + N × (U + A)
Total session input cost ≈ Σ(k=1..N) [S + k × (U + A)]
                         = N×S + (U+A) × N×(N+1)/2

This quadratic growth is why session-length budgets must account for the compounding cost of history, not just per-turn cost.

7. Streaming Token Counting

When using ConverseStream or InvokeModelWithResponseStream, token counts are not available during the stream — they arrive in a metadata event at the end.

ConverseStream metadata event

The stream closes with a ConverseStreamMetadataEvent containing:

{
  "usage": {
    "inputTokens": 39,
    "outputTokens": 187,
    "totalTokens": 226,
    "cacheReadInputTokens": 3500,
    "cacheWriteInputTokens": 0,
    "cacheDetails": []
  },
  "metrics": {
    "latencyMs": 1234
  }
}

Extracting usage from a streaming response in Python:

import boto3

bedrock = boto3.client("bedrock-runtime")

stream_response = bedrock.converse_stream(
    modelId="anthropic.claude-sonnet-4-20250514-v1:0",
    messages=[{"role": "user", "content": [{"text": "Explain transformer attention in 3 sentences."}]}]
)

output_text = ""
usage = None

for event in stream_response["stream"]:
    if "contentBlockDelta" in event:
        delta = event["contentBlockDelta"].get("delta", {})
        if "text" in delta:
            output_text += delta["text"]
            print(delta["text"], end="", flush=True)
    elif "metadata" in event:
        usage = event["metadata"].get("usage", {})

print()  # newline after streamed output

if usage:
    print(f"Input tokens:       {usage.get('inputTokens', 0)}")
    print(f"Output tokens:      {usage.get('outputTokens', 0)}")
    print(f"Cache read tokens:  {usage.get('cacheReadInputTokens', 0)}")
    print(f"Cache write tokens: {usage.get('cacheWriteInputTokens', 0)}")

InvokeModelWithResponseStream headers

For models invoked via InvokeModelWithResponseStream, Bedrock injects token counts in HTTP response headers after the stream completes:

x-amzn-bedrock-input-token-count: 39
x-amzn-bedrock-output-token-count: 187

These headers are available when the response object is first received (before the stream body is consumed), but the counts they reflect are only accurate after generation completes, so in practice the stream must be fully consumed before the values are stable for billing reconciliation.

Implication for billing reconciliation

In streaming pipelines, the reconciliation window between when the request starts and when actual token counts become available spans the full generation time. For budget enforcement mid-stream (e.g., cutting off a runaway generation), the only lever is max_tokens set pre-flight — there is no in-flight token budget enforcement via the API.

8. CloudWatch Token Metrics

Available runtime metrics

Bedrock automatically publishes the following to CloudWatch under the AWS/Bedrock namespace:

Metric Type Description
InputTokenCount Sum Input tokens (non-cached) per invocation
OutputTokenCount Sum Output tokens generated per invocation
CacheReadInputTokens Sum Tokens served from cache (not billed at full input rate)
CacheWriteInputTokens Sum Tokens written to cache
Invocations Count Total invocations
InvocationLatency Milliseconds End-to-end latency
TimeToFirstToken Milliseconds TTFT (added March 2026)
EstimatedTPMQuotaUsage Percent Real-time TPM quota consumption (added March 2026)

Dimensions

Metrics can be filtered by:

  • ModelId — specific model ARN or inference profile ID
  • Region — implicit from the endpoint

CloudWatch Logs Insights query — token spend by IAM principal

Enable model invocation logging, then query the log group:

fields identity.arn as principal,
       input.inputTokenCount as inTokens,
       output.outputTokenCount as outTokens
| stats sum(inTokens) as totalInput,
        sum(outTokens) as totalOutput,
        count() as calls
        by principal
| sort totalInput desc

CloudWatch Logs Insights query — per-model token spend

fields modelId,
       input.inputTokenCount as inTokens,
       output.outputTokenCount as outTokens
| stats sum(inTokens) as totalInput,
        sum(outTokens) as totalOutput,
        count() as calls
        by modelId
| sort totalInput desc

Metric math for estimated daily cost (Claude Sonnet 4 example)

In the CloudWatch console or via GetMetricData, use metric math:

input_cost  = InputTokenCount  * 0.000003   # $3.00/M
output_cost = OutputTokenCount * 0.000015   # $15.00/M
total_cost  = input_cost + output_cost

Aggregate with Sum over the desired period. Cache read and cache write costs require separate metric math expressions with their respective multipliers.

Per-request metadata tagging

Bedrock allows passing a requestMetadata object with arbitrary key-value pairs per request. These appear in the invocation log as requestMetadata and survive to the CloudWatch log group, enabling cost attribution by team, feature, or customer:

bedrock.converse(
    modelId=MODEL_ID,
    messages=[...],
    additionalModelRequestFields={},
    # requestMetadata is passed as a top-level parameter on Converse
    # check SDK version — field name varies
)

The requestMetadata field is the only field in the invocation log supplied by the caller; all other fields are populated by Bedrock automatically.

9. LiteLLM Token Tracking and Divergence

How LiteLLM counts tokens for SpendLogs

LiteLLM’s proxy maintains a SpendLogs table that records token counts per request. For Bedrock-routed requests, LiteLLM historically used a generic tokenizer fallback (gpt-3.5-turbo BPE) when a model-specific tokenizer was not available. This produced systematic divergence from actual Bedrock-billed counts:

  • Claude models use a different BPE vocabulary than GPT tokenizers
  • Divergence was reported at 5–15% on typical prompt structures (GitHub issue #8140, January 2025)
  • Tool-heavy payloads showed larger divergence because tool schema serialization formats differ between OpenAI and Anthropic conventions

Current status

A pull request (BerriAI/litellm #14557) implemented native Bedrock CountTokens API support in LiteLLM. After this change, LiteLLM routes token counting for Bedrock models through the CountTokens endpoint before inference, eliminating the tokenizer mismatch. Verify this is merged in your LiteLLM version before trusting SpendLogs for cost attribution.

/v1/messages/count_tokens proxy endpoint

LiteLLM exposes Anthropic’s count tokens API at /v1/messages/count_tokens for clients that call the Anthropic SDK through LiteLLM as a proxy. Internally, LiteLLM translates this to a CountTokens call to the appropriate Bedrock endpoint.

Reconciliation pattern

For FinOps purposes, do not rely solely on LiteLLM SpendLogs for Bedrock cost attribution. Cross-reference against:

  1. CloudWatch InputTokenCount + OutputTokenCount metrics per ModelId
  2. AWS Cost Explorer with AmazonBedrock service filter
  3. Invocation logs in CloudWatch Logs Insights or S3 (for cache token breakdown)

LiteLLM SpendLogs are useful for per-user or per-key attribution that Bedrock does not natively provide, but total token counts should be validated against AWS-native sources.

10. Batch Inference Token Counting

Pre-submission cost estimation

Bedrock batch inference (CreateModelInvocationJob) processes a JSONL file of requests asynchronously at 50% of on-demand pricing. The batch API does not expose a pre-flight token counting endpoint for the full job.

The recommended pre-submission pattern is to sample records and extrapolate:

import json
import random
from pathlib import Path

def estimate_batch_cost(
    jsonl_path: str,
    model_id: str,
    sample_rate: float = 0.05,
    input_price_per_million: float = 3.00,
    batch_discount: float = 0.50,
) -> dict:
    records = [json.loads(l) for l in Path(jsonl_path).read_text().splitlines() if l.strip()]
    sample = random.sample(records, max(1, int(len(records) * sample_rate)))

    token_counts = []
    for record in sample:
        payload = record.get("modelInput", {})
        # Wrap in invokeModel format for CountTokens
        body = json.dumps(payload)
        resp = bedrock.count_tokens(
            modelId=model_id,
            input={"invokeModel": {"body": body}}
        )
        token_counts.append(resp["inputTokens"])

    avg_tokens = sum(token_counts) / len(token_counts)
    est_total_input = avg_tokens * len(records)

    effective_price = input_price_per_million * (1 - batch_discount) / 1_000_000
    est_input_cost = est_total_input * effective_price

    return {
        "records": len(records),
        "sample_size": len(sample),
        "avg_input_tokens": round(avg_tokens, 1),
        "est_total_input_tokens": int(est_total_input),
        "est_input_cost_usd": round(est_input_cost, 4),
        "note": "Output tokens not counted; add estimated output cost separately"
    }

Post-job actuals

Actual token counts for completed batch jobs are not emitted to CloudWatch. The only source is the manifest.json.out file written to the output S3 bucket, which contains aggregate statistics for the job. Parse this file for reconciliation:

import boto3, json

s3 = boto3.client("s3")

def get_batch_job_actuals(output_bucket: str, job_id: str) -> dict:
    key = f"batch-output/{job_id}/manifest.json.out"
    obj = s3.get_object(Bucket=output_bucket, Key=key)
    manifest = json.loads(obj["Body"].read())
    return manifest.get("statistics", {})

The manifest includes total input tokens processed, total output tokens, success/failure counts, and per-record metadata.

11. Decision Guide: When to Call CountTokens

Scenario Use CountTokens? Rationale
Single short prompt, no budget constraint No Overhead not worth it
Request that may exceed context window Yes Prevent 400 errors from exceeding model limit
Multi-turn agent with cost budget per session Yes Count each turn before sending
Batch job — sample for pre-submission estimate Yes (sample) Exact count per record is too slow; 5% sample gives ±10% accuracy
Streaming response, need mid-stream cost gate No Streaming has no in-flight stop mechanism beyond max_tokens; set max_tokens pre-flight instead
Tool-heavy agentic workflow Yes Tool schema overhead is large and non-obvious; count the full payload including toolConfig
CRIS-only model Use bedrock-mantle endpoint bedrock-runtime CountTokens not available for CRIS-only models

12. Invocation Log Schema Reference

Invocation logging must be explicitly enabled. Logs go to CloudWatch Logs, S3, or both. The canonical log entry format (schema version 1.0):

{
    "schemaType": "ModelInvocationLog",
    "schemaVersion": "1.0",
    "timestamp": "2026-06-18T10:23:00Z",
    "accountId": "123456789012",
    "region": "us-east-1",
    "requestId": "abcd-1234-...",
    "operation": "Converse",
    "modelId": "anthropic.claude-sonnet-4-20250514-v1:0",
    "identity": {
        "arn": "arn:aws:sts::123456789012:assumed-role/MyRole/session"
    },
    "requestMetadata": {
        "team": "finance-agent",
        "session_id": "abc123"
    },
    "input": {
        "inputContentType": "application/json",
        "inputBodyJson": {},
        "inputTokenCount": 512
    },
    "output": {
        "outputContentType": "application/json",
        "outputBodyJson": {},
        "outputTokenCount": 128
    }
}

Note: cacheReadInputTokenCount and cacheWriteInputTokenCount are present in the output body JSON for Converse requests that use prompt caching, but are not top-level fields in the invocation log schema (they appear inside outputBodyJson). For cache-specific analytics, parse the embedded body or use CloudWatch CacheReadInputTokens / CacheWriteInputTokens metrics directly.

Logging is disabled by default. Enable via Bedrock console → Settings → Model invocation logging, or via PutModelInvocationLoggingConfiguration.

Key Findings

  1. CountTokens is exact for input and costs nothing to call. It is the correct foundation for any pre-flight budget gate.

  2. Output tokens cannot be pre-counted. Budget systems must either cap via max_tokens, use historical ratios, or accept output cost uncertainty.

  3. The four token types (input, output, cache-read, cache-write) have different billing rates and different quota contributions. CacheReadInputTokens are the only type that contribute neither to quota nor to the standard input rate — they are billed at ~10% of input price.

  4. Claude 3.7 Sonnet and later models carry a 5× output token burndown rate against TPM/TPD quotas. This is quota-only, not billing; you are billed for actual output tokens at the output price, not 5× them. Miscalibrating max_tokens on these models has an amplified quota impact.

  5. Tool definitions inflate input tokens per turn, invisibly, every turn. Always include toolConfig when calling CountTokens for tool-enabled agents.

  6. The streaming metadata event (ConverseStreamMetadataEvent) is the authoritative source of actual token counts for streaming requests. It fires at the end of the stream.

  7. LiteLLM SpendLogs diverged from actual Bedrock counts before PR #14557 due to tokenizer mismatch. Cross-reference SpendLogs against CloudWatch metrics for FinOps reporting.

  8. Batch job actuals are in manifest.json.out in the output S3 bucket; they are not emitted to CloudWatch.

Sources