← Agent Frameworks 🕐 20 min read
Agent Frameworks

Bedrock Converse API vs InvokeModel — Cost Differences, Tool Token Overhead, Migration Patterns (2026)

Executive Summary · March 2026

AWS Bedrock exposes two primary inference surfaces: `InvokeModel` (and its streaming variant `InvokeModelWithResponseStream`) and the newer `Converse`/`ConverseStream` family.

AWS Bedrock exposes two primary inference surfaces: InvokeModel (and its streaming variant InvokeModelWithResponseStream) and the newer Converse/ConverseStream family. The underlying inference compute is identical — the same GPU processes the same tokens at the same per-token price. No pricing delta exists between APIs. However, the APIs are not token-count neutral when you migrate between them: format changes, tool schema serialization, and multimodal block structures all affect how many input tokens are billed per request. This note maps those differences quantitatively and gives enterprise teams a migration checklist.

The tl;dr cost posture:

Factor InvokeModel Converse
Per-token price Same Same
Request formatting overhead Model-specific (you control the wire format) Unified (Bedrock normalizes)
Tool schemas as input tokens Manual — you embed in the prompt Automatic — toolConfig block counted as input tokens
Multimodal block encoding Model-specific base64 Unified imageBlock/documentBlock — same token formula
Streaming cost Same as non-streaming Same as non-streaming
Payload limit ~5 MB ~3.75 MB
LiteLLM default route bedrock/invoke/ bedrock/converse/ (for supported models)

1. Converse vs InvokeModel Fundamentals

What Converse Adds

The Converse API was introduced in May 2024 as a model-agnostic interface that abstracts away per-provider request/response shapes. Before Converse, each model family (Claude, Llama, Titan, Mistral, Cohere) required developers to construct and parse a unique JSON payload format. Claude on Bedrock, for example, required anthropic_version, messages, and specific content block shapes; Llama required a different prompt template; Titan had its own inputText field structure.

Converse normalizes this into:

  • A unified messages array with typed content blocks (text, image, document, toolResult, toolUse)
  • A single system array for system prompts across all models
  • A toolConfig block for function/tool definitions with a standardized JSON Schema input schema format
  • A guardrailConfig block that applies Bedrock Guardrails uniformly
  • Consistent usage response fields: inputTokens, outputTokens, cacheReadInputTokens, cacheWriteInputTokens

InvokeModel passes your raw JSON body directly to the model provider’s native inference endpoint. You are responsible for constructing the correct format and parsing the raw response. For Claude, this means the Anthropic Messages API JSON format (anthropic_version: "bedrock-2023-05-31"). For Llama, it means assembling the <|begin_of_text|> prompt template manually.

Models Supporting Converse vs InvokeModel-Only

As of June 2026, Converse supports virtually all text generation models on Bedrock. Notable model families confirmed on Converse:

  • Anthropic Claude: All Claude 3+ variants (Haiku, Sonnet, Opus), Claude Sonnet 4.6, Claude Opus 4.7 — full Converse support including tool use, vision, document blocks, streaming tool use, and guardrails
  • Amazon Nova: Nova Premier, Nova Pro, Nova Lite, Nova 2 Lite — full Converse support
  • Meta Llama: Llama 3.1, 3.2, 3.3, Llama 4 Maverick, Llama 4 Scout — Converse support; vision supported on Llama 3.2 and Llama 4
  • Mistral AI: Mistral Large, Mistral Large 2, Mistral Small, Pixtral Large — Converse support
  • AI21 Labs: Jamba 1.5 Large, Jamba 1.5 Mini — Converse support
  • Cohere: Command R, Command R+ — Converse support
  • Writer: Palmyra X4, Palmyra X5 — Converse support

InvokeModel-only (no Converse support):

  • Embedding models (Titan Embeddings G1, Amazon Titan Multimodal Embeddings, Cohere Embed)
  • Image generation models (Stable Diffusion, Amazon Titan Image Generator)
  • Older Claude models (claude-instant-v1, claude-v1/v2 with InvokeModel native format)

This means: if your use case requires text generation from any current foundation model, Converse is available. If you need embeddings or image generation, InvokeModel remains the only path.

No Token Cost Difference at the API Level

The underlying inference compute is identical. AWS documentation confirms this explicitly: “The token count returned by [CountTokens] will match the token count that would be charged if the same input were sent to the model in an InvokeModel or Converse request.” The API surface does not add a billing premium. What changes is how your prompt content is structured, which affects the token count of the payload itself.

2. Converse API Cost Considerations in Depth

System Message Formatting

InvokeModel (Claude native format): the system field is a string. The text you write is tokenized directly, with no wrapper tokens beyond what Claude’s tokenizer applies.

Converse: the system field is an array of SystemContentBlock objects, each with a text string. The JSON serialization is done by the SDK before the payload reaches Bedrock’s normalization layer. You do not pay for the JSON structure itself — Bedrock strips the envelope before passing tokens to the model. The system content tokenizes identically to the equivalent InvokeModel system string.

Net impact: zero additional tokens from Converse system prompt handling, assuming the text content is unchanged.

Tool Definitions as Input Tokens

This is the largest hidden cost in Converse adoption. When you pass a toolConfig block, Bedrock serializes your tool definitions and injects them into the model’s context as input tokens. These tokens are billed at the full input token rate.

How the injection works:

  • Each tool definition includes a name, description, and inputSchema (a JSON Schema object describing the parameters)
  • Bedrock converts the entire toolConfig into a token-counted prompt prefix before the messages array
  • The model never “sees” the raw JSON; it sees a structured representation, but the tokenizer processes the full schema content

Token cost estimation per tool:

A minimal tool (name + short description + 2-3 string parameters):

{
  "name": "get_stock_price",
  "description": "Retrieves current stock price for a ticker symbol",
  "inputSchema": {
    "json": {
      "type": "object",
      "properties": {
        "ticker": {"type": "string", "description": "Stock ticker symbol"},
        "currency": {"type": "string", "description": "Currency code", "enum": ["USD","EUR","GBP"]}
      },
      "required": ["ticker"]
    }
  }
}

Estimated token cost: 80–150 input tokens per tool for a minimal definition like this.

A medium-complexity tool (e.g., a database query tool with 6-8 parameters, nested schema, enum values, long description): Estimated: 300–600 tokens.

A complex tool (e.g., code execution with nested object parameters, 15+ fields, extensive description to guide the model): Estimated: 800–1,500 tokens.

At scale:

  • 10 minimal tools: ~1,000–1,500 input tokens per request
  • 20 medium tools: ~6,000–12,000 input tokens per request
  • 50 mixed tools (realistic agent tool catalog): 5,000–15,000 input tokens per request

At Claude Sonnet pricing ($3.00/MTok input), 50 tools averaging 250 tokens each = 12,500 tool tokens per request:

  • 1,000 requests/day = 12.5M tool tokens = $37.50/day in tool overhead alone
  • 100,000 requests/day = $3,750/day in tool overhead

This cost is present whether you use Converse or InvokeModel — if you were already passing tool schemas via XML in Claude’s native format, you were paying the same. The difference is that Converse makes this cost opaque (tool schemas disappear into toolConfig) whereas in InvokeModel you saw them in your prompt string.

Token-efficient tool use (available on Claude models via Bedrock): Pass anthropic_beta: ["token-efficient-tools-2025-02-19"] in the additionalModelRequestFields. This compresses the tool schema representation in the model context, yielding approximately 13% average reduction in tool-related input tokens. For a 50-tool catalog at 12,500 tokens, this saves ~1,625 tokens per request — meaningful at scale but not transformative for catalogs under 20 tools.

Important: AWS Bedrock’s CountTokens API does not currently include tool schemas in its token count. When you call CountTokens with a toolConfig, the tool definitions are not counted in the returned value. You must estimate tool token overhead separately using the formulas above or by comparing inputTokens in actual inference responses.

Tool Result Tokens

When a tool call completes and you return the result as a toolResult content block, that result is counted as input tokens in the next conversational turn. This is identical to InvokeModel behavior — tool outputs flowing back to the model have always been input tokens.

For structured tool results (JSON objects with multiple fields), token costs can be significant:

  • Simple result (a single string or number): 10–50 tokens
  • Medium result (a JSON object with 5-10 fields): 100–400 tokens
  • Large result (a list of records, a SQL query result, a document excerpt): 500–5,000+ tokens

Multi-turn tool use compounding: In a conversation where the model calls 3 tools in sequence, by turn 4 the messages array contains:

  • Original user message
  • Assistant message with tool call 1 (output tokens from turn 1, now input tokens)
  • Tool result 1 (input tokens)
  • Assistant message with tool call 2
  • Tool result 2
  • Assistant message with tool call 3
  • Tool result 3
  • Assistant’s final response

The input token count at turn 4 includes the full conversation history. For a 3-tool agent loop with medium-complexity tools and results, input token consumption can reach 8,000–20,000 tokens before the model generates its final answer — on top of tool schema tokens that are injected fresh at every turn.

ConverseStream vs Converse — No Cost Difference

Streaming (ConverseStream) and non-streaming (Converse) incur identical per-token charges. The streaming interface returns tokens incrementally via server-sent events, but the billing unit is the same input/output token count. This mirrors the relationship between InvokeModelWithResponseStream and InvokeModel. Choose based on latency and UX requirements, not cost.

3. Migration from InvokeModel to Converse

Why Enterprises Migrate

The primary drivers are not cost-related:

  1. Vendor portability: A single Converse-based codebase can switch between Claude, Nova, and Llama by changing a model ID. InvokeModel code is tightly coupled to each provider’s request/response schema.
  2. Built-in tool use standardization: Converse provides a single toolConfig + toolResult pattern that works identically across all supported models. On InvokeModel, tool use required model-specific XML (Claude legacy) or JSON (Claude 3 native) or prompt engineering for models without native tool support.
  3. Guardrails integration: Converse’s guardrailConfig is the preferred path for applying Bedrock Guardrails uniformly. InvokeModel supports guardrails but requires model-specific header injection.
  4. Batch inference: As of February 2026, Bedrock batch inference supports Converse API format, enabling large-scale offline workloads using the same code as online inference.
  5. Framework alignment: LangChain, Spring AI, and the official AWS Bedrock SDK all default to Converse for new integrations.

Prompt Format Changes and Token Count Impact

The critical migration risk: Claude on InvokeModel (pre-Converse) used the Anthropic native format with XML-style tags in prompt text. Teams that used \n\nHuman: and \n\nAssistant: alternation, or injected <tool_inputs> XML blocks manually, will see token count changes when converting to Converse’s structured messages array.

Scenarios that typically reduce token count on migration:

  • Removing manual XML wrappers (<human>, <assistant>) — these added 5–20 tokens per turn
  • Replacing hand-rolled tool schema XML with Converse’s toolConfig — if the XML was verbose (>300 tokens per tool), Converse’s normalized schema format is often more compact
  • Removing prompt engineering hacks (e.g., repeating “You are an assistant” across every message to compensate for no system prompt support in legacy InvokeModel) — Converse’s first-class system array handles this cleanly

Scenarios that typically increase token count on migration:

  • Adding tool use via Converse when previously you had no tool use (obvious)
  • Moving from a hand-crafted, minimally verbose prompt to Converse’s structured content blocks — the type: "text" wrapper on every content block adds overhead (mitigated by the SDK, not visible in billing, but worth noting)
  • Using Converse’s document blocks where you previously passed document text inline as a string — document blocks trigger Bedrock’s document processing pipeline, which may split content differently than a manual string paste

Net effect: For workloads migrating from InvokeModel with Anthropic native format to Converse with tools, a 0–10% increase in input tokens is typical. For workloads migrating without adding tool use, changes are negligible (within ±2%). Use CountTokens on representative prompts in both formats to validate before migration.

Step-by-Step Migration Checklist

  1. Inventory your InvokeModel callers: Search CloudTrail for eventName = InvokeModel filtered to Bedrock. Identify unique model IDs, request patterns, and call volumes.
  2. Classify by complexity:
    • Simple text → text: trivial Converse migration, no token risk
    • Text with injected tool XML → Converse tool use: measure token diff before/after
    • Multimodal with base64 images: map to Converse imageBlock, verify token formula (see Section 5)
    • Streaming: swap InvokeModelWithResponseStream for ConverseStream, identical semantics
  3. Run CountTokens comparison: For 20–50 representative prompts, compare token counts under InvokeModel native format vs Converse messages format. Flag any request that shows >15% increase.
  4. Test tool schema token cost: If using tools, measure inputTokens in the Converse response’s usage field for tool-heavy requests. Adjust max_tokens headroom accordingly.
  5. Update CloudWatch alarms: After migration, InputTokenCount metric covers Converse calls identically to InvokeModel calls. No alarm changes needed.
  6. Validate CUR attribution: The line_item_usage_type format does not encode the API used (Converse vs InvokeModel). See Section 6 for how to distinguish post-migration.

4. Tool Use Cost Patterns in Converse

The Tool Catalog Size Problem

Enterprise agents often expose large tool catalogs to accommodate diverse user intents. A customer service agent might register 30–60 tools covering account lookup, order management, billing, shipping, and escalation flows. Every single inference request injects the full tool catalog as input tokens, regardless of which tools are actually needed for that particular query.

Token cost at catalog scale (estimated, Claude Sonnet 4.6 pricing as of June 2026, $3.00/MTok input):

Catalog Size Avg Tokens/Tool Total Tool Overhead Cost per 1K requests
10 tools 150 1,500 $0.0045
20 tools 200 4,000 $0.012
50 tools 250 12,500 $0.0375
100 tools 250 25,000 $0.075

At 100,000 requests/day with a 50-tool catalog: $3,750/day = ~$112,500/month in tool overhead. This is a real line item for high-volume agents.

Tool Pruning: Dynamic Tool Selection

The most effective cost reduction strategy is to avoid sending the full catalog. Instead:

  1. Intent classification first pass: Run a lightweight, cheap model (Nova Lite, Haiku) against the user’s query to identify the intent category. Based on intent, load only the 5–10 relevant tools for the main inference call.
  2. Tool routing via embeddings: Pre-embed tool descriptions, embed the user query at request time, retrieve top-K tools via cosine similarity. Send only retrieved tools to Converse.
  3. Tiered tool registration: Separate tools into “always-on” (authentication, session management, ~5 tools) and “domain-specific” groups. Route to domain-specific groups based on conversation context.

Expected savings from tool pruning: Reducing from 50 tools to 10 for 80% of requests cuts tool overhead by ~80% on those requests. If the intent classification call costs 500 input + 100 output tokens, and the main call saves 10,000 tool tokens, the net savings per request is ~9,400 tokens — a >90% tool overhead reduction.

Tool Description Compression

Tool descriptions are the largest token consumer within a tool definition. Engineering norms often lead to verbose descriptions written for developer clarity rather than model efficiency. Strategies to reduce description token cost:

  • Remove redundant information: The model knows common programming patterns. “Returns a JSON object” is usually unnecessary.
  • Use imperative format: “Get current stock price for ticker.” (6 tokens) vs “This function retrieves the current stock price for the given ticker symbol from our pricing service.” (21 tokens).
  • Trim enum value documentation: Instead of describing each enum value in the description, list only the enum values themselves and trust the model to interpret them.
  • Test with CountTokens: Use CountTokens (noting tool schemas are not counted) + a manual inference call to measure description token cost per tool.

A 30% reduction in average description length across a 50-tool catalog saves ~1,800–2,500 tokens per request. At 100K requests/day with Claude Sonnet pricing: ~$540–$750/day recovered.

Prompt Caching for Tool Schemas

Bedrock prompt caching (available for Claude models) allows you to cache the tool schema portion of your input. Since tool schemas rarely change between requests in a session, this is a high-ROI caching target.

Spring AI’s BedrockCacheStrategy.TOOLS_ONLY option places cache breakpoints at the end of the toolConfig block. Cached tokens are billed at ~90% discount (cache read tokens vs. input tokens). For a 12,500-token tool catalog:

  • Without caching: 12,500 tokens × $3.00/MTok = $0.0375 per request
  • With caching (after first request): 12,500 tokens × $0.30/MTok = $0.00375 per request
  • 10x cost reduction on tool schema tokens after the first request in a cache-eligible session

Cache TTL on Bedrock is 5 minutes (configurable in some configurations). For interactive agentic sessions with multiple turns, caching tool schemas is essentially free optimization.

5. Document and Vision Costs in Converse

Document Block Type

The Converse API supports a documentBlock content type that allows passing documents (PDF, HTML, plain text, Word, and other formats) as structured blocks rather than manually extracting and inserting text.

How Bedrock counts PDF tokens:

Bedrock’s PDF processing runs a two-track pipeline for Claude models:

  • Text extraction track: OCR/parsed text is tokenized as a text block. A dense text-only PDF page averages ~250–400 tokens.
  • Visual track (Claude PDF support): Pages are rendered as images and processed through the vision pipeline. A single page via visual processing costs approximately 2,000–2,500 tokens (equivalent to a large image tile).

For a 3-page document:

  • Text-only extraction: ~1,000 tokens total
  • Visual-mode PDF processing: ~7,000 tokens total

The default mode for documentBlock in Converse is text extraction. Visual PDF mode must be explicitly enabled and is only supported on Claude models with vision capability. Choose the mode that matches your use case — visual mode is needed for charts, diagrams, and tables that lose meaning in text form; text-only suffices for prose documents.

There is no cost difference between passing a document via documentBlock in Converse vs. passing the extracted text via InvokeModel’s messages array — assuming the same content ends up as tokens. The block type is a formatting convenience; billing is on token count.

Image Tokens: Claude Vision Formula

Claude’s vision token cost formula applies equally to images passed via Converse imageBlock or InvokeModel native image content blocks. The formula:

For images processed at their natural size (up to maximum dimensions):

  • Images are tiled into 512×512 px tiles
  • Each tile costs ~1,601 tokens (the actual model cost includes base overhead + tile tokens)
  • Minimum cost: 1 tile = 1,601 tokens (for images ≤512×512)
  • A 1024×1024 image: 4 tiles = ~6,404 tokens
  • A 1920×1080 image: ~6 tiles = ~9,606 tokens

At Claude Sonnet 4.6 input pricing ($3.00/MTok):

  • Single 1024×1024 image per request: $0.019
  • 100K requests/day with one image each: $1,920/day

For cost-sensitive vision workloads:

  • Resize images to ≤512×512 before sending if detail requirements allow. This reduces from 4+ tiles to 1 tile — a 4x token reduction.
  • Use low detail mode if the model supports explicit detail settings (Claude Converse passes images at the model’s native resolution; resize at the application layer).
  • For PDF visual mode, prompt caching on repeated document queries drops vision token costs dramatically.

Converse vs. InvokeModel for multimodal: No cost difference. The imageBlock wrapper in Converse and the image block in Claude’s InvokeModel native format both route to the same vision tokenization. The tile formula is model-specific, not API-specific.

6. CUR Representation and CloudTrail Differences

CUR: No API-Level Distinction

The line_item_usage_type field in AWS Cost and Usage Report (CUR 2.0) does not differentiate between Converse and InvokeModel calls. The format is:

{region}-{model}-{token-type}[-{tier}][-cross-region-global]

Examples:

  • USE1-Claude4.6Sonnet-input-tokens — input tokens, Claude Sonnet 4.6, standard tier, us-east-1
  • USE1-Claude4.6Sonnet-output-tokens — output tokens
  • USE1-Claude4.6Sonnet-cache-read-input-token-count — cache read tokens
  • USE1-Claude4.6Sonnet-cache-write-input-token-count — cache write tokens

There is no converse-input-tokens vs invoke-input-tokens distinction in CUR. All inference against a model is aggregated by token type. This means:

  • You cannot split Converse vs InvokeModel spend in a CUR query without joining to an external source
  • Pre- and post-migration cost comparison requires correlating CloudTrail event timestamps with CUR aggregation windows
  • The recommended approach for attributing costs by workload (not by API) is application inference profiles or IAM principal tagging — neither of which tracks API surface

CloudTrail: Clear eventName Distinction

CloudTrail logs InvokeModel, InvokeModelWithResponseStream, Converse, and ConverseStream as distinct eventName values in management events. These are logged by default (no trail configuration needed for management events).

CloudTrail log entry shapes:

// InvokeModel call
{
  "eventName": "InvokeModel",
  "requestParameters": {
    "modelId": "anthropic.claude-sonnet-4-6-20251117-v1:0"
  }
}

// Converse call
{
  "eventName": "Converse",
  "requestParameters": {
    "modelId": "anthropic.claude-sonnet-4-6-20251117-v1:0"
  }
}

Note: CloudTrail logs do not include token counts or cost data for runtime operations. Token counts appear in:

  1. The inference response usage field (real-time, per-request)
  2. CloudWatch InputTokenCount / OutputTokenCount / CacheReadInputTokens / CacheWriteInputTokens metrics (aggregated)
  3. Model invocation logs (if enabled via PutModelInvocationLoggingConfiguration) — these include per-request token counts and the full request/response body

To distinguish Converse vs InvokeModel in spend analysis:

  1. Enable model invocation logging to CloudWatch Logs or S3
  2. Join invocation log records (which include requestId and apiName) to CUR at the model and time-window grain
  3. Use apiName in invocation logs to split Converse vs InvokeModel token volumes
  4. Multiply by the model’s per-token rate from the CUR line items

Tool Result Token Tracking in CloudWatch

CloudWatch’s InputTokenCount metric includes tool result tokens — they are part of the input context on the subsequent turn. There is no separate ToolResultTokenCount metric. To understand tool result token overhead in an agentic loop:

  1. Compare InputTokenCount across turns — the delta between turn N and turn N+1 equals the assistant’s tool call output tokens (now context) plus the tool result content tokens you injected
  2. Use model_invocation_logs fields inputTokenCount per turn to reconstruct per-turn token accumulation
  3. For budget alerts, set CloudWatch alarms on InputTokenCount per model per hour with a threshold based on expected non-tool baseline

7. LiteLLM’s Converse vs InvokeModel Routing

Default Routing Behavior

LiteLLM’s routing behavior for Bedrock has evolved across versions and depends on the model prefix:

  • bedrock/<model-id> — LiteLLM’s automatic routing. For most Claude and Nova models, LiteLLM defaults to the Converse API (as of LiteLLM v1.55+). For models that don’t support Converse (embeddings, older legacy Claude v1/v2), it falls back to InvokeModel.
  • bedrock/converse/<model-id> — Forces Converse explicitly
  • bedrock/invoke/<model-id> — Forces InvokeModel explicitly
  • bedrock/llama/<model-id> — Llama-specific route (uses InvokeModel with Llama prompt format)
  • bedrock/deepseek_r1/<model-id> — DeepSeek R1 specific route

Practical guidance: For Claude Code traffic or any integration requiring maximum reliability, bedrock/invoke/ was historically the smoother path in LiteLLM due to fewer edge cases in the Converse translation layer. The LiteLLM team has steadily closed the Converse feature gap through 2025 and the bedrock/converse/ path is production-stable as of early 2026.

When LiteLLM Forces Converse

Two cases force Converse regardless of prefix:

  1. Application Inference Profile ARNs: When model is an ARN (arn:aws:bedrock:...), the bedrock/ prefix cannot parse it correctly. Use bedrock/converse/<arn> explicitly.
  2. Streaming tool use: Some streaming tool use flows are better supported on the Converse path in LiteLLM due to how the streaming event accumulator works.

Known Issues at the LiteLLM/Converse Boundary

Issues observed through mid-2026:

  1. /messages endpoint with bedrock/converse/: A regression in LiteLLM circa August 2025 caused the /messages passthrough endpoint to fail when using the bedrock/converse/ route prefix. Fix was merged in PR #13627. If running LiteLLM <1.52, pin to bedrock/invoke/ or upgrade.

  2. Application Inference Profile ARN routing: ARNs using the bedrock/ route (without explicit converse/) fail with “Unknown provider” in LiteLLM versions prior to the fix in issue #18258. Workaround: always use bedrock/converse/<arn> for ARN-based model identifiers.

  3. Token count reporting discrepancy: LiteLLM’s internal token counting (used for budget enforcement) uses its own tokenizer estimate before the Bedrock response returns actual usage counts. For Converse calls with large tool schemas, LiteLLM’s pre-call estimate significantly underestimates actual input tokens (because tool schema tokens aren’t pre-estimated). This can cause budget enforcement gaps in LiteLLM’s proxy spend tracking. Use LiteLLM’s usage_tracking: true mode to pull actual token counts from the Bedrock response rather than relying on pre-call estimates.

  4. Structured outputs: Native Bedrock structured outputs via outputConfig.textFormat (issue #21208) were not supported in LiteLLM as of mid-2026. Use Converse directly (boto3) for structured output requirements.

Latency Considerations

No published head-to-head benchmark exists comparing Converse vs InvokeModel latency in isolation. Bedrock’s request normalization layer adds a small overhead when translating Converse’s unified format to model-specific internal formats, but this occurs pre-GPU and is sub-millisecond. Time-to-first-token (TTFT) is dominated by the model’s inference latency, not the API normalization overhead.

Practical latency guidance:

  • For streaming workloads, TTFT measurement uses CloudWatch’s TimeToFirstToken metric, available on both ConverseStream and InvokeModelWithResponseStream
  • First-request latency (cold connection) is driven by TCP/TLS handshake and CRIS routing (~50–100ms overhead) — identical for both APIs
  • Connection reuse via HTTP/2 persistent connections reduces per-request overhead by ~25% — applicable to both APIs equally

8. Synthesis: When to Use Each in 2026 Enterprise Deployments

Use Converse When

  • Building multi-model systems: Swapping between Claude, Nova, and Llama without code changes
  • Implementing tool use / function calling: toolConfig is cleaner than hand-rolling tool XML, and token-efficient tools beta further reduces overhead
  • Using managed frameworks: LangChain (aws module), Spring AI, Amazon Bedrock Agents, and most enterprise LLM frameworks default to Converse — align with the ecosystem
  • Deploying with Guardrails: Converse’s guardrailConfig is the recommended Guardrails integration path
  • Batch inference workloads: Bedrock batch inference supports Converse format as of February 2026
  • New greenfield builds: There is no reason to start with InvokeModel for text generation today

Use InvokeModel When

  • Embedding models: InvokeModel is the only option for Titan Embeddings, Cohere Embed, etc.
  • Image generation: Stable Diffusion and Titan Image Generator are InvokeModel-only
  • Payload size > 3.75 MB: Converse’s payload limit is ~25% smaller than InvokeModel’s ~5 MB limit; high-resolution image batches or very long documents may require InvokeModel
  • Legacy compatibility: Existing production systems with stable InvokeModel code and no multi-model portability requirement have no cost reason to migrate

Cost Optimization Priority Stack for Converse Deployments

  1. Tool catalog pruning (highest ROI): Dynamic tool selection reduces tool token overhead by 60–90% for multi-tool agents
  2. Prompt caching on tool schemas: Applies after pruning; 10x cost reduction on tool tokens for cache-eligible sessions
  3. Token-efficient tools beta (Claude only): ~13% reduction on tool schema tokens; enable via additionalModelRequestFields
  4. Image resizing before send: Resize to ≤512×512 for vision tasks where full detail is not needed; 4x tile reduction
  5. Document block mode selection: Use text extraction mode for prose documents; visual mode only when diagrams are essential
  6. max_tokens tuning: Reducing max_tokens to match expected output size reduces initial TPM quota deduction (billing is on actual tokens, but quota headroom matters for concurrency)

Cost Estimation Template

For a Converse-based agent with tools:

Total input tokens per request =
  system_prompt_tokens
  + message_history_tokens (grows per turn)
  + tool_schema_tokens (full catalog, every turn unless cached)
  + tool_result_tokens (if prior tool calls in this turn)
  + current_user_message_tokens

Total output tokens per request =
  assistant_response_tokens
  + tool_call_tokens (if tool was invoked)

Daily cost =
  (avg_input_tokens × input_price_per_token
   + avg_output_tokens × output_price_per_token)
  × requests_per_day

For Claude Sonnet 4.6 at $3.00 input / $15.00 output per million tokens:

Example: 10-turn agentic conversation, 50 tools, no caching

  • System: 500 tokens
  • History (10 turns avg, 300 tokens/turn): 3,000 tokens
  • Tool schemas (50 tools × 250 tokens): 12,500 tokens
  • Tool results (3 tool calls × 500 tokens): 1,500 tokens
  • User message: 200 tokens
  • Total input: ~17,700 tokens → $0.0531/request
  • Output (1,500 tokens): $0.0225/request
  • Total: ~$0.076/request

With tool caching (90% discount on tool schema tokens after turn 1):

  • Tool schema tokens: 12,500 → billed at 10% = 1,250 effective
  • Input with caching: ~6,450 tokens → $0.0194/request
  • Total with caching: ~$0.042/request (45% reduction)

Sources and References