← Agent Frameworks 🕐 14 min read

Agent Frameworks

Bedrock Converse API vs InvokeModel — Cost Differences and Migration (2026)

Executive Summary · March 2026

The Converse API and InvokeModel share identical per-token billing rates.

The Converse API and InvokeModel share identical per-token billing rates. The choice between them is not a pricing decision — it is an engineering architecture decision with indirect cost consequences. Converse wins on maintainability and cross-model portability. InvokeModel retains a narrow but real advantage for model-specific inference parameters and non-text workloads. As of January 2026, AWS CUR now surfaces distinct operation strings for each API path, making the split auditable for the first time. Teams running LiteLLM against Bedrock should verify their routing: the library uses Converse by default for chat models but falls back to InvokeModel for embeddings, reranking, and image generation.

1. Billing Equivalence: Are Converse and InvokeModel Priced Identically?

Yes — the per-token price is identical. AWS prices Amazon Bedrock inference on the tokens consumed by the underlying model, not on which API surface was used to reach it. Whether a request arrives via bedrock-runtime:InvokeModel, bedrock-runtime:Converse, bedrock-runtime:ConverseStream, or bedrock-runtime:InvokeModelWithResponseStream, the line-item charge is the same: input tokens at the model’s input rate, output tokens at the model’s output rate.

AWS has confirmed this parity in two ways:

The CountTokens API, added to Bedrock Runtime in 2025, explicitly states that “the token count returned will match the token count that would be charged if the same input were sent to the model in an InvokeModel or Converse request.” The parity is documented in the API contract, not inferred.
Flex inference pricing — a 50% discount tier for latency-tolerant workloads — is available across both Converse and InvokeModel. Discount eligibility does not depend on which API surface is used.

The indirect cost vector. While the rate is identical, the total tokens consumed per call can differ between the two APIs depending on how tool definitions are structured and whether the caller is sending redundant model-specific boilerplate that Converse handles internally. This is the lever FinOps teams should measure, not the rate card.

Prompt caching parity. Prompt caching (GA April 2025) is supported on both API surfaces via CachePointBlock objects in the Converse API and via model-native cache control fields in InvokeModel for Anthropic models. Cache hit rates reduce effective cost by up to 90% and latency by up to 85%. The Converse API surfaces cache checkpoints in the messages, system, and tools fields. As of January 2026, supported models for Bedrock prompt caching include Claude Sonnet 3.5, Sonnet 3.7, and Amazon Nova Pro/Lite/Micro. Claude Haiku 4.5, Claude Sonnet 4.5, and Claude Opus 4.5 support a 1-hour TTL; all other supported models use a 5-minute TTL.

2. Tool Use Token Overhead: Converse vs InvokeModel

This is the one area where the two APIs diverge in effective token consumption, and the difference is often misunderstood.

How InvokeModel handles tools

With InvokeModel, the caller constructs the raw model-native request body. For Anthropic Claude on Bedrock, that means the caller writes the tools array in Anthropic’s JSON schema format directly, exactly as they would hit the Anthropic API. No translation layer sits between the caller’s JSON and what the model tokenizes.

How Converse handles tools

Converse accepts tool definitions in a normalized schema (toolConfig.tools[].toolSpec) and then translates them into the model’s native format before tokenization. The caller writes the tool spec once; Bedrock translates it per model internally.

The key implication: the token count for a tool definition is determined by what the model ultimately tokenizes, not by the API’s wrapper schema. Because Converse translates to the native format, the tokenized representation — and therefore the billed token count — is equivalent to what InvokeModel would send for the same tool.

Where overhead actually appears

The real overhead risk with tool use is not API-surface-dependent. It is tool count and schema verbosity. AWS’s own internal measurement showed 134K tokens consumed by tool definitions alone before optimization. A single MCP server with a modest tool set can contribute 2,000–5,000 tokens per turn. At scale with three or four tool servers, overhead reaches 10,000–18,000 tokens per message.

Converse-specific mitigation: Spring AI’s BedrockCacheStrategy enum exposes TOOLS_ONLY and SYSTEM_AND_TOOLS cache placement options specifically for Converse. Because tool definitions are stable across turns in an agentic loop, caching them cuts the amortized input cost per turn proportional to how many turns share the same tool set. A 10-turn agent loop with 8,000 tokens of tool definitions saves 72,000 input tokens if the first turn populates the cache and the next nine hit it — at a 90% cost reduction on that fraction of input.

Bottom line on tool overhead: Converse does not add token overhead relative to InvokeModel for the same effective prompt. The Converse API’s normalization layer does not inject additional tokens into the model’s context. However, Converse’s caching integration is easier to activate and reason about for stable tool schemas, creating a practical cost advantage for agentic workloads with repetitive tool definitions.

3. CUR Appearance: Do Converse and InvokeModel Show Differently?

Yes, as of January 15, 2026. AWS Data Exports added granular operation visibility for Amazon Bedrock model usage. Before this change, both API surfaces produced generic Usage strings in the operation column, making it impossible to audit which code paths were responsible for which costs.

Current operation strings (post-January 2026)

API call	CUR `line_item_operation`	FOCUS `x_Operation`
`InvokeModel` (non-streaming)	`InvokeModelInference`	`InvokeModelInference`
`InvokeModelWithResponseStream`	`InvokeModelStreamingInference`	`InvokeModelStreamingInference`
`Converse` (non-streaming)	`ConverseInference`	`ConverseInference`
`ConverseStream`	`ConverseStreamingInference`	`ConverseStreamingInference`

These strings appear in:

Legacy CUR: line_item_operation column
CUR 2.0: line_item_operation column
FOCUS Data Exports: x_Operation column
AWS Cost Explorer API: Operation dimension

Practical FinOps use cases unlocked

Attribution by API surface. Teams can now write Cost Explorer queries or Athena queries against CUR 2.0 to split spend by API surface. A query grouping by line_item_operation across a Bedrock-heavy account will reveal whether agentic workloads (typically ConverseInference or ConverseStreamingInference) are dominating versus batch inference or direct model calls.

Detecting unintended InvokeModel usage. In organizations that have mandated Converse for all new development, a spike in InvokeModelInference becomes an alert signal — a team may have shipped code that bypasses the standard client.

Combining with Application Inference Profiles. Application inference profiles (AIPs) let teams tag Bedrock spend by application or team. When AIPs are combined with the new operation strings, a single CUR query can answer: “How much did team A’s agentic workflow cost via streaming Converse calls last week?”

4. ConverseStream vs InvokeModelWithResponseStream: Cost Comparison

No cost difference. Both streaming variants bill identically: input tokens at the model’s input rate, output tokens at the model’s output rate. The streaming transport does not affect token billing. Output tokens cost approximately 3–4x more than input tokens across most Bedrock foundation models because generation requires more compute than prefill.

Behavioral differences that affect total cost indirectly

Latency to first token (TTFT). AWS added TTFT observability to Bedrock CloudWatch in March 2026. ConverseStream and InvokeModelWithResponseStream exhibit comparable TTFT for the same model and prompt. The difference is negligible for cost but matters for perceived latency in interactive applications.

Stream event structure. ConverseStream returns typed event blocks (contentBlockDelta, messageStop, metadata) that include token usage in the metadata block at stream end. InvokeModelWithResponseStream returns model-native chunks that require parsing per model format. Neither format affects billing, but ConverseStream’s metadata.usage field makes per-request token accounting easier to implement correctly without per-model parsing logic.

Error handling overhead. A coding error in InvokeModelWithResponseStream response parsing (e.g., incorrectly handling Claude’s streaming delta format vs Llama’s format) can cause silent token waste — the request completes and bills, but the application discards or re-requests. ConverseStream’s normalized event format reduces this class of bug.

5. Cross-Model Portability Value: Quantifying Engineering Cost Savings

The Converse API’s primary value proposition is write-once, run-on-any-supported-model code. This has measurable engineering cost implications.

The multi-model maintenance problem

Without Converse, each model family requires a distinct request serializer and response deserializer:

Anthropic Claude: messages array with role/content structure, tool blocks in content, anthropic_version header
Amazon Nova: similar to Claude but with inferenceConfig instead of max_tokens
Meta Llama: prompt string with special tokens (<|begin_of_text|>, <|eot_id|>), no native tool schema
Cohere Command R: chat_history array with distinct turn structure
AI21 Jamba: messages array with provider-specific parameter names

A team supporting three active models with InvokeModel must maintain three serializers, three deserializers, and three sets of integration tests. When AWS adds a new model, onboarding requires net-new format work.

Converse’s normalization guarantee

Converse normalizes all of the above into a single request schema. The messages array, the system field, the toolConfig block, and the inferenceConfig object work identically regardless of the underlying model. Switching from Claude Sonnet 3.7 to Amazon Nova Pro requires changing one string (the model ID) rather than rewriting a serializer.

Engineering cost estimate

A conservative estimate for a team currently maintaining InvokeModel integrations for three models:

Initial serializer/deserializer per model: 2–4 days of engineering
Regression test suite per model: 1–2 days
Per-model maintenance per year (schema drift, new parameters, deprecation handling): 3–5 days per model

Three models = 9–15 days of annual maintenance. Migrating to Converse collapses this to one integration. At a blended engineering rate of $150/hour, the maintenance reduction is worth $27,000–$45,000 per year for a three-model stack — before counting onboarding time for new models.

Limits of portability

Converse’s normalization covers the intersection of capabilities across supported models. Model-specific behaviors that fall outside the normalized surface — specific inference parameters, provider-native extensions, non-text modalities — require InvokeModel or the additionalModelRequestFields escape hatch. The portability guarantee applies to the common case; edge cases still require per-model handling.

6. When InvokeModel Is Still Required

Despite Converse’s advantages, InvokeModel remains the only option for several categories of work.

Non-text modalities

The Converse API supports text generation models only. The following workload types require InvokeModel:

Workload	Example models	API required
Text embeddings	Amazon Titan Embeddings, Cohere Embed	InvokeModel
Image generation	Stable Diffusion, Amazon Nova Canvas	InvokeModel
Image-to-image	Stable Diffusion XL	InvokeModel
Reranking	Cohere Rerank	InvokeModel
Video generation	Amazon Nova Reel	InvokeModel

Any pipeline that mixes chat with embeddings (e.g., a RAG system) must maintain both API surfaces regardless of Converse adoption for the chat layer.

Model-specific inference parameters not exposed in Converse

Converse exposes a normalized inferenceConfig with a small set of universally supported parameters: maxTokens, temperature, topP, stopSequences. Parameters that exist in a model’s native API but fall outside this set require the additionalModelRequestFields escape hatch in Converse, or InvokeModel directly.

Examples of parameters requiring the escape hatch or InvokeModel:

Anthropic Claude: top_k, thinking (extended thinking mode), betas (feature flags)
Amazon Nova: inferenceConfig.topK
Meta Llama: top_k, seed (for reproducible outputs)
Cohere Command R: k, frequency_penalty, presence_penalty, seed

The additionalModelRequestFields field in Converse passes these through, but it bypasses Converse’s normalization for those fields — the caller must know the model-native field name and type. This makes additionalModelRequestFields a partial escape hatch, not a replacement for InvokeModel.

Models not yet on the Converse supported model list

AWS expands Converse compatibility over time, but not all Bedrock foundation models are Converse-compatible at launch. When a new provider or model family joins Bedrock, it may only be available via InvokeModel for the initial period. Teams should check the Bedrock “Supported models and model features” page before assuming Converse coverage.

Batch inference format choice

As of February 2026, Bedrock batch inference supports both InvokeModel format and Converse format as selectable invocation types. Teams running batch jobs that predate February 2026 will have used InvokeModel format; they can migrate to Converse format without changing billing outcomes, but existing tooling that parses batch output will need to handle the different response structure.

7. Migration Checklist: InvokeModel to Converse

This checklist covers a team migrating an existing InvokeModel-based integration to Converse for a supported text generation model.

Pre-migration assessment

[ ] Confirm the target model is on the Converse supported model list (conversation-inference-supported-models-features in Bedrock docs)
[ ] Inventory all inference parameters in use — identify any that fall outside Converse’s inferenceConfig set
[ ] Identify any non-text API calls in the same codebase (embeddings, image, rerank) — these will remain on InvokeModel
[ ] Confirm IAM permissions: bedrock:InvokeModel covers both Converse and InvokeModel; bedrock:InvokeModelWithResponseStream covers both streaming variants — no new permissions required
[ ] Check if Application Inference Profiles are in use; if so, confirm AIP ARN routing works with Converse (known LiteLLM issue: explicit bedrock/converse/ route prefix required for AIP ARNs)

Request schema migration

[ ] Replace model-native messages array with Converse messages array (role: user/assistant, content: array of ContentBlock)
[ ] Move system prompt from model-native location (e.g., Anthropic’s system field, Llama’s [INST] prefix) to Converse’s top-level system array field
[ ] Convert tool definitions from model-native schema to Converse toolConfig.tools[].toolSpec with name, description, inputSchema.json
[ ] Move max_tokens / maxTokens / max_gen_len (model-dependent naming) into inferenceConfig.maxTokens
[ ] Move temperature, top_p into inferenceConfig.temperature, inferenceConfig.topP
[ ] Map remaining model-specific parameters to additionalModelRequestFields using native field names
[ ] Remove anthropic_version header or equivalent provider versioning fields (Converse handles versioning internally)

Response parsing migration

[ ] Replace model-native response parsing with Converse output.message.content array parsing
[ ] Handle ContentBlock variants: text, toolUse, toolResult, image, document
[ ] Replace model-native token usage fields with Converse usage.inputTokens / usage.outputTokens
[ ] Replace model-native stop reason field with Converse stopReason: end_turn, tool_use, max_tokens, stop_sequence, content_filtered
[ ] For streaming: replace model-native chunk parsing with ConverseStream event handling (contentBlockStart, contentBlockDelta, contentBlockStop, messageStop, metadata)

Prompt caching integration (if applicable)

[ ] Insert CachePointBlock objects at stable context boundaries (system prompt tail, tool definitions tail, long static context tail)
[ ] Verify cache hit via usage.cacheReadInputTokenCount and usage.cacheWriteInputTokenCount in the response usage block
[ ] Set cache TTL via model-appropriate configuration (1-hour TTL available for Claude 4.x Haiku, Sonnet, Opus; 5-minute TTL for others)

CUR and observability

[ ] Update Cost Explorer dashboards to split on line_item_operation: expect ConverseInference and ConverseStreamingInference post-migration (replacing InvokeModelInference / InvokeModelStreamingInference)
[ ] Update alerting thresholds — operation string changes can break existing cost anomaly detectors that filter by operation value
[ ] Tag requests with requestMetadata if using per-request cost attribution (supported in Converse as of late 2024)

Validation

[ ] Run token count parity check using CountTokens API: send the same logical prompt via both APIs and confirm matching counts
[ ] Compare response quality on a representative sample (Converse’s normalization should be transparent, but verify for models with complex native schemas)
[ ] Confirm prompt cache hit rates match expectations on first few production requests

8. LiteLLM: Converse or InvokeModel Under the Hood?

LiteLLM routes Bedrock requests differently depending on the model type and the prefix used in the model string.

Default routing behavior

For chat/text generation models: LiteLLM defaults to the Converse API when the model string uses the bedrock/ prefix for supported chat models. This change was implemented in mid-2024 when Converse reached feature parity for most text generation use cases. LiteLLM supports all four Bedrock runtime endpoints (/invoke, /invoke-with-response-stream, /converse, /converse-stream) but uses Converse as the default path for chat models.

For non-text models: Embeddings, image generation, and reranking models route to InvokeModel automatically because Converse does not support these modalities.

Explicit routing control

LiteLLM exposes an explicit Converse route via the bedrock/converse/ prefix:

model = "bedrock/converse/anthropic.claude-3-5-sonnet-20241022-v2:0"

This is required for Application Inference Profile ARNs. A known LiteLLM bug (open as of December 2025) causes AIP ARNs to fail with “Unknown provider” when using the standard bedrock/ prefix — the explicit bedrock/converse/ prefix is required as a workaround.

Cost implications of LiteLLM routing

Because Converse and InvokeModel are priced identically, LiteLLM’s choice of backend has no direct cost impact. The indirect implications are:

CUR audit trail. LiteLLM’s Converse-default behavior means teams using LiteLLM will see ConverseInference and ConverseStreamingInference dominating their CUR post-January 2026. Teams that assumed LiteLLM used InvokeModel and built CUR dashboards around InvokeModelInference will see those dashboards go dark.
Prompt caching. LiteLLM passes through Bedrock’s CachePointBlock mechanism when cache control fields are set in the request. Because LiteLLM routes chat to Converse by default, prompt cache configuration follows Converse’s schema (not InvokeModel’s model-native schema). Teams migrating from direct InvokeModel with Anthropic cache control headers to LiteLLM will need to update cache control placement to Converse-style CachePointBlock positions.
Batch inference. LiteLLM batch support for Bedrock (as of February 2026) has a known bug where Converse API models (Nova, Llama) receive incorrect modelInput format — image_url fields are passed through as-is and inferenceConfig is not applied. Teams using LiteLLM for Bedrock batch inference should test carefully and pin to a patched version.

LiteLLM pass-through mode

LiteLLM offers a boto3 SDK pass-through mode (bedrock (boto3) SDK) that bypasses LiteLLM’s routing and translation entirely, sending raw requests to whichever Bedrock endpoint the caller specifies. This is the escape hatch for teams that need direct control over API surface selection while still using LiteLLM’s proxy infrastructure for auth, rate limiting, and logging.

Reference: Feature Comparison Table

Dimension	Converse API	InvokeModel
Per-token price	Identical	Identical
Text generation	Yes (all supported models)	Yes (all models)
Embeddings	No	Yes
Image generation	No	Yes
Reranking	No	Yes
Video generation	No	Yes
Tool use	Yes (normalized schema)	Yes (model-native schema)
Structured output	Yes (via tool schema)	Yes (model-native)
Prompt caching	Yes (`CachePointBlock`)	Yes (model-native fields)
Streaming	`ConverseStream`	`InvokeModelWithResponseStream`
Batch inference	Yes (as of Feb 2026)	Yes
Model-specific params	Partial (`additionalModelRequestFields`)	Full
Cross-model portability	Yes (single schema)	No (per-model schemas)
CUR operation string	`ConverseInference` / `ConverseStreamingInference`	`InvokeModelInference` / `InvokeModelStreamingInference`
IAM permissions	`bedrock:InvokeModel` / `bedrock:InvokeModelWithResponseStream`	Same
LiteLLM default for chat	Yes	No
LiteLLM default for embeddings/image	No	Yes