← Agent Frameworks 🕐 18 min read
Agent Frameworks

Bedrock RAG Embedding and Data Pipeline Cost Patterns (2026)

1. Embedding Model Pricing: Titan v2 vs Cohere Embed · March 2026

This note covers end-to-end cost modeling for production RAG workloads on AWS Bedrock as of June 2026.

This note covers end-to-end cost modeling for production RAG workloads on AWS Bedrock as of June 2026. It covers embedding model pricing, vector store selection, ingestion pipeline mechanics, chunking strategy cost impact, CUR attribution, and total pipeline TCO. The June 16, 2026 S3 Vectors query pricing reduction (up to 80% for large indexes) reshapes the vector store decision significantly.

Titan Embeddings v2

Amazon Titan Text Embeddings V2 (amazon.titan-embed-text-v2:0) is the default model for Bedrock Knowledge Bases and the lowest-cost option for English workloads.

Dimension Price (per 1M input tokens) Notes
256 $0.020 4× storage reduction vs 1024
512 $0.020 Balanced cost/recall
1024 $0.020 Default; best recall

Key facts:

  • Flat per-token price regardless of output dimension ($0.020/1M input tokens = $0.00002/1K tokens)
  • 8,192-token context window
  • No output token charge (embedding models only bill input)
  • English-first; multilingual quality degrades past 50-token chunks in non-Latin scripts
  • No batch discount via on-demand API; batch inference (async) uses standard rates

Cohere Embed v3 / v4

Cohere Embed v3 (English/multilingual, 1024-dim) is available on Bedrock at $0.10/1M tokens — 5× more expensive than Titan v2 for English text.

Cohere Embed v4 (cohere.embed-v4:0) launched on Bedrock in 2026 and is now the primary Cohere option:

Model Modality Dimensions Price (1M tokens)
Embed v3 Text only 1024 $0.10
Embed v4 Text 1536 $0.12
Embed v4 Image 1536 $0.47/1M images

When Cohere wins over Titan:

  • Multilingual corpora with non-Latin scripts (Titan v2 degrades; Cohere v3/v4 maintains quality)
  • Multimodal pipelines where image embeddings are required (Embed v4 only)
  • Workloads requiring 1536-dim vectors for fine-grained recall at very large corpora

When Titan wins:

  • English-only enterprise document corpora
  • Cost-sensitivity: Titan is 5× cheaper than Cohere v3 and 6× cheaper than Cohere v4
  • Knowledge Base default integration (no additional configuration)

Batch vs Real-Time Embedding

Bedrock does not offer a separate discounted batch embedding tier for Titan or Cohere via the standard on-demand API. Batch inference (InvokeModelWithResponseStream) uses the same token rate. True batch throughput optimization comes from:

  1. Concurrent invocations: Use Bedrock’s provisioned throughput (PT) for sustained high-volume ingestion — locks in capacity at a flat hourly rate, useful when ingesting >500GB in a single window
  2. Async ingestion jobs: Bedrock Knowledge Bases’ StartIngestionJob API handles batching internally; you pay per embedding call but don’t manage concurrency
  3. Self-managed Lambda + SQS fan-out: For custom pipelines, fan out embedding calls across Lambda functions with SQS batching; costs are identical per token but throughput scales

2. S3 Vectors: General Availability Pricing (June 16, 2026)

S3 Vectors GA launched December 2025 and expanded to 31 AWS Regions by March 2026. On June 16, 2026, AWS cut query processing charges for large indexes (>10M vectors) by up to 80%, applied automatically with no application changes required.

Pricing Components (us-east-1, June 2026)

Component Price Trigger
PUT / upload $0.20–0.22 / GB logical Per ingest
Storage (TimedStorage) $0.06–0.07 / GB-month Monthly
Query — data processed (≤100K vectors) $0.004 / TB processed Per query
Query — data processed (100K–10M vectors) $0.002 / TB Per query (tiered)
Query — data processed (>10M vectors) ~$0.0008 / TB (post-June 16 cut) Per query
Data returned Free below 500 KB/query Per query

Vector size math: A 1024-dim vector = 4 bytes × 1024 = 4.096 KB logical data, plus key and metadata overhead (~1 KB typical), so ~5.1 KB effective size per vector.

Worked example — 1M vector index, 10K queries/day:

  • Storage: 1M × 5.1 KB = 5.1 GB × $0.067 = $0.34/month
  • Query data processed: 1M vectors × 5.1 KB = 5.1 GB = 0.0051 TB; 10K queries × 0.0051 TB × $0.002/TB = $0.10/day = $3.10/month
  • PUT (one-time): 5.1 GB × $0.21 = $1.07 one-time
  • Total monthly (steady state): ~$3.44/month

Worked example — 50M vector index, 100K queries/day:

  • Storage: 50M × 5.1 KB = 255 GB × $0.067 = $17.09/month
  • Query (post-June 16, >10M tier): 255 GB = 0.255 TB; 100K queries × 0.255 TB × $0.0008 = $20.40/day = $612/month (large-scale query volume dominates)
  • Total monthly: ~$629/month

S3 Vectors Limits (GA)

  • Max vectors per index: 10B (40× increase from preview)
  • Max indexes per bucket: 10,000
  • Max metadata per vector: 4 KB
  • Supported distance metrics: cosine, euclidean, dot-product
  • No HNSW parameter tuning exposed; AWS manages index structure

What S3 Vectors Does Not Support (June 2026)

  • No BM25 / keyword search (pure vector only)
  • No real-time metadata filtering during approximate nearest neighbor (ANN) search — filtering happens post-retrieval
  • No direct integration with Bedrock Knowledge Bases as a vector store (as of June 2026; re:Post thread confirms integration is on the roadmap, not yet GA)

3. Vector Store Cost Comparison at Scale

Three primary options for Bedrock RAG workloads: S3 Vectors, OpenSearch Serverless (AOSS), and pgvector on RDS PostgreSQL.

Cost Comparison Table

Workload Scale S3 Vectors AOSS (NextGen, scale-to-zero) pgvector on RDS
100K vectors, 1K queries/day ~$0.15/mo ~$0 (scale-to-zero idle) + per-query compute ~$50–80/mo (db.t4g.medium)
1M vectors, 10K queries/day ~$3.44/mo ~$175/mo (1 OCU) ~$100–150/mo (db.r7g.large)
10M vectors, 100K queries/day ~$45/mo ~$350/mo (2 OCU, HA) ~$300–500/mo (db.r7g.4xlarge)
50M vectors, 100K queries/day ~$629/mo ~$700+/mo (3-4 OCU) ~$600–900/mo (db.r7g.8xlarge)

Notes on each:

S3 Vectors:

  • Near-zero idle cost; cost scales with query × index size
  • Best for: bursty workloads, development, cost-sensitive deployments, very large indexes with moderate query rates
  • Worst for: high-frequency queries against large indexes (query cost compounds)
  • No HA configuration needed (S3 durability: 11 nines)

AOSS NextGen (scale-to-zero, 2026):

  • Previous criticism (“$700/month minimum regardless of use”) resolved with NextGen scale-to-zero launch
  • OCU pricing: ~$0.24/OCU-hour (us-east-1); each OCU = 6 GB RAM + 120 GiB ephemeral storage
  • Production HA requires 2 OCUs minimum (~$350/month); disable redundancy for dev/test (~$175/month)
  • Supports hybrid search (BM25 + vector) natively — significant capability advantage over S3 Vectors
  • Best for: always-on production workloads with consistent query volume, hybrid search requirements, OpenSearch feature set (filters, facets, aggregations)

pgvector on RDS PostgreSQL:

  • No vector-specific surcharge; cost is the RDS instance required to hold the HNSW index in memory
  • HNSW index memory rule of thumb: ~1.3–1.5× raw vector data size; 1M × 1024-dim vectors ≈ 6.2 GB HNSW index
  • Cost cliff at ~5–10M vectors when you must move to memory-optimized instances (db.r7g class)
  • Advantage: existing PostgreSQL workloads can add vectors without a new service; familiar ops model
  • Disadvantage: CPU-bound vector search doesn’t parallelize well past ~8 vCPUs on standard RDS; Aurora pgvector with dedicated vector index on io2 improves this

Crossover Analysis

S3 Vectors beats AOSS cost below approximately 20M vectors at steady-state query volumes under 50K/day. Above that, AOSS NextGen with shared OCUs across collections becomes competitive. For very large corpora (100M+ vectors) with complex filtering, OpenSearch with GPU-accelerated vector search (2026 GA feature) substantially reduces per-query latency and cost.

4. Knowledge Base Ingestion Pipeline Cost

The Bedrock Knowledge Bases managed ingestion pipeline (StartIngestionJob) covers: S3 read → document parsing → chunking → embedding → vector store write.

Component Cost Breakdown (per GB of source documents)

Assumptions: average document = PDF, ~300 words/page, ~400 tokens/page, ~50 pages; 1 GB ≈ ~66 documents ≈ 3,300 pages ≈ 1.32M tokens (pre-chunking).

Pipeline Component Service Unit Rate Cost per GB source
S3 GET (source bucket) S3 per 1K GET $0.0004 ~$0.03
Parsing (Bedrock Data Automation, standard) BDA per page $0.010 $33.00
Parsing (default text extraction, no BDA) Free — $0.000 $0.00
Chunking (managed, built-in) Free — $0.000 $0.00
Embedding (Titan v2, 1024-dim, ~512-token chunks) Bedrock per 1M tokens $0.020 ~$0.03
Vector store write — S3 Vectors PUT S3 per GB $0.21 ~$0.05
Vector store write — AOSS AOSS OCU-hour $0.24 ~$0.10 (amortized)

Key insight on parsing cost: Bedrock Data Automation (BDA) standard output at $0.010/page is the dominant cost when enabled. A 1,000-page document costs $10.00 to parse regardless of its size in bytes. For a 1 GB corpus of scanned PDFs requiring OCR/layout extraction, BDA parsing will cost $33, while the embedding step costs only $0.03. For plain-text or pre-extracted documents, skip BDA entirely — the default text extraction is free.

Embedding cost is not the bottleneck. A 1 GB text corpus ≈ 1.32M tokens → $0.026 in Titan v2 embeddings. Even at Cohere Embed v3 rates ($0.10/1M), the same corpus costs $0.13. The parsing layer (BDA) is 100–1,000× the embedding cost for document-heavy workloads.

Total Ingestion Cost per GB (No BDA parser)

Corpus Type Titan v2 (1024-dim) Cohere v3 (1024-dim)
Plain text / pre-extracted ~$0.06 ~$0.20
With S3 Vectors write ~$0.11 ~$0.25
With AOSS write (amortized) ~$0.16 ~$0.30

Total Ingestion Cost per GB (With BDA Standard Parser)

Corpus Type Titan v2 Cohere v3
Scanned PDFs, 300-word pages ~$33.06 ~$33.20
Dense technical docs, 600-word pages ~$16.53 ~$16.67

5. Re-Ingestion Cost Modeling

Not all document updates require full re-embedding. The cost decision depends on change scope and chunking strategy.

Re-Ingestion Triggers and Scope

Change Type Recommended Action Re-embedding Scope
New document added Full document ingest 100% of new doc chunks
Entire document replaced Delete old vectors + full re-embed 100% of doc chunks
Single section updated (fixed-size chunking) Identify affected chunks by byte offset 1–5 chunks typically
Single section updated (semantic chunking) Section boundaries shift; re-embed adjacent chunks 3–10 chunks typically
Metadata-only update (title, date, author) Vector store metadata update; no re-embedding 0%
Structural reformat (same content) Skip re-embed if content hash unchanged 0%

Content-Hashing Strategy to Avoid Redundant Re-Embedding

Store SHA-256 hash of each chunk’s text alongside the vector. On update, hash new chunks before embedding; skip API call if hash matches. For a corpus where 10% of documents update monthly and 20% of chunks per document change:

Monthly re-embedding cost = corpus_tokens × 0.10 (docs updated) × 0.20 (chunks changed) × embedding_rate
Example: 500M token corpus, Titan v2
= 500M × 0.10 × 0.20 × $0.020/1M
= 10M tokens × $0.020/1M
= $0.20/month re-embedding cost

Full re-ingestion of a 500M-token corpus (e.g., quarterly refresh): $10.00 at Titan v2 rates.

Incremental Sync Architecture

Bedrock Knowledge Bases supports incremental sync via the INCREMENTAL_SYNC ingestion mode (added 2025). It reads S3 object last-modified timestamps, re-processes only changed objects, and handles deletions. This eliminates the need to custom-build change detection for most workloads. Cost impact: typically 80–95% reduction in monthly embedding spend vs. full re-ingestion.

6. Embedding Dimensionality: Cost vs Recall Tradeoffs

Titan v2 supports three output dimensions: 256, 512, and 1024. The per-token price is identical ($0.020/1M) regardless of dimension, but the downstream costs differ.

Storage and Query Cost Impact

Dimension Vector size (bytes) Storage cost per 1M vectors Query data processed per 1M vectors
256 1,024 B (1 KB) 1 GB → $0.067/month 1 GB / query
512 2,048 B (2 KB) 2 GB → $0.134/month 2 GB / query
1024 4,096 B (4 KB) 4 GB → $0.268/month 4 GB / query

Dimension choice has zero impact on embedding API cost but 4× impact on vector store storage and query costs.

Recall Quality Benchmarks

Published benchmarks for Titan v2 on MTEB (as of 2026):

Dimension NDCG@10 (avg) vs 1024 recall loss
1024 ~0.545 Baseline
512 ~0.538 -1.3%
256 ~0.521 -4.4%

These are average figures. Domain-specific corpora (legal, clinical, code) show larger recall degradation at 256 dimensions. English enterprise document corpora (policies, contracts, procedures) typically fall within 2% of baseline at 512 dimensions.

Decision Framework

  • Use 256 dimensions when: storage cost is a primary constraint, corpus is >100M vectors, recall degradation is acceptable (general FAQ / tier-1 support)
  • Use 512 dimensions when: balanced cost/quality for English enterprise corpora; 2× cheaper queries than 1024 with <2% recall loss
  • Use 1024 dimensions when: highest recall required; multilingual or technical corpora; cost is not the binding constraint

At 1M vectors, the monthly query cost difference between 256-dim and 1024-dim is ~$0.20/day at 10K queries/day — a real but not decisive factor for most production workloads.

7. CUR Line Items for Bedrock Embedding vs Inference

CUR 2.0 Structure for Bedrock

Bedrock costs appear in Cost and Usage Reports (CUR 2.0) under the AmazonBedrock service. The key columns for separating embedding from inference:

CUR Column Embedding Value Inference (LLM) Value
line_item_operation InvokeModel InvokeModel
line_item_usage_type USE1-InputTokens:amazon.titan-embed-text-v2:0 USE1-InputTokens:anthropic.claude-3-5-sonnet-20241022-v2:0
product_model_name amazon.titan-embed-text-v2:0 model ID
line_item_resource_id bedrock:amazon.titan-embed-text-v2:0 model ARN

CUR 2.0 aggregates by usage type per hour, not per request. You cannot get per-call cost without application-side logging, but you can separate embedding from generation spend by filtering on product_model_name.

Athena Query: Separate Embedding vs LLM Generation Cost

SELECT
  line_item_usage_start_date                          AS usage_date,
  product_model_name,
  CASE
    WHEN product_model_name LIKE '%embed%'            THEN 'embedding'
    WHEN product_model_name LIKE '%rerank%'           THEN 'reranking'
    ELSE 'generation'
  END                                                 AS usage_category,
  SUM(line_item_usage_amount)                         AS total_tokens,
  SUM(line_item_unblended_cost)                       AS total_cost_usd
FROM cur_database.cur_table
WHERE
  line_item_product_code = 'AmazonBedrock'
  AND line_item_usage_start_date >= DATE('2026-06-01')
  AND line_item_line_item_type = 'Usage'
GROUP BY 1, 2, 3
ORDER BY 1, 5 DESC;

Application Inference Profiles for Tag-Based Attribution

For workloads where embedding calls come from multiple services (Knowledge Bases ingestion, custom Lambda embedding, Bedrock Agents), use Application Inference Profiles (AIP) to attach cost allocation tags:

# Create AIP with cost tag
bedrock.create_inference_profile(
    inferenceProfileName="rag-embedding-prod",
    modelSource={"copyFrom": "arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v2:0"},
    tags=[{"key": "cost-category", "value": "rag-embedding"}]
)

The cost-category tag propagates to CUR and Cost Explorer, enabling clean separation between RAG ingestion embedding, RAG query embedding, and LLM generation costs.

IAM Principal Attribution (April 2026)

As of April 2026, Bedrock supports IAM principal-based cost allocation. The line_item_iam_principal column in CUR 2.0 contains the full ARN of the role or user making each Bedrock call. For teams separating Knowledge Bases ingestion (runs as a Bedrock service role) from application-layer embedding calls (runs as an application role), this enables attribution without AIP setup.

8. Chunking Strategy Impact on Cost

Chunking strategy affects embedding cost primarily through chunk count — more, smaller chunks mean more embedding API calls for the same source document.

Chunk Count by Strategy (1,000-token source document)

Strategy Typical chunks Overhead factor Embedding tokens billed
Fixed-size, 512 tokens, 10% overlap 2.2 1.1× 1,100 tokens
Fixed-size, 256 tokens, 10% overlap 4.5 1.1× 1,100 tokens
Recursive character split, 512 tokens 2.2 1.05× 1,050 tokens
Semantic chunking (sentence-based) 3–8 1.5–4× 1,500–4,000 tokens
Hierarchical / parent-child (2 passes) 4.4 2.2× 2,200 tokens

Semantic chunking’s hidden cost: Semantic chunking computes cosine similarity between adjacent sentence embeddings to place boundaries. This requires embedding every sentence independently during preprocessing, then discarding most embeddings (only the final chunks are stored). For a 1M-token corpus:

  • Fixed-size: ~1.05M tokens billed to embed
  • Semantic: ~2M–4M tokens billed (sentence-level embeddings + chunk embeddings)
  • Cost delta at Titan v2 rates: $0.021 vs $0.040–$0.080 per 1M source tokens

Semantic Chunking Cost Justification

The 2–4× embedding cost premium for semantic chunking is justified when:

  • Domain involves dense technical, legal, or clinical text where sentence boundaries rarely align with topic boundaries
  • Retrieval recall improvements at k=5 exceed ~5% vs fixed-size (measured on your eval set)
  • Source documents have irregular structure (mixed tables, figures, prose)

Published benchmarks show fixed-size chunking at 512 tokens outperforms semantic chunking for generic document retrieval (69% vs 54% accuracy). For structured enterprise content (contracts, policies), semantic chunking recovers the gap.

Chunk Size and Vector Store Query Cost

Chunk count also affects vector store query processing. S3 Vectors charges by data processed per query (proportional to index size in bytes). Smaller chunks = more vectors = more data scanned per query:

Chunk size Chunks per 100K-token corpus S3 Vectors query cost (10K q/day)
256 tokens ~400 chunks (256-dim vectors) ~$0.003/day
512 tokens ~200 chunks (512-dim vectors) ~$0.003/day
1024 tokens ~100 chunks (1024-dim vectors) ~$0.003/day

At small corpus size, chunk count has negligible impact on S3 Vectors query cost. At 100M vector scale, the dimension × chunk count interaction matters: a 256-dim, 256-token chunking strategy yields 4× more vectors than a 1024-dim, 1024-token strategy for the same corpus, with 4× higher query data processed cost.

9. Hybrid Search Cost: Vector + BM25

Hybrid search combines dense vector search with keyword (BM25) retrieval, then fuses the ranked lists (typically via Reciprocal Rank Fusion or normalized score fusion).

Infrastructure Cost Delta

Store Pure Vector Hybrid (Vector + BM25) Marginal Cost
AOSS NextGen 1 OCU Same OCU (BM25 index co-located) $0 marginal
pgvector + pg_trgm RDS instance cost +5–15% CPU overhead Minor
S3 Vectors Not supported natively Requires separate BM25 store Added cost of secondary store

S3 Vectors + Hybrid Search Architecture

S3 Vectors does not support keyword search. To achieve hybrid retrieval with S3 Vectors:

Option A — OpenSearch for BM25, S3 Vectors for dense search:

  • Run both retrievers in parallel; fuse results client-side
  • Cost: OpenSearch OCU cost ($175–350/month) + S3 Vectors query cost
  • This eliminates S3 Vectors’ cost advantage for hybrid use cases

Option B — Bedrock Knowledge Bases hybrid retrieval:

  • Bedrock KBs natively supports hybrid search when backed by AOSS
  • The hybridSearchConfiguration parameter in RetrieveAndGenerate handles the fusion automatically
  • Not available with S3 Vectors-backed knowledge bases

Option C — DynamoDB for BM25 keyword index:

  • Store inverted index in DynamoDB; query in parallel with S3 Vectors
  • DynamoDB read cost: ~$0.25/1M read units; BM25 keyword scan at k=100 ≈ ~10 read units per query ≈ $0.0000025/query
  • Effective marginal hybrid cost: <$0.01/day at 10K queries/day

Query Latency Overhead

Running vector and BM25 in parallel adds zero latency vs the slower of the two (parallel fan-out). Fusion (RRF or score normalization) adds <5ms client-side. The practical overhead is the additional network round-trip if BM25 is in a separate service.

10. Reranker Cost: Bedrock Rerank API

Pricing

Model Price Query definition
Amazon Rerank 1.0 $1.00 / 1,000 queries 1 query = up to 100 document chunks
Cohere Rerank 3.5 (on Bedrock) $2.00 / 1,000 queries Same definition

If a request contains more than 100 document chunks, it counts as multiple queries (ceiling division):

  • 100 chunks → 1 query ($0.001 or $0.002)
  • 101–200 chunks → 2 queries ($0.002 or $0.004)
  • 350 chunks → 4 queries ($0.004 or $0.008)

When Reranking Saves Net Cost

The reranker’s cost-saving case rests on reducing the token count passed to the LLM for generation. The math:

Savings from reranking = (k_without_rerank - k_with_rerank) × avg_chunk_tokens × LLM_input_rate
Net savings = Savings from reranking - Reranker cost

Break-even condition:
(k_without - k_with) × avg_chunk_tokens × LLM_rate > Reranker_rate_per_query

Worked example — Claude Sonnet 3.7 at $3.00/1M input tokens:

Without reranker: k=10 retrieval, 10 chunks × 512 tokens = 5,120 tokens per query With reranker: retrieve k=50, rerank to k=3, 3 × 512 = 1,536 tokens per query

LLM input saved per query = (5,120 - 1,536) tokens = 3,584 tokens
LLM cost saved = 3,584 / 1M × $3.00 = $0.0000108 / query
Reranker cost (Amazon Rerank 1.0, 50 chunks = 1 query) = $0.001 / query
Net cost of reranking = $0.001 - $0.0000108 = +$0.000989 / query (LOSS)

At Claude Sonnet rates, the LLM savings from reducing context by 3,584 tokens ($0.0000108) are dwarfed by the $0.001 reranker cost. Reranking adds net cost at standard retrieval scales with cheaper LLMs.

When reranking breaks even or saves money:

  1. Very long context windows in use: If k=10 retrieval passes 20,000 tokens per query (long chunks, many docs) to a premium LLM:

    LLM saved = (20,000 - 2,000) × $3.00/1M = $0.054 saved vs $0.001 reranker = net $0.053 savings

  2. Expensive LLMs (Claude Opus, GPT-4 class at $15/1M+): LLM cost dominates; reranker pays for itself if it reduces context by >70 tokens

  3. Quality-driven cost reduction: Reranking reduces hallucination-correction loops in agentic pipelines — if a bad RAG retrieval forces a retry (full query + generation), the avoidance cost exceeds the reranker cost easily

  4. High-k initial retrieval (k=50-100) to handle sparse corpora: Reranker converts expensive k=100 LLM context to k=5 — breakeven at ~$0.10/1M LLM input

Practical Reranker Decision Rule

Scenario Recommendation
LLM input cost < $3/1M, k ≤ 10 Skip reranker; use smaller k
LLM input cost > $10/1M, k ≥ 10 Reranker likely saves net cost
Agentic pipeline with retry risk Reranker pays for itself in retry avoidance
BM25 + vector hybrid, need final ranking Reranker adds quality; cost is marginal relative to hybrid infra
Bulk document processing (not interactive) Skip reranker; latency not a constraint, use larger k

11. Total RAG Pipeline TCO: Full Model

This section builds a complete TCO model for a production RAG workload: 10M-token corpus (English enterprise docs), 50K queries/day, Titan v2 embeddings, S3 Vectors storage, Bedrock Knowledge Bases for ingestion, Claude Sonnet 3.7 for generation.

Assumptions

Parameter Value
Corpus size 10M tokens source text
Average chunk size 512 tokens
Overlap 10% (effective 460 tokens/chunk)
Total chunks ~21,750 chunks
Embedding dimensions 512
Query volume 50,000 queries/day
Chunks retrieved per query k=5
Tokens per chunk 512
Reranker No (see §10)
LLM Claude Sonnet 3.7 ($3/1M input, $15/1M output)
LLM input per query k=5 × 512 tokens + 200-token system prompt = 2,760 tokens
LLM output per query 300 tokens avg
Monthly re-ingestion 10% corpus monthly (incremental sync)

Monthly Cost Breakdown

One-time ingestion cost (amortized over 12 months):

Item Calculation Cost
Embedding (Titan v2) 10M tokens × $0.020/1M $0.20
S3 Vectors PUT 21,750 chunks × 2 KB (512-dim) = 43.5 MB × $0.21/GB $0.009
BDA parsing (optional, plain text = $0) $0.00 (skipped for pre-extracted text) $0.00
Total one-time $0.21
Amortized monthly (12 months) $0.02

Monthly re-ingestion cost (10% corpus, incremental):

Item Calculation Cost/month
Re-embedding 1M tokens 1M × $0.020/1M $0.02
S3 Vectors delta PUT ~2,175 chunks × 2 KB = 4.35 MB $0.001
Re-ingestion total $0.02

Monthly storage cost:

Item Calculation Cost/month
S3 Vectors (512-dim, 21,750 vectors) 21,750 × 2 KB = 43.5 MB → 0.0435 GB × $0.067 $0.003

Monthly query cost:

Item Calculation Cost/month
Query embedding (Titan v2) 50K queries/day × 30 × 30 tokens/query = 45M tokens × $0.020/1M $0.90
S3 Vectors query (21,750 vectors × 2 KB = 43.5 MB = 0.0000435 TB/query) 50K × 30 × 0.0000435 TB × $0.002/TB $0.13
LLM generation — input 50K × 30 × 2,760 tokens = 4.14B tokens × $3/1M $12,420
LLM generation — output 50K × 30 × 300 tokens = 450M tokens × $15/1M $6,750
Query total $19,171/month

TCO Summary (50K queries/day production workload)

Category Monthly Cost % of Total
LLM generation (input + output) $19,170 99.5%
Query embedding $0.90 0.005%
Vector store (query + storage) $0.13 <0.001%
Re-ingestion (embedding) $0.02 <0.001%
Total $19,171 100%

The dominant insight: LLM generation cost is 99.5%+ of total RAG pipeline TCO. Embedding costs, vector store costs, and ingestion costs are noise relative to generation. Optimization effort should be directed at:

  1. Reducing generation tokens — smaller k, tighter chunks, query compression
  2. LLM model selection — Haiku vs Sonnet vs Opus changes monthly bill by 10× at identical volume
  3. Caching repeated queries — Bedrock prompt caching at 90% discount for cache hits directly on the dominant cost line

The 80% S3 Vectors query pricing reduction, while meaningful in isolation, moves the needle by <0.001% of TCO for standard interactive RAG.

TCO Sensitivity Table (monthly, same query volume)

LLM Model Input rate Output rate Monthly generation cost
Claude Haiku 3.5 $0.80/1M $4/1M $3,330
Claude Sonnet 3.7 $3/1M $15/1M $19,170
Claude Sonnet 4.5 (est.) $3/1M $15/1M $19,170
Claude Opus 4 $15/1M $75/1M $95,850

Prompt Caching Impact on TCO

If 40% of queries share a common system prompt + context (typical for enterprise FAQ RAG), Bedrock prompt caching reduces input token cost on cache hits by 90%:

Uncached queries (60%): 30K/day × 2,760 tokens × $3/1M = $7,452/month
Cached queries (40%): 20K/day × 200 tokens (cache miss portion) × $3/1M = $360/month
Cache write (amortized): negligible for static system prompts
Total with caching: $7,812/month vs $12,420 uncached = 37% savings

Prompt caching delivers far more TCO impact than any vector store optimization.

Key Findings Summary

  1. S3 Vectors June 2026 price cut (80% reduction for >10M vector indexes) meaningfully reduces query costs at scale but does not change the fundamental TCO picture — LLM generation dominates at 99.5%+ of total spend.

  2. Titan v2 is 5× cheaper than Cohere Embed v3 for English text; choose Cohere only for multilingual workloads or when multimodal embeddings (Embed v4) are required.

  3. BDA parsing is the ingestion cost driver, not embedding. At $0.010/page, a 1,000-page PDF corpus costs $10 to parse and $0.03 to embed. Skip BDA for pre-extracted plain text.

  4. 512-dimension vectors hit 98.7% of 1024-dim recall at half the vector storage and query cost; the correct default for English enterprise RAG.

  5. Reranking adds net cost at standard LLM rates (Claude Sonnet and below) when k ≤ 10. Reranking saves money only with expensive LLMs (>$10/1M input) or large initial retrieval sets (k ≥ 50).

  6. Semantic chunking costs 2–4× more to embed than fixed-size chunking due to sentence-level similarity computation during boundary detection. The quality premium is only justified for structured technical/legal text.

  7. CUR attribution: Use Application Inference Profiles with cost allocation tags to cleanly separate embedding, reranking, and generation in Cost Explorer. product_model_name in CUR 2.0 enables Athena-level separation without AIP setup.

  8. Prompt caching is the highest-leverage TCO lever: 37%+ reduction in generation cost when 40% of queries share common context — dwarfs any vector store or embedding optimization.

Sources