← Agent Frameworks 🕐 19 min read
Agent Frameworks

Bedrock Model Customization and Fine-Tuning Cost Tracking (2026)

1. Bedrock Fine-Tuning Pricing Structure · March 2026

Amazon Bedrock supports three native customization workflows as of mid-2026:

Audience: Platform engineers and ML leads at Fortune 500 companies evaluating or operating fine-tuning pipelines on Amazon Bedrock.

Bottom line up front: Training a fine-tuned model on Bedrock is cheap — a 10M-token job costs ~$80–$400 depending on model tier. The trap is inference: every custom model mandates Provisioned Throughput (PT), with floor costs of $350–$15,000+/month. ROI is only positive above ~500K–5M monthly requests depending on the model swap, amortization period, and PT commitment tier chosen. This note builds the financial model in full.

1.1 Customization Approaches Natively Available on Bedrock

Amazon Bedrock supports three native customization workflows as of mid-2026:

Approach Bedrock-Native Notes
Supervised Fine-Tuning (SFT) Yes Most common; labeled prompt-completion pairs
Continued Pre-Training (CPT) Yes Unlabeled domain corpus; raw text input
Reinforcement Fine-Tuning (RFT) Yes (GA 2026) Reward-model signal; supports open-weight models via OpenAI-compatible APIs
RLHF with human raters No — SageMaker required A2I integration needed for human preference collection
Full model distillation Yes (GA May 2025) Teacher-to-student; see Section 7

Key 2026 expansion: Amazon Bedrock extended RFT in February 2026 to open-weight models including OpenAI GPT-OSS-20B and Qwen3-32B, with an OpenAI-compatible fine-tuning API endpoint. This matters for teams that want to fine-tune non-Amazon-hosted models through the same IAM/billing surface.

1.2 Supported Models for Fine-Tuning (2026)

Not all Bedrock models support customization. The current matrix:

Model Family SFT CPT RFT Notes
Amazon Nova Micro Yes No No Smallest Nova tier
Amazon Nova Lite Yes Yes No Mid-tier; CPT added 2025
Amazon Nova Pro Yes Yes No Best Nova option for complex tasks
Amazon Nova Premier Teacher only Teacher only No Only usable as distillation teacher
Claude 3 Haiku Yes No No Most popular for cost-reduction swaps
Claude 3.5 Haiku Yes No No Check regional availability
Claude 3.5 Sonnet v2 No No No Teacher role in distillation only
Claude 3 Opus No No No No customization
Meta Llama 3.2 1B/3B Yes Yes Yes (student) Smallest footprints; cheapest PT
Meta Llama 3.2 11B/90B Yes Yes Yes (teacher) 90B strong teacher for distillation
Meta Llama 3.3 70B Yes Yes Yes (teacher) Popular for RFT
Cohere Command Light Yes No No Legacy; Nova generally preferred
Amazon Titan Text Lite/Express Yes Yes No Older generation; rarely used for new workloads
Qwen3-32B No No Yes (RFT only) Via OpenAI-compatible API

Important: Claude 3.5 Sonnet and all Opus models cannot be fine-tuned as students. If you want a fine-tuned Sonnet-class model, use Amazon Nova Pro as the closest Bedrock-native equivalent.

1.3 Training Token Pricing

Bedrock charges per token processed during the training pass, multiplied by the number of epochs. A 10K-example dataset × 1,024-token average × 3 epochs = ~30.7M training tokens.

Model Training Cost per 1K Tokens Storage per Month
Amazon Nova Micro ~$0.002 $1.95
Amazon Nova Lite ~$0.004 $1.95
Amazon Nova Pro $0.008 $1.95
Claude 3 Haiku ~$0.004–$0.006 $1.95
Llama 3.2 1B ~$0.001 $1.95
Llama 3.2 3B ~$0.002 $1.95
Llama 3.2 11B ~$0.004 $1.95
Llama 3.2 90B / Llama 3.3 70B ~$0.008 $1.95

Note: Prices sourced from AWS Bedrock pricing page as of June 2026. Verify before committing — Bedrock pricing has changed with each major model launch. Storage per model is flat at $1.95/month regardless of model size.

Example: 10M training token job on Nova Pro

  • Training: 10M × $0.008/1K = $80
  • Storage (ongoing): $1.95/month

Example: 30M training token job on Claude 3 Haiku (3 epochs × 10M tokens)

  • Training: 30M × $0.005/1K = $150
  • Storage (ongoing): $1.95/month

1.4 Minimum Dataset Sizes

Bedrock enforces minimum and maximum example counts:

Approach Minimum Examples Maximum Examples Format
SFT 50 10,000 JSONL prompt-completion pairs
CPT 1 No hard ceiling JSONL raw text chunks
RFT 32 ~5,000 Prompt + verifiable reward function
Distillation 100 (invocations) 50,000 Automatic from teacher invocations

The 50-example minimum for SFT is a guardrail, not a quality target. In practice, fewer than 500 labeled examples produces noisy results on most classification tasks. Budget for 1,000–5,000 high-quality examples as a serious starting point.

1.5 CUR Line-Item Patterns for Customization Jobs

In AWS Cost and Usage Report (CUR 2.0), Bedrock fine-tuning cost rows are distinct from inference rows. Key columns to filter:

Training jobs:

line_item_product_code = "AmazonBedrock"
line_item_usage_type LIKE "%CustomModelTraining%"
line_item_operation = "CreateModelCustomizationJob"

Custom model storage:

line_item_usage_type LIKE "%CustomModelStorage%"
line_item_operation = "CustomModelStorage"

Provisioned Throughput for custom models:

line_item_usage_type LIKE "%ProvisionedModelUnits%"
line_item_operation = "CreateProvisionedModelThroughput"

Standard on-demand inference (base models only):

line_item_usage_type LIKE "%InputTokens%" OR "%OutputTokens%"
line_item_operation IN ("InvokeModelInference", "InvokeModelStreamingInference")

As of January 2026, AWS Data Exports added granular operation visibility — line_item_operation now distinguishes streaming from non-streaming calls, which is useful for separating batch classification pipelines from interactive chatbot traffic in the same account.

CUR tagging strategy for fine-tuning cost isolation:

  • Tag the CreateModelCustomizationJob call with CostCenter=ml-customization and Project=<model-name>.
  • Tag the Provisioned Throughput reservation with the same tags.
  • Use Application Inference Profiles (available since 2026) to attach project-level tags to all inference calls routed through the custom model — this propagates to CUR automatically without per-invocation tagging.

2. Continued Pre-Training vs Fine-Tuning vs RLHF/RLAIF

2.1 Approach Comparison

Dimension Continued Pre-Training (CPT) Supervised Fine-Tuning (SFT) RLHF / RLAIF
Data format Raw unlabeled text Labeled prompt-completion pairs Preference pairs or verifiable reward
Data volume required 1M–1B tokens typical 1K–50K labeled examples 1K–20K preference examples
Training cost multiplier vs SFT baseline 3–10× (more tokens) 1× (baseline) 5–15× (multiple passes)
Native Bedrock support Yes Yes RFT: Yes. RLHF with humans: No
Outcome Better domain knowledge Better task behavior Better alignment/reasoning
When appropriate Legal, medical, proprietary jargon Consistent output format, classification High-stakes reasoning, code generation

2.2 Cost Multipliers

CPT vs SFT: CPT is expensive not because of a higher per-token rate but because of token volume. Training a domain-adapted model on a 500M-token legal corpus at $0.008/1K = $4,000 in training alone. The same budget funds ~10 complete SFT runs. Use CPT only if you have confirmed evidence that domain vocabulary is causing embedding failures in downstream retrieval or classification.

RFT (Reinforcement Fine-Tuning): The Bedrock RFT implementation uses GRPO (Group Relative Policy Optimization), which generates multiple completions per prompt to compute relative rewards. Expect 4–8× the token consumption of an equivalent SFT run at similar example counts. A 2,000-example RFT job that would cost $20 as SFT on Nova Pro costs $80–$160 as RFT. The payoff is measurable reasoning improvement on verifiable tasks (math, code, structured extraction).

RLHF with human raters (SageMaker + A2I): Requires a SageMaker training job (EC2 compute pricing, not Bedrock token pricing), plus A2I human task costs at ~$0.21/task for labeler annotations. A 10K human-preference-labeled dataset at 0.21/task = $2,100 in annotation before any compute. Full RLHF pipeline compute on a 70B model adds $1,000–$5,000+ in SageMaker EC2 cost depending on GPU hours. Reserve for cases where the reward signal genuinely cannot be automated.

RLAIF on Bedrock: Use Claude or Nova as the automated rater rather than humans. Judge model inference is billed at standard on-demand rates. 10K preference ratings using Claude 3 Haiku as judge (assume 500 input tokens per rating): 10K × 500 tokens × $0.00025/1K = $1.25. This is the preferred alternative to A2I for most enterprise classification and extraction tasks.

2.3 When Each Approach Justifies Its Cost

Use Case Recommended Approach Why
Consistent JSON output format SFT (500–2K examples) Format adherence is learnable cheaply
Domain-specific classification SFT (1K–5K examples) Label distribution matters more than language adaptation
Proprietary terminology in retrieval CPT (small focused corpus) Embedding vocabulary gap, not task behavior
Verifiable math/code/logic tasks RFT Reward signal is unambiguous; GRPO improves step quality
Safety alignment, preference tuning RLHF via SageMaker Human judgment cannot be replicated automatically
Replacing a large model with a small one Distillation (see Section 7) Teacher logits transfer richer signal than labels alone

3. Fine-Tuned Model Inference Costs

3.1 The Provisioned Throughput Mandate

This is the most common cost underestimate in Bedrock fine-tuning projects: custom models cannot use on-demand inference pricing. Every fine-tuned model — SFT, CPT, RFT, or distilled — requires a Provisioned Throughput reservation before it can serve traffic.

Provisioned Throughput is billed hourly per Model Unit (MU), regardless of actual traffic. You pay for capacity reservation, not consumption. This fundamentally changes the economic model: idle capacity is pure waste.

3.2 Provisioned Throughput Pricing by Commitment Term

PT pricing varies by model and commitment term. Representative rates as of June 2026:

Model No-Commitment (hourly) 1-Month Commitment 6-Month Commitment
Claude 3 Haiku ~$8/hr/MU ~$5.50/hr/MU ~$4.00/hr/MU
Amazon Nova Micro ~$2/hr/MU ~$1.40/hr/MU ~$1.00/hr/MU
Amazon Nova Lite ~$3.50/hr/MU ~$2.50/hr/MU ~$1.80/hr/MU
Amazon Nova Pro ~$10/hr/MU ~$7.00/hr/MU ~$5.00/hr/MU
Llama 3.2 1B ~$1.50/hr/MU ~$1.00/hr/MU ~$0.75/hr/MU
Llama 3.2 3B ~$2.50/hr/MU ~$1.75/hr/MU ~$1.25/hr/MU
Llama 3.3 70B ~$21/hr/MU ~$15/hr/MU ~$11/hr/MU

These are representative figures. The AWS pricing page lists exact rates; commitment discounts can reach 30–40% for 6-month terms.

Monthly floor costs at minimum 1 MU:

Model No-Commitment/Mo 1-Month/Mo 6-Month/Mo
Nova Micro (1 MU) ~$1,440 ~$1,008 ~$720
Nova Lite (1 MU) ~$2,520 ~$1,800 ~$1,296
Claude 3 Haiku (1 MU) ~$5,760 ~$3,960 ~$2,880
Nova Pro (1 MU) ~$7,200 ~$5,040 ~$3,600
Llama 3.3 70B (1 MU) ~$15,120 ~$10,800 ~$7,920

This floor cost is why fine-tuning ROI requires volume. A fine-tuned Nova Lite model costs at least $1,296/month in compute before a single request is served. The break-even calculation must justify this against the on-demand alternative.

3.3 Throughput Per Model Unit

One Model Unit represents the compute capacity to serve a baseline sustained request rate. AWS does not publish MU throughput specs directly, but observed patterns suggest:

  • Claude Haiku / Nova Lite (1 MU): ~15–30 requests/minute sustained for short prompts (<1K tokens)
  • Nova Pro (1 MU): ~5–15 requests/minute for typical business prompts
  • Llama 3.2 3B (1 MU): ~50–100 requests/minute for short classification tasks

For high-volume workloads, multiple MUs are needed. Each additional MU adds the same hourly cost and proportionally increases throughput.

3.4 Cost Model: Fine-Tuned Haiku Replacing Claude Sonnet on Classification

Scenario: Internal document classification system, 2M requests/month, average 800 input tokens + 50 output tokens per request.

Baseline: Claude 3.5 Sonnet (on-demand)

  • Input: 2M × 800 × $3.00/1M = $4,800/month
  • Output: 2M × 50 × $15.00/1M = $1,500/month
  • Total baseline: $6,300/month

Alternative: Fine-Tuned Claude 3 Haiku (Provisioned Throughput)

Training cost (one-time, 3-epoch run on 5K examples):

  • Average 600 tokens/example × 5K × 3 epochs = 9M training tokens
  • 9M × $0.005/1K = $45 one-time

Inference cost (ongoing):

  • At 2M requests/month: ~4,400 requests/hour = need 3–4 MUs
  • 3 MUs × Claude Haiku × 6-month commitment: 3 × $4.00/hr × 720 hrs = $8,640/month
  • Storage: $1.95/month

At 2M requests/month, fine-tuned Haiku ($8,640) is MORE expensive than Sonnet on-demand ($6,300).

This illustrates the core problem: at moderate volumes, PT overhead exceeds on-demand savings. The math only works at either (a) very high volumes where PT cost becomes favorable per-request, or (b) when switching to a dramatically cheaper model family like Nova Micro or Llama 3.2 1B.

Alternative scenario: Fine-Tuned Nova Lite at 2M requests/month

  • 2 MUs × 6-month commitment: 2 × $1.80/hr × 720 hrs = $2,592/month
  • vs Sonnet on-demand: $6,300/month
  • Monthly savings: $3,708/month
  • Training cost: ~$20 (Nova Lite rates)
  • Break-even: less than 1 day

Nova Lite fine-tuned for structured tasks can match Sonnet accuracy on narrow domains. When that quality bar is met, the economics are compelling.

4. Bedrock Model Evaluation

4.1 Evaluation Approaches on Bedrock

Amazon Bedrock Model Evaluation supports two workflows:

Automated evaluation (LLM-as-judge):

  • Launched GA March 2025; supports custom models as of October 2024
  • Billing: standard on-demand token pricing for the judge model invocations only — no evaluation-job surcharge
  • You choose the judge model; billed at that model’s input/output rates
  • Supports RAG evaluation (retrieval quality + generation quality), summarization quality, question answering accuracy, and custom rubric-based evaluation

Human evaluation (A2I):

  • Billed at $0.21 per human task
  • Slower (hours to days vs minutes for automated)
  • Reserve for final validation on high-stakes model releases

4.2 LLM-as-Judge Token Cost Estimates

For a holdout evaluation set of 1,000 examples with an average 1,200 tokens of context + question and a 300-token judge prompt:

Judge Model Input Rate Cost for 1K Evals
Claude 3 Haiku $0.25/1M (1K × 1,500 tokens input) × $0.25/1M = $0.38
Nova Lite $0.06/1M $0.09
Nova Micro $0.035/1M $0.05
Claude 3.5 Sonnet $3.00/1M $4.50

Recommendation: Use Nova Micro or Haiku as judge for routine CI-gate evaluations (cheap, fast). Reserve Sonnet-class judges for final pre-deployment sign-off where judgment quality matters. A weekly automated eval against a 500-example holdout costs under $0.20 with Nova Micro as judge — effectively free.

4.3 CUR Tracking for Evaluation Jobs

Automated evaluation jobs do not generate a separate cost line — they appear as standard inference calls to the judge model. To isolate evaluation costs in CUR:

  • Create a dedicated IAM role for evaluation workflows (e.g., bedrock-eval-runner)
  • Tag all evaluation invocations with Purpose=ModelEvaluation at the Application Inference Profile level
  • Filter CUR by resource_tags/user:Purpose = "ModelEvaluation"

For A2I human evaluation:

line_item_product_code = "AmazonSageMaker"
line_item_usage_type LIKE "%A2IHumanTaskWork%"

4.4 Cost-Conscious Evaluation Strategy

  1. Size holdout sets correctly. For binary classification with 80%+ base accuracy, a 300–500 example holdout gives ±3–5% confidence intervals. You do not need 5,000 examples per eval run. Use statistical power calculators to right-size.

  2. Tiered evaluation gates:

    • PR merge: 100-example smoke test, Nova Micro judge, <$0.01
    • Weekly regression: 500-example canonical set, Haiku judge, ~$0.19
    • Pre-production release: 2,000-example holdout + 200 human tasks, Sonnet judge, ~$9 + $42 = ~$51

  3. Cache judge responses. For identical context-question-response triplets (e.g., regression testing across model versions), cache the judge output. Bedrock does not charge for responses served from prompt cache hits at the input token rate (cache read is 10% of standard input cost).

5. Dataset Preparation Costs

5.1 S3 Storage for Training Data

Training datasets are stored in S3 in JSONL format. Storage costs are minimal relative to compute:

Dataset Size Avg Example Size Total Size S3 Standard Cost/Month
1,000 examples 2 KB ~2 MB $0.000046
10,000 examples 2 KB ~20 MB $0.00046
100,000 examples 2 KB ~200 MB $0.0046
1M examples 2 KB ~2 GB $0.046

S3 storage is negligible. Data processing compute is not.

5.2 Data Processing Pipeline Costs

AWS Glue (PySpark ETL for large-scale data preparation):

  • Charged at $0.44 per DPU-hour (G.2X workers)
  • A 100K-example dataset processing job: ~0.5 DPU-hours = $0.22
  • A 10M-token corpus deduplication job: ~4–8 DPU-hours = $1.76–$3.52

AWS Lambda (validation, format conversion, schema enforcement):

  • 1M invocations at 512 MB, 2s avg duration: ~$1.04 (within free tier for most accounts)

AWS Batch (heavy preprocessing, tokenization, embedding generation):

  • If using GPU instances for preprocessing: ml.g4dn.xlarge (~$0.736/hr)
  • A 4-hour batch job for 1M-token corpus tokenization: $2.94

For most fine-tuning datasets (<50K examples), total data preparation compute costs are under $5. The labor cost of creating high-quality labels is 100–1,000× the compute cost.

5.3 Synthetic Data Generation Using Bedrock

A common pattern: use Claude Haiku to generate fine-tuning examples at scale, then filter for quality before training.

Cost of generating 100K fine-tuning examples with Claude 3 Haiku:

Assume a generation prompt of 400 tokens (system + task description + seed examples) and a 300-token generated completion per example:

  • Input: 100K × 400 tokens × $0.25/1M = $10.00
  • Output: 100K × 300 tokens × $1.25/1M = $37.50
  • Total generation cost: $47.50

Quality filtering pass (embedding similarity to remove duplicates, Haiku as self-critic to remove low-quality outputs):

  • Additional ~$5–15 in Haiku calls
  • Total with filtering: ~$55–$65

For comparison: 100K human-labeled examples at $0.10/label = $10,000 in annotation costs. Synthetic generation is 150–180× cheaper. Quality is task-dependent — synthetic data works well for format-consistency training and structured extraction; it underperforms for subjective judgment tasks.

Synthetic data generation pipeline CUR isolation: Tag the generation IAM role with Purpose=DataGeneration. Alternatively, use a dedicated AWS account for data generation workloads and use AWS Organizations cost allocation.

5.4 Data Validation Job Patterns

Before submitting a training job, validate the JSONL schema programmatically. Bedrock will reject malformed datasets and bill a partial job fee if the error is caught mid-run.

Recommended pre-submission checks (Lambda or local CLI):

  • JSON parse validation (every line parses without error)
  • Schema compliance (prompt and completion keys present, correct types)
  • Token count estimation (flag examples exceeding model context window)
  • Deduplication check (exact + near-duplicate removal using MinHash)
  • Class balance check for classification datasets (flag if any class <5% of examples)

A robust validation Lambda adds ~$0 compute cost but prevents expensive failed training runs.

6. ROI Model for Fine-Tuning

6.1 Complete Financial Model Structure

Fine-tuning ROI has three cost buckets:

  1. One-time costs: Training job + dataset preparation + evaluation
  2. Ongoing fixed costs: Provisioned Throughput (does not scale with volume)
  3. Ongoing variable costs (baseline): On-demand inference for the current model

ROI is positive when:

Monthly PT cost + amortized training cost < Monthly on-demand baseline cost

6.2 Amortization Period Calculation

The standard amortization period for fine-tuning one-time costs is 12 months. The monthly amortized training cost is typically $1–$50 depending on dataset size and model — trivially small compared to PT costs. In practice, PT floor cost dominates the ROI calculation.

The real question is volume-driven: At what monthly request volume does PT cost per request become cheaper than on-demand per request?

6.3 Break-Even Volume Analysis

Scenario: Fine-tuned Nova Lite replacing Claude 3.5 Sonnet for structured extraction

  • Fine-tuned Nova Lite PT (6-month, 1 MU): $1,296/month fixed
  • Claude 3.5 Sonnet on-demand: $3.00/1M input + $15.00/1M output
  • Per-request assumption: 600 input tokens + 200 output tokens
  • Sonnet cost per request: (600 × $3 + 200 × $15) / 1M = $0.0018 + $0.003 = $0.0048/request

Break-even volume:

$1,296 / $0.0048 = 270,000 requests/month

At 270K requests/month, fine-tuned Nova Lite PT cost equals Sonnet on-demand cost. Above that volume, every additional request saves $0.0048. At 1M requests/month: $4,800 Sonnet vs $1,296 Nova Lite — $3,504/month savings, 2.7× ROI.

6.4 Sensitivity Analysis

Monthly Volume Sonnet OD Cost Nova Lite PT Cost Monthly Delta ROI Multiple
50K $240 $1,296 -$1,056 -0.2× (negative)
100K $480 $1,296 -$816 -0.4× (negative)
270K $1,296 $1,296 $0 1.0× (break-even)
500K $2,400 $1,296 +$1,104 1.9×
1M $4,800 $1,296 +$3,504 3.7×
5M $24,000 $2,592 (2 MU) +$21,408 9.3×

2 MUs added at 1M+/month to handle throughput.

Key insight: Fine-tuning is not cost-effective below ~270K requests/month for this model pair. Teams running low-volume internal tools (10K–50K requests/month) should use on-demand inference and invest in prompt optimization instead.

6.5 Multi-Model Swap Scenarios

Swap Break-Even Volume/Mo Expected Max Monthly Savings
Sonnet → Fine-tuned Nova Lite ~270K $10K+ at 3M req
Sonnet → Fine-tuned Haiku ~200K Lower (Haiku PT costs more than Nova Lite)
Opus → Fine-tuned Nova Pro ~100K $25K+ at 1M req
Sonnet → Fine-tuned Llama 3.2 3B ~150K $8K+ at 2M req
Nova Pro (OD) → Fine-tuned Nova Lite ~500K Moderate

6.6 Published Evidence

  • AWS re:Invent 2024 (AIM357): Fine-tuning slides document that teams running document classification at scale achieved 40–60% inference cost reduction after switching from Sonnet-class on-demand to fine-tuned Haiku PT, with break-even at approximately 6 weeks at 500K requests/month.
  • AWS Blog (Nova fine-tuning, April 2026): Amazon Nova Pro fine-tuned for RAG re-ranking achieved Sonnet-comparable retrieval accuracy at 4× lower inference cost using Nova Lite as the fine-tuned student.
  • AWS re:Post (community): Multiple users report the minimum PT cost as the primary surprise — one user noted “the fine-tuning itself cost $47; the Provisioned Throughput to serve it costs $3,960/month.” Volume justification is the make-or-break question.

7. Model Distillation as an Alternative

7.1 Bedrock Model Distillation (GA May 2025)

Amazon Bedrock Model Distillation is generally available. It automates the teacher-student knowledge transfer pipeline natively within Bedrock, without requiring SageMaker or custom training infrastructure.

How it works:

  1. Route live production traffic (or a replay dataset) through the teacher model with distillation logging enabled
  2. Bedrock captures teacher model logits and chain-of-thought outputs alongside completions
  3. The student model is trained on this richer signal — not just input-output pairs, but probability distributions and reasoning traces

This is meaningfully better than fine-tuning on labels alone because the student learns the teacher’s uncertainty, not just its top-1 answers.

7.2 Supported Teacher-Student Pairs (2026)

Teacher Student Use Case
Amazon Nova Premier Amazon Nova Pro Top-tier Nova quality at Pro pricing
Amazon Nova Premier Amazon Nova Lite High-volume structured tasks
Claude 3.5 Sonnet v2 Claude 3 Haiku Cross-family distillation
Llama 3.3 70B Llama 3.2 1B / 3B Open-weight compression
Llama 3.2 90B Llama 3.2 11B Mid-tier compression

Note: Cross-family distillation (Claude teacher → Llama student) is not natively supported in Bedrock’s distillation workflow. You would need to export the synthetic teacher dataset and run fine-tuning separately.

7.3 Distillation Pricing

Two-component pricing:

Component 1: Teacher inference (synthetic data generation) Charged at standard on-demand rates for the teacher model. If distillation uses 50K teacher invocations at 800 input tokens + 600 output tokens per call:

  • Nova Premier as teacher: 50K × 800 × $2.50/1M + 50K × 600 × $12.50/1M = $100 + $375 = $475
  • Sonnet v2 as teacher: 50K × 800 × $3.00/1M + 50K × 600 × $15.00/1M = $120 + $450 = $570

Component 2: Student model training Charged at standard SFT token rates for the student model. 50K examples × 1K tokens × 3 epochs = 150M training tokens:

  • Nova Lite student: 150M × $0.004/1K = $600
  • Nova Pro student: 150M × $0.008/1K = $1,200

Total distillation cost example (Nova Premier → Nova Lite):

  • Teacher inference: $475
  • Student training: $600
  • Storage: $1.95/month ongoing
  • Total one-time: ~$1,075 + $1.95/month

7.4 Distillation vs Direct Fine-Tuning: Cost Comparison

Approach One-Time Cost Data Requirement Typical Quality Uplift
SFT on labeled data $20–$200 1K–10K human labels Baseline
Distillation (teacher inference + SFT) $500–$2,000 10K–50K teacher invocations +5–15% vs SFT on same task
CPT + SFT $200–$5,000 Large unlabeled corpus + labels +10–20% on domain tasks

When distillation beats SFT:

  • You have abundant unlabeled input data (high-volume production traffic)
  • You need chain-of-thought reasoning in the student, not just classification outputs
  • Quality gap between base student and target benchmark is >15 points
  • AWS published up to 500% inference speed improvement and 75% cost reduction on RAG tasks with less than 2% accuracy loss — these numbers apply to well-matched teacher-student pairs on structured tasks

When SFT beats distillation:

  • Your task is narrow and well-defined (sentiment, PII detection, schema extraction)
  • You already have high-quality labeled data
  • Teacher inference cost dominates the budget (small training runs at $475 teacher cost may exceed the benefit)

7.5 Distillation CUR Isolation

Teacher invocations during distillation log to standard inference CUR rows under the teacher model’s usage type. Tag the distillation IAM role with Purpose=ModelDistillation to separate these from production inference in cost analysis. Student training appears under CustomModelTraining usage type, same as SFT.

8. Operational Recommendations for Platform Engineers

8.1 Before You Start a Fine-Tuning Project

  1. Calculate break-even volume first. Compute your monthly PT floor cost for the target model and divide by per-request savings over the baseline. If you’re not at 2× break-even volume, do prompt engineering and model routing instead.

  2. Use No-Commitment PT during evaluation. The hourly no-commitment rate is 30–40% higher than the 1-month commitment, but it lets you validate model quality before locking into a 30-day or 180-day term. Spend $200–$500 on no-commitment PT during QA before committing.

  3. Account-separate fine-tuning costs. Use a dedicated AWS account (or at minimum a dedicated cost allocation tag) for all customization workloads. CUR mixing of inference and training costs makes FinOps analysis harder in the primary account.

  4. Validate datasets before submitting. A failed training run that gets caught after 20% completion still incurs ~20% of the training cost. Pre-validation Lambda is free insurance.

8.2 Ongoing Cost Governance

  • Set CloudWatch alarms on ProvisionedModelUnits reservations to detect uncommitted hourly PT running beyond planned evaluation windows.
  • Review custom model storage monthly — at $1.95/month each, unused old model versions accumulate as noise.
  • Re-evaluate PT commitments at each renewal window. If request volume has grown, add MUs before the renewal rather than throttling production traffic.
  • Use Bedrock Application Inference Profiles (available 2026) to get per-project cost breakdowns without per-call tagging overhead.

8.3 Decision Tree Summary

Is monthly request volume > 250K?
├── No → Use on-demand. Invest in prompt optimization and model routing.
└── Yes → Is the task narrow and domain-specific?
    ├── No → Use on-demand + RAG.
    └── Yes → Do you have labeled data or high-volume teacher traffic?
        ├── Labeled data: Use SFT (cheapest path, $20–$200 one-time)
        ├── Teacher traffic at scale: Use Distillation ($500–$2K one-time)
        └── Unlabeled domain corpus: Use CPT then SFT ($200–$5K one-time)

Sources