← Agent Frameworks 🕐 18 min read

Agent Frameworks

AWS AI Cost Optimization — New Features and Announcements H1 2026

Executive Summary · March 2026

H1 2026 marks a structural shift in how AWS prices and governs AI inference spend.

For: Platform engineers and FinOps leads running AI/ML workloads on AWS
Signal type: What is GA or in preview that was not available at the start of 2025
Confidence: High on GA features; moderate on roadmap signals sourced from re:Invent sessions and blog posts

H1 2026 marks a structural shift in how AWS prices and governs AI inference spend. The three most operationally important developments are: (1) a three-tier Bedrock service-tier model (Priority / Standard / Flex) that lets teams route individual API calls to the right cost-performance band without code restructuring; (2) IAM principal-level cost attribution in CUR 2.0, enabling team- and project-level Bedrock charge-backs for the first time; and (3) the AWS FinOps Agent, entering public preview in June 2026, which automates cost anomaly investigation and surfaces Bedrock-specific recommendations through a conversational interface. Separately, Amazon Nova 2 (launched re:Invent December 2025) and Bedrock Model Distillation GA (May 2025) give teams the tools to reduce per-token costs by 50–75% for the right workloads.

1. Bedrock New Pricing Features H1 2026

1.1 Three-Tier Service Model (Priority / Standard / Flex)

The most significant Bedrock pricing restructuring in 2025–2026 was announced at re:Invent 2025 and became broadly available in early 2026. Amazon Bedrock now exposes three explicit service tiers per inference call:

Tier	Price delta vs Standard	Latency profile	When to use
Priority	+75%	Up to 25% faster	User-facing, SLA-bound, low-latency-critical
Standard	Baseline	Best-effort	General workloads (existing default behavior)
Flex	−50%	Higher latency accepted	Evaluations, bulk enrichment, non-real-time pipelines

Teams select the tier per API call via a service_tier parameter in the Converse and InvokeModel APIs — no endpoint or code architecture changes required. This is the key operational advantage over the legacy Batch Inference path, which requires S3 orchestration. For 2026, supported models for Flex and Priority include Amazon Nova and selected third-party models (Qwen, certain OpenAI API-compatible models on Bedrock). Claude family support on Priority and Flex is model-version-dependent; verify at the Bedrock pricing page before committing workflows.

FinOps implication: Audit existing workloads running at Standard that have latency tolerance. Migrating even 30% of throughput to Flex yields a 15% blended inference cost reduction with no accuracy or quality trade-off. Benchmark latency variance before moving any synchronous user-facing path.

1.2 Bedrock Reserved Tier Expansion

The Reserved service tier — committed tokens-per-minute capacity for predictable throughput and performance — expanded in January 2026 with two meaningful changes:

Flexible input/output allocation: Teams can now split TPM commitments between input and output tokens independently, rather than buying a single blended TPM figure. This matters for workloads with asymmetric ratios (e.g., RAG queries with short prompts and long completions).
GovCloud availability: The Reserved tier became available for Claude Sonnet in AWS GovCloud (US-West) as of January 2026, unblocking FedRAMP-scoped deployments that previously had no commitment pricing option.

Discount levels for Reserved tier remain 20–40% versus Standard on-demand pricing, with one-month and six-month commitment windows. Six-month commitment delivers the upper end of that range.

1.3 Batch Inference — Model Roster Expansion

Batch Inference (S3-in / S3-out, 24-hour SLA, 50% off on-demand pricing) has been GA since 2024, but AWS expanded the model roster through H1 2026. Amazon Nova Pro, Nova Lite, Nova Micro, Claude 3.5 Sonnet v2, Llama 3.3 70B, and select Mistral models are all available for batch as of mid-2026. The pricing model remains consistent: 50% discount, JSONL input via S3, results returned to S3 within 24 hours.

Key limitation that remains: Batch Inference does not support streaming, tool use with stateful loops, or Bedrock Agents orchestration. It is suited for offline enrichment, evaluation pipelines, and bulk classification — not interactive or multi-turn workloads.

1.4 Bedrock Model Distillation — GA and Updated Pricing

Bedrock Model Distillation reached GA on May 1, 2025 (technically Q2 2025, but operationally new for teams that deferred adoption). The pricing model has two components:

Synthetic data generation: Teacher model inference billed at standard on-demand rates (e.g., Nova Premier at $2.50/$12.50 per million input/output tokens).
Student model fine-tuning: Billed at customization training rates (example: Nova Pro fine-tuning at $0.008 per 1,000 tokens).

Supported teacher/student pairings at GA: Nova Premier → Nova Pro, Claude 3.5 Sonnet v2 → smaller Anthropic models, Llama 3.3 70B → Llama 3.2 1B/3B.

Performance claims from AWS: Distilled models run up to 500% faster and cost up to 75% less at inference time, with less than 2% accuracy degradation for RAG workloads. Validate the 2% claim on your actual task — it is measured on generic benchmarks. Domain-specific tasks may show higher regression.

On-demand inference for custom models (July 2025): As of July 2025, fine-tuned and distilled Nova models can run via on-demand inference at the same per-token rate as the base model — eliminating the requirement for Provisioned Throughput to serve custom models. This lowers the entry cost for distillation significantly.

2. AWS Cost Optimization Hub for AI Workloads

2.1 AWS FinOps Agent — Preview (June 2026)

The highest-signal announcement for FinOps practitioners in H1 2026 is the AWS FinOps Agent, entering public preview in the week of June 15, 2026. This is a distinct product from Cost Optimization Hub — it is a conversational agent built on Amazon Bedrock that integrates Cost Optimization Hub and AWS Compute Optimizer as data sources.

Capabilities confirmed in preview:

Natural-language queries against AWS cost data (e.g., “What is my Bedrock spend by model family this month compared to last month?”)
Rightsizing and idle resource recommendations surfaced from Cost Optimization Hub, including affected resource ARN, current vs. recommended configuration, estimated monthly savings, and implementation guidance
Savings Plans purchase recommendations
Cost anomaly detection with automated root-cause investigation — when an anomaly fires, the agent investigates and posts findings to a Slack channel
Jira ticket creation from recommendations (optional integration)
Scheduled recurring FinOps workflows (e.g., weekly cost report to a distribution list)

Pricing during preview: Free with a monthly usage cap. Production pricing not yet announced.

AI-specific coverage: Cost Optimization Hub itself does not yet have a dedicated Bedrock recommendation category surfaced in the console — current recommendations are compute-centric (EC2, RDS, Lambda). However, the FinOps Agent wraps both Cost Optimization Hub and CUR 2.0 data, so Bedrock spend analysis is accessible through natural-language queries against billing data even without structured recommendation entries.

Limitation: The FinOps Agent does not yet generate Bedrock-specific recommendations equivalent to “switch this workflow to Flex tier” or “this Knowledge Base vector store is idle.” Those require either manual CUR analysis or third-party FinOps tools. This is a gap to watch — the agent architecture makes adding Bedrock-specific recommendation modules straightforward, and AWS is likely to add them.

2.2 Billing and Cost Management MCP Server

At re:Invent 2025, AWS also launched the Billing and Cost Management MCP Server, enabling natural-language cost analysis inside developer environments (Claude Desktop, Amazon Q Developer). This is distinct from the FinOps Agent — it is an MCP-protocol integration for developer tooling rather than a standalone FinOps product. Platform engineers can query cost data conversationally within their IDE without switching to the AWS console.

2.3 Compute Optimizer Automation

Re:Invent 2025 introduced Compute Optimizer Automation: the ability to auto-apply Compute Optimizer recommendations without manual intervention. Currently scoped to EC2 and EBS — not yet Bedrock Provisioned Throughput or SageMaker instances. Worth monitoring as this automation surface expands.

3. SageMaker HyperPod Cost Updates

3.1 Resilient Training and Cost Mechanics

SageMaker HyperPod remains the recommended AWS path for large-scale distributed training (multi-node GPU/Trainium clusters). The resilient training capability — automatic checkpoint-to-S3 and cluster restart on node failure — has been in production since 2024 and is the primary cost justification for HyperPod over standard SageMaker Training Jobs.

The cost arithmetic:

Standard SageMaker Training Jobs bill from job start to job end. A node failure at hour 47 of a 72-hour training run results in 47 hours of wasted compute (the job fails and must restart from scratch unless you build your own checkpointing).
HyperPod with resilient training checkpoints periodically to S3. A node failure triggers replacement and resumes from the last checkpoint, typically losing 5–15 minutes of training versus 47 hours. For large Trn2 or P5e cluster runs, this alone can reduce effective training cost by 30–60% relative to retry-from-zero on standard jobs.

Pricing structure: HyperPod charges per HyperPod unit per second with no minimum. Spot Instance support adds an up-to-90% discount off on-demand HyperPod pricing for interruption-tolerant workloads — viable when combined with resilient training’s automatic restart capability.

HyperPod Flexible Training Plans: For sustained, predictable training schedules, HyperPod introduced Flexible Training Plans — reserved access to high-demand GPU capacity within specified time windows. These provide predictable access to Trn2 and P5e instance capacity, which has historically been constrained. Pricing details are negotiated; they reduce the spot-vs-on-demand uncertainty for planned training sprints.

3.2 SageMaker Savings Plans Coverage for HyperPod

SageMaker Savings Plans cover SageMaker Training Jobs and inference instances but do not cover HyperPod units directly — HyperPod has its own pricing dimension. Teams should not assume existing SageMaker Savings Plan commitments will apply to HyperPod spend. Verify coverage against the Savings Plans FAQ before migrating training workloads to HyperPod.

3.3 When HyperPod Is Cost-Effective

Training runs longer than ~8 hours on multi-node GPU clusters: resilient training’s node-failure insurance pays for itself
Workloads that can use Spot Instances with auto-restart: 90% discount vs on-demand creates substantial savings even accounting for interruptions
Teams that have previously absorbed failed training job costs at scale — HyperPod’s total cost of ownership story is built around eliminating the hidden cost of failed runs
Not recommended for: single-node fine-tuning jobs under 4 hours, inference workloads (use Inference Components or Bedrock instead), or teams without distributed training expertise to operate the HyperPod cluster

4. AWS re:Invent 2025 AI Cost Announcements

4.1 New Compute Instances

Trainium3 (Trn3) Generally Available AWS announced Trainium3 UltraServers at re:Invent 2025, with general availability in early 2026. Key specs: up to 144 Trainium3 chips per UltraServer, 362 FP8 petaflops of compute, 4x energy efficiency improvement over Trn2. Reported training and inference cost reductions of up to 50% versus Trn2 for supported model architectures.

Trn3 targets pre-training and large-scale fine-tuning of frontier models. For teams currently on Trn2 or P5e: Trn2 offers 30–40% better price-performance than P5e/P5en today; Trn3 extends that gap further for training workloads where Neuron SDK support is mature.

Trainium4 Preview Trainium4 was announced in preview at re:Invent 2025, targeting future large-scale frontier model training. No pricing announced; no GA timeline confirmed.

P6e GB300 (NVIDIA GB200) AWS introduced P6e instances using NVIDIA GB200 NVL72 systems, delivering approximately 20x the compute of prior P5en instances. Targets frontier-scale training and very large inference workloads. Pricing not publicly listed at time of research; expect premium pricing appropriate for frontier use cases.

4.2 Amazon Nova 2 Family (re:Invent 2025)

Amazon launched the Nova 2 family at re:Invent 2025 with four models:

Model	Context window	Input pricing	Output pricing	Notes
Nova 2 Lite	1.0M tokens	$0.30 / 1M	$2.50 / 1M	Fast, cost-effective reasoning
Nova 2 Pro	1.0M tokens	$0.30 / 1M	$10.00 / 1M	Advanced reasoning, agentic
Nova 2 Sonic	—	Speech-to-speech model	—	Real-time conversational AI
Nova 2 Omni	—	Multimodal (text, image, video, audio)	—	First unified multimodal reasoning model

Nova 2 Pro and Lite pricing represents a further reduction versus original Nova Pro ($0.80/$3.20) and Nova Lite ($0.06/$0.24). The Nova 2 series was designed to undercut comparable OpenAI and Google models on price while targeting enterprise accuracy requirements.

4.3 Amazon Kiro (re:Invent 2025)

Amazon Kiro, an AI-native IDE announced at re:Invent 2025, entered GA as Kiro Pro Max in the June 15, 2026 AWS Weekly Roundup. Relevant for FinOps only in that development-time AI tooling spend now appears alongside runtime inference spend in consolidated billing — teams should ensure cost allocation tags distinguish the two categories.

4.4 FinOps Infrastructure Announced at re:Invent 2025

Several infrastructure improvements announced in December 2025 affect how teams govern AI spend:

Database Savings Plans: New savings plan category offering up to 35% savings on RDS and other database services without upfront payment. Indirectly relevant for AI pipelines that rely heavily on Aurora or DynamoDB for vector storage or session state.
Flexible RI and Savings Plans Sharing Preferences: Customers can now configure how Reserved Instance and Savings Plans discounts are shared across accounts within an organization — important for multi-account Bedrock deployments where the AI team holds the commitment but individual business units generate the spend.
Cost Efficiency Score: A new unified metric in AWS Cost Management that aggregates rightsizing, idle resource, and Savings Plans coverage signals into a single account-level score. Useful for establishing a baseline before and after Bedrock cost optimization programs.
NAT Gateway Idle Detection: Surfaces idle NAT Gateway costs — relevant for ML pipelines that route model API calls or data transfers through NAT unnecessarily.

5. Amazon Nova Series Updates H1 2026

5.1 Nova Premier Recap

Nova Premier 1.0 launched October 31, 2025 (not re:Invent — it preceded the conference). Pricing: $2.50 per million input tokens / $12.50 per million output tokens, 1.0M token context window. Positioned as Amazon’s most capable model and the preferred teacher for Bedrock Model Distillation pipelines.

5.2 Nova Fine-Tuning — April 2026

As of April 8, 2026, Amazon Nova models support fine-tuning via Amazon Bedrock directly. Fine-tuning supports supervised fine-tuning (SFT), reinforcement fine-tuning (RFT), and distillation across the Nova model family. This expanded fine-tuning availability (previously only available through SageMaker AI for some Nova models) is meaningful for teams that want to stay within the Bedrock API surface.

Fine-tuning pricing example for Nova Pro: $0.008 per 1,000 tokens for training data, $1.95/month for custom model storage. A 5M-token fine-tuning job costs approximately $40 to train. Custom fine-tuned Nova models serve at the same on-demand inference rate as the base model — no Provisioned Throughput required (as of July 2025).

5.3 Nova 2 Customization

Nova 2 Lite offers the broadest customization surface of any proprietary model in Bedrock as of mid-2026: supervised fine-tuning, reinforcement fine-tuning, and distillation — available through both Bedrock and SageMaker AI. The Nova 2 family also supports multimodal fine-tuning (text, image, video) for domain adaptation to industry terminology and brand voice.

5.4 Gemma 4 Added to Bedrock (June 2026)

Gemma 4 from Google DeepMind became available on Amazon Bedrock in the week of June 15, 2026. Relevant for teams that benchmark against open-weight alternatives to Nova — Gemma 4 pricing on Bedrock has not been confirmed at time of research.

6. AWS AI Cost Management Roadmap Signals

6.1 CUR 2.0 Bedrock Granularity — IAM Principal Attribution (April 2026)

This is the most important cost visibility improvement in H1 2026. As of April 9, 2026, Amazon Bedrock supports cost allocation by IAM user and role in CUR 2.0 and Cost Explorer.

Mechanics: AWS records the caller IAM principal ARN for every Bedrock API call in the line_item_iam_principal column when IAM principal data collection is enabled on a CUR 2.0 export. Teams can tag IAM users and roles with attributes (team, project, cost_center) and activate those as cost allocation tags — they then appear in Cost Explorer and CUR 2.0 as filterable dimensions.

Previously, Bedrock spend appeared as a single line in CUR with no sub-account attribution. Now, a platform team can break down model inference cost by engineering team, product feature, or customer-facing application without requiring per-team AWS accounts or custom tagging schemes in application code.

Operational steps:

Enable IAM principal data in your CUR 2.0 export configuration
Tag IAM roles used by each application or team (cost_center=data-platform, project=search-ranking)
Activate those tags as cost allocation tags in Billing console (24-hour propagation)
Build Cost Explorer views or Athena queries against resourceTags/{key} columns

6.2 FOCUS Export Improvements

AWS CUR 2.0 and FOCUS exports now include granular Bedrock operation names — InvokeModelInference and InvokeModelStreamingInference — as separate line items, enabling per-operation cost attribution. Combined with IAM principal attribution, this provides the foundation for team-level, feature-level, and model-level Bedrock cost allocation without third-party tooling.

FOCUS 1.4 specifically: no confirmed public announcement of FOCUS 1.4 adoption timeline from AWS at time of research. AWS is tracking the FOCUS standard through FinOps Foundation channels but has not publicly committed to a 1.4 export release date.

6.3 Billing and Cost Management MCP Server

Announced at re:Invent 2025, the Billing and Cost Management MCP Server enables Claude (and any MCP-compatible client) to query AWS cost data through natural language. For FinOps practitioners already using Claude Code or Claude Desktop, this is a practical path to ad-hoc cost analysis without building Athena queries manually.

6.4 AgentCore Pricing — Gap to Watch

AWS AgentCore (announced 2025/2026, targets agentic AI orchestration infrastructure) does not yet have a transparent public pricing page comparable to Lambda or Bedrock. AgentCore workloads consume Bedrock inference, Lambda, and potentially custom runtime compute — the billing is disaggregated across those services rather than surfaced as a unified AgentCore cost center. Teams running agentic workloads at scale should implement CUR tagging strategies that can bridge AgentCore session IDs to application-level cost attribution before spend grows. AWS has signaled simplification in roadmap sessions but no concrete timeline.

6.5 Generative AI Cost Optimization — AWS Blog Signal (June 2026)

The AWS Cloud Financial Management blog has been publishing structured guidance on Bedrock cost optimization through H1 2026, including:

Model tiering and intelligent routing (25–40% cost reduction)
Prompt caching (up to 90% input token reduction for repeated context)
Application Inference Profiles for cross-region load balancing and cost allocation

Application Inference Profiles — introduced in late 2024 — allow teams to create named inference profiles that route requests across regions for resilience and that appear as distinct line items in CUR, enabling product-level cost attribution without IAM role tagging.

7. New SageMaker Cost Features

7.1 Inference Components — Multi-Model Cost Architecture

SageMaker Inference Components (launched late 2023, matured through 2025) enable hosting multiple large language models on a single GPU endpoint with dynamic model loading. Salesforce engineering published findings of up to 8x cost reduction versus dedicated-endpoint-per-model architectures. Inference Components load and unload model weights dynamically based on request traffic, maximizing GPU utilization across a model portfolio.

Key mechanism: Rather than keeping every model hot in GPU memory simultaneously, Inference Components evict idle models and load requested models from S3 on demand. Cold-start latency (typically 5–30 seconds for multi-billion-parameter models) is the trade-off. For low-QPS or bursty workloads with multiple models, this shifts the cost structure from fixed (all models hot) to semi-variable (GPU amortized across active models).

When it beats Bedrock on-demand: For teams with custom fine-tuned models, proprietary model weights, or models not available on Bedrock, Inference Components provide a cost-efficient self-managed inference path. For commodity foundation model inference, Bedrock on-demand (especially Flex tier) is typically simpler and comparable in cost.

7.2 Graviton for ML Inference

SageMaker supports Graviton3-based instances (ml.c7g family) for ML inference:

Up to 50% cost savings versus C5/C6i for PyTorch, TensorFlow, XGBoost, and scikit-learn inference
20% better price-performance versus x86 for CPU-bound inference
Supports SageMaker Savings Plans (same discount structure applies)

Applicability: Graviton3 is relevant for embedding generation, lightweight classification, scikit-learn pipelines, and ONNX models that run on CPU. Not applicable to GPU-dependent transformer inference. Teams running embedding pipelines for RAG at scale should benchmark ml.c7g against ml.m6i or ml.c6i — the savings can be substantial.

7.3 SageMaker Savings Plans — Coverage Scope

SageMaker Savings Plans offer up to 64% discount on a 3-year all-upfront commitment, covering:

SageMaker Training Jobs instances
SageMaker Inference endpoints (including Inference Components)
SageMaker Studio compute

Not covered: HyperPod units (separate pricing), Bedrock inference, SageMaker Feature Store (charged separately). Teams managing a mixed SageMaker + Bedrock portfolio should model Savings Plan coverage separately for each surface — a single SageMaker Savings Plan commitment does not cover Bedrock Provisioned Throughput.

Operational Priorities for Platform Engineers and FinOps Leads

Based on the above, the highest-ROI actions for H1 2026:

Enable IAM principal cost allocation in CUR 2.0 immediately — zero cost, high visibility payoff. Prerequisite for any Bedrock charge-back or team-level attribution program.
Audit Standard-tier Bedrock workloads for Flex eligibility — model evaluation pipelines, offline enrichment, batch classification, and any non-real-time inference should migrate to Flex (50% off). Requires only adding service_tier=flex to the API call.
Benchmark Bedrock Model Distillation for your top-cost task — if a Nova Premier or Claude 3.5 Sonnet v2 workflow runs at high volume, distilling to a smaller student model can reduce per-call inference cost by 75% after a one-time training investment. The break-even point at $0.008/1K training tokens is typically a few weeks of production inference volume.
Activate the AWS FinOps Agent preview — free during preview. The cost anomaly investigation and Cost Optimization Hub integration provide immediate value for teams without a dedicated FinOps toolchain.
Tag IAM roles with project and cost_center before Bedrock spend scales — retroactive tagging is difficult. Establish the taxonomy now while usage is still growing.
Evaluate Trn3 for next training cycle — 50% cost reduction vs Trn2, 4x energy efficiency. For teams already running HyperPod on Trn2, Trn3 migration is a straightforward compute upgrade with Neuron SDK compatibility.

Key Gaps and Limitations to Track

Cost Optimization Hub native Bedrock recommendations: Not yet available as a structured category. Current Hub recommendations remain compute-centric. The FinOps Agent partially fills this gap through natural-language querying but does not produce structured Bedrock-specific recommendation entries.
AgentCore cost transparency: No consolidated AgentCore billing surface. Spend is disaggregated across Lambda, Bedrock, and supporting services.
FOCUS 1.4 timeline: No confirmed AWS adoption date.
Flex tier model coverage: Not all Bedrock models support Flex or Priority tier as of mid-2026. Verify model compatibility before building tier-selection logic.
HyperPod Savings Plans gap: SageMaker Savings Plans do not cover HyperPod units — teams that shift training to HyperPod lose their existing SageMaker Savings Plan coverage on training compute.