← Agent Frameworks 🕐 14 min read
Agent Frameworks

Bedrock Application Inference Profiles — Multi-Tenant Cost Attribution (2026)

Executive Summary · March 2026

Application Inference Profiles (AIPs) are the primary mechanism for attributing Amazon Bedrock inference costs to specific teams, applications, or cost centers.

Application Inference Profiles (AIPs) are the primary mechanism for attributing Amazon Bedrock inference costs to specific teams, applications, or cost centers. An AIP is a user-created resource that wraps a foundation model (or system-defined cross-region inference profile) and carries AWS cost allocation tags that flow automatically into Cost and Usage Reports (CUR 2.0). Every InvokeModel and Converse API call made through an AIP ARN instead of a raw model ID generates billing records tagged with the AIP’s metadata. For multi-tenant platforms — shared Bedrock accounts serving multiple internal product teams — AIPs are the only native mechanism for per-application cost accountability without building a custom token-accounting sidecar.

As of April 2026, AWS added IAM principal-based cost allocation alongside AIPs, giving organizations two orthogonal tagging dimensions: resource-level (AIP tags) and identity-level (IAM user/role). The two mechanisms compose; a well-structured deployment uses both.

1. What AIPs Are

An Application Inference Profile is an AWS resource (AWS::Bedrock::ApplicationInferenceProfile) that:

  1. References a source model — either a single-region foundation model ARN or a system-defined cross-region inference profile ARN.
  2. Carries AWS resource tags (key-value pairs) that are registered as cost allocation tags in the Billing console.
  3. Exposes its own ARN in the format arn:aws:bedrock:{region}:{account-id}:application-inference-profile/{profile-id}.
  4. Acts as a drop-in replacement for the model ID string in InvokeModel, InvokeModelWithResponseStream, Converse, and ConverseStream calls.

The tagging is the entire point. When a caller invokes Bedrock using an AIP ARN, the resulting CUR line items inherit the AIP’s resource tags. This is equivalent to tagging an EC2 instance and having those tags appear on every hour of compute — except here it happens per billing aggregation period (daily) rather than per request.

AIPs do not change inference routing behavior on their own. They are a billing and observability wrapper. The actual routing decision (which region, which model variant) is handled by the underlying model source the AIP points to.

2. AIP vs System-Defined Inference Profiles (CRIS)

Bedrock exposes two profile types under the InferenceProfile resource family. The type field in GetInferenceProfile responses distinguishes them.

Dimension System-Defined (CRIS) Application (AIP)
Created by AWS You (account owner)
Purpose Cross-region traffic routing and capacity pooling Cost attribution and observability tagging
ARN prefix arn:aws:bedrock:{region}::foundation-model/... or region-set prefix arn:aws:bedrock:{region}:{account-id}:application-inference-profile/{id}
Carries user tags No Yes
Appears in CUR as tagged resource No Yes, with resourceTags/{key} columns
Supports cross-region routing Yes — distributes traffic across region set Only if copyFrom points to a CRIS ARN
Visible in AWS console cost views No (appears as raw model usage) Yes, once tags activated as cost allocation tags
IAM resource-based conditions Not applicable bedrock:InferenceProfileArn condition key

The critical relationship: An AIP wrapping a CRIS ARN gets both benefits — cross-region capacity pooling from the CRIS and cost attribution from the AIP tags. An AIP wrapping a single-region foundation model ARN gets cost attribution only, with no cross-region failover.

Naming note: “CRIS” is the informal name for system-defined cross-region inference profiles. AWS documentation also calls them “cross-Region inference profiles.” The us. and eu. prefixes on model IDs (e.g., us.anthropic.claude-3-5-sonnet-20241022-v2:0) are shorthand ARNs for system-defined profiles covering US or EU region sets.

3. Creating AIPs: boto3, CLI, and CloudFormation

boto3 (Bedrock control-plane client, not bedrock-runtime)

import boto3

bedrock = boto3.client("bedrock", region_name="us-east-1")

# AIP wrapping a single-region foundation model
response = bedrock.create_inference_profile(
    inferenceProfileName="payments-team-claude-sonnet",
    description="Claude Sonnet 3.5 v2 for the Payments engineering team",
    modelSource={
        "copyFrom": "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0"
    },
    tags=[
        {"key": "Team",        "value": "payments"},
        {"key": "CostCenter",  "value": "cc-4420"},
        {"key": "Environment", "value": "production"},
        {"key": "Application", "value": "fraud-detection-agent"},
    ]
)

profile_arn = response["inferenceProfileArn"]
# e.g. arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/cypje2y15yrd
print(f"Created: {profile_arn}")

# AIP wrapping a system-defined cross-region profile (recommended for production)
response = bedrock.create_inference_profile(
    inferenceProfileName="payments-team-claude-sonnet-xr",
    description="Cross-region Claude Sonnet for Payments — us.* region set",
    modelSource={
        "copyFrom": "arn:aws:bedrock:us-east-1::foundation-model/us.anthropic.claude-3-5-sonnet-20241022-v2:0"
    },
    tags=[
        {"key": "Team",        "value": "payments"},
        {"key": "CostCenter",  "value": "cc-4420"},
        {"key": "Environment", "value": "production"},
    ]
)

AWS CLI

aws bedrock create-inference-profile \
  --inference-profile-name "analytics-team-nova-pro" \
  --description "Nova Pro for the Analytics team data pipeline" \
  --model-source '{"copyFrom": "arn:aws:bedrock:us-east-1::foundation-model/amazon.nova-pro-v1:0"}' \
  --tags '[
    {"key": "Team",        "value": "analytics"},
    {"key": "CostCenter",  "value": "cc-8810"},
    {"key": "Environment", "value": "production"}
  ]' \
  --region us-east-1

CloudFormation / CDK

# CloudFormation snippet
AnalyticsTeamInferenceProfile:
  Type: AWS::Bedrock::ApplicationInferenceProfile
  Properties:
    InferenceProfileName: analytics-team-nova-pro
    Description: Nova Pro for Analytics team data pipeline
    ModelSource:
      CopyFrom: arn:aws:bedrock:us-east-1::foundation-model/amazon.nova-pro-v1:0
    Tags:
      - Key: Team
        Value: analytics
      - Key: CostCenter
        Value: cc-8810
      - Key: Environment
        Value: production

Tag activation (mandatory step)

Tags on AIPs do not appear in CUR automatically. After creating the AIP and its tags, activate each tag key as a cost allocation tag:

AWS Console → Billing → Cost Allocation Tags → User-Defined Cost Allocation Tags → Activate

Or via CLI:

aws ce create-cost-category-definition  # use Cost Explorer activation flow
# OR
aws billing activate-cost-allocation-tag \
  --tag-keys "Team" "CostCenter" "Environment"  # (Billing API, newer)

Tag activation propagates within 24 hours. CUR data before activation is not backfilled with the new columns.

Naming conventions

Adopt a consistent naming scheme across all AIPs to simplify querying and policy authoring:

{team}-{model-shortname}-{env}
# e.g.
payments-claude-sonnet-prod
analytics-nova-pro-prod
infra-claude-haiku-dev
shared-claude-sonnet-staging

The inferenceProfileName is mutable (update_inference_profile); the ARN is immutable. Store the ARN in your secrets manager or SSM Parameter Store, not the name.

4. How AIP Tags Appear in CUR

CUR 2.0 (the successor to CUR 1.0, now the default for new accounts) adds per-tag columns when tags are activated. For Bedrock AIPs:

CUR Column Source Example Value
resourceTags/user:Team AIP resource tag payments
resourceTags/user:CostCenter AIP resource tag cc-4420
iamPrincipal/arn Caller IAM principal (April 2026) arn:aws:iam::123456789012:role/payments-service
lineItem/ResourceId AIP ARN arn:aws:bedrock:us-east-1:...:application-inference-profile/cypje2y15yrd
lineItem/UsageType Bedrock usage dimension USE1-Bedrock-Claude:InputTokens
lineItem/UnblendedCost Dollar amount 0.0432

The resourceTags/user: prefix is standard AWS CUR format for user-defined tags. AWS-managed tags use resourceTags/aws: prefix. The prefix in CUR columns is resourceTags/user: even though the tag key itself is just Team.

Granularity note: CUR reports to daily granularity per usage type. There is no per-request line item. A day’s worth of InputTokens through a given AIP appears as a single aggregated cost record.

IAM principal columns (added April 2026) appear as iamPrincipal/arn and iamPrincipal/type (IAM User vs Role). These are independent of AIP tags — they track the identity of whoever called the API, not which AIP was used. Both columns appear on the same CUR row, enabling two-axis slicing (which team’s profile AND which role/service called it).

5. AIP Per-Team Architecture

Option A: One AIP per team/application (recommended for charge-back)

Create a dedicated AIP for each billing unit. This produces clean, unambiguous CUR rows with no post-hoc attribution logic.

Account: bedrock-shared-prod (123456789012)
│
├── AIP: payments-claude-sonnet-prod     → tags: Team=payments, CostCenter=cc-4420
├── AIP: analytics-nova-pro-prod         → tags: Team=analytics, CostCenter=cc-8810
├── AIP: fraud-claude-haiku-prod         → tags: Team=fraud, CostCenter=cc-4420
└── AIP: infra-claude-sonnet-dev         → tags: Team=infra, CostCenter=cc-0001, Env=dev

Each team is issued their AIP ARN and granted IAM permission to invoke it. Direct model invocation (bypassing an AIP) is denied via SCP.

SCP snippet to enforce AIP-only access:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowBedrockOnlyViaTaggedAIP",
      "Effect": "Deny",
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream",
        "bedrock:Converse",
        "bedrock:ConverseStream"
      ],
      "Resource": "*",
      "Condition": {
        "Null": {
          "bedrock:InferenceProfileArn": "true"
        }
      }
    }
  ]
}

This SCP denies any invocation that doesn’t specify an inference profile ARN — effectively requiring all callers to go through a registered AIP. Apply it at the OU level, exempting the profile-creation role.

Option B: Shared AIP with request-level tags

A single AIP can be used by multiple callers if you supplement with IAM principal attribution (April 2026 feature). This trades granular CUR attribution for operational simplicity — fewer profiles to manage, but cost breakdown requires joining on iamPrincipal/arn.

This pattern suits platforms where roles are already per-team (e.g., payments-service-role, analytics-pipeline-role) and you don’t want to maintain an AIP per team.

Option C: AIP per model tier, IAM principal per team

A middle path: one AIP per model (claude-sonnet-prod, nova-pro-prod, haiku-prod), plus IAM roles per team. Costs are broken down by model tier via AIP tags and by team via principal columns. This works well when teams share models and the finance query is “how much did Team X spend on Claude Sonnet?”

6. AIP Limitations

6.1 No support for custom models (fine-tuned / continued pre-training)

AIPs cannot wrap fine-tuned model ARNs directly. Fine-tuned models in Bedrock require Provisioned Throughput before they can be served; the resulting resource is a Provisioned Model ARN (arn:aws:bedrock:{region}:{account-id}:provisioned-model/{id}). AIP copyFrom accepts foundation model ARNs and system-defined profile ARNs only — not provisioned model ARNs.

Workaround: Tag the Provisioned Throughput resource itself with cost allocation tags. Provisioned Throughput charges (per-hour model unit costs) appear in CUR under lineItem/UsageType containing ProvisionedThroughput. Inference charges on top of a provisioned model are attributed to the provisioned model ARN, not to a profile.

6.2 No Guardrails passthrough tagging

Bedrock Guardrails apply at the request level via the guardrailIdentifier and guardrailVersion parameters. Guardrail evaluation costs appear as separate CUR line items (Bedrock-Guardrails:Units) and are attributed to the calling account’s Guardrail resource, not to the AIP ARN. You cannot tag Guardrail invocations through AIP tags — they need separate Guardrail resource tags.

Implication: If your platform charges teams for both inference and guardrail costs, you need two attribution mechanisms in parallel: AIP tags for inference + Guardrail resource tags for safety evaluation.

6.3 Region constraints

An AIP is a regional resource. You create it in a specific region and it can only be invoked in that region. If your platform serves multiple regions, create one AIP per region per team:

us-east-1: payments-claude-sonnet-prod   (arn:aws:bedrock:us-east-1:...)
eu-west-1: payments-claude-sonnet-prod   (arn:aws:bedrock:eu-west-1:...)

The profile ID will differ; the tags should be identical. Region appears in CUR via lineItem/UsageType prefix (e.g., USE1-, EUW1-).

6.4 Bedrock Agents and Knowledge Bases

Bedrock Agents internally invoke foundation models. If an Agent is configured to use a raw model ID, those invocations bypass AIPs. You must configure the Agent’s foundationModel field with an AIP ARN to get attribution. Test this before applying an SCP that blocks direct model invocation — Agent deployments can silently lose access.

6.5 No per-AIP spend limits

AIPs have no native budget or spend-cap functionality. You can apply AWS Budgets alerts filtered by the AIP’s tag values, but Bedrock will not throttle or reject requests when a budget threshold is crossed. Rate limiting at the AIP level requires an API Gateway or LiteLLM-layer enforcement mechanism.

6.6 bedrock-mantle endpoint exclusion

AIPs apply to the bedrock-runtime endpoint (InvokeModel, Converse). They do not apply to the bedrock-mantle endpoint (OpenAI-compatible chat/completions and responses APIs). For OpenAI-compatible workloads, use Bedrock Projects instead (see Section 7).

7. Bedrock Projects vs AIPs

Bedrock Projects (GA’d alongside the bedrock-mantle endpoint) and Application Inference Profiles serve overlapping but distinct purposes. The primary fork is which API your callers use.

Dimension Application Inference Profile Bedrock Project
API endpoint bedrock-runtime (InvokeModel, Converse) bedrock-mantle (OpenAI-compatible: chat/completions, responses)
Primary purpose Cost attribution via CUR tags Application isolation, access control, OpenAI API compatibility
Cost allocation CUR resourceTags columns CUR resourceTags columns (similar mechanism)
Access control granularity IAM + SCP on profile ARN Project-level IAM policies, API key scoping
Model support Foundation models + system-defined CRIS profiles Foundation models (no PT)
Guardrails Separate (not inherited) Can be attached at Project level
Cross-region Via CRIS source profile Project-level routing policies
Suitable for Internal platform teams using AWS SDK Teams migrating from OpenAI, using 3rd-party clients that speak OpenAI

Decision rule:

  • SDK/boto3/LangChain callers using InvokeModel or Converse → use AIPs.
  • Callers using openai.ChatCompletion pointing at a Bedrock endpoint → use Bedrock Projects.
  • Hybrid platform → create both, use the appropriate resource per call path.

What Bedrock Projects add over plain AIPs:

  • OpenAI-compatible endpoint routing (no client code changes for OpenAI SDK users).
  • Project-level Guardrails attachment (guardrail costs attributed to the Project).
  • API key management within the Project boundary.
  • Simpler multi-tenant isolation for non-AWS-SDK consumers.

Projects do not replace AIPs for SDK-native callers. The two coexist in the same account.

8. LiteLLM AIP Configuration

LiteLLM’s bedrock/ prefix route cannot parse ARN strings. Use bedrock/converse/ prefix or the model_id override parameter.

Config file (litellm_config.yaml)

model_list:
  # Per-team AIP mappings — each team gets a named virtual model
  - model_name: bedrock-payments-sonnet
    litellm_params:
      model: "bedrock/converse/anthropic.claude-3-5-sonnet-20241022-v2:0"
      model_id: "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/cypje2y15yrd"
      aws_region_name: "us-east-1"

  - model_name: bedrock-analytics-nova-pro
    litellm_params:
      model: "bedrock/converse/amazon.nova-pro-v1:0"
      model_id: "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/xwpqk7r9abcd"
      aws_region_name: "us-east-1"

  - model_name: bedrock-fraud-haiku
    litellm_params:
      model: "bedrock/converse/anthropic.claude-3-haiku-20240307-v1:0"
      model_id: "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/mnop3s4tuvwx"
      aws_region_name: "us-east-1"

Python SDK invocation

import litellm

response = litellm.completion(
    model="bedrock/converse/anthropic.claude-3-5-sonnet-20241022-v2:0",
    model_id="arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/cypje2y15yrd",
    messages=[{"role": "user", "content": "Summarize this quarter's fraud report."}],
    aws_region_name="us-east-1",
)

Known issue: bedrock/ vs bedrock/converse/

The bedrock/ prefix (which uses InvokeModel) fails with ARN model IDs — it treats the ARN as an unknown provider string. Always use bedrock/converse/ (the Converse API route) when specifying AIP ARNs via model_id. This is tracked in LiteLLM issue #18258 and #8911. As of LiteLLM v1.x in 2026, the model_id parameter on the bedrock/converse/ route is the stable path.

Router-level team isolation in LiteLLM

For a multi-tenant LiteLLM proxy, map API keys to virtual models backed by team-specific AIP ARNs:

router_settings:
  routing_strategy: "simple-shuffle"

model_list:
  - model_name: claude-sonnet  # generic name exposed to callers
    litellm_params:
      model: "bedrock/converse/anthropic.claude-3-5-sonnet-20241022-v2:0"
      model_id: "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/PAYMENTS_PROFILE_ID"
      aws_region_name: "us-east-1"
    model_info:
      id: payments-sonnet-aip
      team_id: payments  # LiteLLM virtual team

This binds each LiteLLM virtual team to its Bedrock AIP — spend tracked in LiteLLM’s own DB and in AWS CUR simultaneously.

9. Cost Attribution Precision: What AIPs Capture vs What They Miss

What AIP tags capture accurately

  • On-demand inference token charges: input tokens, output tokens, cache-read tokens, cache-write tokens (each appears as a separate CUR usage type).
  • All invocations through InvokeModel, InvokeModelWithResponseStream, Converse, ConverseStream when called with the AIP ARN.
  • Cross-region inference charges when the AIP wraps a CRIS profile (charges appear in the origin-region CUR).

What AIP tags miss or misattribute

Gap Reason Mitigation
Provisioned Throughput charges AIP cannot wrap PT models; PT billed per-hour regardless of usage Tag PT resources directly
Guardrail evaluation charges Guardrail CUR lines attributed to Guardrail resource, not AIP Tag Guardrails with matching team tags
Bedrock Agents overhead Agent orchestration token charges attributed to Agent ARN, not AIP, unless Agent’s FM is set to AIP ARN Configure Agent foundationModel with AIP ARN
Knowledge Base retrieval RAG retrieval charged per query, attributed to KB resource Tag KB resources with team tags
Fine-tuned model training Training jobs billed to the fine-tuning job ARN Tag fine-tuning jobs with cost allocation tags
Cross-account invocations AIP cannot be invoked cross-account; charges always in AIP’s account See Section 10
Sub-daily granularity CUR aggregates to daily; no per-request cost visible Use CloudWatch Estimated Quota Consumption metrics for real-time proxy

Coverage estimate: For a typical Bedrock deployment using only on-demand foundation models via SDK, AIPs capture ~85–90% of total Bedrock charges. The remaining 10–15% is Guardrails, Knowledge Bases, and Agent overhead.

10. Multi-Account AIP Strategy

Can you create an AIP in Account A and invoke it from Account B?

No. Application Inference Profiles are account-scoped resources. The ARN contains the account ID and the invoking principal must be in the same account as the AIP. Cross-account resource-based policies for AIPs are not supported — Bedrock does not implement an bedrock:InferenceProfile resource policy.

Consequence: In a multi-account AWS Organization where each product team has its own account, each account must create its own AIPs. You cannot centralize AIPs in a shared-services account and have spoke accounts invoke them.

Multi-account architecture patterns

Pattern 1: AIP per spoke account (recommended)

Each team account creates its own AIPs. Bedrock model access is granted via an Organization-level Bedrock policy or each account enables the model independently. CUR data flows to the management account’s Cost and Usage Report via consolidated billing.

Management Account
└── CUR aggregated cost view (all accounts, all AIP tags)

Payments Account (111111111111)
└── AIP: payments-claude-sonnet-prod → tags: Team=payments

Analytics Account (222222222222)
└── AIP: analytics-nova-pro-prod → tags: Team=analytics

The management account’s CUR contains all line items from all accounts. Athena queries on the management account CUR partition by lineItem/UsageAccountId + resourceTags/user:Team to produce cross-account, per-team dashboards.

Pattern 2: Centralized Bedrock account with cross-account IAM role assumption

Teams in spoke accounts assume a role in the Bedrock account to make inference calls. The Bedrock account holds all AIPs. Cost attribution is by AIP tag (team identity) rather than by account. The assumed role identity appears in iamPrincipal/arn for secondary validation.

This pattern reduces the number of AIPs (one per team regardless of account count) but requires cross-account role assumption in every inference call path — adds ~5ms latency and a credential refresh cycle.

Pattern 3: Per-account AIPs with SCP enforcement at OU level

Apply the SCP from Section 5 at the OU level. Each account is required to have AIPs before any inference calls work. Account vending automation creates a standard AIP set during account provisioning.

11. Athena Queries for AIP-Based Cost Breakdown

Assumes CUR 2.0 is configured to export to S3 and crawled by Glue into an Athena database named aws_cur with table cost_and_usage_report.

Query 1: Total Bedrock spend by team (monthly)

SELECT
    "resourceTags/user:Team"                          AS team,
    SUM("lineItem/UnblendedCost")                     AS total_cost_usd,
    SUM(CASE WHEN "lineItem/UsageType" LIKE '%InputTokens%'  THEN "lineItem/UsageAmount" ELSE 0 END) AS input_tokens,
    SUM(CASE WHEN "lineItem/UsageType" LIKE '%OutputTokens%' THEN "lineItem/UsageAmount" ELSE 0 END) AS output_tokens
FROM aws_cur.cost_and_usage_report
WHERE
    "lineItem/ProductCode" = 'AmazonBedrock'
    AND "lineItem/LineItemType" IN ('Usage', 'DiscountedUsage')
    AND DATE_TRUNC('month', CAST("lineItem/UsageStartDate" AS TIMESTAMP)) = DATE_TRUNC('month', CURRENT_DATE)
    AND "resourceTags/user:Team" IS NOT NULL
GROUP BY "resourceTags/user:Team"
ORDER BY total_cost_usd DESC;

Query 2: AIP-level daily spend (last 30 days, per AIP ARN)

SELECT
    DATE("lineItem/UsageStartDate")                   AS usage_date,
    "lineItem/ResourceId"                             AS aip_arn,
    "resourceTags/user:Team"                          AS team,
    "resourceTags/user:CostCenter"                    AS cost_center,
    "lineItem/UsageType"                              AS usage_type,
    SUM("lineItem/UnblendedCost")                     AS cost_usd,
    SUM("lineItem/UsageAmount")                       AS usage_amount
FROM aws_cur.cost_and_usage_report
WHERE
    "lineItem/ProductCode" = 'AmazonBedrock'
    AND "lineItem/LineItemType" IN ('Usage', 'DiscountedUsage')
    AND CAST("lineItem/UsageStartDate" AS TIMESTAMP) >= CURRENT_TIMESTAMP - INTERVAL '30' DAY
    AND "lineItem/ResourceId" LIKE '%application-inference-profile%'
GROUP BY 1, 2, 3, 4, 5
ORDER BY 1 DESC, 6 DESC;

Query 3: Two-axis breakdown — team + IAM principal (April 2026 columns)

SELECT
    "resourceTags/user:Team"                          AS team,
    "iamPrincipal/arn"                                AS caller_arn,
    DATE_TRUNC('week', CAST("lineItem/UsageStartDate" AS TIMESTAMP)) AS week_start,
    SUM("lineItem/UnblendedCost")                     AS cost_usd
FROM aws_cur.cost_and_usage_report
WHERE
    "lineItem/ProductCode" = 'AmazonBedrock'
    AND "lineItem/LineItemType" IN ('Usage', 'DiscountedUsage')
    AND DATE_TRUNC('month', CAST("lineItem/UsageStartDate" AS TIMESTAMP)) = DATE_TRUNC('month', CURRENT_DATE)
GROUP BY 1, 2, 3
ORDER BY 4 DESC
LIMIT 100;

Query 4: Bedrock cost not attributed to any AIP (governance gap detection)

SELECT
    DATE("lineItem/UsageStartDate")                   AS usage_date,
    "lineItem/UsageAccountId"                         AS account_id,
    "product/modelId"                                 AS model_id,
    SUM("lineItem/UnblendedCost")                     AS unattributed_cost_usd
FROM aws_cur.cost_and_usage_report
WHERE
    "lineItem/ProductCode" = 'AmazonBedrock'
    AND "lineItem/LineItemType" IN ('Usage', 'DiscountedUsage')
    AND (
        "lineItem/ResourceId" NOT LIKE '%application-inference-profile%'
        OR "resourceTags/user:Team" IS NULL
    )
    AND CAST("lineItem/UsageStartDate" AS TIMESTAMP) >= CURRENT_TIMESTAMP - INTERVAL '7' DAY
GROUP BY 1, 2, 3
HAVING SUM("lineItem/UnblendedCost") > 0.01
ORDER BY 4 DESC;

This last query is a governance health check — any rows returned represent inference spend that bypassed the AIP system and has no team attribution. Integrate it into a weekly FinOps report.

Notes on CUR table schema variance

  • Column names in CUR 2.0 use the format "lineItem/UsageType" with a forward slash. In legacy CUR 1.0 exports crawled by Glue, the columns may be normalized to line_item_usage_type (underscores, lowercase). Adjust queries accordingly.
  • Tag columns only exist after activation. If a tag was not activated before the billing period, the column is absent and the WHERE clause on it returns no rows rather than an error — silence that can mislead.
  • product/modelId in CUR reflects the raw model (not the AIP). Use lineItem/ResourceId to identify the AIP ARN.

12. AIP Quota Behavior

Quota inheritance

An AIP does not have its own independent TPM or RPM quota. It inherits the quota of the underlying foundation model (or system-defined profile) it wraps. All AIPs wrapping the same model share the same account-level quota pool for that model.

Example: Three AIPs all wrapping anthropic.claude-3-5-sonnet-20241022-v2:0:

  • payments-claude-sonnet-prod
  • analytics-claude-sonnet-prod
  • fraud-claude-sonnet-prod

All three draw from the same account quota for Claude Sonnet 3.5 v2 in that region. Heavy usage by the analytics team can throttle the payments team — the AIP boundary is a billing boundary, not a rate-limit boundary.

Quota mechanics (as of March 2026)

Bedrock quota consumption uses a reservation model: when a request arrives, Bedrock deducts input_tokens + cache_write_tokens + max_tokens from the TPM counter. After the response completes, the counter is corrected down to actual output token count. This means setting max_tokens=32000 when typical output is 500 tokens causes 64x over-reservation during the request window — a common cause of spurious throttling that AIPs do not protect against.

Monitor EstimatedTPMQuotaUsage in CloudWatch (available as a custom metric from the Bedrock quota dashboard sample project at aws-samples/sample-quota-dashboard-for-amazon-bedrock).

Per-AIP rate limiting

Since AIPs share model quotas, per-team rate limiting must be enforced at a layer above Bedrock:

  • LiteLLM proxy: Configure tpm_limit and rpm_limit per virtual key or team in litellm_config.yaml. LiteLLM enforces these client-side before forwarding to Bedrock.
  • API Gateway: Throttle by API key (one key per team) at the Gateway level before the request reaches the Bedrock-calling Lambda.
  • Service Quotas increase request: If total platform usage routinely hits model quotas, request a quota increase for that model via AWS Service Quotas console. Quotas are per-model per-region per-account.

Quota impact of cross-region AIPs

An AIP wrapping a CRIS system-defined profile draws quota from multiple regions. If us-east-1 is throttled, Bedrock routes the request to us-west-2 — drawing from that region’s quota. Total effective capacity is the sum across regions in the CRIS set. This is the primary operational reason to wrap CRIS profiles rather than single-region FMs.

Architecture Decision Matrix

Scenario Recommended AIP Pattern
Single-account, 3–10 teams, charge-back required One AIP per team per model tier, SCP enforcement
Single-account, teams share roles One AIP per model tier + IAM principal attribution
Multi-account org, teams in spoke accounts AIPs in each spoke account; aggregate via CUR consolidated billing
Multi-account org, centralized Bedrock Centralized AIP account with cross-account role assumption per team
OpenAI SDK callers Bedrock Projects (not AIPs)
Mixed SDK + OpenAI callers AIPs for SDK path, Projects for OpenAI path, same underlying model
Custom/fine-tuned models Tag PT resources directly; AIPs not applicable
Need per-team rate limits AIP for billing attribution + LiteLLM/API GW for rate limits
Compliance: all inference must be tagged SCP requiring bedrock:InferenceProfileArn condition

Key References