← Agent Frameworks 🕐 14 min read
Agent Frameworks

NVIDIA Agentic AI Safety Blueprint — and Its Deprecation

Brandon Sneider · May 2026

> **Source credibility:** NVIDIA TIER 2 — vendor-produced technical documentation and open-source code (Apache 2.0).

See also (wiki): wiki/agentic-ai-governance.md · wiki/ai-cybersecurity.md · wiki/model-risk-management.md · wiki/vendor-security-questionnaires.md · wiki/mcp-security-governance.md · wiki/hitl-deployment-pattern.md · wiki/data-readiness.md

Source credibility: NVIDIA TIER 2 — vendor-produced technical documentation and open-source code (Apache 2.0). Architecture claims are reproducible via the public GitHub repo (NVIDIA-AI-Blueprints/safety-for-agentic-ai). Evaluation dataset quality (garak prompts, WildGuardMix) is independently verifiable on HuggingFace. No independent red-team validation of post-training effectiveness claims. The deprecation notice (April 22, 2026) and successor product direction are from NVIDIA’s own announcement.

The Lead: This Blueprint Is Deprecated — and That Is the Story

On April 22, 2026, NVIDIA deprecated the Safety for Agentic AI blueprint. The replacement is three separate microservices: NeMo Auditor (pre-deployment vulnerability scanning), NeMo Guardrails (runtime safety, now standalone), and Safe Synthesizer (safe training data generation).

This deprecation is not a failure — it is the natural productization of a reference implementation. The blueprint showed enterprises what a 4-stage safety lifecycle looks like. The successors make each stage a purchasable, maintained service. Understanding the blueprint is still essential because it defines the vocabulary and decision points that the successor products are built around.

Why this matters for enterprise technology leaders: The pattern of the blueprint — scan before deployment, harden with post-training, deploy with guardrails, monitor at runtime — is becoming the vendor consensus for AI safety architecture. NVIDIA is not alone: this matches the structure of JPMC’s Lethal Trifecta risk classifier, Singapore’s MGF agentic AI framework, and OWASP’s LLM risk categories. The question for enterprise teams is not whether to implement this lifecycle but which vendor’s toolchain executes each stage.

The 4-Stage Safety Lifecycle

The blueprint organized model safety into four sequential stages, each targeting a different point in the AI development and deployment lifecycle:

Stage 1: Evaluate          Stage 2: Post-Train        Stage 3: Deploy          Stage 4: Runtime
garak vulnerability   →    SFT on curated        →    Package as NIM    →    NeMo Guardrails
scanning + dataset         safety datasets             (trusted, signed)       at inference
evaluation                 to close gaps

This is a lifecycle design principle, not just a toolchain recommendation. Each stage addresses failures the previous stage cannot catch:

  • Pre-training evaluation finds known vulnerability classes before any investment in the model
  • Post-training closes gaps that evaluation surfaces, using curated datasets aligned to the deployment policy
  • NIM deployment ensures the hardened model is cryptographically packaged and cannot be substituted
  • Runtime guardrails catch adversarial inputs and policy violations that got past training

Stage 1: garak Vulnerability Scanning

garak is NVIDIA’s open-source red-teaming tool. It runs automated adversarial probes against a model before deployment and produces an easy-to-understand safety and security report.

What garak scans for:

  • Hallucination: generates confidently stated falsehoods, factual inversions
  • Prompt injection: susceptibility to instructions embedded in retrieved content or user input designed to override system behavior
  • Jailbreaks: prompt patterns that bypass content safety restrictions
  • Goal misalignment: responds in ways inconsistent with the declared use case
  • Data exfiltration vectors: whether the model can be manipulated into leaking information from context or training data

“Curated risk prompts” in the blueprint context means garak does not use generic red-team prompts. The evaluation pipeline uses domain-specific prompt sets matched to the enterprise’s deployment context. This is the “Model Behavior Guide” translated into test cases — each behavior the guide defines as out-of-scope gets a corresponding adversarial probe.

Enterprise thresholds: The blueprint does not publish universal pass/fail scores. Instead, it calibrates thresholds against the enterprise’s own risk tolerance, defined in the Model Behavior Guide (see below). A customer service bot and a clinical decision support system have categorically different acceptable failure rates for prompt injection.

Evaluation datasets used (Stage 1):

  • Nemotron Content Safety Dataset V2 (nvidia/Aegis-AI-Content-Safety-Dataset-2.0 on HuggingFace): NVIDIA’s curated dataset for content safety classification
  • WildGuardMix (allenai/wildguardmix): Allen AI’s dataset covering harmful/safe instruction pairs with multi-perspective labels
  • NeMo Eval framework (NVIDIA/NeMo) for structured pipeline execution

Stage 2: Post-Training (Safety SFT)

The problem post-training solves: Evaluation in Stage 1 finds gaps. Post-training closes them. A base LLM that scores poorly on prompt injection resistance can be fine-tuned to refuse injected instructions — but only if the fine-tuning dataset contains representative examples of the attack pattern and the correct refusal behavior.

The SFT recipe uses a blend of four dataset types:

Dataset Source Purpose
Nemotron Content Safety Dataset V2 NVIDIA / HuggingFace Safe/unsafe interaction classification with human labels
Gretel Synthetic Safety Alignment gretelai/gretel-safety-alignment-en-v1 Synthetically generated safety-aligned instruction pairs
HarmfulTasks CrystalEye42/eval-safety (GitHub) Tasks with known harmful completion patterns; teaches refusal
Llama Nemotron Post Training Dataset nvidia/Llama-Nemotron-Post-Training-Dataset NVIDIA’s curated instruction-following and safety alignment data

Additional datasets for red-team coverage:

  • JailBreakV-28k / RedTeam 2K: 2,000 adversarial jailbreak prompts for measuring resistance
  • WildGuardMix: dual-use dataset covering both harmful and benign instruction variants

On-policy prompt generation: The recipe also generates new training prompts using the target model itself — the model being hardened generates responses to novel safety-relevant prompts, which are then labeled and added to the training blend. This is the “on-policy” approach: training data reflects the actual distribution of the model’s outputs, not just a static dataset.

Post-training framework: NVIDIA NeMo RL (NVIDIA/NeMo-RL), which scales from 1 GPU to 100B+ parameter models. Minimum hardware for the post-training stage: 8× H100 or A100 80GB. This is not a developer laptop operation.

What post-training addresses that Stage 1 cannot: Evaluation identifies that a model is vulnerable to prompt injection. Post-training actually changes the model’s weights to be more robust. The distinction matters for enterprise governance: evaluating a model and finding it vulnerable without closing the gap is insufficient due diligence for high-stakes deployments.

What post-training does not fix: Runtime adversarial inputs that are novel relative to the training distribution. This is why Stage 4 (NeMo Guardrails) exists — the assumption is that training will reduce vulnerability but not eliminate it.

Stage 3: NIM Packaging and Deployment

After post-training, the hardened model is packaged as a NIM (NVIDIA Inference Microservice). This step matters for enterprise governance for reasons beyond performance:

  • Cryptographic signing: NIM packages are signed. The NIM deployed in production is verifiably the model that was evaluated and hardened in Stages 1 and 2, not a substituted or modified version.
  • Deployment standardization: NIM containers expose standard OpenAI-compatible APIs. The safety-hardened model deploys identically to an off-the-shelf model — no custom integration required.
  • NGC provenance: Models pulled from NVIDIA NGC have a verifiable chain of custody. Enterprise risk teams can document where the model came from and what validation it underwent.

For regulated industries (banking, healthcare, insurance), the audit trail from Stage 1 evaluation through Stage 2 fine-tuning to Stage 3 signed NIM deployment is the documentation needed for model risk management review. See model-risk-management.md.

Stage 4: NeMo Guardrails at Inference Runtime

NeMo Guardrails is the runtime safety layer. It runs programmable logic before and after every LLM call, operating as middleware between the application and the model.

What it blocks at runtime:

  • Content safety violations: Detected by the NemoGuard 8B Content Safety model (Llama 3.1-based, multilingual). Covers explicit content, dangerous instructions, and policy-violating requests.
  • Topic drift: Topic Control model constrains the system to its defined domain. If a customer service bot receives a prompt asking it to discuss competitors or provide legal advice, Topic Control intercepts.
  • Jailbreak attempts: Prompt patterns that attempt to override the system prompt or change the model’s declared identity. (Note: dedicated jailbreak detection NIM is listed as not yet generally available in the blueprint version.)

How NeMo Guardrails works architecturally:

  1. User input arrives at the application
  2. Guardrails middleware intercepts the input and runs classification (content safety, topic control)
  3. If the input is safe, it passes to the LLM; if unsafe, a configured refusal response is returned without the LLM ever seeing the input
  4. LLM generates a response
  5. Guardrails middleware intercepts the output and runs output-side classification
  6. Safe output is returned; unsafe output triggers a fallback response

The “programmable logic” framing means guardrails policies can be expressed as Colang rules (NeMo Guardrails’ policy language), not just model inference. This allows deterministic rules (“if the query contains [competitor name], return [specific response]”) alongside model-based classification. This is a material advantage over pure model-based safety — deterministic rules cannot be jailbroken.

NeMo Guardrails hardware requirements in the RAG blueprint context:

  • Content Safety model: dedicated GPU with 48GB VRAM (H100 or A100)
  • Topic Control model: dedicated GPU with 48GB VRAM (H100 or A100)
  • B200 GPUs not supported for guardrails (as of the blueprint’s deprecation date)

The Deprecation: What the Successors Do

As of April 22, 2026, NVIDIA recommends three distinct microservices in place of the monolithic blueprint:

NeMo Auditor

Replaces Stage 1 (garak scanning) with a managed service. Key differences from the blueprint’s evaluation pipeline:

  • Managed SaaS delivery (no self-hosted garak runner required)
  • Continuous scanning against updated vulnerability catalogs (the blueprint required manual refresh of garak probes)
  • Structured reporting aligned to enterprise GRC workflows, not just raw pass/fail scores
  • API-first design for integration with CI/CD pipelines — run audit on every model version update

Enterprise implication: NeMo Auditor industrializes what the blueprint required manual engineering to set up. For enterprises that cannot staff an AI red-team, this is materially lower operational cost.

NeMo Guardrails (Standalone)

Previously embedded in the blueprint as Stage 4, now a standalone product. Key additions beyond the blueprint:

  • Colang 2.0 policy language (more expressive than the blueprint’s configuration)
  • Support for multi-agent workflows — guardrails can be applied at each agent-to-agent handoff, not just at the human-facing API boundary
  • Managed policy update cadence (new threat patterns pushed as updates, not requiring redeployment)

Enterprise implication for agentic AI: In multi-agent systems, the attack surface is every inter-agent call, not just the user-facing endpoint. Guardrails at the human interface only is insufficient for agentic architectures where one compromised agent can instruct others. This connects directly to the threat models in jpmc-lethal-trifecta-agentic-ai-security.md (Lethal Trifecta: autonomy + tool access + inadequate oversight) and a2a-mcp-protocol-security.md (A2A trust escalation, cross-agent contamination).

Safe Synthesizer

Addresses Stage 2 (post-training data generation) as a managed capability. Instead of the blueprint’s manual dataset blending:

  • Automated synthetic data generation aligned to a provided Model Behavior Guide
  • On-policy generation managed by NVIDIA infrastructure (not requiring the enterprise’s own 8× H100 cluster for SFT)
  • Integration with NeMo Auditor’s evaluation outputs — gaps found in auditing automatically inform what synthetic data to generate

Enterprise implication: The blueprint’s post-training recipe required 8× H100/A100 and significant MLOps capability to execute. Safe Synthesizer moves this to a managed service. For most enterprises, this is the only viable path to policy-aligned fine-tuning — few have the GPU cluster and ML engineering staff to run SFT in-house.

Best Practices for Model Behavior Guide

NVIDIA’s best-practices-model-behavior-guide.md defines the north star document that every stage of the safety lifecycle references. Synthesized here because it defines the inputs to garak thresholds, post-training datasets, guardrails topic rules, and NeMo Auditor’s evaluation criteria.

A Model Behavior Guide is a codification of deployment context, not a generic safety policy. It answers:

  1. Who is the user? Role, industry, use case, language, country of deployment. A model deployed for healthcare clinicians in the EU has different behavior requirements than one deployed for retail customer service in the US.

  2. What behavior is required?

    • Core principles: factuality, coherence, instruction following
    • Persona: how the model presents itself
    • Tone: professional, conversational, neutral
    • Formatting: structured outputs, response length, codeblock usage
    • Verbosity: specific to the use case (50-100 characters for summaries vs. multi-paragraph for technical explanations)
    • Content safety standards: what the model will and will not produce
    • Product security standards: how the model handles adversarial inputs

  3. What are the edge cases? Ambiguous prompts, technically impossible requests, adversarial probes, multi-language inputs, inputs with errors or typos. The guide specifies desired behavior for each category.

The guide’s role in the lifecycle:

  • Informs garak probe selection (which attack types to prioritize based on deployment context)
  • Defines the labeling criteria for post-training datasets (what is “safe” for this specific deployment)
  • Configures NeMo Guardrails topic control rules
  • Establishes the evaluation criteria for NeMo Auditor scans

“Model safety becomes enforceable, not aspirational” — NVIDIA’s framing. The argument is that safety claims without a Model Behavior Guide, a garak scan against it, and runtime guardrails calibrated to it are marketing, not governance. This is operationally correct: a generic “we use responsible AI practices” claim cannot be audited; a Model Behavior Guide with garak scan results and guardrail configurations can be.

Mapping to Existing Research Frameworks

Singapore Model AI Governance Framework — Agentic AI Addendum (see singapore-model-ai-governance-agentic-ai-2026.md)

Singapore’s MGF defines four dimensions for agentic AI governance: Transparency, Human Oversight, Accountability, and Security. The NVIDIA safety lifecycle maps:

MGF Dimension NVIDIA Stage
Transparency Model Behavior Guide (defines intended behavior explicitly) + NIM provenance chain
Human Oversight Not addressed by the blueprint — HITL must be inserted at application layer
Accountability Stage 1 evaluation reports + Stage 2 training logs + Stage 3 NIM signing
Security Stage 1 garak (pre-deployment) + Stage 4 NeMo Guardrails (runtime)

Gap: The blueprint has no HITL mechanism. Singapore’s MGF explicitly requires human oversight checkpoints for high-risk agentic decisions. The blueprint’s architecture assumes automated pipeline execution. Enterprises combining the NVIDIA blueprint with agentic AI deployments must insert HITL at the application layer — the toolchain will not do it. See enterprise-agentic-ai-operations-playbook.md for the HITL vs. HOTL pattern.

JPMC Lethal Trifecta (see jpmc-lethal-trifecta-agentic-ai-security.md)

JPMC’s risk classifier flags systems as high-risk when they combine: (1) autonomous decision-making, (2) tool/API access, (3) inadequate oversight. The NVIDIA blueprint directly addresses factor 3 — NeMo Guardrails + the evaluation pipeline constitute the “adequate oversight” layer for models deployed as NIMs. But factors 1 and 2 are outside the blueprint’s scope: the blueprint does not constrain what the agent decides or which tools it can call. Those require the ZSP/JIT permission controls analyzed in zsp-jit-agent-permissions-enterprise.md.

OWASP LLM Top 10 and NIST AI RMF

OWASP LLM Top 10 threat categories addressed by the blueprint:

OWASP Threat Blueprint Stage
LLM01 — Prompt Injection Stage 1 (garak jailbreak/injection scans) + Stage 4 (runtime guardrails)
LLM02 — Insecure Output Handling Stage 4 (output-side content safety filtering)
LLM04 — Model Denial of Service Partially (topic control limits scope)
LLM06 — Sensitive Information Disclosure Stage 2 (post-training for data exfiltration resistance)
LLM08 — Excessive Agency Not addressed — blueprint does not constrain agent autonomy
LLM09 — Overreliance Not addressed — human oversight is out of scope

NIST AI RMF alignment:

  • Govern: Model Behavior Guide = the documented policy artifact NIST requires
  • Map: Stage 1 garak scanning = AI risk identification
  • Measure: Evaluation datasets (WildGuardMix, AEGIS) = AI risk measurement
  • Manage: Stage 2 post-training + Stage 4 guardrails = risk treatment

Cisco AI Defense (see cisco-ai-defense-token-audit.md)

Cisco’s AI Defense implements a four-point lifecycle interception. The NVIDIA blueprint maps closely:

Cisco Stage NVIDIA Equivalent
Discover No equivalent (Cisco inventories all AI in the enterprise; NVIDIA operates on a single model)
Validate Stage 1: garak scanning + dataset evaluation
Protect Stage 4: NeMo Guardrails at inference
Monitor Prometheus/Zipkin observability stack

The Cisco approach adds the enterprise inventory dimension (what AI is running, where) that the NVIDIA blueprint does not address. For enterprise-wide AI governance, the Cisco AI Defense architecture + NVIDIA’s per-model safety lifecycle are complementary, not competitive.

Hardware Requirements Summary

Stage What Runs Hardware Required
Stage 1: Evaluation garak + NeMo Eval + vLLM for target model 8× H100 or A100 80GB (for model under test)
Stage 2: Post-Training NeMo RL SFT 8× H100 or A100 80GB, 300GB storage, 128GB RAM, Python 3.12
Stage 3: NIM Packaging NIM container build Moderate (build machine, not serving cluster)
Stage 4: Guardrails NemoGuard Content Safety 8B + Topic Control 8B 2× dedicated GPUs, 48GB VRAM each

The implication: full implementation of the 4-stage lifecycle requires an H100/A100 cluster at two points (evaluation and post-training). This is not accessible to most enterprises without cloud GPU access. The successor products (NeMo Auditor, Safe Synthesizer) are clearly positioned to move stages 1 and 2 to managed services specifically because of this hardware barrier.

Enterprise Decision Framework: Where This Fits in Your AI Governance Stack

The NVIDIA safety lifecycle fills one specific gap: it tells you how to harden a specific open-weight model for a specific deployment context. It does not tell you which models to evaluate, how to govern your overall AI portfolio, or how to implement human oversight.

Use the blueprint/successor products when:

  • Deploying open-weight models (Llama, Nemotron, Mistral) that need safety fine-tuning before enterprise deployment
  • Regulated industry deployment (finance, healthcare) where model risk management requires documented evaluation and hardening
  • Agentic AI deployment where runtime guardrails are required as a condition of deployment
  • Building an AI safety program from scratch and need a reference architecture to start from

Do not rely on this alone when:

  • Deploying closed-API models (OpenAI, Anthropic, Google) — post-training is not an option; focus on Stage 4 guardrails and procurement-side controls
  • The governance requirement is enterprise AI inventory and policy (use Cisco AI Defense, Wiz AI-SPM, or equivalent)
  • HITL checkpoints are required by policy — insert at the application layer, the NVIDIA stack will not do this for you
  • The threat model includes inter-agent attacks in a multi-agent system — NeMo Guardrails standalone is designed for this; the deprecated blueprint was not

The “enforceable, not aspirational” bar: For Fortune 500 technology and risk leaders, the practical test of this framework is whether a risk committee can review the outputs of each stage and make an approval decision. Stage 1 garak reports are readable by non-engineers. Stage 2 training logs are auditable. Stage 3 NIM signing provides provenance. Stage 4 guardrail configurations are documentable as policy. This lifecycle produces the artifacts that model risk management frameworks (including OCC SR 11-7 equivalent interpretations for AI) require.

Sources

  • NVIDIA-AI-Blueprints/safety-for-agentic-ai GitHub repository (Apache 2.0), accessed May 2026
  • NVIDIA deprecation notice, April 22, 2026 (embedded in README)
  • NVIDIA garak open-source repository (NVIDIA/garak)
  • NVIDIA NeMo RL (NVIDIA/NeMo-RL)
  • NVIDIA NeMo Guardrails (NVIDIA/NeMo-Guardrails), developer.nvidia.com/nemo-guardrails
  • HuggingFace: nvidia/Aegis-AI-Content-Safety-Dataset-2.0
  • HuggingFace: allenai/wildguardmix
  • HuggingFace: nvidia/llama-3.1-nemoguard-8b-content-safety
  • best-practices-model-behavior-guide.md, NVIDIA-AI-Blueprints/safety-for-agentic-ai/docs
  • NeMo Auditor, Safe Synthesizer product announcements (NVIDIA, April 2026)