← Agent Frameworks 🕐 10 min read

Agent Frameworks

Enterprise Agentic AI Operations Playbook: Six Decisions Before Any Agent Goes Live

Brandon Sneider · May 2026

> **Source credibility: MIXED — HIGH for sourced components. TIER 1.**

See also (wiki): wiki/enterprise-agent-runtime-infrastructure.md · wiki/agentic-ai-governance.md · wiki/agent-operations-observability.md

Source credibility: MIXED — HIGH for sourced components. TIER 1. This note synthesizes practitioner conference observations (May 2026) with sourced frameworks: Lenovo Hybrid AI Advantage (Lenovo Press/StoryHub), Singapore MGF for Agentic AI (IMDA Jan 2026), JPMC agentic security blog (March 2026), and published HITL/HOTL governance literature. Conference-derived observations are marked [Practitioner obs.] — directionally validated by multiple independent sources but not independently citeable.

Executive Summary

This playbook consolidates the operational patterns that distinguish enterprises successfully running agents in production from those still piloting. Six decision points that must be answered before an agent goes live:

Where does inference run? (Lenovo hybrid routing model)
What is the agent’s identity and permission scope? (JIT-revokable, not standing)
HITL or HOTL? (oversight model by risk tier)
Does the agent hit data first? (retrieval before generation — re-ranking architecture)
Is there a kill/contain/scale protocol? (defined before deployment, not after)
What is the audit trail? (agent registry + tamper-evident logs)

1. Hybrid Inference Routing — Where Does the Inference Run?

Lenovo’s framework, published in its 2026 hybrid AI guidance (Lenovo Press, LP2368), defines three inference tiers with different cost, latency, and governance profiles:

Tier	Location	Best for	Cost/token	Governance
Device	AI PC / workstation / Mac	Dev, validation, sensitive data	Lowest	Highest control
Edge / On-prem	Local server, private cloud	Production workloads >4h/day	Low	High control
Cloud	Hyperscaler or API	Burst, experimentation, low-volume	Highest	Lowest control

Routing decision rule (Lenovo): Place inference as close to the data source as possible. Unnecessary cloud hops add latency, token cost, and data-transfer risk.

Economic trigger (Lenovo TCO 2026): At ≥4 hours daily GPU utilization, on-prem beats cloud on TCO. At high utilization: 8× lower cost/token vs. cloud IaaS, 18× vs. frontier MaaS. Break-even: 4 months in 2026 (was 17 months in 2024).

Practical routing logic:

Data is classified / regulated → on-prem or device, no exceptions
Workload >1M tokens/day sustained → run the on-prem math
Burst or one-off → cloud
Development and red-teaming → device tier (no production data)

Source: Lenovo Hybrid AI StoryHub · Lenovo TCO 2026

2. HITL vs. HOTL — Oversight Model by Risk Tier

The most important governance decision for each deployed agent is not whether to include human oversight, but what form it takes:

Human-in-the-loop (HITL): Human must approve before the agent acts. Used for:

Irreversible actions (send email, execute transaction, delete data, call external API)
High-stakes decisions (above a dollar threshold, customer-facing, regulatory-touching)
Novel situations outside the agent’s tested scope
Any action in the JPMC Lethal Trifecta zone (untrusted input + sensitive data + external action)

Human-on-the-loop (HOTL): Agent acts; human monitors and can intervene. Used for:

High-volume, low-risk, reversible actions
Well-tested, narrow-scope tasks with defined expected outputs
Batch processes where per-action review is operationally impossible

Why the distinction matters: A HITL agent that makes a bad recommendation costs minutes. A HOTL agent that mishandles 500 customer interactions before a human notices costs customer relationships and regulatory exposure. The oversight model must match the blast radius.

Enterprise default: Most organizations in 2026 use HITL for novel deployments and migrate to HOTL only after (a) the agent has a validated track record in production, (b) semantic observability is in place to catch confidence decay, and © a kill protocol exists. [Practitioner obs.]

Source: Strata.io HITL Guide 2026 · Singapore MGF Dimension 2 · EU AI Act Article 14

3. Data First — Retrieval Before Generation

The single most common agent failure mode in production is not hallucination — it is the agent generating an answer from its training distribution when it should have retrieved current, specific organizational data first. [Practitioner obs.]

The pattern: retrieval → re-ranking → generation, not generation alone.

Agent receives task
Retrieves relevant documents, records, or data from internal sources
Re-ranks retrieved results by relevance to the specific task (not just embedding similarity)
Generates response grounded in retrieved context

Why re-ranking matters: embedding similarity retrieval returns semantically adjacent documents, not always the most relevant ones for a specific enterprise task. A re-ranking model (cross-encoder or LLM-as-judge) scores retrieved candidates against the actual query before passing them to the generation model. This step catches cases where the top embedding match is topically related but factually wrong for the question.

Operational implication: Agents that skip or shortcut retrieval amplify hallucination risk at scale. The data retrieval architecture should be validated before the generation model is. Build the retrieval pipeline first; stress-test it. The generation model can only be as accurate as what it’s given.

4. Kill / Contain / Scale Protocol

Document this before you need it. Enterprises that define escalation protocols after a production incident are responding under pressure with no framework. [Practitioner obs.]

Three states, three responses:

State	Trigger criteria	Response
Kill	Agent produces harmful output; fails safety checks; breaches authorization boundary; confidence decay detected below threshold	Immediate halt. Agent removed from production. Incident log opened. Root cause required before redeployment.
Contain	Agent behaves unexpectedly but within safe bounds; output quality degrading; edge case not in test coverage	Restrict scope. Elevate to HITL. Notify operator. Log all actions until resolution. No scale-out.
Scale	Agent has track record (N runs, M% acceptance rate, L human overrides); semantic quality metrics stable; HITL has been progressively relaxed and holds	Authorized for increased volume, reduced oversight frequency, or migration from HITL to HOTL.

Pre-deployment requirements for each state:

Kill: define the trigger conditions, the notification path, and who has authority to halt
Contain: define what “unexpected” looks like operationally — not “something goes wrong” but specific measurable signals (override rate >X%, confidence score <Y, latency spike, etc.)
Scale: define the acceptance criteria explicitly. “It seems to be working” is not a criterion.

Singapore MGF alignment: Dimension 1 (assess and bound risks upfront) requires defining action scope before deployment — this is the same requirement operationalized at the runtime level.

5. Agent Registry and Audit Trail

An agent registry is a catalog of every deployed agent with its identity, permission scope, oversight model, and operational history. This is the governance artifact that makes everything else auditable. [Practitioner obs., corroborated by Singapore MGF + JPMC security blog]

Minimum viable agent registry entry:

Agent ID:           [unique, traceable]
Accountable human:  [named individual + role]
Permission scope:   [explicit list — read/write access per system]
Oversight model:    HITL / HOTL / hybrid
Kill criteria:      [defined threshold conditions]
Contain criteria:   [defined threshold conditions]
Scale criteria:     [acceptance gates]
Deployment date:    [date]
Last audit:         [date + result]
Incident log:       [linked]

Audit trail requirements (from JPMC):

Tamper-evident runtime logs — every tool call, every external action
Traceable from action → agent → authorizing human
Sufficient to support incident investigation and regulatory inquiry

The off-switch requirement: Every agent must have an identified off-switch: a named person, a system flag, or an automated trigger that can halt the agent immediately. The Singapore MGF calls this “meaningful human accountability” — it requires not just that a human is nominally responsible, but that the human has the technical capability to stop the agent.

6. Validation Gates: What Gets Checked Before and During Production

Two distinct validation moments — often conflated, leading to underpowered production monitoring:

Pre-deployment validation gates:

Accuracy on task-specific test set (not just a public benchmark)
Policy compliance testing — does the agent respect organizational rules, not just avoid factual errors?
Adversarial testing — prompt injection attempts on the specific data sources and tools it will use
Red-team by someone outside the build team

Production validation gates (continuous):

Semantic quality monitoring: output quality metrics, not just latency and error rate (see confidence decay in jpmc-lethal-trifecta-agentic-ai-security.md)
Human override rate — if humans are consistently overriding the agent, the HITL check is catching something the pre-deployment testing missed
Authorization boundary violations — any attempt to access a system outside the registered scope
Confidence score distribution — flag when average confidence on a task class shifts downward

Decision Checklist: Agent Pre-Deployment

Before any agent goes to production, this checklist must be complete:

[ ] Inference tier decided (device / edge+on-prem / cloud) and documented with rationale
[ ] Agent identity registered in the agent registry with accountable human named
[ ] Permission scope explicit, least-privilege, JIT-revokable (not standing grants)
[ ] Oversight model determined: HITL or HOTL, with written rationale
[ ] Retrieval architecture validated before generation model testing
[ ] Kill / contain / scale criteria written and approved
[ ] Off-switch tested — someone actually ran it in staging
[ ] Audit trail enabled and tested — action log is tamper-evident
[ ] Adversarial testing complete (prompt injection on production data sources)
[ ] Semantic observability in place — quality metrics, not just latency metrics

What This Means for Your Organization

The six-decision framework is the pre-flight checklist for agentic AI. Organizations that ask these questions before deployment avoid the most common failure modes: agents with excessive standing permissions, oversight models mismatched to blast radius, and production incidents with no defined kill protocol.

The registry requirement is the highest-value and most-skipped step. A registry entry takes 30 minutes. An incident investigation without one takes months. The off-switch requirement — that someone must be able to halt the agent immediately, not in principle but technically — is the test that reveals whether governance is real or performative. If the test reveals no one knows how to stop the agent, that is the answer to whether the agent should be in production.

The HITL/HOTL distinction matters more than organizations realize at deployment time. The correct oversight model depends on reversibility, blast radius, and the agent’s track record — not on operational convenience. Defaulting to HOTL to avoid latency without first establishing a validated track record is how production incidents accumulate.

If the agent pre-deployment checklist surfaces gaps in an existing program, it is worth a focused conversation — brandon@brandonsneider.com.

Key Data Points

Finding	Value	Source	Date	Tier
Inference routing threshold	≥4h/day GPU utilization → on-prem	Lenovo LP2368	2026	TIER 1
HITL default for novel deployments	High blast-radius / irreversible actions require human review	Singapore MGF + JPMC blog	Jan 2026 / Mar 2026	TIER 1
Lethal Trifecta criteria	Untrusted input + sensitive data + external action = HITL required	JPMC Securing Agentic AI	Mar 2026	TIER 1
Kill protocol requirement	Written before deployment, not after incident	Practitioner consensus	May 2026	TIER 2
Audit trail standard	Tamper-evident, traceable to named human	JPMC + Singapore MGF	Mar 2026 / Jan 2026	TIER 1
Off-switch requirement	Named person or system flag, tested in staging	Singapore MGF Dim. 4	Jan 2026	TIER 1

Sources

Emerging Thesis: Agents Building Agents

[Practitioner obs., Institute for AI Transformation, May 2026]

The prevailing near-term prediction from enterprise AI practitioners: agents will soon design, configure, and deploy other agents autonomously — without human architects specifying the agent’s tools, permissions, retrieval scope, or oversight model.

Why this is a structural shift, not an incremental one:

Today’s agent deployment model assumes a human answers the six pre-deployment questions (inference tier, permission scope, HITL vs HOTL, retrieval architecture, kill protocol, audit trail) before any agent goes live. That assumption is the foundation of every governance framework published in 2026 — Singapore MGF, JPMC security blog, SR 26-2.

If agents begin spawning other agents autonomously, the governance model collapses at the point of agent creation, not agent operation. The kill switch, the agent registry, the named accountable human — all of these assume a human made a deliberate deployment decision. Auto-spawned agents may have none of these.

The three governance gaps this opens:

Registry gap — An agent spawned by another agent may never appear in the human-maintained agent registry. No named accountable human. No defined permission scope. No kill criteria.
Permission inheritance gap — What permissions does a spawned agent inherit from its parent? If the parent has broad tool access, does the child? The ZSP/JIT model (zero standing privileges, task-scoped, auto-expiring) becomes the only safe default — but most enterprises have not implemented it even for human-deployed agents.
Audit trail gap — The Singapore MGF requirement for tamper-evident logs traceable to a named human breaks when the originating decision was made by another agent. The chain of accountability has no human at the root.

The implication for the pre-deployment checklist:

The six-decision checklist in this playbook was designed for human-initiated deployments. Organizations planning to allow any form of agent-initiated agent deployment must add a seventh decision before the others: What is the governance model for auto-spawned agents? Specifically: Can agents spawn other agents at all (policy decision)? If yes, under what constraints? Who authorizes the spawning event? What permissions can a spawned agent hold? What is the automatic kill mechanism if a spawned agent has no human-registered accountable owner?

Current state (May 2026): No published governance framework addresses auto-spawned agent governance at the deployment level. Singapore MGF covers sub-agent authorization cascades (Dimension 3) but assumes a human authorized the top-level orchestrator. The gap is identified; the framework does not yet exist.

Brandon Sneider | brandon@brandonsneider.com May 2026