See also (wiki): wiki/agentic-ai-governance.md, wiki/model-risk-management.md
Executive Summary
- MIT CISR’s January 2026 briefing, based on 62 executive interviews across 41 organizations, maps the full surface area where generative AI risk emerges — from training data through foundation models, prompts, outputs, use decisions, RAG systems, and autonomous agents.
- The central distinction: embedded risks come with the technology (training data bias, hallucination, vendor-driven model drift) and enacted risks emerge from organizational choices (prompt design, access controls, verification requirements, agent permissions).
- Each risk type demands a different management response. Embedded risks require vendor engagement, contractual transparency clauses, and independent evaluation. Enacted risks require internal governance, capability building, and coordinated controls.
- RAG deployments expose pre-existing access control gaps that went unnoticed when data was harder to find. AI agents introduce autonomy creep — the gradual, unreviewed expansion of what automated systems are authorized to do.
- The practical starting point: inventory every GenAI deployment, document model provenance, system prompt design, data connections, authorized use contexts, human review gates, and accountability assignments.
The Risk Space Framework
MIT CISR identifies eight components where GenAI risk emerges, each with distinct characteristics that demand specific attention.
Core Components (Present in Every GenAI Use)
| Component | Risk | Who Controls It |
|---|---|---|
| Training data | Inherited bias, outdated practices, inaccuracies absorbed from internet-scale datasets | Vendor (not the deploying organization) |
| Foundation model | Probabilistic outputs, hallucination, opacity of reasoning, unannounced vendor updates that break working implementations | Vendor (with limited organizational influence via contracts) |
| User prompt | Confidential data leakage, prompt injection via compromised documents, poor instructions producing poor outputs | End users (governable via policy and training) |
| System prompt | Single point of failure if poorly designed; prompt leakage exposing proprietary logic and security guardrails | Organization’s technical team |
| Output | Polished appearance masks errors; biased language, fabricated content, inflated claims look authoritative | Requires evaluator expertise to challenge |
| Use decision | Internal errors are correctable; external-facing errors damage trust and reputation | End users (governable via verification gates) |
One executive captured the output evaluation challenge directly: “The probability of hallucination may be low, but the negative consequences of hallucination are really high for us.” Some organizations enforce this with a blunt gate: “Do you have the expertise to challenge the output? If not, you’re not allowed to use GenAI.”
Extended Components (RAG and Agents)
RAG and vector databases inherit whatever quality issues exist in underlying data. The more consequential risk: they surface data users could technically access but previously had no practical way to find. One executive described the discovery: “Sometimes it turns out that the user has access to more than they knew. It can show you that data inadvertently, when previously it wasn’t as easily discoverable.” RAG does not create unauthorized access — it reveals pre-existing access control gaps that went unnoticed when search was harder.
AI agents introduce autonomy creep: organizations delegate tasks incrementally, each step seeming reasonable, without revisiting the aggregate authorization. Permissions granted for one task get reused across agents, enabling actions that were never evaluated in combination. One executive was direct: “We need to make sure AI-based recommendations are not generated in a vacuum, or acted on autonomously. The minute we take an action without that verification, we’re on the hook.”
Embedded vs. Enacted: The Management Fork
This is the briefing’s most actionable insight. Not all GenAI risks respond to the same controls.
Embedded risks — training data quality, model behavior, performance drift from vendor updates — exist before any deployment decision. Organizations cannot eliminate them. They manage them through:
- Vendor selection criteria that evaluate data provenance and model transparency
- Contractual requirements for change notification and transparency
- Independent evaluation of model outputs against known benchmarks
- Proactive vendor engagement (one executive noted: “For the first time, because of AI, we have quarterly roadmap meetings with our vendors. We never used to do this anywhere near as actively.”)
Enacted risks — system prompt design, user data handling policies, verification requirements, agent permissions, authorized use contexts — emerge from organizational choices. They reflect the quality of internal governance capabilities. Managing them requires:
- Coordinated governance from guiding principles through technical platform controls
- Clear policies on what data may be included in prompts
- Mandatory human verification before external-facing outputs
- Explicit, reviewed permission boundaries for agents
- Regular audit of the gap between authorized and actual use
The distinction matters because most organizations default to treating all GenAI risk as a procurement or vendor management problem. Embedded risks are a vendor problem. Enacted risks are an operations and governance problem. Conflating the two means either over-relying on vendor controls for risks the organization actually owns, or under-investing in vendor engagement for risks the organization cannot fix internally.
Key Data Points
| Data Point | Source | Date | Credibility |
|---|---|---|---|
| 62 executive interviews across 41 organizations | MIT CISR Data Research Advisory Board | Q1–Q2 2025 (published Jan 2026) | HIGH — independent academic research, qualitative methodology |
| 8 risk components mapped (training data, foundation model, user prompt, system prompt, output, use decision, RAG/vector databases, agents) | MIT CISR briefing | Jan 2026 | HIGH |
| 2 risk categories (embedded vs. enacted) requiring distinct management approaches | MIT CISR briefing | Jan 2026 | HIGH |
| “Autonomy creep” identified as key agent risk — incremental delegation without aggregate authorization review | MIT CISR briefing | Jan 2026 | HIGH |
| RAG exposes pre-existing access control gaps rather than creating new unauthorized access | MIT CISR briefing | Jan 2026 | HIGH |
Temporal tier: TIER 1 (published January 2026; interviews conducted Q1–Q2 2025). Findings reflect current-generation model deployments.
Source credibility: HIGH. MIT CISR is an independent academic research center funded by consortium membership, not vendor sponsorship. The Data Research Advisory Board consists of practicing executives, not consultants or vendors. Qualitative methodology (semi-structured interviews) provides depth on how organizations experience risk rather than statistical prevalence — appropriate for a taxonomy-building exercise, though the 41-organization sample means specific risk frequencies are not generalizable.
What This Means for Your Organization
The embedded vs. enacted distinction gives risk owners a practical sorting mechanism. Most mid-market companies (200–2,000 employees) are running GenAI with a single governance approach — usually an acceptable use policy that treats all risk as a user behavior problem. That misses the embedded half entirely. Training data bias, hallucination rates, and model drift from vendor updates are not things your acceptable use policy can address. They require vendor-facing controls: evaluation criteria, contractual clauses, and the kind of quarterly roadmap meetings MIT CISR’s executives described.
The autonomy creep finding deserves particular attention as organizations move from copilot-style tools to agentic deployments. Each individual delegation — letting an agent query a database, call an API, draft a communication — looks reasonable in isolation. The aggregate authorization often has not been reviewed by anyone. The practical check: for every agent workflow, can you name the person accountable for the combined outcome, and has that person reviewed the full permission chain?
The RAG access control finding is a useful early-warning signal. If your organization is deploying RAG over internal data, the first risk is not hallucination — it is discovering that your access controls were never as clean as you assumed. Treat the RAG deployment as a forced audit of data permissions.
If any of this raised questions specific to how your organization is mapping its own GenAI risk surface, I’d welcome the conversation — brandon@brandonsneider.com.
Sources
-
Van der Meulen, N., Lefebvre, H., and Wixom, B. H. “Mapping the Generative AI Risk Space.” MIT CISR Research Briefing, Vol. XXVI, No. 1, January 15, 2026. https://cisr.mit.edu/publication/2026_0101_GenerativeAIRisk_VanderMeulenLefebvreWixomLegner. Credibility: HIGH — independent academic research center; qualitative study based on 62 semi-structured interviews with data and technology executives from 41 organizations (MIT CISR Data Research Advisory Board members), conducted Q1–Q2 2025.
-
Van der Meulen, N. and Wixom, B. H. “Managing the Two Faces of Generative AI.” MIT CISR Research Briefing, Vol. XXIV, No. 9, September 2024. Referenced as companion briefing distinguishing GenAI tools from GenAI solutions.
Brandon Sneider | brandon@brandonsneider.com April 2026