Who Runs AI at a Company Your Size: Three Governance Models for Companies Without a Chief AI Officer

Executive Summary

• Only 36% of companies with 500 or fewer employees have a dedicated AI governance role. The other 64% bolt AI onto someone’s existing job — and the choice of whose job determines whether AI adoption succeeds or stalls. (Gradient Flow/Pacific AI, n=351, February-May 2025)
• Three models work at 200-2,000 employees: IT-led, operations-led, and cross-functional committee. Each fits a specific organizational profile. Companies that pick the right model based on where their operational complexity sits — not based on who has the lightest calendar — report measurably stronger outcomes. IBM’s survey of 600+ organizations finds hub-and-spoke governance structures (the cross-functional model) deliver 36% higher ROI on AI spend than decentralized structures. (IBM IBV/Oxford Economics, n=600+, Q1 2025)
• The wrong default is the most expensive mistake. At most mid-market companies, AI governance lands on the CIO’s desk by inertia — not by design. This works when AI is a technology procurement problem. It fails when AI becomes a workflow redesign problem, which is where the actual value sits. RSM’s 2025 survey finds 92% of middle market companies encounter implementation challenges, and “insufficient internal skills and expertise” — not technology — is the barrier for 35%. (RSM, n=966, February-March 2025)
• The decision takes 30 minutes. The consequences last years. A CEO or COO who answers three diagnostic questions can identify the right governance home for AI at their company. This document provides the decision tree.

The Three Models

Every mid-market company that successfully governs AI without a dedicated Chief AI Officer uses one of three structures. The choice depends not on company size alone but on three variables: where the operational complexity sits, who controls the workflow, and whether the company’s AI ambition is efficiency (doing the same things faster) or transformation (doing different things).

Model 1: IT-Led Governance

Best fit: Companies where AI is primarily a tool selection and deployment problem — the existing workflows stay the same, but people use AI tools to execute them faster.

How it works: The CIO, VP of IT, or most senior technology leader owns AI governance as an explicit 20-30% workload allocation. They control tool evaluation, vendor management, security review, and acceptable use policy. Department heads request AI tools through IT; IT evaluates, approves, deploys, and monitors.

Where it succeeds: Companies in the early stages of AI adoption where the primary need is getting approved tools into employees’ hands safely. Organizations with strong IT leadership that already manages a disciplined procurement and security process. Companies whose AI use cases are concentrated in productivity tools (M365 Copilot, Google Workspace AI, general-purpose assistants) rather than operational transformation.

Where it fails: The IT-led model breaks when AI moves beyond productivity tools into workflow redesign. IT departments manage technology infrastructure. They do not typically redesign how the sales team qualifies leads, how the finance team closes the books, or how the operations team schedules production. When AI’s value depends on changing how work gets done — not just which tools people use — IT governance creates a bottleneck. The CIO can deploy the tool but cannot mandate the workflow change.

Karina Arteaga, an AI strategy advisor quoted in InformationWeek (March 2026), captures the failure mode: companies treat AI “as a technology problem when it is fundamentally an operating-model problem.” This misclassification leads to fragmented decisions and what she calls “a perfect storm: high expectations, low organizational readiness.”

The evidence: The RSM Middle Market AI Survey (n=966, February-March 2025) finds 39% of middle market companies cite lack of in-house expertise as their primary barrier to AI readiness, and 34% cite absence of clear AI strategy. Both point to a governance gap that IT alone cannot close — the expertise gap is about business process knowledge, not technology knowledge.

Model 2: Operations-Led Governance

Best fit: Companies where AI’s value comes from changing how work gets done — process automation, workflow redesign, capacity reallocation — and where the COO or VP of Operations already owns the processes AI is supposed to improve.

How it works: The COO or senior operations leader owns AI governance with IT in a supporting technical role. The COO decides which workflows to target, designs the pilot structure, measures operational outcomes, and manages the change management process. IT handles tool procurement, security, integration, and technical support. The COO owns the “what” and “why”; IT owns the “how” and “is it safe.”

Where it succeeds: Operations-heavy companies (manufacturing, distribution, logistics, services delivery) where the path to AI value runs through the core business process. Companies where the COO already leads continuous improvement or operational excellence initiatives — AI governance slots into an existing change management infrastructure. Organizations where the CEO needs a single point of accountability for AI results measured in operational terms (cycle time, error rates, capacity utilization), not technology terms (adoption rates, license utilization).

PwC’s 2026 COO strategy report finds seven out of ten COOs are already engaging with agentic AI, and the COO is becoming the executive most responsible for how AI embeds into core operations. The COO owns “whether employees actually use AI tools, whether workflows actually change, and whether promised efficiency gains actually materialize.”

Where it fails: The operations-led model struggles at companies where AI use cases span functions without a single operational throughline. If the marketing team, the finance team, and the customer service team all want AI tools for different purposes, and none of their workflows connect to the COO’s operational domain, the COO becomes an awkward landlord for initiatives outside their expertise.

The evidence: Harvard Business Review (March 2026) describes the turf war directly. At a Fortune 500 insurance company, the COO argued that “an agentic workforce is the definition of ops” while the CIO countered that agentic AI systems are technology infrastructure. The resolution: neither executive owned AI. The company moved to a decision-rights framework where each executive owned specific AI-related decisions based on functional expertise.

Model 3: Cross-Functional Steering Committee

Best fit: Companies where AI use cases are distributed across multiple departments with no single operational owner, and where governance requires coordinating legal, IT, HR, finance, and operations perspectives simultaneously.

How it works: A steering committee of 4-6 executives meets monthly (or biweekly during the first 12 months) to approve AI initiatives, set risk appetite, allocate investment, and resolve cross-functional conflicts. One executive — typically the CEO, COO, or CIO — chairs the committee and serves as the day-to-day AI decision-maker between meetings. The committee does not build or deploy AI; it governs priorities, policies, and resource allocation.

Where it succeeds: Companies with distributed AI ambitions where no single function dominates the use case portfolio. Organizations navigating regulatory complexity where legal, compliance, HR, and IT must coordinate on data governance, employment law, and industry-specific requirements. Companies where the CEO wants shared executive accountability rather than a single throat to choke.

IBM’s research finds that hub-and-spoke governance models — structurally similar to a steering committee with a central coordinator — deliver 36% higher ROI on AI spend than decentralized structures. The centralized coordination layer prevents duplicated tool purchases, conflicting policies, and shadow AI proliferation. (IBM IBV/Oxford Economics, n=600+ CAIOs, Q1 2025)

Where it fails: Committees fail when they become approval bottlenecks. If every AI tool request requires committee review, adoption stalls. The IAPP’s AI Governance Profession Report (n=671, 2025) finds that 45% of respondents cite “prioritization of speed to market over governance concerns” as the primary obstacle to governance maturity — committees that slow deployment without adding proportional risk reduction get bypassed. The committee also fails when roles are ambiguous: as the ISACA governance triad research notes, “collapsing roles creates blind spots — when the person building the AI system is also the person approving it for production, oversight is nominal.”

The committee must have teeth. EY’s 2026 AI survey (n=500) finds only 50% of AI governance leaders have independent authority to halt a high-priority project. At the other half of organizations, stopping a failing initiative requires board or CEO approval — which means politically championed projects survive past their evidence-based expiration date.

The Decision Tree: Three Questions

A CEO or COO can identify the right governance model by answering three questions honestly. The answers should reflect how the company operates today, not how leadership wishes it operated.

Question 1: Where does AI value come from at your company?

Question 2: Who controls the workflows AI is supposed to change?

Question 3: What is the CEO’s AI governance tolerance?

Scoring

If two or three answers point to the same model, the decision is clear. If answers split across models, start with the model that matches Question 1 (where the value comes from) and add coordination mechanisms from the other models as needed. The most common hybrid: operations-led governance with a quarterly cross-functional review.

Key Data Points

What This Means for Your Organization

The governance ownership question is not abstract. It determines who approves AI tools, who kills failing pilots, who manages the vendor relationship when Microsoft raises Copilot prices mid-contract, and who answers the board when they ask “what is our AI strategy?”

For a company with 200-2,000 employees, the realistic path forward is not hiring a Chief AI Officer. IBM’s data shows the CAIO role delivers measurably better outcomes, but the role commands $250,000-$400,000 in total compensation at the enterprise level. The mid-market alternative is choosing the right existing executive and giving them explicit authority, a bounded time commitment (6-10 hours per week is the researched minimum), and a quarterly reporting cadence that keeps the board informed without creating bureaucracy.

The most common mistake is not choosing wrong — it is not choosing at all. When AI governance lands on the CIO’s desk by default, the company gets technology governance without business transformation. When it lands on no one’s desk, shadow AI proliferates: the Gradient Flow survey found only 55% of small companies have AI policies in place, and IAPP data shows 27% of employees at small companies use AI tools without approval. The risk is not that employees use AI badly. The risk is that no one knows what they are using, on what data, with what exposure.

If this raised questions about where AI governance should sit at your organization specifically — the answer depends on variables no framework can capture from the outside — I would welcome that conversation at brandon@brandonsneider.com.

Sources

1. Gradient Flow / Pacific AI — 2025 AI Governance Survey. n=351, February-May 2025. 91% U.S.-based organizations. Company size breakdown: 32% small (≤500), 41% medium (501-5,000), 26% large (5,000+). Independent survey. High credibility for governance maturity data by company size. https://pacific.ai/2025-ai-governance-survey/

2. IBM Institute for Business Value / Oxford Economics / Dubai Future Foundation — Chief AI Officer Study. n=600+ CAIOs across 21 industries and 22 countries, Q1 2025. Broader survey of 2,300 organizations. Vendor-affiliated but rigorous methodology. Key finding: hub-and-spoke governance delivers 36% higher ROI; organizations with CAIOs report 10% higher AI ROI. https://www.ibm.com/thought-leadership/institute-business-value/en-us/report/chief-ai-officer

3. RSM Middle Market AI Survey 2025. n=966 (762 US, 204 Canada), February-March 2025. ±3.2% margin of error. Respondents hold decision-making authority on technology investments. Independent. Primary source for mid-market adoption and implementation data. https://rsmus.com/insights/services/digital-transformation/rsm-middle-market-ai-survey-2025.html

4. IAPP / Credo AI — AI Governance Profession Report 2025. n=671. 77% of organizations actively working on AI governance; 50% of governance professionals assigned to privacy/legal/compliance teams. Independent. High credibility for governance team structure data. https://iapp.org/resources/article/ai-governance-profession-report

5. Deloitte — State of AI in the Enterprise, 2026. n=3,235 business and IT leaders, 24 countries, 6 industries, August-September 2025. Only 21% report mature agent governance; governance preparedness at 30%. Consulting-affiliated but large sample. https://www.deloitte.com/us/en/what-we-do/capabilities/applied-artificial-intelligence/content/state-of-ai-in-the-enterprise.html

6. Harvard Business Review — “Who in the C-Suite Should Own AI?” March 2026. Toby Stuart, Helzel Chair at UC Berkeley Haas. Case study analysis applying Abbott’s System of Professions theory to AI ownership conflicts. Recommends decision-rights framework over single-owner models. Academic source. High credibility. https://hbr.org/2026/03/who-in-the-c-suite-should-own-ai

7. PwC — 2026 COO Operations Strategy. 7 in 10 COOs engaging with agentic AI. Consulting-affiliated but based on primary executive research. https://www.pwc.com/us/en/executive-leadership-hub/coo.html

8. EY — AI Survey 2026. n=500 AI governance leaders. 50% have independent authority to halt AI projects. Consulting-affiliated. https://www.ey.com (referenced via HBR and secondary sources)

9. InformationWeek — “The CIO Hot Seat: How to Lead AI Without Becoming the Scapegoat.” March 2026. Karina Arteaga quoted on AI as operating-model problem vs. technology problem. Trade publication. https://www.informationweek.com/it-leadership/2026-cio-trend-from-seat-at-the-table-to-the-ai-hot-seat

10. ISACA — “Collaboration and the New Triad of AI Governance.” 2025. Privacy-security-legal triad framework. Professional association research. High credibility for governance structure recommendations. https://www.isaca.org/resources/news-and-trends/industry-news/2025/collaboration-and-the-new-triad-of-ai-governance

Brandon Sneider | brandon@brandonsneider.com March 2026