Building an AI Governance Function from Scratch

Five steps in the right order, starting with the leader rather than the framework.

8 min read
Short answer

Build an AI governance function in five steps: hire the leader first, inventory every AI system, classify each by risk, build the governance infrastructure, and earn the organization's trust. The common mistake is writing the framework first and hiring someone to run it; the right leader should shape the design. Start early enough to be running before the EU AI Act's high-risk obligations apply on December 2, 2027.

Free AI leadership role brief

Before you write a job description, get a one-page brief on your role after a 30-minute call: scope, reporting line, a pay range from public filings, and where the candidates are.

Get my role brief →

Where do you start?

Usually with a trigger: the board asked who owns AI risk and nobody had a good answer, a regulatory deadline is approaching, or an AI incident or near-miss made the need visible. However it starts, building AI governance is mainly a leadership and organization design problem. The first decisions are about people, authority and structure, not technology.

Step 1: Why hire the leader before designing the program?

Because the right leader's experience and judgment should shape the design. A leader who inherits a framework drawn up without them spends months rebuilding it to fit reality. Hire first, then give them the mandate to assess the current state, set priorities and propose the structure. How to find a Head of AI Governance covers where to look.

The level depends on scale. A large enterprise may need a Chief AI Officer; in certified FY2025 US Department of Labor wage filings, those roles were offered between $105,269 and $400,000 in base wage (see the table). A mid-size organization may need a VP or senior director of AI governance with the pragmatism to build on a small budget. Either way, look for governance expertise, technical fluency, credibility inside the organization and the willingness to say no when it matters.

Step 2: How do you inventory your AI?

List every AI system in use: models built in-house, third-party AI products, software with AI features built in, and pilots that drifted into production without formal review. For each, record the business function, the data it uses, the decisions it influences and the current oversight. This step usually turns up more AI than anyone expected, especially inside vendor tools.

Step 3: How do you classify AI systems by risk?

The EU AI Act gives a useful starting framework with four tiers: unacceptable risk (prohibited), high risk, limited risk (transparency duties) and minimal risk. It works as an internal yardstick even outside the EU. Put governance effort on the highest-risk systems first: anything influencing hiring, credit or healthcare decisions needs attention now, while a tool that summarizes meeting notes can wait.

Step 4: What governance infrastructure do you need?

The NIST AI Risk Management Framework gives risk, compliance, technology and legal teams a shared structure. It isn't prescriptive, but many organizations use it as an organizing principle.

Step 5: How does the governance function earn its mandate?

A governance function without credibility is a policy department without influence. The leader should build relationships before enforcing standards: learn engineering workflows, sit with product managers to see how AI decisions get made, and align with legal and compliance. Governance earns its place by helping the organization deploy AI with more confidence, not by slowing everything down.

How long does it take?

There's no reliable public benchmark, and it depends on how much AI you already use and how fast you hire. Hiring the leader, the inventory and the risk classification come first; the infrastructure follows; embedding it in daily work never really ends. Work back from the dates that bind you, such as December 2, 2027 for the EU AI Act's Annex III uses. If you need outside help during the build, read how to hire AI governance consultants first.

What does a working impact assessment program look like?

Canada’s federal government is a public example. Under its Directive on Automated Decision-Making, departments complete an Algorithmic Impact Assessment before an automated decision system goes live, and the score sets one of four impact levels. By November 14, 2026, 39 assessments had been published: 23 at Level II (moderate impact), 15 at Level I and none at Level III or IV. See all 39, by department and level.

Frequently asked questions

What is the first step in building AI governance?

Hire the leader before designing the program. The right leader's experience should shape the framework, and a leader who inherits a framework drawn up without them usually has to rebuild it.

How should we classify AI systems by risk?

The EU AI Act's four tiers are a practical starting point even outside the EU: unacceptable (prohibited), high risk, limited risk with transparency duties, and minimal risk. Govern the high-risk systems, such as those influencing hiring, credit or healthcare decisions, first.

Which framework should an AI governance program use?

Many organizations use the NIST AI Risk Management Framework as a shared structure across risk, compliance, technology and legal teams, and the EU AI Act's risk tiers and obligations where they apply.

Written by Mariya with research and drafting help from AI tools. Every figure was checked against the sources below.

Sources

Share this article LinkedIn X Email Copy Link
← All Articles Get a free role brief →

Ready to talk about who should lead your AI?

Flat-fee retained search. No percentage of compensation. Aligned with your interests.

Get a free role brief