AI governance is the set of decisions, roles and checks an organisation uses to control how it builds, buys and uses artificial intelligence. It covers where AI may be used, who is accountable for each use, what has to be checked before and after, and how the organisation would show a customer, an auditor or a regulator that all of this is happening.
This guide explains what AI governance covers, why it has become urgent, which UK rules and international standards apply, and what a mid-sized organisation should put in place first.
Why AI governance matters now
Three findings from the last two years explain the urgency.
AI is being used faster than it is being governed. A 2025 study by KPMG and the University of Melbourne surveyed more than 48,000 people in 47 countries, including 1,029 in the UK. In the UK, 65% of workers said they intentionally use AI at work. 58% said they had relied on AI output without checking its accuracy, and 39% said they had uploaded company information, such as financial, sales or customer data, to a public AI tool [1].
Incidents are rising. The Stanford AI Index 2026 reports that the AI Incident Database recorded 362 incidents in 2025, up from 233 in 2024 [2].
Having a policy is different from being in control. The same Stanford report, drawing on a survey run with McKinsey, found that the share of businesses with no responsible AI policies fell from 24% to 11% in a year. Over the same period, the share rating their own incident response as "excellent" fell from 28% to 18% [2].
The financial stakes are visible too. In EY's Responsible AI Pulse survey of C-suite leaders at companies with over US$1 billion in annual revenue, nearly 98% of the 100 UK respondents said their organisation had lost money through unmanaged AI risks. EY estimates the average loss at US$3.9 million. Only 17% of UK respondents correctly identified the appropriate controls (such as non-compliance with AI regulations, poor-quality trained data, or cybersecurity vulnerabilities) for a set of AI risks [3]. The respondents were at very large companies, so the loss figure will not transfer directly to a mid-sized organisation. The knowledge gap, however, probably does translate.
What AI governance covers
We group AI governance into six areas.
| Area | The question it answers |
|---|---|
| Purpose and scope | Where will we use AI, and where will we not? |
| Accountability | Who owns each use, and who signs off a new one? |
| Risk | What could go wrong with this use, and how serious would it be? |
| Data | What data goes in, is it good enough, and are we allowed to use it? |
| Oversight | What does a person check, and what is monitored after launch? |
| Evidence | What records would show that the above is happening? |
A small organisation can cover these with a register of AI uses, a one-page policy, a named approver and a quarterly review. A bank needs far more. The areas are the same in both.
Who is responsible
Accountability sits with senior leadership. In the Bank of England and Financial Conduct Authority's 2024 survey of 118 financial firms, 84% reported having an accountable person for their AI framework, and 72% said executive leadership was accountable for AI use cases [4].
Outside regulated sectors, the usual arrangement in a mid-sized organisation is this:
- The leadership team decides the organisation's position on AI and reviews the register.
- The head of data or IT runs the register and the sign-off process.
- The data protection lead assesses uses that involve personal data.
- Business owners are accountable for each use in their area.
- Everyone follows the rules of use and reports problems.
The UK rules that apply
The UK has taken a different route from the European Union. It has no single AI law. It relies on existing regulators applying existing law, guided by a common set of principles.
The five principles. The government's 2023 white paper on AI regulation set out five principles for regulators to apply: safety, security and robustness; appropriate transparency and explainability; fairness; accountability and governance; and contestability and redress [10].
Data protection law. UK GDPR applies whenever AI processes personal data. The Data (Use and Access) Act 2025 changed the rules on automated decision-making, and regulations in force since 12 May 2026 require the Information Commissioner to prepare a statutory code of practice on AI and automated decision-making, including guidance on children's data [5]. At the time of writing (November 2026), the ICO is still preparing the code, so check its website for the latest position.
Sector regulators. Financial firms answer to the FCA and the Prudential Regulation Authority. In their 2024 survey, firms named data protection and privacy as the largest regulatory constraint on their use of AI [4].
The EU AI Act. If you sell AI-enabled products or services into the EU, the EU's AI Act may apply to you as well. Take advice on your own position.
The standards worth knowing
Two voluntary standards come up often in procurement questionnaires and audits.
ISO/IEC 42001. Published in December 2023, this is the international standard for an AI management system. It is certifiable, in the way ISO 27001 is for information security [9]. In the Stanford and McKinsey survey, 36% of organisations cited it as an influence on their responsible AI practice, the first year it appeared [2].
The NIST AI Risk Management Framework. Published by the US National Institute of Standards and Technology in 2023, it organises AI risk work into four functions: govern, map, measure and manage [11]. It is free and often used as a checklist. NIST is revising it, so check for the latest version.
Neither is required by UK law. Both are useful as a structure, and customers may ask about them.
AI governance and data governance
The two are closely related, and a mid-sized organisation should run them together.
Data governance decides who owns each data set, what it means, how good it has to be and who may use it. AI governance adds the questions that are specific to AI: what the system is used for, how its output is checked, and who is accountable for decisions it informs.
The link between them is strong. In the Bank of England and FCA survey, four of the top five risks firms saw in their use of AI were about data: privacy and protection, quality, security, and bias [4]. An organisation that has not sorted out ownership and quality will find its AI governance has little to stand on.
The agent question
AI systems that take actions for themselves, usually called agents, are spreading quickly, and governance is behind. Deloitte's 2026 State of AI in the Enterprise survey of 3,235 leaders found that close to three-quarters of companies plan to deploy agentic AI within two years, and only 21% of those companies report a mature model for agent governance [6].
EY found that 64% of UK companies allow employees to build or deploy AI agents themselves, and 53% have formal policies covering that activity [3].
If staff in your organisation can build agents with the tools they already have, your register and sign-off process need to cover agents explicitly.
Common mistakes
Writing a policy and stopping. The Stanford figures above show policies spreading while confidence in incident response falls. A policy without a register, an owner and a review is just paper.
Banning tools without offering one. A Microsoft survey of 2,003 UK employees found that 71% had used unapproved consumer AI tools at work, and 28% said their employer provides no work-approved option [7].
Skipping training. Only 27% of UK respondents in the KPMG study had received any AI education or training [1]. The Stanford survey found knowledge and training gaps were the most-cited obstacle to responsible AI, named by 59% of organisations [2].
Treating it as an IT matter. The decisions that matter are about purpose and accountability, and those belong to the leadership team, but they need to be adopted throughout the organisation.
Where to start
- Find out what is in use. Ask every team which AI tools they use and for what.
- Name an approver and an owner for each use.
- Publish one page of rules. Cover approved tools, data that must stay out, what a person must check, and how to propose a new use.
- Sort uses by risk. Anything that informs a decision about a person, faces customers or uses personal data at scale needs sign-off first.
- Review every quarter with the leadership team.
The UK government's AI Adoption Research notes that its own survey “will not provide insights into shadow AI adoption”, meaning staff using unapproved chatbots or public large language models [8]. This is compounded by many software applications integrating their own AI features. For example, Salesforce’s Agentforce (previously Einstein) provides a chatbot to query the data held in their platform. Providing training on what AI is and creating a safe place for people to talk about what they’re using is a better way to not only determine the current state but where opportunities for AI use are possible.
Common questions
Is AI governance a legal requirement in the UK?
There is no single law requiring an AI governance framework. Data protection law, equality law, consumer law and sector rules all apply to AI use, and governance is how you meet them and show that you have.
What is the difference between AI governance and responsible AI?
Responsible AI describes the aim: AI that is safe, fair, ethical and accountable. AI governance is the practical arrangement that gets you there.
Does a small organisation need AI governance?
Yes, in proportion. A register, a one-page policy and a named approver are enough to begin.
Who should own AI governance?
A member of the leadership team should be accountable, with the head of data or IT running it day to day.
This guide is general information and not legal advice.
Citations
- KPMG and the University of Melbourne, Trust, attitudes and use of artificial intelligence: A global study 2025, UK findings. Fieldwork November 2024 to January 2025. https://kpmg.com/uk/en/insights/ai/uk-attitudes-to-ai.html
- Stanford Institute for Human-Centered AI, The 2026 AI Index Report, chapter 3, "Responsible AI". Survey data from the AI Index and McKinsey & Company. https://hai.stanford.edu/ai-index/2026-ai-index-report
- EY, "Responsible AI governance drives business performance", UK findings from the EY Responsible AI Pulse survey, 14 October 2025. https://www.ey.com/en_uk/newsroom/2025/10/responsible-ai-governance-business-performance
- Bank of England and Financial Conduct Authority, Artificial intelligence in UK financial services - 2024, 21 November 2024. https://www.bankofengland.co.uk/report/2024/artificial-intelligence-in-uk-financial-services-2024
- The Data Protection Act 2018 (Code of Practice on Artificial Intelligence and Automated Decision-Making) Regulations 2026, SI 2026/425. https://www.legislation.gov.uk/uksi/2026/425/made
- Deloitte, State of AI in the Enterprise 2026, press release, 21 January 2026. https://www.deloitte.com/us/en/about/press-room/state-of-ai-report-2026.html
- Microsoft UK, "Rise in 'Shadow AI' tools raising security concerns for UK organisations", 13 October 2025. Survey of 2,003 UK employees by Censuswide. https://ukstories.microsoft.com/features/rise-in-shadow-ai-tools-raising-security-concerns-for-uk
- Department for Science, Innovation and Technology, AI Adoption Research, IFF Research and Technopolis, updated 13 February 2026. https://www.gov.uk/government/publications/ai-adoption-research/ai-adoption-research
- ISO/IEC 42001:2023, Information technology - Artificial intelligence - Management system, first edition, December 2023. https://www.iso.org/standard/42001
- Department for Science, Innovation and Technology, A pro-innovation approach to AI regulation, white paper, presented to Parliament 29 March 2023, updated 3 August 2023. https://www.gov.uk/government/publications/ai-regulation-a-pro-innovation-approach/white-paper
- National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, 26 January 2023. https://www.nist.gov/itl/ai-risk-management-framework
Contains public sector information licensed under the Open Government Licence v3.0.
