Model governance has a reputation problem: engineers hear the phrase and picture a 40-page policy PDF that nobody who ships code has ever read. Meanwhile the regulatory clock is real - the EU AI Act's obligations for general-purpose AI models have applied since August 2025, and the high-risk system requirements land in August 2026. Companies that sell software into Europe, as we do from Estonia, no longer get to treat governance as optional paperwork.
The good news is that effective governance looks less like policy writing and more like engineering discipline you mostly already have. Version control, review gates, CI checks, incident response - the same machinery, pointed at models and prompts instead of only code.
Start With an Inventory You Can Trust
You cannot govern what you cannot enumerate, and every organization we have assessed underestimates its AI surface. The inventory must cover more than the flagship chatbot:
- ▸Hosted model APIs: Which providers, which model versions, which regions, under which data-processing agreements. Provider-side model deprecations are a change you do not control - track them.
- ▸Embedded AI in SaaS: The copilot features your CRM and helpdesk vendors switched on, which quietly process your customer data.
- ▸Fine-tuned and self-hosted weights: Where checkpoints live, what data trained them, who can pull them.
- ▸Prompts and agent configurations: System prompts, tool definitions, and retrieval corpora are behavior-defining artifacts; they belong in git with owners, not in a dashboard someone edits live.
An inventory row should name an accountable owner, the use case, the data categories involved, and a risk tier. It costs about half a day per system, and it becomes the index for everything else.
Model Cards as Living Engineering Docs
For each inventoried system, write a short model card: intended use, known failure modes, evaluation results, data categories, and explicit out-of-scope uses. Keep it in the repo next to the code and require updates in the same pull request that changes model, prompt, or tooling - documentation that can drift from reality is worse than none, because auditors will find the drift.
This is also where standards stop being abstract. ISO/IEC 42001 asks for documented lifecycle management of AI systems; the Map and Measure functions of the NIST AI RMF ask for exactly the context and evaluation evidence a good model card holds. Writing it once, in the repo, satisfies several masters.
Evaluation Gates and Change Control
The core governance question is: what stops a bad change from shipping? For models, the answer is evaluation gates wired into the same CI that guards your code.
1. Define a golden evaluation set per use case - real inputs, expected behaviors, and known failure cases including safety and injection probes. 2. Run it on every change to model version, system prompt, or tool configuration, and block merge on regression beyond agreed thresholds. 3. Pin model versions explicitly and treat provider upgrades as changes that go through the same gate - the API silently getting smarter is also the API silently changing behavior. 4. Store evaluation results as build artifacts so any production behavior can be traced back to the gate that approved it. 5. Route higher risk tiers to heavier review: a marketing-copy assistant needs a green pipeline; a system that scores loan applications needs a human sign-off and a documented rationale.
This risk-tiered routing is what keeps governance survivable. Uniform maximal process gets bypassed; proportionate process gets followed.
Mapping to the Regulations Without Drowning
Resist building a separate compliance universe. Take your existing artifacts - inventory, model cards, eval reports, incident runbooks - and maintain a mapping table to the frameworks you face: EU AI Act articles for anything user-facing or high-risk, ISO/IEC 42001 if you pursue certification, NIST AI RMF as the common vocabulary customers increasingly reference in security questionnaires. In our experience the mapping exercise reveals gaps measured in documents, not in months of new process. The trap to avoid is the reverse order: starting from the framework and generating paperwork that no engineering workflow keeps current.
Incident response deserves explicit extension too: define what a model incident is - harmful output reaching a user, a data leak through a completion, an eval gate bypassed - and give it the same severity ladder, on-call path, and postmortem discipline as an outage.
Governance as a Sales Asset
The forward-looking case is blunt: AI governance is moving from differentiator to procurement requirement. Enterprise buyers now send AI-specific due-diligence questionnaires; regulators want evidence, not intentions; and the August 2026 high-risk deadline will catch teams that postponed this work. An engineering organization that can produce its model inventory, show eval gates in CI, and hand over current model cards answers in days what unprepared competitors answer in quarters. Governance done as engineering is not overhead - it is the paperwork that lets you say yes to the biggest customers.