Governance-First Approval Systems for AI: What They Prove, What They Miss, and Where Runtime Evidence Fills the Gap

Adding AI to a workflow raises two questions. Is the AI change approved? What did the AI do on this specific case?

Teams often treat these as one problem. They are not. The first controls what goes live. The second records what happened during a live decision. Treating them as interchangeable produces paper assurance, not operational proof.

The distinction matters for every team, from a 10-person fintech to a regulated enterprise. A startup using AI-assisted triage for refunds must prove what happened on a case as much as a bank must. The scale differs. The question is the same: who approved what, and why?

The standards make the distinction explicit. ISO 42001 Clause 8, NIST AI RMF Govern and Map functions, and EU AI Act Article 9 require pre-deployment approval. ISO 42001 Clause 9, NIST AI RMF Measure and Manage functions, and EU AI Act Articles 12 and 26 require post-deployment monitoring. Both layers are necessary. The case-level connection is missing. No framework says how to link the approval record to the live decision.

For a detailed comparison, see How Four AI Governance Frameworks Handle Approval vs. Runtime Evidence.

The Short Version: Approval Is Not Runtime Evidence

A Governance-First Approval System Gates What Reaches Production

A governance-first approval system sits above the model runtime. It decides whether an AI system or AI change can reach production. It sets the conditions, the sign-off and the review cadence.

In practice it answers concrete questions:

• Can we switch the model behind AI triage for this queue?
• Can we change the prompt bundle?
• Can we add a retrieval source?
• Can we expose a plugin?
• Can we broaden the cases where AI may recommend execution?

Think of it as change control, model risk governance and release management combined. It sits in a different layer from the runtime control layer. Runtime control decides what an operator can see, approve or execute on a live case. The governance layer decides whether that runtime behaviour may exist in production at all. An operator acts inside a runtime boundary. That boundary itself had to be approved before the system touched a live case.

What Usually Sits Inside It: Six Objects Carry the Control

A serious deployment typically contains six objects.

1. AI system registry. Lists every AI capability in scope: owner, intended use, queue or workflow, users, risk tier, model or provider, retrieval dependencies, tool integrations and operational status. Distinguish triage classification, summary generation, suggested next steps, and workflows that surface or trigger a plugin action. ISO 42001 Annex A defines this layer.

2. Policy and risk layer. Records AI policy, acceptable use, impact assessments, risk assessments and treatment decisions. It states which workflows may use AI. It states what data may reach the model. It marks outputs as advisory, human-reviewed, or barred from execution without a second control. NIST AI RMF Map requires this context. EU AI Act Article 9 requires a documented risk management system for the AI lifecycle.

3. Approval workflow. The sign-off route. Who reviews? In what order? For which release type? A prompt tweak for low-risk categorisation should not follow the same path as a new model plus a new plugin in a finance exception case. Route changes by blast radius.

4. Release packet. The evidence bundle attached to the approval. You do not approve "the model". You approve a bundle: model, provider and version; prompt bundle; tool allowlist or plugin scope; retrieval configuration or corpus snapshot; guardrail settings; evaluation results and known limitations; rollback plan and operator guidance. The release packet is the unit of approval. It stays with the system until the next release.

5. Exception and waiver management. Covers emergency releases, temporary bypasses, compensating controls and expiry dates. Without it, approvals move to chat. Nobody can tell later whether a bypass was temporary or still active. Weak exception management leaves permanent emergency configurations with no start date.

6. Periodic review and corrective action. Reviews system ownership, model drift, incidents, override rates, denied paths, exception expiry and control fit. ISO 42001 Clause 10 and NIST AI RMF Manage require it. It keeps the approval layer honest.

Governance-First Systems Prove Ex Ante Control

Governance-first systems answer the pre-deployment questions. Who owns this AI workflow? Who approved the prompt change? Which policy and risk treatment were in force? What evidence supported the release? Which exception was granted, by whom, and until when? When was the last formal review?

An auditor can ask "Show me who approved this AI change." A strong system returns owner, reviewers, policy, risk assessment, evidence bundle, approval date and sign-off.

That is why it works in regulated, multi-stakeholder and audit-sensitive deployments. It creates ownership. It reduces undocumented changes. It gives legal, compliance, privacy, security and operations teams one approval structure. For sensitive workflows such as finance exceptions, medical triage and benefit determinations, approval records and release packets show that a workflow was reviewed before it touched customers, funds or regulated processes.

Governance-First Systems Do Not Address Runtime Reality

Used alone, governance-first systems do not prove runtime reality. They do not tell you the exact prompt used on case C-48192. They do not show which retrieved documents shaped the recommendation. They do not show whether a tool was visible, blocked or unavailable on that case. They do not show whether an operator overrode the AI. They do not show which plugin action ran, what the downstream system returned, or whether the outcome matched the approved intent.

The limitation is architectural. An approval record says what was allowed into production at release time. It does not describe one live decision. Approval without runtime evidence proves governance, not operations. Runtime logs without approval records prove what happened, but not whether it was allowed. Both are required. Neither is sufficient alone.

EU AI Act Article 12 is the most prescriptive existing runtime logging requirement. High-risk systems need tamper-resistant automatic event recording with a minimum six-month retention period. But Article 12 focuses on system-level event logs, not case-level decision records. The case-level gap remains.

Governance-First Alone Is Sufficient for Stable Pipelines

Some deployments can rely on governance-first approval alone.

The fit is stable pipelines with defined inputs and outputs: classic document processing, OCR extraction, batch classification. The approval unit is clearer. Runtime behaviour is more predictable. Change velocity is lower. If the system's behaviour is fully characterised by its configuration at deploy time, the release packet is the primary evidence.

The need for runtime decision records grows with the pace of AI change, the use of retrieval or external tools, the presence of human-in-the-loop decisions, the sensitivity of downstream actions and the regulatory exposure of the workflow. If AI is purely advisory with no downstream action execution, or the workflow is internal-only with no customer, regulatory or financial exposure, governance-first alone may carry the full weight. A quarterly-reviewed document classifier may need only approval governance. A daily-updated triage system handling payment exceptions does not.

Approval Without Runtime Evidence Breaks Down as AI Changes

Fast-changing AI workflows break this model. Prompt bundles change. Retrieval sources change. External tools change. Provider behaviour changes. If the case-level trail is weak, the result is a strong review packet from three months ago and no explanation of the decision made yesterday.

The gap widens as AI moves from advisory to autonomous execution within guardrails. An advisory recommendation that an operator ignores has limited blast radius. An autonomous action that executes within approved boundaries but produces an unexpected outcome needs a complete decision record. Was the action within the approved scope? Did the guardrails behave as expected? Which external system returned the unexpected result?

Governance tooling is beginning to respond. Some platforms now offer continuous agent trace evaluation, runtime policy violation monitoring and active enforcement through guardian agents. The gap is narrowing. For most organisations, the integration between governance platform and operational systems does not yet exist in practice.

Two Scenarios Show Where Governance-First Is Enough and Where It Fails

Scenario 1: where governance-first is sufficient.

A team runs a document classification pipeline. The model reads a PDF. It classifies it into one of twelve categories. It routes it to the right queue. The model updates quarterly. The prompt is versioned and stable. The taxonomy does not change.

The release packet, with model version, prompt, taxonomy, evaluation results and routing rules, is the primary evidence. Runtime monitoring matters for drift and queue distribution. The approval record substantially describes the system's behaviour. A case-level decision record is useful for auditing. It is not needed for proving control. The approval system can stand alone.

Scenario 2: where it breaks down.

A team handles payment exceptions. It wants AI to help with failed-transaction reprocessing. It switches to a new model for issue summarisation. It revises the prompt that recommends the next best action. It exposes a reprocess plugin for a narrow class of cases. It requires human review before execution.

The governance-first approval system captures the release packet correctly. It cannot answer the manager's later question: "What happened when the operator handled this reprocess case?"

The runtime decision record must show the original case context. It must show the AI recommendation shown to the operator. It must show whether the reprocess action was visible, blocked or unavailable. It must show who reviewed the case and who executed the action. It must show what the downstream payment system returned. It must show how the case changed afterwards. It must show whether the reprocess succeeded or failed.

The approved release record says the workflow was allowed. The case-level runtime record says what actually happened. Both are required for a complete control model.

This is why what to log for external actions matters. Once a workflow crosses a system boundary, a governance packet alone is not enough.

The Right Rollout Order: Evidence First, Approval Second

For LLM, retrieval and agentic workflows, the better sequence is usually:

1. Instrument runtime decisions first.
2. Create a decision ledger tied to the case record.
3. Wrap releases in a governance-first approval system.
4. Review exceptions, overrides, denied paths and drift on a cadence.

This order beats starting with approval paperwork. Runtime evidence shows what is actually happening. Approval governance shows what should be happening. You need both. Evidence first gives the governance layer something real to review.

For narrower workflows such as document processing, OCR and batch classification, governance-first can come earlier. Runtime behaviour is more stable. The approval unit is clearer.

Three Layers Form the Design Rule

• Approved release record: what was allowed into production.
• Runtime decision record: what happened on a specific case.
• Immutable evidence archive: what you can still prove later.

Most teams skip the third layer. Immutability means append-only writes. Once evidence is recorded, the actors in the decision cannot modify or delete it. In practice this means write-once storage for action records. It means cryptographic timestamping or hash chains for integrity. It means separating the evidence store from the operational runtime, so a system failure or bad actor cannot alter the record later.

EU AI Act Article 12 requires tamper-resistant logs. ISO 42001 Clause 9 requires that monitoring evidence be preserved for management review. Neither prescribes the full architecture. That is an implementation decision. Make it early, not as a retrofit.

Where Latch Fits: At the Boundary Between Approval and Live Decisions

Latch is a runtime control layer: the operational system where approved AI meets live decisions and evidence survives. For a full definition of this emerging category, see The Runtime Control Layer: What This Category of Software Is and Why It Exists.

The approval layer sits above Latch: AI registry, policy, approval workflows, release packets. Latch preserves the runtime decision record. The model proposes. The governed workflow decides. Unified intake keeps case context on one record. AI recommendations stay bounded by policy. Approval and role enforcement stay attached to the case. Plugin action execution captures structured results. An immutable case-linked audit trail holds the evidence.

Latch does not replace an enterprise AI management system. It gives the approval layer somewhere real to land. The governance shell sits above. The proof of the live decision stays with the case.

The Case Record as a System of Action makes the same argument from an AI-governance angle. The case is not merely where someone types notes. It is where operational memory has to survive.

Continue Reading

• How Four AI Governance Frameworks Handle Approval vs. Runtime Evidence
• The Runtime Control Layer: What This Category of Software Is and Why It Exists
• AI Triage Needs a Control Plane, Not Better Prompts Alone
• Why Approval, Auth, and Audit Logic Must Stay in the Core
• What to Log When Operators Trigger External Actions
• What Auditors Need to See in a High-Risk Approval Workflow