Chapter 12 – Governance, Policy and Lifecycle

Purpose: Turn the inner loop (scaffold + runtime + tools) into decision rights, policy, evidence, and repeatable lifecycle gates. This lets you ship and operate agents safely over time.

TL;DR

Agentic systems are probabilistic, tool-wielding, and operate in sensitive domains, so governance must answer not just "can we build it?" but "should we ship it, on what terms, and how do we live with it over time?"
Every agent needs an AI dossier — a living evidence bundle covering use-case objectives, risk classification, data/privacy posture, architecture, evaluation results, and framework mappings — that grows richer as the agent advances through lifecycle gates.
A seven-stage lifecycle (Idea, Discovery & Design, Lab/Prototype, Pilot, Limited Production, Broad Production, Continuous Improvement/Retirement) with explicit governance checkpoints at each gate ensures that harness results, OWASP threat models, and sector-specific reviews are completed before autonomy increases.
Policy frameworks — including RBAC decision rights, tenant-aware approval matrices, policy-as-code guardrails, and data-isolation rules — are encoded in configuration registries and enforced at both design time and runtime, not buried in static PDF documents.
Risk classification drives control depth (prevention, detection, response layers), while ongoing telemetry, drift detection, red-team re-runs, and blameless incident postmortems close the loop from production monitoring back into evaluation harnesses and governance reviews.

12.0 Why Governance, Policy, and Lifecycle Matter for Agentic Systems

By now you have seen how to build agentic systems. Chapters 2–4 covered concepts, scaffolds, tools, and inner/outer loops. Chapter 5 introduced harnesses for testing and observing agents. Chapters 6–7 explained metrics, judges, datasets, and evaluation playbooks. Chapter 8 covered optimization. Chapter 9 covered best-practice design patterns. Chapter 10 focused on UX patterns. Chapter 11 addressed rollout, safety controls, and production operations.

Chapter 12 shifts the question:

Not "can we build the HR Leave / Invoice / Contact Centre agent?" But "should we ship it, on what terms, and how do we live with it over time?"

We keep the same portfolio in view: HR Leave, IT Service Desk, Invoice/Procurement, Contact Centre, CRM Co-pilot, Contract Review, and Scheduling/Rostering.

Each may be technically impressive yet still unacceptable. Legal, ethical, or safety concerns might block deployment. Governance is about structuring those judgements.

12.0.1 From "can we build it?" to "should we ship it?"

Traditional software is mostly deterministic. Given input X, you know output Y. You test for conformance, deploy, and monitor.

Agentic systems differ in three ways.

First, they are probabilistic. The same input produces slightly different outputs each time. You reason in terms of distributions of behaviour, not exact rules.

Second, they have loops and tools. Inner loops (Chapter 4) plan, call tools, observe effects, and iterate. Tools can read and write into HR systems, invoice platforms, contract repositories, CRM, and scheduling engines.

Third, they sit close to policy and people. Many operate in sensitive domains: employment, money, legal exposure, and citizen interactions.

A helpful bridge to Chapter 10: we wrap a probabilistic core with deterministic shells. These shells include clear controls, previews, guardrails, and audit logs. Governance is the organisation-level shell around the product shell. It decides what autonomy is allowed, what evidence is required to increase it, and how to respond when things go wrong.

That is why the big questions become: What autonomy level is acceptable for this agent, task, and sector? What evidence do we need before increasing autonomy or scale? How do we bound security and safety risks, including OWASP LLM risks like prompt injection, data leakage, and "excessive agency"? (OWASP Foundation)

12.0.2 Socio-technical scope (and where OWASP fits)

We use three layers.

Governance addresses who decides whether an HR, Finance, Legal, or Contact Centre agent is acceptable. It covers how sectoral rules and organisational risk appetite apply. It defines who is accountable.

Guardrails are design-time and runtime constraints baked into the system. These include content and safety filters, tool whitelists and permissions, policy-as-code rules, and evaluation thresholds.

Runtime controls are the live control surfaces from Chapter 11. These include kill switches, autonomy dials, routing and cohort controls, and rate limits and quotas.

For security, we assume a global baseline: OWASP's Top 10 for Large Language Model Applications. This taxonomy identifies LLM-specific risks such as prompt injection, insecure output handling, data/model poisoning, supply-chain vulnerabilities, sensitive-information disclosure, "excessive agency", misinformation, and unbounded resource consumption. (OWASP Foundation)

This chapter covers five areas. First, a lifecycle model for moving agents from idea to production to retirement. Second, a checkpoint structure for governance reviews at each stage. Third, documentation artefacts (the "AI dossier") that show decision-makers what the agent is, what it can do, and what evidence supports each autonomy grant. Fourth, mapping to Singapore frameworks and sectoral overlays. Fifth, using harnesses, metrics, and judges from Chapters 6–9 as primary governance evidence.

By the end, you should be able to position your HR Leave, Invoice, or Contact Centre agent in a lifecycle state and know what checkpoint comes next. You should be able to describe what artefacts and reviews are required to graduate that agent. You should be able to map governance requirements to the Singapore landscape and relevant security taxonomies. And you should be able to use evaluation infrastructure as the evidentiary basis for every governance gate.

12.1 Lifecycle: Idea → Pilot → Production → Continuous Improvement

All your agents move through states. Make these explicit.

Idea / Scoping – Someone proposes an agentic use case such as an HR Leave bot, Invoice auto-approval, or Contact-Centre advisor. Do a quick assessment: is this a good candidate for agentic AI? Chapter 3 heuristics apply.
Discovery & Design – If green-lit, scope the journeys. Identify data sources. Define the risk tier. Design the evaluation plan (Chapters 6–9 harness early outline). Map to relevant OWASP LLM categories. (OWASP Top 10 for LLM)
Lab / Prototype – Build a technical proof of concept. Run the scaffold in isolation. Do initial evals and SME-marked sets. Complete first security sanity checks.
Pilot (Controlled) – Deploy to a small internal population with heavy guardrails. Use shadow-mode or human-in-the-loop only. Run A/B tests where safe. Do deep monitoring and iterative improvement.
Limited Production – Expand rollout to broader but still restricted populations or lower autonomy tasks. Harness results justify stepping up. OWASP and sector checks have passed.
Broad Production – Deploy at full scale with high autonomy where appropriate. Maintain ongoing monitoring and red-team exercises. Continue evaluation via harnesses and online metrics (Chapter 7).
Continuous Improvement / Retirement – This may involve scaling down, retraining, retiring in favour of a successor system, or tightening autonomy after incidents.

12.1.1 Mapping lifecycle states to our portfolio

Table 11.1: Portfolio lifecycle states

Agent	Current state (example)	Near-term target state
HR Leave	Broad Production	Continuous improvement
IT Service Desk	Limited Production	Broad Production
Invoice / Procurement	Pilot (Controlled)	Limited Production
Contact Centre	Pilot (Controlled)	Limited Production
CRM Co-pilot	Discovery & Design	Lab / Prototype
Contract Review	Idea / Scoping	Discovery & Design
Scheduling / Rostering	Lab / Prototype	Pilot

Each agent can be placed on a one-page "registry card" containing: current lifecycle state, key risks and sector overlays, current autonomy bands by task type and cohort, date of last harness snapshot or risk review, and next scheduled checkpoint.

You can align to organisational rhythms. Annual planning determines which agents graduate from idea to pilot, or pilot to production. Quarterly risk reviews determine which high-risk agents get deep dives. Sectoral milestones such as new MAS, MOH, or IMDA guidance trigger re-review of affected agents.

12.2 Governance Checkpoints and Documentation Artefacts

Lifecycle states tell you when you decide. Checkpoints and artefacts define how you decide and what evidence you see.

12.2.1 Governance checkpoints along the lifecycle

We anchor concrete checkpoints to the states above.

1. Idea & Scoping Checkpoint

The inputs are a short concept note. The key questions are: Is this a suitable problem for agentic AI per Chapter 3 heuristics? Who are the impacted stakeholders? What could go very wrong? The output is green, amber, or red to proceed into Discovery & Design, plus an initial risk tier.

2. Design Checkpoint

The inputs are early journeys, data inventory, skeleton architecture, and draft evaluation plan. The key questions are: Are data sources and privacy obligations understood (PDPA, sector rules)? Is there a basic OWASP LLM threat model for the design? Have we planned harnesses and SME-driven metrics (Chapters 7–9)? The output is approval (or not) to build a lab prototype, plus a list of design-time conditions.

3. Pre-Pilot Checkpoint

The inputs are the lab prototype, initial harness results, and security/DPIA notes. The key questions are: Do offline results meet baseline thresholds? Are high-impact failure modes understood and mitigated (or kept in assist-only mode)? Is there a clear oversight UX and escalation path (Chapter 10)? The output is approval to run a controlled pilot with defined scope, autonomy caps, and monitoring requirements.

4. Pre-Production Checkpoint

The inputs are the pilot report, richer harness results, user feedback, and near-complete dossier. The key questions are: Do results meet per-metric targets, especially in high-risk cohorts? Are Singapore frameworks mapped and addressed? Have OWASP LLM risks been tested with red-team harnesses, and are controls satisfactory? The output is an explicit decision on whether to go to Limited or Broad Production, which autonomy bands are allowed, and which controls and monitoring must be in place.

5. Change / Release Checkpoints

The inputs are a change description, regression harness runs, OWASP-tagged security tests, and cost/latency impacts. The key questions are: What changed (model, scaffold, tools, configs)? What are the deltas in metrics including security-focused tests? The output is approve, reject, or defer, plus an update to the dossier and config registries.

6. Retirement / Down-scoping Checkpoint

The inputs are incident history, performance drift, and strategic shifts. The key questions are: Should we retire or significantly reduce autonomy for this agent? Do we have decommissioning and data-retention plans? The output is a decision plus changes to routing and autonomy dials.

You can implement these as checklists or one-page forms. They do not have to be heavy. But they do have to exist.

📌 Note: For multi-tenant deployments, see additional considerations in Section 11.3.6.

12.2.2 Core documentation artefacts: the AI dossier

Each significant agent should have an AI dossier. This is a living document that everything else plugs into. At minimum it contains:

Use case & objectives – in business terms
Risk classification & impact analysis – including OWASP LLM categories in scope and sector overlays
Data & privacy view – sources, flows, PDPA assessment, retention decisions
Architecture & scaffold – models, tools, loops, autonomy levels, dependencies
Evaluation & harness plan – metrics, rubrics, datasets, thresholds, cohorts (Chapters 6–9)
UX & oversight design – escalation, explainability, failure handling (Chapter 10)
Ops & control surfaces – kill switches, autonomy dials, logs, SLOs, cost budgets (Chapter 11)
Framework mapping – how Model AI Governance Framework, AI Verify, MAS FEAT etc. apply
Decision and incident log – key lifecycle decisions, incidents, and remediations

The dossier starts light at Idea/Design and becomes richer as you approach production. Governance checkpoints should always refer back to it, not a scattering of slide decks and ad-hoc docs.

12.2.3 Roles, decision rights, and RACI

Four groups of players are involved. Domain owners come from HR, IT, Finance, Legal, Contact Centre, or Operations. AI/Data science leads handle architecture, models, scaffolds, and evaluation. Risk & Compliance/DPO cover legal/regulatory matters, PDPA, and sectoral rules. Security/Architecture handle infra security, network/data zones, tool permissions, and SDLC.

For each checkpoint, be explicit about roles. The Accountable person (A) gives final sign-off. The Responsible person (R) does the work and collates evidence. The Consulted parties (C) must be heard. The Informed parties (I) must be kept aware.

In practice, for Pre-Production of the Invoice Agent: Finance owner is A, AI lead is R, Risk/Compliance + DPO + Security are C, and others are I. For Pre-Production of the Contact Centre Agent: Contact-centre head is A, AI lead is R, Risk + DPO + Security are C, and HR is I if HR data is involved.

This structure avoids both "shadow AI" and "AI did it" blame-shifting.

12.2.4 Policy-as-code and configuration registries

Static PDF policy docs will not keep up. You need elements of policy-as-code and central registries.

Policy-as-code examples include rules like "No invoice above $N can be auto-approved", "All maternity or medical-leave questions escalate to HR humans", and "Contact Centre must never send PHI fields outside protected channels". These become checks in scaffolds (pre/post-conditions on tool calls), validators on outputs (guardrails), and CI rules (block build if a change violates them).

Configuration registries track guardrail settings (safety filters, toxicity thresholds, PDPA redaction rules), autonomy levels per agent × task × cohort, approved tools and their scopes (read-only vs write, test vs prod), and mappings to OWASP LLM categories for which each control is relevant.

Changes to these registries go through the same change-management and release checkpoints. They are referenced in evidence packages.

In Chapter 11 we called out a practical production rule: the unit of release is a versioned configuration bundle (often an AgentBundle) plus an evidence bundle. Governance should require both. The AgentBundle captures the runnable system state including model, prompts, tool contracts, policy configs, and memory rules. The evidence bundle captures the proof that justified shipping that bundle, including harness results, red-team outcomes, and risk acceptance notes.

12.3 Mapping to Singapore and Sectoral Frameworks

Now that we have an internal lifecycle and governance structure, we plug it into Singapore's AI governance ecosystem and key sector rules.

12.3.1 Overview of key Singapore frameworks

Four pillars are most useful in practice.

1. Model AI Governance Framework for Generative AI (Singapore) sets out nine governance dimensions for GenAI: accountability, data, trusted development and deployment, incident reporting, testing and assurance, security, content provenance, safety and alignment R&D, and AI for public good. You can treat these dimensions as headings for your dossier and as prompts for what evidence your gates should demand. (Model AI Governance Framework GenAI)

2. AI Verify is a testing framework and toolkit. It assesses AI systems against governance principles through technical tests and process checks. (AI Verify Foundation)

3. IMDA Starter Kit for Safety Testing of LLM-Based Applications and Project Moonshot provides practical guidance and baseline tests for common LLM-app risks: hallucination, undesirable content, data disclosure, and vulnerability to adversarial prompts. Project Moonshot is the open-source test toolkit that operationalises many of these recommended tests. (Infocomm Media Development Authority) (AI Verify Foundation)

4. Sectoral frameworks and principles include MAS FEAT principles (Fairness, Ethics, Accountability, Transparency) for financial institutions using AI. They also include public-sector policies, healthcare clinical governance, and education and HR policy constraints. (Monetary Authority of Singapore)

OWASP LLM Top 10 sits alongside these. It is not a governance principle set but a security threat taxonomy your security and AI teams should adopt. (OWASP GenAI Security Project)

12.3.2 Mapping the lifecycle to these frameworks

Table 11.2: Lifecycle × Framework × Activities (concept)

The rows are Model AI Governance Framework, AI Verify, MAS FEAT (for financial agents), and OWASP LLM Top 10. The columns are lifecycle stages from Section 11.1.

During Discovery & Design, use the Model Framework to clarify purpose, stakeholders, and human-in-the-loop patterns. Use AI Verify to decide which principles will be most critical to test later. For MAS FEAT (if in finance), articulate fairness and transparency expectations. For OWASP, do initial threat modelling and decide which Top-10 categories are relevant.

During Pre-Pilot / Pilot, use the Model Framework to design communication and explanation for early users. Use AI Verify to start running early technical tests. For MAS FEAT, check early fairness and accountability behaviours for invoice or credit scoring agents. For OWASP, implement basic mitigations and pilot red-team scenarios for prompt injection, info disclosure, and excessive agency.

During Pre-Production / Production, use the Model Framework to show how explainability, transparency, and human-centric design are implemented. Use AI Verify to produce a testing report as part of the evidence package. For MAS FEAT, map decisions to FEAT assessment methodologies such as Veritas. For OWASP, show how each in-scope Top-10 item is mitigated or why residual risk is accepted.

12.3.3 Sectoral overlays

For each agent, sector overlays tighten requirements.

For Invoice/Procurement and Contract Review in Financial Institutions, you must align to MAS FEAT and related guidelines, often using Veritas assessment methodologies. Expect richer evidence on fairness, explainability, and accountability in credit/approval decisions. Expect strict auditability of decisions and logs. (Monetary Authority of Singapore)

For Contact Centre and HR/IT in Public Agencies, expect higher standards of fairness, non-discrimination, transparency, and recourse. Citizens may need clear communication when interacting with AI-supported systems.

For Healthcare-adjacent use cases, safety, privacy, and clinical governance dominate. These agents may never go beyond assistive/autocomplete modes without heavy oversight.

For HR and Scheduling/Rostering, labour law, union agreements, and fairness concerns heavily shape acceptable autonomy. You may need explicit commitments around appeal mechanisms and periodic fairness reviews.

Sector overlays usually mean stricter thresholds, more frequent checkpoints, more detailed dossiers, and sometimes external assurance.

12.3.4 "Compliance plumbing" vs real safety

A risk with all these frameworks is "tick-the-box" behaviour: a one-off AI Verify run, a nominal mapping to the Model Framework, or a PDF saying "we've considered OWASP", with little change to design, evaluation, or operations.

This chapter takes the opposite view. Compliance artefacts are by-products of doing the engineering properly. The real work lives in picking the right problems (Chapter 3), structuring behaviour with good scaffolds (Chapters 4–5), building strong evaluation regimes (Chapters 6–9), designing UX and control surfaces that keep humans meaningfully involved (Chapters 9–11), and embedding OWASP/sector risks into harnesses and red-team suites, not slideware.

Frameworks then become views on this work – not replacements for it.

12.3.5 International framework mappings

While this chapter centres on Singapore frameworks, the governance principles apply across jurisdictions. Table 11.3 maps the handbook's governance artefacts to three widely adopted international frameworks.

Table 11.3: International framework quick-reference

Framework	Scope and status	Key requirements for agentic systems	Handbook mapping
EU AI Act (2024, phased enforcement 2025–2027)	Risk-based regulation; mandatory for high-risk AI systems deployed in or affecting the EU	Risk classification and conformity assessment; human oversight and transparency obligations; technical documentation and quality management; post-market monitoring and incident reporting	Risk classification → §12.1 lifecycle gates and §12.5 risk taxonomy. Human oversight → Ch3 Principle 6 (Oversight by Design), Ch9 §9.5C (HITL Interrupts). Technical documentation → §12.2 AI dossier. Post-market monitoring → §12.4.3 telemetry and Ch7 §7.2.5 drift detection
NIST AI RMF 1.0 (US, voluntary, 2023)	Four-function framework (Govern, Map, Measure, Manage) for AI risk management	Establish AI governance structures; categorize and map AI risks; measure with metrics and evaluation; manage residual risks through controls and monitoring	Govern → §12.2 checkpoints and RACI. Map → §12.5.1 portfolio risk taxonomy. Measure → Ch6–7 evaluation foundations and playbooks. Manage → §12.5.3 prevention/detection/response layers
ISO/IEC 42001:2023 (international, certifiable)	AI management system standard; certifiable like ISO 27001	AI policy and leadership commitment; risk assessment and treatment; AI system lifecycle management; performance evaluation and continual improvement	AI policy → §12.2 governance checkpoints. Risk assessment → §12.5 risk and safety engineering. Lifecycle management → §12.1 lifecycle model. Performance evaluation → Ch6–7 evaluation as evidence (§12.4)

Key insight: The handbook's governance artefacts — lifecycle gates, AI dossiers, harness-based evidence packages, risk taxonomies, and ongoing telemetry — map naturally to all three frameworks. If you implement governance as described in this chapter, you produce evidence that satisfies multiple regulatory regimes with minimal additional effort. The framework-specific work is primarily in documentation format and submission procedures, not in the underlying engineering.

📚 Related sections: - NIST AI RMF is already cited in §12.1 (lifecycle) and §12.5 (risk engineering) - ISO/IEC 42001 is referenced in §12.2 (governance checkpoints) - For Singapore-specific frameworks, see §12.3.1–11.3.3 above - For OWASP LLM Top 10 security mapping, see §12.5.1

12.3.6 Multi-Tenant Considerations

Enterprise agents often serve multiple business units, subsidiaries, or jurisdictions. Each has distinct policies, approval matrices, and compliance requirements. Multi-tenancy is not just an infrastructure concern – it is a governance requirement.

12.3.6.1 What "tenant" means for agentic systems

A tenant boundary defines four things. Policy scope determines which rules apply to this user or request. Data isolation determines what information this request can access. Approval authority determines who can approve actions for this tenant. Audit segregation determines how to attribute and query activity.

Table 11.4: Common tenant dimensions in enterprise agents

Dimension	HR Leave example	Invoice example	Contact Centre example
Legal entity	Singapore subsidiary vs US subsidiary	Purchasing entity	Service region
Business unit	Engineering vs Sales vs Operations	Cost center	Product line
Employment type	Permanent vs contract vs intern	Vendor tier	Customer segment
Jurisdiction	SG labor law vs US FMLA vs EU Working Time Directive	Tax jurisdiction	Regulatory regime

12.3.6.2 Policy versioning per tenant

Each tenant may have different base policies, different policy versions (BU-A updated last month while BU-B still runs the old version), and tenant-specific overrides or addenda.

Implementation pattern:

# Policy resolution hierarchy
policy_resolution:
  order:
    1: "tenant_override"      # BU-specific exceptions
    2: "jurisdiction_base"    # Legal requirements by location
    3: "company_default"      # Global company policy

  hr_leave:
    company_default:
      version: "v2024.3"
      source: "global_hr_policy.md"

    jurisdictions:
      singapore:
        version: "v2024.3-SG"
        overrides:
          - childcare_leave_days: 6  # Statutory minimum
          - paternity_leave_days: 2

      us_california:
        version: "v2024.3-US-CA"
        overrides:
          - cfra_leave_eligible: true
          - paid_family_leave: true

    tenant_overrides:
      engineering_sg:
        - additional_wellness_days: 2
        - flexible_leave_policy: true

Retrieval must be tenant-aware. Filter the policy corpus by tenant at query time. Include tenant context in retrieval queries. Validate that retrieved documents match tenant scope.

12.3.6.3 Context isolation

Tenant boundaries must prevent data leakage.

Table 11.5: Isolation requirements

Data type	Isolation rule	Violation example
Personal data	Accessible only to data subject + authorized roles	Agent reveals colleague's leave balance
Tenant policies	Accessible only within tenant	Agent cites BU-A policy when helping BU-B user
Approval history	Scoped to tenant + need-to-know	Agent references cross-BU approval patterns
Aggregated metrics	Anonymized or tenant-scoped	Agent reveals "most people in your team take mental health days"

Three implementation patterns address these requirements.

Pattern 1: Tenant context injection

def build_context(user_id: str, tenant_id: str) -> Context:
    # Validate user belongs to tenant
    assert user_tenant(user_id) == tenant_id

    # Filter all data sources by tenant
    policies = get_policies(tenant_id=tenant_id)
    user_data = get_user_data(user_id=user_id, tenant_id=tenant_id)

    # Never include cross-tenant data
    return Context(
        tenant_id=tenant_id,
        policies=policies,
        user_data=user_data,
        # No global or cross-tenant fields
    )

Pattern 2: Tool-level tenant enforcement

tools:
  get_leave_balance:
    required_params:
      - employee_id
      - tenant_id  # Explicit, not inferred
    validation:
      - employee_belongs_to_tenant(employee_id, tenant_id)
    audit:
      - log_access(tool, employee_id, tenant_id, requester_id)

Pattern 3: Output filtering

Post-process responses to redact cross-tenant references. Validate that citations reference only tenant-scoped documents.

12.3.6.4 Tenant-specific approval matrices

Different tenants may have different approval authorities.

approval_matrices:
  engineering_sg:
    leave_request:
      - days: "1-3"
        approver: "direct_manager"
      - days: "4-10"
        approver: "direct_manager + skip_level"
      - days: ">10"
        approver: "hr_partner"

  sales_us:
    leave_request:
      - days: "1-5"
        approver: "direct_manager"
      - days: ">5"
        approver: "regional_director"
      - type: "extended_leave"
        approver: "hr_business_partner + legal"

The agent must resolve the correct approval matrix for the tenant, route to the correct approvers, and validate approvals against tenant-specific authority.

12.3.6.5 Audit trail segregation

Audit logs must support three capabilities. Tenant-scoped queries answer questions like "Show me all HR Leave agent activity for BU-A". Cross-tenant analytics answer questions like "What is our overall agent success rate?" with appropriate aggregation. Compliance exports answer questions like "Export all activity for Singapore entity for MOM audit".

Audit schema:

audit_record:
  run_id: "abc123"
  timestamp: "2024-01-15T10:30:00Z"

  # Tenant identification
  tenant_id: "engineering_sg"
  jurisdiction: "singapore"
  legal_entity: "Acme Pte Ltd"

  # User identification (redactable for aggregation)
  user_id: "emp_456"
  user_role: "software_engineer"

  # Action details
  agent: "hr_leave"
  action: "submit_leave_request"
  outcome: "approved"

  # Evidence
  policy_version: "v2024.3-SG"
  approver: "mgr_789"

  # Access control for the audit record itself
  visibility: ["hr_admin", "compliance", "tenant_admin"]

12.3.6.6 Multi-tenant testing requirements

Your evaluation harness must test tenant isolation explicitly.

Table 11.6: Isolation test suite

Test category	Test case	Expected outcome
Data isolation	User A queries User B's balance (same tenant)	Denied
Data isolation	User A queries User B's balance (different tenant)	Denied
Policy isolation	BU-A user triggers query, retrieval returns BU-B policy	Blocked at retrieval filter
Approval isolation	BU-A request routed to BU-B approver	Blocked at routing
Audit isolation	BU-A admin queries BU-B audit logs	Denied

Table 11.7: Cross-tenant edge cases

Scenario	Challenge	Required handling
Transfer between tenants	Which policy applies during transition?	Clear effective dates, policy handoff logic
Shared services	HR shared across multiple BUs	Explicit cross-tenant permissions, logged
Merger/acquisition	Combining tenant policies	Migration plan, parallel policy support
Matrix organization	Employee in multiple BUs	Primary tenant designation, explicit rules

12.3.6.7 Multi-tenant governance checklist

Before deploying a multi-tenant agent, verify the following.

Policy layer:

[ ] Policy resolution hierarchy defined and tested
[ ] Tenant-specific overrides documented and versioned
[ ] Policy retrieval filtered by tenant at query time
[ ] Policy update process includes tenant notification

Data layer:

[ ] All data sources tagged with tenant ownership
[ ] Cross-tenant queries blocked at infrastructure level
[ ] User-tenant membership validated on every request
[ ] Personal data access logged and auditable

Approval layer:

[ ] Approval matrices defined per tenant
[ ] Approver resolution tested for each tenant
[ ] Cross-tenant approval escalation paths defined (if any)

Audit layer:

[ ] Audit records include tenant identification
[ ] Tenant-scoped audit queries supported
[ ] Compliance export per jurisdiction available
[ ] Audit retention matches tenant/jurisdiction requirements

Testing:

[ ] Isolation tests: user A cannot see user B's data (same tenant)
[ ] Isolation tests: tenant A cannot see tenant B's policies
[ ] Policy resolution tests per tenant
[ ] Approval routing tests per tenant

📚 Related sections: - Internationalization → Chapter 14, §14.2.7 - Context isolation patterns → Chapter 9, §9.4 - Audit and observability → Chapter 11, §11.4

12.4 Harnesses and Evaluation (Chapters 6–9) as Governance Evidence

In Chapter 3 we framed "Evaluation is a Feature". In Chapters 6–9 we built harnesses, judges, datasets, and metrics. Chapter 12 treats those same artefacts as primary governance evidence.

12.4.1 Harness runs as primary evidence objects

For each agent, a harness configuration plus its results is an auditable object. It states what you tested: scenarios (routine, edge, adversarial, OWASP-linked), cohorts (by role, region, vendor class), and tools and flows (read-only vs write, sensitive vs non-sensitive data). It shows how you scored: business success metrics, safety/policy compliance, and fairness, latency, cost, and UX metrics.

This is what should sit in front of governance boards or risk committees, not only demo videos.

For high-risk agents, you might maintain families of harnesses: a baseline regression suite (does behaviour remain acceptable?), a red-team suite (OWASP LLM threats), and a sector-specific suite (for example, MAS FEAT fairness scenarios).

12.4.2 Evidence packages per release

Every significant change to an agent should produce a standard evidence package. For example, for the Contact Centre Agent:

1. Change summary describes what changed: "New base model, updated safety classifier, added tool for CRM notes, slightly adjusted prompt around benefit explanations."

2. Harness diffs show metrics before vs after: task success and CSAT proxy, hallucination rate and safety-filter triggers, and OWASP LLM categories in scope such as prompt injection and sensitive-info leakage.

3. Risk and mitigation notes document newly discovered failure modes (such as a subtle jailbreak vector) and guardrails or autonomy changes deployed.

4. Cost and performance show changes in latency and cost per 1,000 calls (Chapter 11.6).

5. Updated dossier pointers indicate which sections changed and which framework mappings are impacted.

The evidence package is small enough for decision-makers to read but grounded in the full harness data if they drill down.

12.4.3 Telemetry and online evaluation as ongoing proof

Harnesses provide pre-deployment evidence. Governance also needs ongoing assurance.

Telemetry (Chapter 11.4) includes per-agent dashboards showing performance, safety triggers, escalation rates, and fairness/complaint signals. Alerts fire when metrics cross thresholds, such as a sudden spike in safety blocks or an anomaly in approval rates for certain groups.

Online evaluation includes sampling interactions for SME review, using LLM-as-judge internally to triage samples with humans checking borderline or high-risk ones, and running A/B tests for UX or scaffold tweaks under explicit guardrails.

Drift detection compares against "golden sets" collected from real traffic and against security scenarios such as a bank of OWASP-style attacks replayed periodically.

This telemetry feeds into lifecycle checkpoints, risk reviews, and incident postmortems.

12.4.4 Mini-example: HR Leave Agent dossier slice

An abbreviated slice of the HR Leave Agent dossier, as it might look going into Broad Production:

Purpose & objectives: Reduce routine HR leave queries by 40%. Improve response time and clarity. Provide consistent policy-aligned advice across all employee cohorts.

Risk & sector overlays: Medium risk, internal only. Dominant themes are fairness, privacy, policy misapplication, and staff relations. OWASP items in scope are sensitive-info disclosure, excessive agency (bounded), and misinformation.

Architecture & autonomy: RAG-based scaffold over HR policy corpus with tools for read-only leave balance and ticket creation. Autonomy levels: auto-answers for low-risk FAQs, suggest-only for complex/sensitive topics, no auto-approval of leave.

Evaluation & harnesses: SME-designed rubrics for policy correctness, clarity, tone, and fairness. Harness coverage includes 1,200 historical anonymised queries, 300 synthetic edge cases, and 150 adversarial/OWASP-style prompts. Results: ≥95% correctness on core scenarios, ≤2% hallucination rate, zero policy-breach events in latest run, fairness parity across cohorts within threshold.

Governance signoffs: HR owner sign-off (A), AI lead (R), DPO + Security (C). Framework mapping: Model AI Governance Framework dimensions addressed, AI Verify mini-report attached.

This slice can be expanded to a full dossier or condensed into a one-pager for exec briefing.

12.5 Risk and Safety Engineering for Agentic Systems

Here we connect security, safety, and risk engineering into the governance lifecycle for HR, Invoice, Contact Centre, CRM, and beyond.

12.5.1 Portfolio risk taxonomy

Each agent carries risk profiles that map to the OWASP LLM Top 10 and beyond.

Table 11.8: Risk profiles by agent

Risk category	HR Leave	Invoice	Contact Centre	Contract Review
Prompt injection	Medium	Medium	High (external)	High (external docs)
Sensitive-info disclosure	High (personal)	High (financial)	High (PHI, PII)	High (legal, commercial)
Excessive agency	Low (no approvals)	High (auto-pay)	Medium	Low
Supply-chain (model, plugin)	Medium	Medium	Medium	Medium
Misinformation	Medium	Low	High	Medium

You assess each risk, decide on treatment (avoid, mitigate, accept, transfer), and document in the dossier.

12.5.2 Hazard analysis on loops and tools

For each agent, enumerate where things can go wrong.

In the inner loop, the model may hallucinate, the planner may pick the wrong tool, or the loop may not terminate.

With tools, a tool may return wrong data, a tool call may mutate state incorrectly, or a tool call may be blocked or time out.

With context, retrieval may return irrelevant or outdated policy, or the context window may overflow and truncate key info.

With memory, stale or incorrect user context may lead to bad recommendations.

At the autonomy boundary, the agent may act when it should ask, or fail to escalate when it should.

For each hazard, assess likelihood (how often might this occur?), impact (what is the worst outcome?), controls (what mitigations are in place?), and residual risk (is this acceptable given controls?). This is classic safety engineering adapted for agentic AI.

12.5.3 Layers of control: prevention, detection, response

Think in three layers.

Prevention includes input validation on prompts and tool inputs, output validation via guardrails and safety classifiers, tool permission scoping (read-only vs write, test vs prod), and policy-as-code checks in the scaffold.

Detection includes online metrics and alerts (Chapter 11.4), anomaly detection on agent behaviour, periodic harness re-runs for regression and adversarial tests, and user feedback signals such as thumbs down, escalations, and complaints.

Response includes kill switches and autonomy dials, incident response runbooks, postmortem and remediation processes, and communication plans for internal, external, and regulatory audiences.

Each agent should have documented controls across all three layers.

12.5.4 Scenario-based risk testing and red-teaming

Go beyond static evaluation.

Scenario-based testing walks through realistic attack scenarios. For example: an employee tries to extract another employee's salary, or a vendor injects malicious content in an invoice description.

Red-teaming means dedicated adversarial testing, either internal or external. Use OWASP LLM categories as a checklist. Use Project Moonshot or similar toolkits to automate common attack patterns. (AI Verify Foundation)

Tabletop exercises walk leadership through "what if" scenarios. For example: the agent leaks sensitive data, approves a fraudulent invoice, or gives harmful advice.

Document findings, remediation actions, and residual risk acceptance.

12.5.5 Incident management and postmortems

When things go wrong, follow six steps.

Detection and triage: How do you know? Who is alerted?

Containment: Use kill switch, reduce autonomy, communicate with users.

Investigation: Review trace logs, replay in harness, do root-cause analysis.

Remediation: Fix the issue, re-evaluate, update harnesses, update dossier.

Postmortem: Conduct a blameless review, capture lessons learned, make systemic improvements.

Regulatory/legal: Handle notification obligations (PDPC, sector regulators) if applicable.

Incident logs feed back into the AI dossier and inform future checkpoints.

Where Chapter 12 Leaves You

You now have a lifecycle model for moving agents from idea to production to retirement. You have a checkpoint structure for governance reviews at each stage. You have documentation artefacts (the "AI dossier") that let decision-makers see what the agent is, what it can do, and what evidence supports each autonomy grant. You can map to Singapore frameworks and sectoral overlays. You can use harnesses, metrics, and judges from Chapters 6–9 as primary governance evidence. You have risk and safety engineering practices adapted for agentic AI.

The next chapter (Chapter 13) will look at team structures and operating models for building and running agentic systems at scale.

Appendix 11.A Tables

Table 11.A.1: Lifecycle gates summary

Gate	Key questions	Required artefacts	Primary owner (A)	Consulted (C)
Idea & Scoping	Is this a suitable problem? What could go wrong?	Concept note	Domain owner	AI lead, Risk
Design	Data sources clear? OWASP threat model? Eval plan?	Early journeys, architecture sketch, eval plan	AI lead	Domain owner, DPO, Security
Pre-Pilot	Offline results meet baseline? High-impact failures mitigated?	Lab prototype, initial harness results, security notes	AI lead	Domain owner, Risk, Security
Pre-Production	Per-metric targets met? Frameworks mapped? OWASP tested?	Pilot report, richer harness results, near-complete dossier	Domain owner	AI lead, Risk, DPO, Security
Change / Release	What changed? Metric deltas? Security tests?	Change description, regression harness runs, cost/latency impacts	AI lead	Domain owner, Security
Retirement	Should we retire or reduce autonomy? Decommissioning plan?	Incident history, performance drift analysis	Domain owner	AI lead, Risk

Table 11.A.2: Framework mapping summary

Framework	Primary focus	When to use	Key outputs
Model AI Governance Framework (GenAI)	Governance dimensions	Throughout lifecycle	Dossier structure, checkpoint prompts
AI Verify	Technical testing	Pre-pilot onwards	Testing reports
MAS FEAT	Fairness, ethics, accountability, transparency	Financial agents	FEAT assessment
OWASP LLM Top 10	Security threats	Design onwards	Threat model, red-team results
IMDA Starter Kit / Moonshot	Safety testing	Lab onwards	Baseline test results

Table 11.A.3: Risk treatment summary

Risk category	Treatment options	Example controls
Prompt injection	Mitigate	Input validation, output guardrails, sandboxed execution
Sensitive-info disclosure	Mitigate	PII redaction, access controls, audit logging
Excessive agency	Avoid/Mitigate	Autonomy caps, human-in-the-loop, approval workflows
Supply-chain	Mitigate	Vendor assessment, model versioning, dependency scanning
Misinformation	Mitigate	Fact-checking, citation requirements, human review

Reference Map

📚 Full reference map: The detailed source reference map for Chapter 12 has been consolidated in Appendix F, §F.4.

Section-to-Source Citations

Section	Topic	Key source anchors
11.0 Why Governance Matters	Socio-technical framing for agentic systems	OWASP Top 10 for LLM Apps. (OWASP Foundation)
11.1 Lifecycle	Lifecycle states and maturity model	NIST AI RMF 1.0. (NIST Publications)
11.2 Governance Checkpoints	Gate structure, dossiers, RACI	NIST AI RMF 1.0, ISO/IEC 42001. (ISO)
11.3 Singapore Frameworks	Model Framework, AI Verify, MAS FEAT, IMDA	Model GenAI Framework. (AI Verify Foundation) AI Verify. (AI Verify Foundation) MAS FEAT. (Monetary Authority of Singapore) IMDA Starter Kit. (Infocomm Media Development Authority)
11.3.5 Multi-Tenant	Policy versioning, context isolation, audit segregation	Enterprise architecture practices, data governance standards
11.4 Harnesses as Evidence	Using evaluation as governance proof	Anthropic Engineering: Demystifying evals. (Anthropic) OpenAI Evals guide. (OpenAI Platform)
11.4.3 Telemetry and online evaluation	Continuous monitoring and risk management	NIST AI RMF 1.0. (NIST Publications) NIST Generative AI Profile. (NIST Publications)
11.4.4 Mini-example: HR Leave dossier	Privacy and data governance for AI	PDPC Advisory Guidelines. (PDPC)
11.5 Risk and Safety Engineering	Risk management and security for agentic systems	NIST Generative AI Profile. (NIST Publications)
11.5.1 Portfolio risk taxonomy	AI-specific risk identification	ISO/IEC 23894. (ISO)
11.5.2 Hazard analysis	Agent/tool specific attacks and failures	OWASP LLM Prompt Injection Prevention. (OWASP Cheat Sheet Series)
11.5.3 Layers of control	Security-by-design, defense in depth	Google Secure AI Framework (SAIF). (Safety Center)
11.5.4 Red-teaming	Practical red-teaming and benchmarking	Project Moonshot. (AI Verify Foundation) Microsoft AI Red Teaming Agent. (Microsoft Learn)
11.5.5 Incident management	Incident reporting and learning loops	Model AI Governance Framework for Generative AI. (AI Verify Foundation)

Key Takeaways

Governance is the organisational shell around the product shell — because agentic systems are probabilistic, tool-wielding, and sit close to people and policy, you need structured decision rights that answer "should we ship it?" before "can we build it?"
The AI dossier is your single source of truth — a living evidence bundle (use-case objectives, risk tier, data/privacy view, architecture, evaluation results, framework mappings, and incident log) that starts light at ideation and deepens through every lifecycle gate.
Lifecycle gates enforce graduated autonomy — moving an agent from Idea through Discovery, Lab, Pilot, Limited Production, Broad Production, and eventually Retirement requires passing explicit checkpoints with concrete artefacts and sign-offs at each transition.
Policy belongs in code and configuration registries, not slide decks — RBAC matrices, approval workflows, data-isolation rules, and guardrail thresholds must be versioned, tenant-aware, and enforced at both design time and runtime.
Risk classification drives control depth — categorising each agent by OWASP LLM threat profile and sector overlay determines which prevention, detection, and response layers are required and how stringent evaluation thresholds must be.
Evaluation harnesses are primary governance evidence — the same harness results, red-team outcomes, and SME-judged metrics from Chapters 6-9 serve as the evidentiary basis for every checkpoint, replacing demo videos and ad-hoc slide decks.
Incidents close the loop back to governance — blameless postmortems, drift detection, and ongoing telemetry feed directly into dossier updates, harness expansions, and future checkpoint reviews, making governance a continuous cycle rather than a one-time gate.

Next → Chapter 13 covers the people side — teams, roles, skills, and operating models that make agentic AI a durable organisational capability.

References

Suggested minimum per-section citations

11.0 Why Governance, Policy, and Lifecycle Matter: 101, 115
11.1 Lifecycle: idea → pilot → production → continuous improvement: 115, 116, 117
11.2 Governance checkpoints and documentation artefacts: 115, 128, 130
11.3 Mapping to Singapore and sectoral frameworks: 106, 107, 109, 110, 111, 112, 113
11.3.5 Multi-tenant considerations: Enterprise architecture practices, data governance standards
11.4 Using harnesses and evaluation as evidence for governance: 121, 122, 123, 125
11.5 Risk and safety engineering for agentic systems: 101, 104, 118, 119, 120, 126, 127, 128, 117