HAIAMM v3.0, Software Domain Handbook

AI/HAI Software Assurance, security of the AI/HAI software the organization builds

Version: 3.0 Domain: Software Audience: Security, AppSec, AI/ML Engineering, MLOps / Platform Engineering, Product, Privacy/Legal, SRE Use: Conduct a maturity assessment of the AI/HAI Software Assurance program, and build the practices that move it from Foundational to Industry-Leading.

Preface

This handbook is a self-contained, practitioner-facing document. Read it end to end to understand the Software domain of HAIAMM, or jump to Part IV to perform an assessment.

The handbook makes three commitments to the reader:

1. Fundamentals first. It teaches the load-bearing practices an organization must have to claim mastery of the security of AI/HAI software it builds, not a catalog of everything one could do.
2. Measurable by default. Every activity prescribed in these pages is paired with at least one outcome metric (with a baseline, a target, and a source). Activity counts are not metrics; outcomes are.
3. Self-contained. The document does not require the reader to follow links, open companion files, or chase references. Every concept used in the assessment is defined inside these pages.

If a statement in this handbook treats AI as a tool performing security rather than the subject being secured, that statement is wrong. Flag it.

Table of Contents

Part I, Domain Overview

1. About this handbook
2. The Software domain in v3.0 terms
3. Why a domain-specific handbook
4. The seven AI/HAI software archetypes
5. Domain boundary rules
6. Stakeholders and roles
7. How to use this handbook

Part II, Foundations

1. The four Business Functions in this domain
2. The three maturity levels
3. HAI-specific threat tactics (EA, AGH, TM, RA)
4. The priority compliance map
5. Shadow AI in the Software domain (unsanctioned internal AI builds)
6. Metrics taxonomy

Part III, The Twelve Practices in the Software Domain

1. Strategy & Metrics (SM)
2. Policy & Compliance (PC)
3. Education & Guidance (EG)
4. Threat Assessment (TA)
5. Security Requirements (SR)
6. Secure Architecture (SA)
7. Design Review (DR)
8. Implementation Review (IR)
9. Security Testing (ST)
10. Environment Hardening (EH)
11. Issue Management (IM)
12. Monitoring & Logging (ML)

Part IV, Maturity Assessment Workbook

1. How the assessment works
2. Scoring methodology
3. The questionnaire (108 questions)
4. Practice-level rollup
5. Domain-level rollup
6. Improvement roadmap template

Part V, Reference

1. Glossary
2. Reference frameworks
3. Change log

Part I, Domain Overview

1. About this handbook

HAIAMM is the Human-Assisted Intelligence Assurance Maturity Model. It is an AI assurance maturity model, structured after OWASP SAMM and BSIMM in shape, and scoped to AI/HAI in content. HAIAMM has six domains, Software, Data, Endpoints, Infrastructure, Vendors, Processes, and twelve practices that apply across all six.

This handbook covers the Software domain. It contains:

• A definition of what the Software domain is and is not.
• The twelve practices, each described in Software-domain terms with three maturity levels (Foundational, Comprehensive, Industry-Leading).
• A complete maturity assessment workbook with 108 yes/no questions and a scoring methodology.
• A reference section with a glossary and the major frameworks HAIAMM aligns with.

This is one of six domain handbooks. Software-specific assessment questions live only in this handbook; the Vendors handbook contains only Vendors questions, the Data handbook only Data questions, and so on.

2. The Software domain in v3.0 terms

The Software domain governs the AI/HAI software the organization builds, not the AI tools it consumes from third parties, and not the use of AI to perform security work.

In scope:

• LLM-integrated applications and product features the organization ships, customer-facing chatbots, internal copilots, AI-assisted workflows embedded in the org's own product.
• Autonomous AI agents the organization builds, tool-using agents with autonomy and scope of effect on org or customer systems, including single-purpose and multi-agent orchestrations.
• Retrieval-augmented-generation (RAG) pipelines the organization operates, vector stores, retrievers, and generators wired into the org's own retrieval flows over org-controlled or org-supplied corpora.
• Fine-tuning and training workloads the organization runs, supervised fine-tuning, RLHF, LoRA, and pre-training jobs on org infrastructure or vendor APIs that produce models the org owns.
• Evaluation harnesses the organization builds, automated evaluation pipelines that score model outputs against test sets and gate model promotion.
• Model-serving services the organization operates, inference endpoints the org runs (own-hosted or vendor-hosted-with-org-fine-tune) and that other internal services or external customers consume.
• Classical ML models the organization ships in production, non-generative ML (fraud, ranking, recommendation, anomaly detection) integrated into product surfaces.

Out of scope of the Software domain:

• AI tools and services the organization consumes from third parties, that is the Vendors domain (a Vendors-supplied AI feature embedded inside an org-built application is a Vendors artifact at the consumption layer and a Software artifact at the application layer; cross-references are expected).
• Data flowing into and out of AI systems, that is the Data domain (data is special and frequently cross-references Software).
• AI inference infrastructure the organization hosts as a shared platform, that is the Infrastructure domain (a model registry, an org-wide vector store, an inference platform serving many internal apps).
• Business workflows that embed AI, that is the Processes domain (an AI-augmented hiring workflow, claims-adjudication pipeline, etc.).
• AI-enabled endpoints and user interfaces in scope of endpoint management, that is the Endpoints domain.

The subject of every cell in this handbook is the AI/HAI software the organization ships. The organization is the developer; the system is what is being secured.

3. Why a domain-specific handbook

Building AI/HAI software is not the same as building classic web services or consuming AI from vendors. Five reasons motivate the standalone handbook:

• AI-specific failure modes are first-party risks. Prompt injection, training-data leakage, tool misuse, excessive agency, agent goal hijack, rogue-agent drift, and output-integrity regression are not covered by classic AppSec curricula or by vendor-risk reviews. They are owned by the engineering teams that write the code, train the models, and ship the features.
• Engineering adopts AI faster than security and platform can follow. LLM calls land in customer-facing features in a sprint. Researchers fine-tune from notebooks on side branches. Agents wired to internal APIs ship behind feature flags. The classic SDLC was never designed to gate this path.
• Deployer duties belong to whoever ships the system. EU AI Act Art. 26 deployer duties, GDPR Art. 22 automated-decisioning safeguards, ISO/IEC 42001 AIMS operational obligations, and sector-specific rules (FDA AI/SaMD, FINRA/SEC model risk, NYDFS Part 500) apply to the organization that ships the AI-enabled product, not the foundation-model vendor. The program must produce evidence on demand.
• Shadow AI in engineering is the program's primary L1 outcome. Unsanctioned LLM integrations, ungoverned agents, demo pipelines accumulating production data, and AI features shipped behind feature flags without security review are the central problem the L1 program exists to solve. This is unusual among maturity model domains and is structurally embedded in this handbook.
• Seven archetypes, one program. The seven AI/HAI software archetypes (LLM-integrated app, AI agent, RAG pipeline, fine-tuning/training workload, eval harness, model-serving service, classical ML) behave differently enough that threats, requirements, reference architectures, design checklists, and tests are archetype-keyed throughout the handbook.

4. The seven AI/HAI software archetypes

Most of the practices in this handbook key their content to seven archetypes. Knowing the archetypes well is a prerequisite for using the handbook.

1. LLM-integrated app. An application that calls an LLM (org-hosted or vendor-hosted) as part of its product behavior. Examples: a customer-facing chatbot answering product questions, an internal copilot answering employee help-desk queries, an AI-assisted summarization feature in a SaaS product, an AI-enhanced search experience. Risk shape: prompt injection through user input, data exfiltration via prompts and completions, output integrity, hallucinated harmful content, cost runaway.

2. AI agent. An autonomous or semi-autonomous loop that takes actions using tools. Examples: a tool-using customer-service agent that resolves tickets by querying internal APIs and updating customer records, an internal devops agent that triages incidents by querying observability and remediation systems, a multi-step research agent. Risk shape: the full HAI TTP surface (EA, AGH, TM, RA) plus tool-scope sprawl, indirect injection via tool returns, long-session drift, multi-agent miscoordination, kill-switch failure.

3. RAG pipeline. A retrieval-augmented generation pipeline: vector store plus retriever plus generator. Examples: internal knowledge-base Q&A over the org's docs, customer-facing support assistant retrieving from product documentation, AI-enhanced search over the org's catalog. Risk shape: indirect prompt injection via poisoned corpus content, retrieval-time data leakage, cross-tenant retrieval bleed, embedding inversion, retrieval-source provenance.

4. Fine-tuning / training workload. Supervised fine-tuning, RLHF, LoRA, full pre-training, or domain-adaptation training on org infrastructure or vendor APIs producing models the org owns. Risk shape: training-data leakage (memorization → extraction), data poisoning, supply-chain attacks on base models, label-flipping, integrity of the fine-tune artifact, no-train compliance on training data sourced from vendor APIs.

5. Eval harness. An automated evaluation pipeline that scores model outputs against test sets and gates model promotion. Examples: a CI eval harness that runs regression corpora on every model-version promotion, a continuous output-integrity evaluator running golden test sets daily. Risk shape: eval-set contamination, metric gaming, drift in evaluator-model judgments, false sense of safety, reproducibility failure.

6. Model-serving service. An inference endpoint that exposes a model to other internal services or external customers. Examples: an internal feature-extraction service backed by an embedding model, a customer-facing inference API for an org-fine-tuned LLM, a classical ML scoring service. Risk shape: inference-API abuse (model extraction, jailbreaks, DoS), authn/authz at the endpoint, tenant isolation, model-version churn, silent model-family swap regressions.

7. Classical ML. Non-generative ML models in production (fraud, ranking, recommendation, anomaly detection). Risk shape: adversarial examples, model decay, training-serving skew, evasion attacks. Still in scope of HAIAMM Software because the org builds and operates these models.

A single product is often more than one archetype simultaneously, for example, a customer-facing copilot is both an LLM-integrated app and an AI agent, and may also embed a RAG pipeline. Threat libraries, requirements packs, reference architectures, design checklists, and tests in this handbook accommodate that.

5. Domain boundary rules

When in doubt about whether something belongs in the Software domain, ask: who is responsible for the security of the inside of this thing?

• If the organization is responsible: it is a Software artifact (with possible cross-references to Data, Infrastructure, Processes, or Endpoints depending on what the artifact touches).
• If a vendor is responsible: it is a Vendors artifact (with possible cross-references to Endpoints when the consumption point is a managed endpoint).

Common boundary cases:

• An internal LLM-powered build-failure analyzer is a Software artifact, even if its inference runs on a vendor API. The vendor API itself, separately, is a Vendors artifact.
• A shared vector store hosted in the organization's cloud and used by multiple internal AI features is an Infrastructure artifact. The embeddings inside it may also be a Data concern. Each internal AI feature that queries it is a Software artifact.
• A fine-tuning workload that produces a model the org ships is a Software artifact. The training corpus is a Data artifact. The vendor API used to perform the fine-tune is a Vendors artifact (consumed). The model-serving endpoint the org runs is a Software artifact.
• A customer-facing AI feature that calls an external LLM API is a Software artifact (the org owns the feature and the deployer duties); the external API is separately a Vendors artifact.
• An AI assistant deployed on engineer endpoints (e.g., a coding assistant) is a Vendors artifact (consumed from a third party); only when the org builds and ships its own coding assistant is that a Software artifact.
• Data flowing through an organization-built AI service is a Software concern at the service level and a Data concern at the corpus, log, or embedding level. Cross-references are expected.

6. Stakeholders and roles

The AI/HAI Software Assurance program is cross-functional by design. The following roles appear throughout this handbook:

• Executive sponsor. Typically the CISO co-sponsored by the CTO, Head of Engineering, or Chief AI Officer; co-signed by Privacy/Legal where applicable. Owns budget, scope, and decision rights for the program.
• Program lead. Operationally accountable for the program day-to-day. Often the AppSec lead or AI Security lead. Maintains the AI/HAI software inventory, runs the working group, owns the metrics.
• Cross-functional working group. Security, Engineering (one rep per product line shipping AI), Data/ML platform, MLOps, Product, Privacy/Legal, Site Reliability, and an application-architect reviewer. Meets at least monthly.
• Intake reviewers. A small population trained to assess AI/HAI software artifacts against the threat library, the requirements pack, and the priority compliance map. Drawn from AppSec, AI/ML engineering, and architecture.
• Architect reviewers. Senior engineers with sign-off authority on design reviews for AI/HAI software integrations.
• AI/ML engineering leads. Own per-archetype technical reference patterns and engineering standards.
• MLOps / Platform engineering. Owns the eval harness, model registry, training infrastructure, and model-serving platform, cross-cuts with Infrastructure domain.
• Product owners. Sign off on intended use, user surface, decision-affecting use claims, and deployer-duty obligations for customer-facing or decision-affecting artifacts.
• Deployer-duty owner. A named human-oversight owner for each customer-facing or decision-affecting AI/HAI software artifact in production, accountable for EU AI Act Art. 26 obligations.
• Integration owners. The engineering owner of each AI/HAI software artifact in the inventory, named in the SM inventory and accountable for maintaining the artifact's posture.

7. How to use this handbook

Three modes of use are supported:

• Read it linearly. Parts I and II ground the reader in the domain and foundations. Part III walks the twelve practices in the Software-domain context. Part IV provides the assessment instrument. Part V is reference.
• Run an assessment. Skim Parts I and II for context (one to two hours), then go directly to Part IV. The 108 questions are organized by practice and by maturity level, with explicit evidence prompts. Scoring methodology and rollup tables follow.
• Build a program from scratch. Read Part II carefully, then implement the twelve practices' Level 1 in the order described in Part II's dependency text. Use the Level 1 questions in Part IV as a self-check for completeness.

The questions in Part IV are duplicated as a per-practice "Practice Maturity Questions" section at the end of each practice in Part III. They are the same questions; the duplication is deliberate so the practice-by-practice reader sees the assessment instrument inline.

Part II, Foundations

8. The four Business Functions in this domain

The twelve practices group into four Business Functions. Each function exists for a distinct intent. Every practice belongs to exactly one function.

Governance, Strategy & Metrics (SM), Policy & Compliance (PC), Education & Guidance (EG). Establish why, what, who, and how: the program's strategic frame, its enforceable rules, and the workforce literacy that makes everything downstream possible. In this domain, Governance answers: who owns AI/HAI software risk, what policies apply, what training every engineer and every reviewer must complete, and how AI/HAI software intake gates production.

Building, Threat Assessment (TA), Security Requirements (SR), Secure Architecture (SA). Decide what could go wrong, what the artifact must do about it, and how the artifact is shaped to do it, before code is written or models are trained. In this domain, Building answers: what threats AI/HAI software archetypes carry, what requirements every artifact must meet, what reference patterns engineering teams should reach for.

Verification, Design Review (DR), Implementation Review (IR), Security Testing (ST). Prove that the designed artifact, the implemented artifact, and the running artifact actually meet the Building-function outputs. In this domain, Verification answers: did the team pick the reference pattern at design time, do the live code and configuration match the design, and does the artifact actually behave correctly under adversarial probes.

Operations, Environment Hardening (EH), Issue Management (IM), Monitoring & Logging (ML). Run the program safely in production, harden the runtime, manage the issues, and watch what is actually happening. In this domain, Operations answers: which controls keep sanctioned AI/HAI software development frictionless and unsanctioned development observable, where AI/HAI software issues go, and what telemetry produces deployer-duty evidence on demand.

Cross-function rule: progress in one function without the others is unstable. A mature Verification function on top of weak Governance yields reviews nobody honors. A mature Operations function on top of weak Building chases effects without causes. The handbook is balanced across the four by design.

L1 build order within the Software domain follows the dependency graph: SM precedes everything; PC and EG follow SM; TA, SR, and SA follow Governance; DR and ST run in parallel after SA L1 exists; IR follows DR; EH, IM, and ML form the Operations layer that depends on SM inventory, SA patterns, and PC policies all being in place.

9. The three maturity levels

Every cell in this handbook (each combination of practice and level in the Software domain) is one of three maturity levels. The levels are cumulative, Level 2 assumes Level 1 is in place; Level 3 assumes Level 2 is in place.

Level 1, Foundational. Stand up the minimum viable capability. Discover what AI/HAI software the organization ships, publish the core policies, run the first version of the controls, baseline the metrics. Typical outputs: inventories of all seven archetypes, short published policies (engineering standards, AUP, intake gate), per-archetype threat models, per-archetype requirements packs, per-archetype reference patterns, per-archetype design checklists, per-archetype implementation review checklists, per-archetype test batteries, per-archetype logging baselines, first detections, the AI-specific incident playbook. Reality check: if the program cannot answer "what AI/HAI software do we ship, what rules apply to it, and who is accountable" within a week, it is not at Level 1.

Level 2, Comprehensive. Calibrate the program's intensity by risk tier. Move from one-size-fits-all to differentiated depth. Replace point-in-time activities with continuous validation. Typical outputs: a published risk-tier rubric, a tier-treatment matrix specifying what each tier receives from each practice, per-tier calibrated activities, per-artifact deep threat models for Critical-tier, quantitative/binary requirements packs, IaC-encoded reference patterns, scenario-based design reviews, continuous implementation-drift detection, per-tier red-team cadence, tier-calibrated hardening and logging, post-incident learning loops feeding back to SA/SR/EG/ML. Reality check: if the same review effort goes to a Low-tier internal RAG used by a small team and to a Critical-tier customer-facing agent acting on customer accounts, the program is not at Level 2.

Level 3, Industry-Leading. Automate the substrate. Benchmark externally against peers. Contribute back to the AI-assurance ecosystem. Typical outputs: signal-driven inventory and tier updates, machine-readable requirements pack with CI/CD attestation, continuous configuration attestation, automated adversarial testing, IaC-driven hardening with adaptive tightening from ML and IM signals, detection-as-code with ML-driven anomaly detection on prompt/completion and tool-call corpora, telemetry-driven policy refresh, external benchmarking briefs, contributions to MITRE ATLAS, OWASP LLM/Agentic Top 10, NIST AI RMF Playbook, AI Vulnerability Database, OpenSSF AI, CSA AI Safety Initiative, and sector ISACs. Reality check: if all activity is still internally generated, no external contributions, no benchmarking deltas, no automation replacing routine review work, the program is mature for its own purposes but is not industry-leading.

10. HAI-specific threat tactics (EA, AGH, TM, RA)

Four AI-specific threat-tactic categories appear throughout this handbook. Threat libraries, requirements, reference architectures, design checklists, tests, and detections are tagged to them.

EA, Excessive Agency. The AI or agent has more capability than its use case requires. Tool scopes too broad, permission model wider than any individual human's, effects reaching systems not in scope. The classic example in own-built software: an internal agent issued an org-wide read service account "because that's what the demo used", a level of authority no single human in the organization holds. Mitigated in Software via tool-scope minimization at the SA pattern level, least-privilege IAM at the EH layer, and tool-scope boundary testing in IR and ST.

AGH, Agent Goal Hijack. The agent's benign goal is redirected into an attacker's goal via content injected along a trusted-looking path. The injection might arrive in a retrieved document (RAG corpus), a tool response (an external API's payload), multi-turn history, or any input the agent treats as authoritative. Mitigated in Software via injection-defense at the SA RAG pattern, multi-turn history bounds, retrieved-content distrust at the prompt-template layer, and AGH detection in ML monitoring.

TM, Tool Misuse. Tools available to the AI or agent are invoked for attacker purposes, argument smuggling, unexpected combinations, crafted parameters, recursive invocation. Differs from Excessive Agency in that the scope of the tool may be appropriate; the tactic is the misuse of a legitimate tool. Mitigated in Software via per-tool argument-schema validation at the SA agent pattern, tool-call rate-limit and circuit-breaker defaults at the EH layer, and tool-misuse regression corpus in ST.

RA, Rogue Agents. Autonomous agents drift from intended behavior across long sessions, reflective loops, or multi-agent miscoordination, producing harmful effects nobody explicitly instructed. Drift is the through-line; the trigger may be model update, prompt staleness, or emergent multi-agent interaction. Mitigated in Software via session memory bounds at the SA agent pattern, recursive-invocation guardrails, kill-switch design, and RA detection (long-session drift, multi-agent miscoordination) in ML monitoring.

The four categories are not exhaustive of AI threat surface, they sit alongside classic prompt injection, training-data leakage, output integrity, and supply-chain concerns, but they are the categories most under-represented in classic AppSec frameworks and most useful for organizing the Software domain's threat library.

11. The priority compliance map

Every Software-domain Policy & Compliance practice at Level 1 publishes (and downstream practices reference) a one-page priority compliance map. The set below is the priority set for the Software domain. Sector-specific items are added as applicable.

The map's purpose is traceability: an auditor or regulator asking "how is Article 26 addressed?" should reach a single cell in the map and from there one policy and from there one evidence artifact.

12. Shadow AI in the Software domain (unsanctioned internal AI builds)

Shadow AI, AI/HAI adopted outside the program's visibility, attribution, and governance, is the central problem the Level 1 program exists to solve. In the Software domain, shadow AI takes a different shape than in Vendors:

• Shadow AI in Software is unsanctioned internal AI builds. Engineers shipping LLM features behind feature flags without security review. Eval harnesses skipping the go-live gate. Prototype agents promoted to production. Demo pipelines accumulating production data over months. Fine-tunes run from researcher notebooks against datasets that should not have been touched. AI calls embedded in pre-existing services without announcement, surfaced months later in an audit.
• Shadow AI compounds. Every month of unobserved adoption increases the data-class footprint, the user count, and the regulatory exposure inside the engineering organization. Programs that defer shadow AI work to "later" find later is far more expensive than now, deployer duties have been unmet for months, GDPR Art. 33 clocks have started silently, EU AI Act Annex III high-risk classifications have been missed.
• Shadow AI is observable today. The signals already exist in most engineering organizations, source-code scans for LLM SDK imports and vector-store clients, dependency manifests, CI/CD telemetry for training and fine-tuning jobs, runtime egress to AI provider domains, model and prompt registries, cloud spend on inference APIs, and engineer self-disclosure under amnesty. No new tooling is required at L1.
• Shadow AI manifests through more than one practice. The handbook treats it primarily in SM and EG, but it appears in TA (shadow-AI-in-engineering threat view), PC (amnesty path on the intake gate), EH (egress allowlist and DLP-tuned-for-AI controls catch unsanctioned outbound flows), IM (shadow-AI emergence containment play), and ML (shadow-AI emergence detection). Cross-practice coordination matters.

Every Level 1 activity in this handbook contributes to making shadow AI in engineering visible, attributable, and trending down. The Level 1 outcome metric "shadow-AI-in-engineering ratio" appears in Strategy & Metrics, Policy & Compliance, Education & Guidance, Threat Assessment, Environment Hardening, and Monitoring & Logging, six of the twelve practices. That is intentional.

13. Metrics taxonomy

Every level block in this handbook carries three metric types. The taxonomy is the canonical vocabulary; examples and targets are practice-specific.

• Outcome metrics (lagging). Directly measure whether the level's goal was achieved. Reported monthly or quarterly. Stated in a four-column table: Metric · Baseline · Target · Source.
• Process metrics (leading). Predict outcome metrics by measuring execution. Reported weekly or at the cadence of the underlying activity.
• Effectiveness metrics (business value). Measure what the outcome means to the business. Reported quarterly, often qualitative supported by quantitative.

Metric selection in this handbook follows two rules:

• SMART: specific, measurable, achievable, relevant, time-bound.
• Outcome over output: results are preferred to activity counts. "Reviews completed" is an output. "Issues caught at design rather than production" is an outcome.

If a metric in this handbook does not have a baseline column, the baseline is the value the program records on first measurement. The first cycle of measurement is itself an L1 activity.

Part III, The Twelve Practices in the Software Domain

Each practice section follows the same shape:

• Practice Overview. Objective, description, context.
• Maturity Level 1. Objective, activities (A, B, C), outcome metrics, success criteria.
• Maturity Level 2. Same structure.
• Maturity Level 3. Same structure.
• Common Pitfalls. Three to four per level.
• Practice Maturity Questions. Three yes/no questions per level, the same questions also appear in the Part IV assessment workbook.

14. Strategy & Metrics (SM)

Practice Overview

Objective: Stand up an AI/HAI Software Assurance program that discovers, inventories, and strategically governs the AI/HAI software the organization builds, with shadow-AI-in-engineering prevention as the primary L1 outcome and a defensible risk-tier rubric as the primary L2 deliverable.

Description: SM-Software establishes the program charter, the authoritative inventory of AI/HAI software artifacts the organization ships, and the practice-maturity metrics that prove the program is working. The Software domain governs AI capabilities the organization builds itself across seven archetypes: LLM-integrated applications and product features, autonomous AI agents, retrieval-augmented-generation (RAG) pipelines, fine-tuning and training workloads, evaluation harnesses, model-serving services, and classical ML models integrated into product surfaces. SM L2 produces the risk-tier rubric every other Software-domain L2 practice depends on per the v3.0 dependency graph.

Context: Engineering teams adopt AI faster than security, privacy, and platform teams can review them. An LLM call gets added to a customer-facing feature in a sprint; a researcher fine-tunes a model from a notebook on a side branch; a developer wires up an agent that touches three internal APIs and ships it behind a feature flag. None of this is malicious, it is the normal pace of AI-enabled product development. But it bypasses threat modeling, requirements, reference architecture, design review, and the deployer duties that the EU AI Act and GDPR Art. 22 place on whoever owns the production decision. The AI/HAI Software Assurance program makes this surface visible, attaches accountable ownership, and puts a light-touch intake on the path from prototype to production, so sanctioned AI features ship faster and unsanctioned ones cannot quietly accumulate.

Maturity Level 1

Objective: Stand up the AI/HAI Software Assurance program, build an inventory of AI/HAI software the organization builds, and establish baseline metrics that prove shadow AI in engineering is decreasing.

Activities.

A) Charter the AI/HAI Software Assurance program. Publish a short program charter that names the problem (shadow AI in engineering, ungoverned LLM integrations, agents shipped without threat modeling, fine-tunes that consume training data outside governance), defines scope, and assigns accountable ownership. Charter elements include a problem statement grounded in AI-specific failure modes (prompt injection, training-data leakage, tool misuse, excessive agency, agent goal hijack, rogue agents, output-integrity regression, model-family-swap compatibility); in-scope archetypes (LLM-integrated app, autonomous agent, RAG pipeline, fine-tune/training workload, eval harness, model-serving service, classical ML); an executive sponsor (CISO co-sponsored by CTO / Head of Engineering / Chief AI Officer; co-signed by Privacy/Legal where applicable); a working group spanning Security, Engineering, Data/ML platform, MLOps, Product, Site Reliability, and an application-architect reviewer; decision rights for approval, block, exception, and go-live; and a numerical year-one success target tied to the L1 outcome metrics.

B) Build the AI/HAI software inventory and discover shadow AI. Establish a single AI/HAI software inventory as the program's source of truth. Minimum inventory fields are artifact name, owning team, archetype, production status, customer-facing or internal-only flag, AI/HAI capabilities (tool use, autonomy, multi-turn memory, retrieval sources, fine-tuning sources, output-integrity-critical outputs), data classes processed at inference and training, LLM/model provider and version, approval status (Sanctioned / Provisional / Under review / Prohibited / Awaiting Intake), risk tier (populated at L2), and linked artifacts (TA snapshot, SR REM, SA pattern, latest DR decision, latest IR finding, ML logging-baseline status). Discovery at L1 uses signals platform and security teams already have: source-code grep for LLM SDK imports (openai, anthropic, langchain, llama_index, vertexai, transformers, vllm, bedrock) and vector-store clients (pinecone, weaviate, qdrant, chromadb, pgvector); dependency manifests (package.json, requirements.txt, pyproject.toml, go.mod); CI/CD telemetry tagging training, fine-tuning, and eval jobs; runtime egress to AI provider domains; model and prompt registries (MLflow, SageMaker, Vertex AI Model Registry); cloud spend on Bedrock / Vertex / OpenAI / Anthropic / Azure OpenAI routed by tag; and a 60-second self-attestation form publicized to engineering with an amnesty window for previously undisclosed AI/HAI software in production.

C) Establish foundational metrics that measure practice maturity and shadow AI reduction in engineering. Baseline and track a small, automatable set of outcome, process, and effectiveness metrics tied to the L1 outcome (shadow AI reduction in engineering and inventory coverage of what the org ships). Publish a quarterly shadow AI scoreboard to the executive sponsor that reports total inventory by approval status broken out by archetype, new AI/HAI artifacts discovered this quarter and their intake status, the shadow-AI-in-engineering ratio trend across the last four quarters, AUP attestation coverage across engineering headcount, and the top five unmitigated AI-specific risks with owners and remediation status. Keep activity counts (scans run, tickets closed) out of the outcome view, they belong to process metrics.

Outcome Metrics (L1).

Success Criteria.

• Program charter published and sponsored by an accountable executive (CISO co-sponsored by CTO / Head of Engineering / Chief AI Officer) with a cross-functional working group.
• AI/HAI software inventory exists as a single source of truth with ≥90% coverage of discovered artifacts within 12 months, broken out by archetype.
• Shadow-AI-in-engineering ratio baselined and trending down for two consecutive quarters.
• ≥95% of engineering headcount has acknowledged the AI Acceptable Use & Engineering Standards Policy.
• Quarterly shadow AI scoreboard delivered to the executive sponsor with archetype-level breakdown.

Maturity Level 2

Objective: Risk-tier the AI/HAI software inventory, calibrate the program's intensity per tier, and measure practice maturity and shadow-AI reduction per tier, establishing the tier rubric every other Software-domain L2 practice depends on.

Activities.

A) Define the AI/HAI software risk-tier rubric. Four tiers, Critical / High / Medium / Low, assigned from a small set of auditable dimensions specific to AI/HAI software: data sensitivity processed at inference or training (regulated PHI / PCI / regulated PII / customer source code / customer confidential elevates tier); decision-affecting use (GDPR Article 22 automated decisioning, EU AI Act Annex III high-risk categories such as hiring, credit, education, biometrics, critical infrastructure, law enforcement, immigration, justice, essential services) → Critical; agentic capability (tool surface, autonomy, scope of effect on org or customer systems); user exposure (customer-facing → elevate; partner / B2B → elevate; internal → neutral; developer/eval-only → lower); training-data posture (fine-tuning on customer or regulated data → elevate); production-load-bearing role on a revenue, onboarding, identity, or regulated-control surface; and concentration/criticality as sole AI capability behind a product. The rubric is documented as a short table; tier is derived deterministically; human overrides are allowed but recorded with rationale and reviewed by the working group.

B) Calibrate program intensity per tier. Publish a tier-treatment matrix specifying what each tier receives from each downstream practice, intake depth, TA depth, SA pattern adherence, design review lane, IR cadence and re-review triggers (model swap, new tool, scope change), ST battery and red-team cadence, EH controls (per-tenant isolation, egress allowlist, secrets vault, PII redaction at logging), ML detection set, and IM SLAs by severity. Critical artifacts receive the full program (deep per-artifact threat model, full SR pack with REM, full-lane DR with named architect, semi-annual IR plus on-material-change re-review within 14 days, full ST battery plus quarterly red-team, all detections, executive sign-off); Low artifacts use the fast-track (archetype-level threat snapshot, base SR pack, no required DR, spot-check ST, baseline logging). Each downstream Software-domain L2 practice inherits this calibration; the rubric and the matrix are authored here in SM L2 and changes flow through the SM working group.

C) Per-tier scoreboard and governance. The L1 shadow AI scoreboard becomes tier-aware. Inventory state is reported by tier and by archetype, a Critical-tier customer-facing agent is its own row, the count of Low-tier internal-only RAG prototypes is one line. Shadow-AI-in-engineering ratio is reported per tier, a Critical-tier unsanctioned artifact is a headline, a Low-tier one is a line item. Per-tier SLA adherence across intake, DR, IR, ST, ML, and IM is reported monthly. The tier-movement log records upgrades (an artifact that gained agentic capability, customer exposure, or regulated data) and downgrades with rationale, reviewed by the program sponsor. Quarterly executive review explicitly discusses tier-balance: is the program's effort matching the program's risk profile?

Outcome Metrics (L2).

Success Criteria.

• Risk-tier rubric published and applied; tier assigned to 100% of inventory from auditable dimensions.
• Tier-treatment matrix published; downstream practices (PC, TA, SR, SA, DR, IR, ST, EH, ML, IM) calibrated to it.
• Per-tier shadow AI ratio reported quarterly; Critical-tier unsanctioned AI in production = 0.
• Per-tier SLA adherence ≥90% across practices.
• Tier-movement governance active, changes logged with rationale and reviewed by the sponsor.

Maturity Level 3

Objective: Automate inventory and tier maintenance from live build/deploy/runtime signals, benchmark the program against external peers, and contribute anonymized AI/HAI software ecosystem intelligence back to the industry.

Activities.

A) Continuous inventory and tier automation from build/deploy/runtime signals. Inventory auto-updates from CI/CD events (job-type tags for training, fine-tuning, eval, deploy), model-registry events (new model registered, version promoted, deprecated), dependency-manifest scanning on commit (new LLM SDK or vector-store client imported), runtime egress (new outbound flow to AI provider domain), prompt/completion log volumes (a new artifact emitting prompts is a discovery signal), self-attestation, and the intake queue. Tier assignments are rule-based on the L2 rubric inputs; rule changes are versioned and replayable; tier changes auto-trigger downstream practice obligations within 24 hours (a Medium-to-Critical upgrade triggers DR, ST, and ML reconfiguration). Human curation handles new archetypes, ambiguous discoveries, and dimensional-input conflicts. A data-quality SLO is published: ≥99% of active AI/HAI artifacts correctly tiered within 48 hours of a material change; ≥95% inventory completeness against discovery-source reconciliation.

B) External benchmarking. Program metrics are compared against peer benchmarks through OWASP SAMM AI extensions, OpenSSF AI working groups, CSA AI Safety Initiative, BSIMM-style observational data on what comparable orgs ship in AI/HAI software, MITRE ATLAS practitioner data exchanges, sector ISACs with AI working groups (FS-ISAC, H-ISAC, IT-ISAC), and formal peer roundtables (CISO communities, AI safety practitioner circles). A semi-annual "how we compare" brief covers inventory coverage, shadow-AI ratio, per-tier SLA adherence, automation level, IR drift detection rate, ST coverage rate, and time-from-intake-to-provisional-approval. Benchmark deltas inform program investment, board-level narrative, and next-year L2 / L3 work priorities.

C) Contribute anonymized AI/HAI software ecosystem intelligence. Contribute to MITRE ATLAS (new TTPs observed in own-built software, tagged to HAI TTPs EA / AGH / TM / RA where applicable), OWASP LLM Top 10 and Agentic Top 10 (review, comment, real-world telemetry from prompt/completion and tool-call corpora), NIST AI RMF Playbook and successor editions, the AI Vulnerability Database (AVID) for own-discovered software-side issues responsibly disclosed where they touch upstream models or libraries, OpenSSF AI working groups (reference patterns, dependency-manifest signatures, supply-chain advisories), and ISO/IEC 42001 AIMS community where applicable. Target minimum four substantive contributions per year; quality over volume; every contribution anonymized and legally vetted.

Outcome Metrics (L3).

Success Criteria.

• Inventory auto-update SLO published and met; tier-rule change-log versioned and replayable.
• Tier-assignment automation operational with exception-based human review; tier changes auto-trigger downstream obligations within 24 hours.
• Semi-annual external-benchmarking brief published to the sponsor with ≥5 peer-comparable metrics.
• ≥4 substantive industry contributions per year, anonymized and cited.
• Executive / board ROI narrative refreshed at least annually with external benchmarks and avoided-loss examples.

Common Pitfalls

Level 1. - Inventory seeded only from "AI features the product team announced", misses unannounced LLM calls in pre-existing features, internal-tool integrations, eval-harness data flows, and fine-tunes from researcher notebooks. - Treating AI features inside own apps that call vendor APIs as a Vendors-domain concern, they are first-party Software (the org ships the feature and owns deployer duties); the vendor question is a sub-concern. - Program positioned as a blocker, intake SLA unpublished, engineering cycle time balloons, product teams route around the program by shipping behind feature flags. - Metrics count activity (scans run, tickets closed, reviews completed) instead of outcomes (shadow-AI-in-engineering ratio down, AUP coverage up, AI-specific incidents trending down).

Level 2. - Tier-rubric inputs are subjective ("important," "sensitive"), reviewers tier differently, auditors do not trust it, tier movements feel political. - Tier-treatment matrix published but not enforced, Critical artifacts routed to the same queue as Low; calibration exists on paper only. - Scoreboard still reported in aggregate, hiding Critical-tier shadow AI because overall averages look fine. - Downstream practices treat tier as advisory, not operational, DR / IR / ST / ML do not differentiate scope by tier, defeating the purpose of L2.

Level 3. - Automation runs without a data-quality SLO, signal-driven inventory silently drifts and humans stop trusting it. - Benchmarking chooses peers that flatter the program instead of stretching it (Series-A startups when shipping to enterprise; internal-tool builders when shipping customer-facing AI). - Industry contributions are press releases and conference talks, not technical artifacts that land in MITRE / OWASP / NIST / AVID / OpenSSF. - Tier-change downstream-trigger automation fires too noisily, every prompt-template tweak triggers re-review; engineering teams disable the signal-source rather than fix rule sensitivity.

Practice Maturity Questions

Level 1. 1. Is there a published AI/HAI Software Assurance program charter with a named executive sponsor (CISO co-sponsored by CTO / Head of Engineering / Chief AI Officer), a cross-functional working group, and clear decision rights for approval, block, exception, and go-live? Evidence: charter document with sponsor signatures and working-group roster. 2. Does a single AI/HAI software inventory exist, seeded from source-code, dependency-manifest, CI/CD, runtime-egress, model-registry, and cloud-spend signals, covering all seven in-scope archetypes with ≥90% coverage of discovered artifacts within 12 months? Evidence: inventory export reconciled against discovery-source query results. 3. Are L1 outcome metrics baselined and reported quarterly to the sponsor, inventory coverage, shadow-AI-in-engineering ratio (≤15% trending down), AUP attestation (≥95% of engineering), named-owner coverage (100%), and data-exposure events? Evidence: most recent quarterly shadow AI scoreboard deck.

Level 2. 1. Is every AI/HAI software artifact in the inventory assigned a risk tier based on an auditable rubric covering data sensitivity, decision-affecting use (EU AI Act Annex III / GDPR Art. 22), agentic capability, user exposure, training-data posture, production-load-bearing role, and concentration? Evidence: rubric document plus inventory column showing tier and derivation inputs per artifact. 2. Is there a published tier-treatment matrix driving differential intensity across PC, TA, SR, SA, DR, IR, ST, EH, ML, and IM, with ≥95% of Critical-tier artifacts receiving full-scope treatment in the last 12 months? Evidence: tier-treatment matrix plus cross-practice adherence report for Critical artifacts. 3. Does the quarterly shadow AI scoreboard report per tier and per archetype (with Critical-tier unsanctioned AI in production tracked at zero), and is tier-movement logged and reviewed by the sponsor? Evidence: tier-aware scoreboard and tier-movement log for the prior two quarters.

Level 3. 1. Do inventory and tier assignment auto-update from live build/deploy/runtime signals (CI/CD, model registries, dependency manifests, runtime egress, prompt/completion telemetry, intake, self-attestation) with a published data-quality SLO, and is ≥80% of curation handled automatically with exception-based human review? Evidence: pipeline diagram, SLO dashboard, curation-source breakdown. 2. Do you publish a semi-annual external-benchmarking brief comparing the program against ≥5 peer-comparable metrics via OWASP SAMM AI / OpenSSF / MITRE ATLAS / sector ISACs, and does it drive investment decisions? Evidence: most recent brief and a budget or staffing decision traceable to a benchmark delta. 3. Does the program contribute ≥4 substantive, anonymized artifacts per year to the AI/HAI software ecosystem (MITRE ATLAS, OWASP LLM/Agentic Top 10, NIST AI RMF, AVID, OpenSSF AI, sector ISACs), and does the ROI narrative cite external benchmarks? Evidence: contribution log with acceptance confirmations and the most recent ROI narrative.

15. Policy & Compliance (PC)

Practice Overview

Objective: Publish the priority policies and compliance map that make the AI/HAI Software Assurance program enforceable, so every AI/HAI artifact the organization builds is governed by documented rules, gated before production, and defensible to auditors and regulators.

Description: PC-Software codifies three priority policies specific to engineering AI/HAI software, an AI Engineering Standards policy governing what controls are required per archetype, an AI Acceptable Use & Engineering Standards policy governing what engineers may do with LLM SDKs, model registries, and tool-using agents, and an AI Software Intake / Go-Live Gate policy defining what every artifact must produce to enter production. It maps those policies to the regulations that directly apply to the software the organization ships: EU AI Act Art. 26 deployer duties, Art. 50 transparency, Annex III high-risk triggers, Art. 9 risk management, and Art. 15 accuracy/robustness/cybersecurity; GDPR Art. 22 automated decision-making, Art. 32 security, Art. 33 breach, and Art. 44–49 international transfers; NIST AI RMF GOVERN/MAP/MEASURE/MANAGE; ISO/IEC 42001 AIMS; SOC 2 CC9.2; and sector-specific rules where applicable.

Context: Organizations that build AI-enabled software inherit a generic Secure Development Lifecycle and a generic AUP. Neither answers the questions AI/HAI software raises: which archetypes need a threat snapshot before shipping, who may authorize a customer-facing agent, what data classes are permissible as fine-tuning inputs, what a go-live gate produces as evidence, or how EU AI Act Art. 26 deployer-duty compliance flows from the team that ships the feature to the security review that approved it. Without AI-specific policies and an explicit compliance map, shadow AI accumulates inside the engineering organization, deployer duties go unmet, and auditors cannot trace a regulation to a control. PC-Software governs what the organization builds, in contrast to PC-Vendors, which governs what it consumes.

Maturity Level 1

Objective: Publish the three priority AI/HAI software engineering policies, map them to the priority compliance requirements, and operate the go-live gate that prevents ungated AI/HAI software from reaching production.

Activities.

A) Publish the three priority AI/HAI software engineering policies. Ship each in its smallest useful form, short, readable, specific enough to be enforceable against engineering decisions. The AI Engineering Standards policy specifies the minimum required controls per archetype (TA threat snapshot, SR requirements-evidence map, SA reference-pattern adherence or DR-approved deviation, IR readiness attestation, ST evidence, ML logging-baseline), the data classes permitted per archetype and deployment context (regulated data at inference requires privacy sign-off; regulated data as fine-tuning input requires privacy-officer approval and a no-train-on-production-PII confirmation), agentic scope constraints (no agent acts on customer accounts, executes external writes, or calls APIs outside its declared tool scope without an SR-approved boundary in the REM), output-integrity-critical designations that trigger GDPR Art. 22 safeguards, and model/provider version-logging at go-live with model-family swaps triggering re-review. The AI Acceptable Use & Engineering Standards policy enumerates sanctioned LLM SDKs and providers, lists actions requiring approval (fine-tuning on customer or regulated data, wiring tool-using agents that touch internal APIs, shipping customer-facing AI features, calling foundation-model APIs not on the sanctioned list), prohibits actions without explicit sign-off (fine-tuning on customer PII without privacy approval, agents acting on customer accounts without DR approval, automated decisions with legal/significant effect without Art. 22 safeguards, regulated data piped through non-DPA-covered inference endpoints), imposes a disclosure obligation to the SM-Software inventory, and requires attestation at hire and annually. The AI Software Intake / Go-Live Gate policy makes intake mandatory before production deployment for all in-scope archetypes, lists required go-live artifacts by archetype, exposes an amnesty path for previously ungated production artifacts (which are routed as open IM findings), and names the program sponsor (or delegated AppSec lead) as the go-live decision authority.

B) Map the three policies to the priority compliance requirements. Build a one-page priority compliance map that an auditor can read in 60 seconds. The map ties EU AI Act Art. 26 deployer duties to AI Engineering Standards (archetype controls, output-integrity-critical flag, human-oversight assignment) plus the Intake Gate (artifact checklist, deployer-duty owner); Art. 50 transparency to Engineering Standards (transparency requirement for customer-facing features) plus the AUP (disclosure obligation); Annex III high-risk classification to the Intake Gate (Annex III assessment required for affected archetypes); Art. 9 risk management to Engineering Standards (TA + SR + SA required artifacts) plus the gate checklist; Art. 15 accuracy/robustness/cybersecurity to Engineering Standards (ST evidence required, SR includes robustness). GDPR Art. 22 maps to Engineering Standards (output-integrity-critical flag triggers safeguards) plus the gate (Art. 22 safeguards checklist); Art. 32 security to Engineering Standards plus AUP prohibited flows; Art. 33 breach notification to the gate (IR-readiness attestation); Art. 44–49 international transfers to AUP plus Engineering Standards (model/provider residency at go-live). NIST AI RMF GOVERN/MAP/MEASURE/MANAGE traces to the full three-policy stack and the gate; ISO/IEC 42001 AIMS traces to the program charter (from SM) plus all three L1 policies; SOC 2 CC9.2 traces to the gate's foundation-model vendor DPA confirmation. Sector-specific rules (HIPAA BAA for PHI in clinical AI, PCI-DSS 12.8 for AI in payment flows, FINRA/SEC model risk, FDA AI/SaMD) flow into the archetype controls or the gate's required-artifacts checklist for affected archetypes.

C) Operate the intake gate and track foundational compliance outcomes. Run a single intake ticket queue with a published SLA (triage within 5 business days; provisional approval within 10 business days for Low-tier archetypes with no regulated data and no customer exposure). The artifacts checklist is archetype-keyed, the engineer submitting intake receives the checklist for their archetype and missing artifacts block go-live. Gate approval creates or updates the SM-Software inventory record with artifact links. The amnesty path is linked from the intake form, the AUP, and the eng-all-hands communications from SM. Exceptions are logged with owner, rationale, and review date; no exception may remain open longer than 90 days without re-review. Foundation-model provider DPA status, training-on-data posture, and named subprocessor list are confirmed at go-live rather than trusted from a marketing page.

Outcome Metrics (L1).

Success Criteria.

• Three priority policies (AI Engineering Standards, AI AUP & Engineering Standards, AI Software Intake / Go-Live Gate) published, approved by Legal/Privacy and Security, communicated to all engineers.
• One-page priority compliance map published, covering EU AI Act Art. 26/50/Annex III/Art. 9/Art. 15, GDPR Art. 22/32/33/44–49, NIST AI RMF GOVERN/MAP/MEASURE/MANAGE, ISO/IEC 42001, SOC 2 CC9.2, and applicable sector-specific obligations; linked from each policy.
• Go-live gate operational with a per-archetype artifacts checklist, published SLA, and visible amnesty path.
• ≥95% of engineering headcount has acknowledged the AI AUP in the current year.
• ≥85% of AI/HAI software artifacts reaching production in the last 12 months passed the gate; 100% for Critical/High-tier; every customer-facing or decision-affecting artifact has a named deployer-duty owner.

Maturity Level 2

Objective: Deepen policy controls and compliance evidence per AI/HAI software risk tier, automate artifact assembly from the SM-Software tier rubric, and produce audit-ready evidence trails continuously.

Activities.

A) Tier-calibrated policy depth and sign-off requirements. Extend the three L1 policies with tier-specific addenda using the SM-Software L2 tier rubric. Critical artifacts require full SR pack with REM, executive (CISO or CTO) and privacy-officer sign-off before go-live, EU AI Act Annex III high-risk assessment reviewed by Legal, GDPR Art. 22 safeguards reviewed by Privacy, foundation-model inference provider DPA and training-data posture attestation on file at go-live, a kill-switch and human-override path confirmed and tested, and mandatory re-review within 14 days on every material change (model swap, new tool added to an agent, new data class, scope expansion). High artifacts require full SR pack plus REM with fast-track exemptions, CISO-delegated AppSec lead sign-off, EU AI Act and GDPR assessments, provider attestation, and re-review within 30 days on material change. Medium artifacts use base SR pack plus REM with fast-lane DR (or DR waiver for sanctioned reference-pattern implementations) and re-review annually or within 60 days on material change. Low artifacts use base SR pack with self-attested checklist and re-review at annual review. Policy exceptions require named owner, compensating control, Legal/AppSec reviewer acknowledgment, and expiry date (max 12 months); Critical-tier artifacts have no amnesty for missing go-live artifacts after L2 is established, missing artifacts become blocking findings routed through IM.

B) Continuous compliance evidence assembly and foundation-model attestation tracking. For every Critical and High AI/HAI software artifact, maintain a live compliance evidence bundle that auto-assembles the current TA snapshot, the SR REM with gap status and owner, the SA reference-pattern confirmation or DR-approved deviation, the latest DR decision and date, the latest IR attestation or finding log if IR found drift, ST evidence (test battery last-run date, prompt-injection regression corpus last-run date, data-egress canary last-run date), ML logging-baseline confirmation with last-validated date, the deployer-duty record (named human-oversight owner, disclosure mechanism, Art. 26 obligations checklist), and the foundation-model provider record (current DPA, training-on-data posture statement, subprocessor list, model-version log). Staleness rules trigger PC-Software findings routed to IM: Critical TA snapshot 90 days, IR attestation 6 months, ST evidence 30 days, provider DPA status 90 days. The evidence bundle is the primary artifact a regulator or auditor receives when asking about a specific artifact.

C) Exception management and tier-aware enforcement. Integrate the exception register with the intake gate: no exception approved without tier-appropriate compensating control and expiry. Monthly exception aging review escalates exceptions >90 days past expiry to the program sponsor. Sector-specific evidence bundles (HIPAA PHI-in-clinical-AI, PCI-DSS 12.8, FDA AI/SaMD, FINRA/SEC model-risk) are generated from the compliance evidence bundle for the artifacts they apply to; completeness tracked. Enforcement asymmetry: Critical-tier artifacts with missing go-live artifacts are blocking findings; no amnesty applies post-L2.

Outcome Metrics (L2).

Success Criteria.

• Three priority policies extended with tier-specific addenda; 100% of Critical artifacts carry executive plus privacy-officer sign-off at go-live in the last 12 months.
• Compliance evidence bundle live for every Critical/High artifact; staleness inside tier-specific targets.
• Exception register comprehensive and reviewed monthly; zero exceptions past expiry un-escalated; Critical-tier missing go-live artifacts treated as blocking findings.
• Sector-specific evidence bundles (HIPAA / PCI-DSS / FDA / FINRA as applicable) complete for in-scope artifacts.
• Regulatory or auditor inquiry evidence SLA (≤5 BD) met in the last 12 months.

Maturity Level 3

Objective: Automate compliance attestation from CI/CD, model-registry, and runtime telemetry; drive policy updates from monitoring signals and external regulatory motion; and contribute to AI software standards development.

Activities.

A) Continuous compliance attestation from CI/CD and model-registry signals. Evidence bundles auto-update from CI/CD go-live events (artifact checklist attached to release record), model-registry promotion events (a new model version triggers re-checks of TA snapshot age, IR attestation currency, and ST evidence), dependency-manifest changes (a new LLM SDK import auto-opens a PC finding if the artifact is not yet in inventory), runtime-egress signals (a new AI provider domain auto-opens intake), and prompt/completion log volume events. The attestation-generation pipeline produces a provenance-complete evidence pack for any artifact, regulation-keyed (EU AI Act evidence pack, GDPR processor-obligation pack, ISO 42001 AIMS evidence set) or artifact-keyed, within 3 business days. The currency SLO is ≤24 hours latency after a triggering event; completeness is ≥99% of active Critical/High artifacts.

B) Telemetry-driven policy refresh and regulatory-motion tracking. Operate a quarterly policy-refresh cycle driven by ML-Software detection trends (which AI-specific violation classes are rising), IM-Software incident learnings (which policy gaps created the incident conditions), tier-movement data (which archetypes are growing fastest and at what risk level), and external regulatory and standards updates (EU AI Act implementing acts, EDPB AI guidance, NIST AI RMF Playbook updates, US Executive Orders on AI, state AI laws, sector-specific guidance from FDA/FINRA/OCC/NYDFS/HHS). Refresh output is a versioned changelog for each of the three policies approved by Legal/Privacy and Security; EG-Software training content updates within 30 days of any policy change; the SM-Software inventory archetypes and tier rubric are reviewed for needed updates. A regulatory-motion tracker maintains a log of open regulatory instruments with expected effective dates, mapped to the policy each will affect; the working group reviews quarterly.

C) Standards contribution and external engagement. Participate in AI software standards and regulatory forums: EU AI Act deployer-guidance consultations (Art. 26 implementing acts), GDPR EDPB AI guidance rounds, NIST AI RMF Playbook working groups, ISO/IEC 42001 community, and sector regulators (FDA AI/SaMD, FINRA/OCC model risk, NYDFS Part 500, HHS). Contribute AI-software-specific artifacts to public standards, go-live gate schemas, compliance evidence bundle templates, archetype-keyed policy addendum patterns, deployer-duty evidence records, through CSA AI Safety Initiative, OpenSSF AI, Shared Assessments, and the OWASP AI governance track. Target at least two substantive public comments or standards contributions per year on AI/HAI software policy and compliance topics.

Outcome Metrics (L3).

Success Criteria.

• On-demand attestation pack generation inside 3 business days for any active AI/HAI software artifact; SLA met in last 12 months.
• Continuous attestation pipeline operational with ≤24h currency SLO; completeness ≥99% of Critical/High artifacts.
• Quarterly telemetry-driven policy-refresh cycle operating with a versioned, externally-auditable changelog.
• ≥2 substantive public regulatory or standards contributions per year on AI/HAI software policy; external recognition documented.
• Zero material audit findings on AI/HAI software controls in the last 12 months.

Common Pitfalls

Level 1. - Reusing the generic AUP and SDLC without AI-specific clauses, no rule on fine-tuning data, no archetype-specific controls, no deployer-duty owner requirement; auditors cannot trace a regulation to an artifact. - Go-live gate applies only to new AI features announced through product management, misses LLM calls embedded in existing services behind feature flags, eval harnesses reaching production traffic, and fine-tunes run from researcher notebooks. - Compliance map lists frameworks but does not say which policy carries which regulation, auditors trace coverage themselves and typically conclude it is untraceable. - Gate checklist is archetype-agnostic, an agent and a classical ML model receive the same list; agent-specific controls (kill-switch, tool-scope boundary, human-override) are never actually required.

Level 2. - Tier-specific addenda published but sign-off requirements never enforced, Critical artifacts ship with only the base L1 checklist because no one enforces the executive sign-off rule. - Compliance evidence bundle is a folder of PDFs only the compliance lead can navigate, a second reviewer cannot assemble the regulator pack without them. - Evidence staleness thresholds exist on paper but no alert fires when exceeded, a Critical TA snapshot ages past 90 days and nobody notices until an audit. - Sector-specific bundles treated as "covered by the DPA", HIPAA BAA specifics or FDA AI/SaMD documentation are not operationalized.

Level 3. - Attestation pipeline generates evidence that is technically complete but narratively thin, a regulator still needs a human to explain what the artifacts mean; the 3 BD SLO is met but a follow-up hearing is needed. - Policy refresh is cadence-only, quarterly ritual without real telemetry input; the changelog reads like formatting updates. - External regulatory contributions are deadline-only comment letters rather than technical artifacts implementing bodies actually use. - Contributed policy templates and schemas are published once and then go stale, external practitioners find outdated versions and stop trusting the program.

Practice Maturity Questions

Level 1. 1. Have you published and formally approved the three priority AI/HAI software engineering policies (AI Engineering Standards, AI AUP & Engineering Standards, AI Software Intake / Go-Live Gate) with archetype-specific controls, data-class restrictions, and a deployer-duty owner requirement, and is there a one-page compliance map tying each priority requirement (EU AI Act Art. 26/50/Annex III/Art. 9/Art. 15, GDPR Art. 22/32/33/44–49, NIST AI RMF, ISO/IEC 42001, SOC 2 CC9.2, sector-specific) to the specific policy that carries it? Evidence: published policy set, approval signatures, and one-page compliance map. 2. Is the go-live gate operational with a per-archetype artifacts checklist, a published intake SLA (triage ≤5 BD; provisional approval ≤10 BD for Low-tier), and an amnesty path, and does ≥85% of AI/HAI software reaching production in the last 12 months have a gate record (100% for Critical/High)? Evidence: intake queue export reconciled against SM-Software inventory. 3. Are ≥95% of engineering headcount covered by a current-year AI AUP acknowledgment, and does every customer-facing or decision-affecting AI/HAI artifact in production have a named deployer-duty owner logged in SM-Software inventory? Evidence: LMS attestation report and inventory column showing deployer-duty owners for affected artifacts.

Level 2. 1. Have the three priority policies been extended with tier-specific addenda (per the SM-Software L2 rubric), and do Critical artifacts carry explicit executive plus privacy-officer sign-off at go-live with a live compliance evidence bundle covering TA snapshot, SR REM, SA pattern confirmation, DR decision, IR attestation, ST evidence, ML logging-baseline, deployer-duty record, and foundation-model provider attestation? Evidence: tier addenda, gate records showing dual sign-off, and a sample evidence bundle for a Critical artifact. 2. Is the compliance evidence bundle continuously maintained for every Critical/High artifact with staleness inside tier-specific targets, and can a regulatory or auditor inquiry be satisfied with provenance-complete artifacts within 5 business days? Evidence: evidence-registry staleness report and last inquiry-response log. 3. Is an exception register operated with named owners, compensating controls, and expiry dates, reviewed monthly, with Critical-tier missing go-live artifacts treated as blocking findings (no amnesty), and sector-specific evidence bundles (HIPAA / PCI-DSS / FDA / FINRA as applicable) complete for in-scope artifacts? Evidence: exception register, monthly review minutes, and sector-bundle completeness report.

Level 3. 1. Does a continuous attestation pipeline auto-update evidence bundles from CI/CD events, model-registry promotions, and runtime signals, with attestation currency ≤24h latency and ≤3 BD on-demand pack generation, and is ≥99% of Critical/High artifacts continuously attested? Evidence: pipeline architecture, SLO dashboard, currency and completeness metrics. 2. Does the program operate a quarterly, telemetry-driven policy-refresh cycle (ML detection trends + IM incident learnings + regulatory-motion tracker + tier-movement data) with a versioned changelog where 100% of changes are traceable to a named signal or regulatory update? Evidence: most recent policy changelog with rationale entries citing telemetry or regulatory source. 3. Does the program contribute ≥2 substantive public comments or standards artifacts per year on AI/HAI software policy topics (EU AI Act implementing guidance, GDPR EDPB AI guidance, NIST AI RMF Playbook, ISO/IEC 42001, sector regulators, or community standards bodies) with documented external recognition? Evidence: contribution log with publication links and recognition citations.

16. Education & Guidance (EG)

Practice Overview

Objective: Build the AI-assurance literacy every engineer touching AI/HAI software needs and the practitioner skills the smaller population performing threat modeling, secure code review, security testing, architecture review, and red-teaming of AI systems must have, with shadow AI in engineering awareness as the primary L1 cultural outcome.

Description: EG-Software covers two audiences. The first is the entire engineering population building or shipping in-scope AI/HAI software archetypes (LLM-integrated applications, agents, RAG pipelines, fine-tune/training workloads, eval harnesses, model-serving services, classical ML models); they need AI-assurance literacy covering what the archetypes are, what the HAI TTPs (EA / AGH / TM / RA) mean for the code they write, what the AI AUP requires, how the go-live gate works, and what a sound requirement-evidence map looks like. The second is the practitioner population, AppSec reviewers, AI/ML platform engineers, application architects doing AI-feature reviews, and red-teamers, who need deep, hands-on skills covering MITRE ATLAS tactics, OWASP LLM/Agentic Top 10, prompt-injection patterns, agent goal-hijack recognition, tool-misuse detection, training-data poisoning indicators, output-integrity testing, and kill-switch design.

Context: AI-specific vulnerabilities, prompt injection (AGH), excessive agency (EA), tool misuse (TM), rogue agent drift (RA), training-data leakage, output-integrity regression, are not covered by classic AppSec curricula. Engineers adopting LLM features, RAG pipelines, and agent platforms learn the API surface but rarely the adversarial model. A developer who has only taken the org's OWASP Top 10 course will build an agent without thinking about goal-hijack scenarios; an AppSec reviewer trained only on SAST and DAST findings will not recognize a prompt-injection vector in a tool-call argument. Without a deliberate EG practice targeted at these gaps, AI security surfaces late, at incident time, in external audits, or in customer questionnaires.

Maturity Level 1

Objective: Deliver foundational AI-assurance literacy to ≥95% of the engineering workforce building AI/HAI software and role-based practitioner training to 100% of the reviewer population, with an active shadow-AI-in-engineering awareness campaign.

Activities.

A) Ship engineering workforce AI-assurance literacy training. A single short course (≤20 minutes) every engineer takes on hire and refreshes annually, tied to the AI AUP attestation from PC-Software L1. Content covers what the seven AI/HAI software archetypes are with concrete examples from the org's own inventory; the AI AUP in five rules (sanctioned archetypes and SDKs, prohibited data-class flows, approvals required before fine-tuning on regulated data or wiring up an agent, disclosure obligation to the inventory, attestation requirement); the HAI TTPs in plain language (Excessive Agency, the agent can do more than it should; Agent Goal Hijack, injected content redirects the agent's goal; Tool Misuse, the agent's tools are invoked for attacker purposes; Rogue Agent, autonomous drift from intended behavior), plus prompt injection, training-data leakage, and output-integrity regression with one concrete engineering example per TTP matched to a relevant archetype; how the go-live gate works (how to submit intake, what the per-archetype artifacts checklist requires, what provisional approval means, how the amnesty path works); a one-screen example of a good requirement-evidence map (REM); and a before-you-connect decision aid. Delivery is an LMS module plus a one-page reference card pinned in engineering Slack/Teams plus a brief at engineering all-hands when the program launches; no role gating.

B) Deliver role-based practitioner training for the reviewer population. A deeper module (approximately 2 hours) for the practitioner population only, AppSec reviewers, AI/ML platform engineers running model-serving and fine-tuning infrastructure, application architects reviewing AI-feature designs in DR, and red-teamers running ST exercises. Completion is a prerequisite to approving intakes. Content covers a MITRE ATLAS tactics walkthrough across all 14 tactics applied to the org's archetypes, with one example per tactic; the OWASP LLM Top 10 and OWASP Agentic Top 10 mapped to the archetype(s) where each entry is most relevant; prompt injection in depth (direct and indirect patterns; retrieved content, tool responses, and multi-turn history as injection vectors); agent goal-hijack scenarios (multi-step goal drift, crafted documents, multi-agent coordination channels, AGH + RA combined); tool-misuse pattern recognition (argument smuggling, unexpected combinations, recursive invocation, crafted parameters); training-data poisoning indicators and provenance-record reading; output-integrity testing (regression corpora for jailbreaks and prompt injections, data-egress canary design, kill-switch and human-override path testing, logging-completeness verification); kill-switch and human-override design at the agent-deployment level; the priority compliance map in practice (given an archetype, which requirements apply and where the evidence lives); and a calibration exercise where three sample archetype intakes (a customer-facing agent, a fine-tune on internal data, a RAG over public docs) are scored independently with instructor-facilitated debrief.

C) Run the shadow-AI-in-engineering awareness campaign. An always-on communications program that makes it uncomfortable to ship AI/HAI features outside the program and easy to surface them. Elements include a launch moment with the executive sponsor naming shadow AI in engineering, announcing the amnesty window, and publishing the sanctioned-archetype catalog with explicit framing that the program is an enabler not a blocker; recurring monthly short content (new archetype approved, a fast-track win such as intake-to-provisional in 3 BD, an anonymized example of a TTP caught during intake review, an external incident reframed as "what would we find in our own inventory?"); an "Is this AI?" series calling out features silently shipping in internal tooling or behind feature flags; an amnesty path visibly linked from the AUP, the intake form, and engineering channel pins; a feedback channel for engineers to nominate archetypes or SDK patterns for the sanctioned catalog (triaged and acknowledged within 5 BD); and deployer-duty micro-content for teams shipping customer-facing or decision-affecting AI features. Campaign channel links are tagged so attribution of intake submissions and amnesty disclosures to campaign touchpoints is tracked.

Outcome Metrics (L1).

Success Criteria.

• Workforce AI-assurance literacy module launched; ≥95% current-year completion sustained.
• Practitioner training launched, completion gated on intake-approval permissions, and reviewer calibration drift inside target for two consecutive quarters.
• Shadow-AI-in-engineering awareness campaign running with at least monthly content cadence and measurable attribution.
• Deployer-duty micro-content deployed for every customer-facing or decision-affecting AI/HAI archetype active in the inventory.
• Training content owner named; content updated within 30 days of any change to policies, archetype list, or compliance map.

Maturity Level 2

Objective: Deepen practitioner skill through scenario-based training from real intake cases, deliver product-line-specific engineering tracks calibrated to SM-Software L2 risk tiers, and run seasonal shadow-AI-in-engineering campaigns tied to release cycles.

Activities.

A) Scenario-based reviewer training from real intakes. Build a scenario library from anonymized real intakes from the org's own queue; each scenario includes the as-submitted archetype description, the original reviewer decisions (tier, TTP identifications, SR gaps), any reviewer disagreement, and the resolved outcome after calibration or post-launch review. Organize scenarios per archetype (agent, RAG-pipeline, fine-tune, model-serving) and per TTP cluster (EA-heavy, AGH-heavy, RA-heavy, training-data-leakage-heavy). Run paired calibration exercises in which two reviewers independently score the same scenario, with instructor-facilitated debrief on tier delta, TTP-identification deltas, and SR gap list differences. Weight curriculum to tier: Critical-tier agent and fine-tune scenarios dominate the advanced modules; Medium/Low scenarios stream-line fast-track calibration. Practitioners graduate by running three live intakes end-to-end with a senior-reviewer shadow and producing a passing TA snapshot and SR REM.

B) Product-line-specific engineering tracks. Deliver distinct training tracks for engineering product lines building AI/HAI software. The mobile track covers on-device model serving, RAG with local vector store, agent calls from mobile clients, EA and TM TTPs in mobile-surface context, and SA reference-pattern differences from server-side. The web/SaaS track covers LLM-integrated web features, browser-side RAG, customer-facing chatbots, AGH via user-supplied content, Art. 50 transparency in UX, and output-integrity in customer-visible responses. The ML platform track covers fine-tuning pipelines, training workloads, model-serving infrastructure, eval harnesses, training-data leakage patterns, no-train verification in provider DPAs, and data-egress canary design for prompt/completion logs. The backend services track covers agentic pipelines calling internal APIs, multi-agent coordination, tool-using backend workers, RA + EA TTP patterns in long-running agent sessions, and kill-switch and human-override architecture for server-side agents. Each track is paired with the SA reference pattern for the relevant archetype. Required for any team owning a Critical or High-tier artifact in the applicable product line; target ≥1 trained practitioner per artifact.

C) Seasonal, behavior-driven shadow-AI-in-engineering campaigns. Tie campaigns to observed shadow-AI risk windows in the engineering cycle: major release windows (sprint-to-ship pressure leads to ungated LLM additions), Q1 OKR planning (teams add AI features to roadmaps without intake), hiring surges (new engineers arrive with pre-existing habits), and post-external-incident moments (a public prompt-injection or training-data-leak incident creates a teachable window). Each campaign carries a pre-measured behavior target (for example, "reduce ungated LLM SDK imports in the monorepo by 50% in Q3" or "increase Critical-tier intake submissions before sprint start by 30%") and a post-campaign measurement. Amnesty windows run alongside campaigns; disclosure volume and source attributed to campaign channels. Campaigns missing behavior targets by more than 20% are redesigned by the program sponsor.

Outcome Metrics (L2).

Success Criteria.

• Scenario library of ≥30 real-sourced scenarios across archetypes; reviewer calibration drift inside target for two consecutive quarters.
• Product-line training tracks (mobile, web, ML platform, backend) delivered; ≥1 trained practitioner per Critical/High-tier artifact.
• ≥2 behavior-driven campaigns run in the last 12 months with measured outcomes; ≥70% of campaigns hit behavior target.
• Training content refresh cadence met; ≥80% of content updated in last 90 days.

Maturity Level 3

Objective: Operate continuous calibration at scale, externalize the AI-assurance curriculum and reviewer rubric as industry-shared artifacts, and contribute to emerging AI-engineering certification pathways.

Activities.

A) Externalize the curriculum, scenario library, and reviewer rubric. Publish the workforce AI-assurance literacy module (learning objectives, assessment questions, reference-card template), the practitioner role-based training curriculum (module outlines, ATLAS tactic coverage matrix, per-archetype reviewer job aids), the anonymized scenario library (scenario format, per-archetype examples, calibration debrief format), and the reviewer rubric (tier-assignment criteria, TTP-identification scoring, SR-gap-list completeness scoring) under a permissive license or as a consortium deliverable through CSA AI Safety Initiative, OpenSSF AI, the OWASP AI security track, or applicable sector ISAC (FS-ISAC, H-ISAC, IT-ISAC). Accept community contributions; flow changes back into internal content within 30 days. Track adoption via citations in external publications, forks, downloads, and direct adoption acknowledgment from other organizations.

B) Continuous live calibration. Run monthly calibration rounds: a current anonymized intake sampled from the program's live queue is shared with the reviewer cohort; each reviewer independently scores tier, TTPs, and top three SR gaps; drift is reported to the program sponsor. Individual reviewer drift is a development signal, not a performance metric, reviewers with persistent drift on specific archetype types receive targeted coaching and additional scenario exposure. Calibration results feed the scenario library directly; new scenarios drawn from intakes where calibration revealed drift are added within 30 days.

C) AI-engineering certification contribution. Contribute to AI-engineering and AI-assurance certification pathways as they emerge: CSA AI Safety, ISACA AI Audit / AI Risk certificates, sector-specific ISAC credentials, OWASP AI Security curriculum, OpenSSF AI Practitioner path. Align the org's practitioner capstone with certification-grade rubrics where credentials exist; support reviewers pursuing external credentials. Contribute MITRE ATLAS new-technique candidates and confirmed-technique instances from own-built AI/HAI software observations (minimum one per year where novel observations exist). Target ≥2 substantive contributions per year to industry curriculum or certification working groups.

Outcome Metrics (L3).

Success Criteria.

• Curriculum, scenario library, and reviewer rubric published externally with documented adoption.
• Monthly live calibration operating; drift inside target for two consecutive quarters; calibration results feeding the scenario library continuously.
• ≥50% of Critical-tier reviewers credentialed where credentials exist.
• ≥2 substantive contributions to industry certification / curriculum per year.
• ≥1 MITRE ATLAS TTP contribution or confirmation per year where novel observations exist.

Common Pitfalls

Level 1. - Workforce training covers classic OWASP Top 10 but not the HAI TTPs (EA / AGH / TM / RA), engineers know about SQL injection but not about agent goal hijack or excessive agency; the AI-specific gap remains open. - Practitioner training is a one-hour "intro to LLMs" rather than a hands-on module covering ATLAS tactics, OWASP LLM/Agentic Top 10, and TTP-recognition exercises against real archetype examples. - Reviewer training is optional, intake-approval permissions granted without training completion; calibration drift is never measured; two reviewers regularly arrive at different tiers for the same archetype. - Shadow-AI campaign launches once with an exec message, then goes silent, no monthly content, no amnesty attribution, no feedback channel; engineers never hear about it again.

Level 2. - Scenario library built from invented examples rather than anonymized real intakes, reviewers learn the shape of a "good" intake but not the actual edge cases that surface in the org's queue. - Product-line tracks are optional; engineering teams skip them and produce designs in DR that do not account for archetype-specific TTPs; DR catches the gaps late and at high cost. - Campaigns launched without a pre-measured behavior target, "shadow AI awareness" claimed as success without data on whether ungated LLM imports decreased or amnesty disclosures increased. - Calibration drift is measured but not acted on, reviewers with persistent drift never receive coaching; the calibration exercise becomes a box-check rather than a development signal.

Level 3. - External publication without ongoing maintenance, other organizations find a stale scenario library and stop trusting the program; citations dry up. - Credentialing becomes performative, reviewers pursue credentials that do not map to the org's actual tier-treatment rubric; credential acquisition is celebrated but calibration drift stays unchanged. - Live calibration becomes a gotcha rather than a development signal, reviewers learn to game the monthly exercise and improve calibration scores without improving actual intake quality. - Contributions to industry working groups do not loop back, what is published externally drifts from what reviewers use internally; practitioners cite the external artifact and contradict the internal rubric.

Practice Maturity Questions

Level 1. 1. Have all engineers building or operating AI/HAI software completed a current-year AI-assurance literacy course covering the seven in-scope archetypes, the HAI TTPs (EA / AGH / TM / RA) plus prompt injection, training-data leakage, and output-integrity regression, the AI AUP rules, and the go-live gate intake process, with ≥95% completion and content updated within 30 days of any policy or archetype change? Evidence: LMS completion report, content change-log, and the most recent literacy module. 2. Has the practitioner population (AppSec reviewers, AI/ML platform engineers, AI-feature architects, red-teamers) completed role-based training covering ATLAS tactics, OWASP LLM/Agentic Top 10, prompt injection patterns, agent goal hijack, tool misuse, training-data poisoning, output-integrity testing, and kill-switch design, with completion gated on intake-approval permissions and calibration drift ≤1 tier step and ≤2 TTP misclassifications per sample for two consecutive quarters? Evidence: practitioner curriculum, permission-gating record, and calibration-exercise results. 3. Is a shadow-AI-in-engineering awareness campaign running with at least monthly content, a visible amnesty path linked from the AUP and intake form, and measurable attribution of intake submissions and amnesty disclosures to campaign channels, with disclosures rising in Q1–Q2 after launch then declining as the sanctioned-archetype program grows? Evidence: campaign content calendar, channel-attribution report, and amnesty disclosure trend.

Level 2. 1. Is there a scenario library of ≥30 anonymized real intake cases powering practitioner training across the org's in-scope archetypes, with paired calibration exercises showing Critical-tier drift ≤1 tier step and ≤1 TTP misclassification per sample for two consecutive quarters? Evidence: scenario library index and quarterly Critical-tier calibration drift report. 2. Have product-line-specific engineering tracks (mobile, web, ML platform, backend as applicable) been delivered to ≥1 practitioner per Critical/High-tier artifact, with team-level training coverage tracked in the SM-Software inventory? Evidence: track rosters reconciled against the inventory's Critical/High artifact list. 3. Are shadow-AI campaigns running on a seasonal, behavior-driven cadence with pre-set behavior targets and post-campaign measurement, with ≥70% of campaigns hitting their target, and is ≥80% of training content updated in the last 90 days? Evidence: campaign plans with pre/post measurements and content-refresh changelog.

Level 3. 1. Has the practitioner curriculum, anonymized scenario library, and reviewer rubric been published externally (CSA, OpenSSF AI, OWASP AI, or sector ISAC) with documented adoption, citations, forks, or direct acknowledgment, and do contributions loop back into internal content within 30 days? Evidence: external publication links, adoption telemetry, and internal update changelog. 2. Is a monthly live calibration cadence operating (anonymized intake from the live queue, independent reviewer scoring, drift reported to sponsor), with calibration results feeding the scenario library within 30 days, and do ≥50% of Critical-tier reviewers hold an external AI-assurance or AI-engineering credential where one exists? Evidence: calibration log, scenario-library update trail, and credential registry. 3. Does the program contribute ≥2 substantive artifacts per year to industry AI-engineering certification or curriculum working groups, and ≥1 MITRE ATLAS TTP contribution or confirmation per year where novel observations exist in own-built AI/HAI software? Evidence: contribution log with acceptance confirmations and the ATLAS submission record.

17. Threat Assessment (TA)

Practice Overview

Objective: Build and maintain a reusable threat library for the AI/HAI software the organization ships, one archetype-level threat model per software artifact type, so every SM intake produces a threat snapshot in minutes rather than a blank-page exercise.

Description: TA-Software catalogs the threats specific to AI/HAI software the organization builds and ships. At L1 the library covers one threat model per software archetype, LLM-integrated app, AI agent, RAG pipeline, fine-tuning/training workload, eval harness, model-serving service, classical ML, mapped to HAIAMM's HAI-specific TTPs (EA, AGH, TM, RA), to MITRE ATLAS tactics (TA0001–TA0014), and to OWASP LLM/Agentic Top 10. Each artifact registered in the SM inventory generates a threat snapshot by pulling the archetype model and adding artifact-specific deltas: specific tool list, specific retrieval sources, specific fine-tuning data classes, output-integrity-critical paths. L2 layers per-artifact deep models for Critical-tier artifacts and red-teams the threat library quarterly against real integrations. L3 automates library maintenance from telemetry and external feeds, and contributes discovered TTPs back to MITRE ATLAS, OWASP, and AVID.

Context: Engineering teams shipping AI/HAI software face failure modes that classic AppSec threat modeling was not designed to enumerate, prompt injection at the application boundary, tool-scope overreach in agent loops, retrieval poisoning in RAG pipelines, training-data leakage in fine-tuning workloads, output-integrity regression when the underlying model family changes. These are first-party risks owned by the teams that write the code and train the models. TA-Software closes the gap by making AI/HAI-specific threats a first-class library that threat modelers pull from at every intake, and by tying every archetype threat to a specific ATLAS tactic so the walk from attacker capability to artifact exposure is concrete rather than narrative.

Maturity Level 1

Objective: Build the AI/HAI software archetype threat library, integrate a threat snapshot into every SM intake, and ensure every artifact's threat surface is documented before it reaches production.

Activities.

A) Build the AI/HAI software archetype threat library. Author one threat model per AI/HAI software archetype. Each is concise (target two pages), explicitly scoped to first-party artifacts the organization builds, and maps threats to HAI TTPs, ATLAS tactic IDs, OWASP LLM/Agentic Top 10 references, and the PC-Software priority compliance map. Archetypes to cover: LLM-integrated application or feature, autonomous AI agent, RAG/retrieval-augmented generation pipeline, fine-tuning or model-training workload, evaluation/red-team harness, model-serving service, and classical ML model in a product surface. Per-archetype threat content covers EA (Excessive Agency) patterns, tool scopes broader than the narrowest function that satisfies the use case, agent permissions wider than any individual human operator's, side-effects reaching systems outside the stated scope; AGH (Agent Goal Hijack) patterns, indirect prompt injection via retrieved document, tool response payload, multi-turn history, or shared memory store, plus system-prompt exfiltration and persona override; TM (Tool Misuse) patterns, argument smuggling, unexpected tool combinations, recursive invocation, crafted parameters exploiting missing input validation; RA (Rogue Agents) patterns, long-session goal drift, reflective-loop divergence, multi-agent miscoordination, autonomous continuation past a natural stop. Beyond HAI TTPs, each archetype model documents the ATLAS full tactic walk (TA0001 Reconnaissance through TA0043 Impact) with techniques selected or excluded with rationale, and cross-references OWASP LLM Top 10 items (LLM01 Prompt Injection, LLM03 Training Data Poisoning, LLM06 Excessive Agency, LLM08 Vector and Embedding Weaknesses, and relevant Agentic Top 10 entries) and compliance linkage to the PC-Software priority compliance map. Owner: named TA-Software library steward; cadence: reviewed quarterly; versioned in a single location linked from every SM inventory record.

B) Produce a per-intake threat snapshot for every SM inventory registration. Bind TA into the SM intake flow, every new artifact registration emits a threat snapshot before Sanctioned status is issued; Provisional-status artifacts receive a snapshot within five business days. Snapshot contents (designed to fit one screen): which archetype(s) apply, an artifact may be composite; artifact-specific deltas over the archetype model covering specific tool list and scope, specific retrieval sources and trust level, specific fine-tuning data classes and consent basis, and output-integrity-critical decision paths; top-five threats for this artifact each with HAI TTP tag, ATLAS tactic ID, OWASP reference, and compliance linkage; controls already evident from the design vs. gaps for SR/SA follow-up; reviewer name, date, and expiry. Time target: one business day per intake with the library available. Most threat content comes pre-written in the archetype model; the reviewer adapts rather than invents.

C) Author the shadow-AI-in-engineering threat view. Shadow AI in engineering, LLM calls shipped without intake, agents wired into products behind feature flags, fine-tunings run from researcher notebooks, has its own threat surface distinct from sanctioned artifacts. The shadow-AI-in-engineering threat document covers entry vectors (unannounced LLM API calls in feature branches, direct SDK imports without security review, researcher-run fine-tuning from Jupyter notebooks, agents wired to production APIs behind feature flags, developer-built eval harnesses consuming production data); elevated threats for shadow artifacts (no threat model, no SR requirements pack, no design review, no security testing, EU AI Act Art. 26 deployer duties unmet because the artifact is unknown to the program); and detections available at L1 from SM discovery sources (LLM SDK imports in feature branches, CI/CD telemetry for model-training jobs, runtime egress to AI provider domains from services without inventory entries, cloud-spend signals for untagged AI API usage). Output: a "Shadow AI in Engineering, Threat View" one-pager reviewed by the program sponsor and feeding the ML-Software detection backlog and IM-Software triage playbook.

Outcome Metrics (L1).

Success Criteria.

• Seven archetype threat models published, each tagged to HAI TTPs (EA/AGH/TM/RA), ATLAS tactic IDs (TA0001–TA0014), OWASP LLM/Agentic Top 10 references, and the PC-Software priority compliance map.
• Threat snapshot gate live in the SM intake flow, 100% of newly Sanctioned AI/HAI software artifacts in the last 90 days have a snapshot attached.
• Shadow-AI-in-engineering threat view published, reviewed by the program sponsor, and feeding the ML-Software detection backlog.
• Named library steward and quarterly refresh cadence operating.
• ≥90% of active AI/HAI software artifacts in the inventory carry a current-year snapshot.

Maturity Level 2

Objective: Layer per-artifact deep threat models on top of archetype snapshots for Critical-tier artifacts, integrate external AI-security threat intelligence, and red-team the threat library quarterly against real integrations.

Activities.

A) Per-artifact deep threat modeling for Critical-tier artifacts. For every Critical-tier artifact in the SM inventory, produce a full per-artifact threat model that goes beyond the archetype snapshot. Coverage: artifact-specific attack trees including per-tool abuse paths, retrieval-source provenance chain and injection surface, and fine-tuning data-class memorization risk quantified by data class and corpus size; an abuse-case catalog with named adversary archetypes (external attacker, malicious insider, compromised subprocessor, compromised vendor model family) and concrete attack narratives for this specific artifact; deployer-duty mapping covering EU AI Act Art. 26 obligations and the threat-control chain specific to this artifact; and a full ATLAS tactic walk for the artifact with technique-level specificity across all 14 tactics. High-tier artifacts receive archetype snapshot plus artifact-specific deltas and an ATLAS full tactic walk; no High-tier artifact remains on archetype-only. Refresh cadence: Critical semi-annual plus change-driven on model-family swap, new tool, new retrieval source, or scope change; High annual plus change-driven.

B) External AI-security threat intelligence integration. Subscribe to and operationalize MITRE ATLAS updates, AVID new entries for techniques relevant to the org's software archetypes, OWASP LLM Top 10 and Agentic Top 10 revisions, academic adversarial-ML venues (IEEE S&P, USENIX Security, NeurIPS ML Safety), and sector ISAC AI working groups. Quarterly triage cadence determines which new items change the archetype library, change per-artifact models, or require updates to dependent SR or ST artifacts. Changes are change-logged and reviewed by the library steward and the IM backlog owner. Intel-to-library update lead time targets 30 days on Critical-impact items.

C) Red-team the threat library itself. Each quarter, ST-Software runs an adversarial probe against an in-scope AI/HAI software artifact using only the threat scenarios documented in the library for that archetype. Threats the exercise identifies that are not in the library are library gaps, not passing findings. Gap closure is a governance activity: every gap becomes a ticket with a named owner and an expiry date; Critical-tier gaps close within 30 days, High-tier within 60 days. The gap rate per quarter trends down as the library matures. Gaps are also reviewed for SR and ST update implications, a threat absent from the library is also likely absent from a requirement and a test.

Outcome Metrics (L2).

Success Criteria.

• Per-artifact deep threat models live for 100% of Critical-tier and ≥90% of High-tier artifacts, with refresh cadences met.
• External threat intel integrated with quarterly triage and documented change-log; intel-to-library update ≤30 days on Critical-impact items.
• Quarterly red-team-the-library exercise operating; every gap carries a named owner and expiry date; Critical-tier gaps close within 30 days.

Maturity Level 3

Objective: Automate threat-library maintenance from telemetry and external feeds, and contribute discovered AI/HAI software TTPs back to MITRE ATLAS, OWASP, and AVID.

Activities.

A) Telemetry-driven library updates. Wire ML-Software detection alerts, IM-Software post-incident review records, external feeds (ATLAS technique additions, AVID new entries, OWASP LLM/Agentic Top 10 revision drafts, sector-ISAC AI advisories), and academic publication scanning into an auto-proposal pipeline. Human curators approve, reject, or defer each auto-proposal. The change-log is machine-readable; downstream SR and ST artifacts subscribe to the change feed and receive update-required notifications when a threat they reference changes. Target: ≥60% of library changes auto-proposed; lead time from signal to update ≤14 days.

B) Industry contribution. Contribute emerging first-party-observed TTPs, attack patterns discovered in own-built AI/HAI software, to MITRE ATLAS following ATLAS evidence-and-provenance requirements; to OWASP LLM Top 10/Agentic Top 10 revision cycles with real-world telemetry evidence; to AVID via structured disclosure submissions; and to NIST AI RMF Playbook successor editions with practitioner input. Target: at least four substantive contributions per year, quality-graded and legally vetted before submission, every contribution anonymized.

C) Shared threat-model artifacts. Publish anonymized archetype threat models (scrubbed of org-specific tool names and data classes) under a permissive license for peer-org adoption. Host or co-host at least one industry tabletop per year tied to the library, ATLAS practitioner table, OWASP AI chapter, or sector ISAC AI working group.

Outcome Metrics (L3).

Success Criteria.

• Library auto-update pipeline operating with ≤14-day lead time from signal to update; ≥60% of changes auto-proposed.
• ≥4 industry contributions per year; ≥2 recognized in external artifacts (ATLAS merge, AVID entry, OWASP revision).
• Anonymized archetype threat models published under permissive license with tracked peer-org adoption.
• Industry tabletop hosted or co-hosted in last 12 months.

Common Pitfalls

Level 1. - Threat models describe "the AI" as the actor performing the security work rather than the first-party AI software artifact as the subject being assessed, the library catalogs what AI tools do rather than what threats face the software the org ships. - Archetype library covers only production-facing archetypes; eval harnesses and fine-tuning workloads are excluded because they are not customer-facing, shadow AI in engineering gets no threat surface. - Threat snapshot is a checklist checkbox rather than a living artifact, snapshots are marked complete without artifact-specific deltas, making them useless for SR and ST follow-through. - ATLAS tactic walk is performed for narrative completeness but no technique IDs are assigned, the walk produces prose, not structured references that ST and IR can act on.

Level 2. - Per-artifact deep model is the archetype snapshot with the artifact name swapped in, no artifact-specific tool list, no retrieval-source provenance analysis, no per-data-class memorization risk; the depth is cosmetic. - External intel is subscribed but never triaged, ATLAS update notifications accumulate unread; the library is frozen at L1 publication while the threat landscape evolves. - Red-team-the-library exercise is a threat-hunting session that adds entries to a finding log but never cross-checks findings against the library, gaps are never surfaced because the comparison was never made. - Deep modeling stops at Critical tier; High-tier artifacts remain on archetype-only snapshots despite carrying regulated data or significant agent scope.

Level 3. - Auto-proposal pipeline accepts signals without curation, false-positive ML-Software detections pollute the library with phantom threats; downstream SR and ST artifacts generate incorrect requirements and tests. - Contributions to MITRE/AVID/OWASP are observer submissions, comments, conference talks, rather than technical artifacts with evidence that produce substantive change. - Published anonymized archetype models are not maintained after release, external adopters build on a stale version while the internal library has advanced; the divergence becomes visible when discrepancies are cited publicly. - Telemetry-driven update loop fires on every minor model-configuration change; engineering teams disable the telemetry feed to stop the noise rather than tune signal sensitivity.

Practice Maturity Questions

Level 1. 1. Are there published, versioned threat models for all seven AI/HAI software archetypes, LLM-integrated app, agent, RAG, fine-tuning/training, eval harness, model-serving service, classical ML, each mapping archetype threats to HAI TTPs (EA/AGH/TM/RA), ATLAS tactic IDs, OWASP LLM/Agentic Top 10 references, and PC-Software compliance items, with a named library steward and a documented quarterly refresh cadence? Evidence: TA library with seven versioned archetype documents and a named owner record. 2. Does every AI/HAI software artifact entering the SM inventory receive a threat snapshot (delivered within one business day of intake) that documents the applicable archetype(s), artifact-specific deltas, top-5 threats with HAI TTP tags and ATLAS tactic IDs, and gaps for SR/SA follow-up, with 100% of newly Sanctioned artifacts carrying a snapshot in the last 90 days? Evidence: SM intake tickets with snapshot attachments dated within intake SLA. 3. Is there a published shadow-AI-in-engineering threat view, reviewed by the program sponsor in the last 12 months, that documents entry vectors, elevated threat scenarios for unreviewed AI/HAI software artifacts, and the specific detections used to surface them? Evidence: Dated threat view document with program-sponsor review record and links to ML-Software and IM-Software backlogs.

Level 2. 1. Does every Critical-tier AI/HAI software artifact have a current-year per-artifact deep threat model covering artifact-specific attack trees, an abuse-case catalog, deployer-duty mapping, and a full ATLAS tactic walk with technique-level specificity, with a semi-annual refresh cadence and change-driven updates on model swap, new tools, or scope change? Evidence: Per-artifact threat model documents dated within cycle, with change-driven update records. 2. Is external AI-security threat intel integrated with a quarterly triage cadence and a documented change-log, with intel-to-library update ≤30 days on Critical-impact items? Evidence: Quarterly triage meeting records and change-log entries with signal-to-update timestamps. 3. Do you run a quarterly red-team-the-library exercise that probes an in-scope AI/HAI software artifact using only library threats and surfaces misses as library gaps, with every gap carrying a named owner and expiry date, and Critical gaps closing within 30 days? Evidence: Quarterly exercise artifacts with gap register showing owner assignments and closure dates.

Level 3. 1. Does the threat library auto-update from telemetry and external feeds via a human-curated auto-proposal pipeline, with ≥60% of changes auto-proposed, a ≤14-day lead time from signal to update, and a machine-readable change-log consumed by downstream SR and ST practices? Evidence: Pipeline telemetry showing proposal rate and lead-time distribution; SR/ST subscription confirmation. 2. Does the program contribute at least four substantive, evidence-backed technical artifacts per year to MITRE ATLAS / AVID / OWASP LLM/Agentic Top 10, with at least two externally recognized in published advisory or standard revisions? Evidence: Contribution log with external recognition citations. 3. Are anonymized archetype threat models published under a permissive license with tracked peer-org adoption, and does the program host or co-host at least one industry tabletop per year tied to the library? Evidence: License artifact, adoption tracking data, tabletop event record.

18. Security Requirements (SR)

Practice Overview

Objective: Translate the threats from TA-Software and the policies from PC-Software into a reusable Requirements Pack for AI/HAI software the organization builds, a base set plus per-archetype deltas, so every artifact entering production carries a testable Requirements-Evidence Map rather than a blank slate.

Description: SR-Software authors a small, archetype-keyed AI/HAI Software Requirements Pack: one base requirement set that applies to every artifact, plus per-archetype deltas for LLM-integrated app, agent, RAG pipeline, fine-tuning/training workload, eval harness, model-serving service, and classical ML. Each requirement is stated as a testable condition, either a measurable SLA or a binary evidence condition, not a narrative aspiration. Every artifact reaching SM intake carries a Requirements-Evidence Map (REM) linking each applicable pack requirement to current evidence, accepted gaps (with a named owner and expiry date), and compensating controls. Downstream practices, SA, DR, IR, ST, inherit the REM rather than re-deriving requirements per artifact.

Context: Without a shared requirements pack, each design review, implementation review, and security test invents the acceptance bar from scratch. Two reviewers score the same agent differently. Contracts miss clauses that should be table stakes. The program cannot demonstrate EU AI Act Art. 26 deployer duties or GDPR Art. 22 automated-decisioning safeguards across its AI/HAI software portfolio because there is no shared traceability from regulation to requirement to evidence artifact. SR-Software closes that gap with the minimum viable pack, not a checklist of 60 items, but the 20-ish requirements that matter for every AI/HAI artifact the org ships, plus archetype-specific additions for agents, fine-tuning workloads, RAG pipelines, and model-serving services.

Maturity Level 1

Objective: Publish the AI/HAI Software Requirements Pack (base plus per-archetype deltas), wire it into the SM intake gate, and produce a Requirements-Evidence Map for every artifact entering production.

Activities.

A) Author the base AI/HAI Software Requirements Pack. The base pack applies to every AI/HAI software artifact the org builds, regardless of archetype. Keep it to 20 or fewer base requirements at L1. Each requirement has: an ID, a statement, a rationale (threat tag from TA-Software and compliance tag from PC-Software), an evidence source, a test method, and an acceptance criterion. Minimum base categories: identity and auth, SSO-backed human access, service-principal model for any agent or automated process, secrets vault for API keys to LLM providers with no keys in source code or CI/CD environment variables, per-tenant data isolation for multi-tenant artifacts; data boundary, explicit declaration of which data classes may reach the model at inference, which may be used in fine-tuning or training with the consent or lawful basis for regulated data, which may populate RAG retrieval indexes, and a no-train assertion for each LLM provider API call confirmed at the API level rather than in contract text alone; logging, prompt, completion, tool-call, admin-audit, and identity events captured for every production deployment with retention meeting the longest applicable regulation and export mechanism available on demand; permissions for the agent archetype, published tool allowlist, per-tool scope minimization, human-in-the-loop gate for destructive or customer-account-affecting tool calls, kill-switch design in place and tested; output integrity, for AI output that gates a decision with legal or significant effect on a person (GDPR Art. 22 trigger) or falls under EU AI Act Annex III high-risk system categories, a defined test corpus, drift-detection schedule, and human-validation cadence; disclosure, EU AI Act Art. 50 user-facing disclosure met and Art. 26 deployer-duty evidence trail in place; and failure modes, documented fallback behavior for vendor outage, model deprecation, and rate-limit events. Every base requirement is tagged to at least one TA-Software archetype threat and at least one item from the PC-Software priority compliance map.

B) Author per-archetype requirement deltas. Each archetype carries a short delta (three to eight additional requirements) reflecting the threat-specific obligations from TA-Software's archetype threat models. The LLM-integrated app delta covers input sanitization policy, output handling safety, and hallucination-consequence classification. The agent delta specifies minimum-viable tool scope per declared tool, human-in-the-loop gate definition with confidence threshold and timeout fallback, agent session termination condition that is not bypassable by injected instructions, and agent memory scope with retention obligations. The RAG pipeline delta requires retrieval-source allow-list with no dynamic source addition outside a change-review gate, retrieval-source provenance with origin and trust classification, injection-defense requiring retrieved context be treated as untrusted, and multi-tenant retrieval isolation verified at query time. The fine-tuning/training workload delta requires training-data provenance for every dataset with documented consent or lawful basis, a no-train assertion for training data sourced from vendor APIs, data minimization scoped to the stated model capability objective, and a privacy evaluation confirming regulated data is not trivially extractable from the fine-tuned model's completions. The eval harness delta covers evaluation-data handling as production data, findings disposition routing to IM-Software, and harness access restricted to named personnel. The model-serving service delta requires model version registry, model-swap notification triggering a re-review gate, and rate-limiting and abuse throttling at the serving layer. The classical ML delta requires a model card for every production model, a drift detection signal, and authentication on the prediction endpoint.

C) Wire the pack into the SM intake gate and produce a REM per artifact. Every artifact approved for production carries a REM. Each applicable pack requirement is marked Met, Met-with-compensating-control, Gap-accepted, or Not-applicable with justification. Each Met row cites specific evidence: code-review finding, architecture-diagram element, admin-console screenshot, test result reference, DPA clause citation, or live-system demonstration note. Each Gap-accepted row names a compensating control, a named owner, and a re-review date (maximum 90 days at L1). The REM is stored with the SM inventory record for the artifact. Material changes, model-family swap, new tool addition, new retrieval source, new data class, scope expansion, trigger REM re-review before the change ships to production.

Outcome Metrics (L1).

Success Criteria.

• Base pack plus seven archetype deltas published, tagged to TA-Software threats and the PC-Software priority compliance map.
• 100% of new AI/HAI software artifacts approved in the last 90 days have a REM on file.
• ≥90% of active AI/HAI software artifacts in the SM inventory carry a current-year REM.
• Named pack owner and quarterly refresh cadence operating.
• Accepted-gap backlog tracked with every gap carrying a named owner and re-review date; median age inside ≤90 days.

Maturity Level 2

Objective: Replace qualitative requirements with quantitative, SLA-bound, and binary-evidence conditions; calibrate the requirements pack per risk tier; and validate REM evidence continuously for Critical and High-tier artifacts.

Activities.

A) Quantitative and binary requirement pack. For every requirement in the base pack and each archetype delta, replace qualitative language with measurable or binary conditions. Logging retention: specify exact retention periods by data class and regulation (e.g., prompt/completion logs retained ≥12 months; admin-audit logs retained ≥24 months). No-train assertion: binary, vendor API contract includes explicit no-train clause and admin-console setting is confirmed OFF with dated screenshot on file. Human-in-the-loop gate: binary, a synchronous approval gate is implemented in the agent loop for each tool in a destructive, external-network, or customer-account-affecting category, verified in code review with commit reference and in ST canary test with test ID. Kill-switch: binary, emergency-halt mechanism exists, is tested quarterly, and is invoked within five minutes from decision to full agent stop, with last test date and result on file. Output integrity: drift detection runs daily on the production evaluation corpus; a performance degradation of ≥5% from baseline triggers a human review gate. Secrets vault: binary, no LLM provider API key appears in source code, CI/CD environment variables, container image layers, or plaintext configuration files, confirmed by the most recent secrets-scanning run with zero findings. Training-data provenance: every dataset used in fine-tuning is listed in the artifact's REM with source, data-class classification, consent basis, and a membership-inference test result at or below the defined threshold.

B) Per-tier requirement depth. Publish a per-tier pack overlay aligned to the SM L2 tier-treatment matrix. Critical tier: full base pack and all applicable archetype deltas; executive sign-off on the completed REM before Sanctioned status is issued; full REM with no rows left blank; accepted-gap aging SLA of 60 days maximum before mandatory escalation to the program sponsor; EU AI Act Art. 26 full deployer-duty checklist as a discrete appendix to the REM; re-validation of all Critical-tier REM evidence quarterly. High tier: full base pack and applicable archetype deltas; accepted-gap aging SLA of 90 days; re-validation of REM evidence semi-annually. Medium tier: base pack and applicable archetype deltas; accepted-gap aging SLA of 120 days; re-validation annually. Low tier: base pack only; fast-track process with abbreviated evidence citations acceptable; re-validation at annual review.

C) Continuous REM-evidence validation. Critical-tier REMs re-validated quarterly; High-tier semi-annually. Validation method: select a stratified sample of REM rows per artifact, at least one row per base category, and verify each cited evidence artifact against current observable reality: logging retention and exportability checked against actual log settings; no-train assertion re-confirmed against admin-console state and contract currency; kill-switch re-run and result verified against SLA; secrets scan re-run with zero findings confirmed; tool allowlist re-verified against deployed agent configuration. Validation deltas (a row claimed Met but evidence fails re-validation) are routed to IM-Software as findings with severity tags and remediation SLAs matching the artifact's tier. Accepted-gap aging reviewed monthly; gaps approaching the escalation threshold notify the named owner before the deadline.

Outcome Metrics (L2).

Success Criteria.

• 100% of pack requirements carry a quantitative or binary evidence condition; all qualitative language removed.
• ≥95% of Critical-tier REMs re-validated against observed reality in the last 90 days; validation deltas routed to IM-Software.
• No Critical-tier accepted gap open beyond 60 days without documented escalation to the program sponsor.
• 100% of Critical-tier artifacts carry full EU AI Act Art. 26 deployer-duty checklist evidence in their REM.
• Per-tier pack overlay published and enforced; SM intake routing verified.

Maturity Level 3

Objective: Express the AI/HAI Software Requirements Pack as a machine-readable artifact, automate REM-evidence validation from CI/CD attestation and runtime signals, and contribute to industry-standard AI software security requirements bodies.

Activities.

A) Machine-readable pack and CI/CD attestation. Express the Requirements Pack (base plus archetype deltas) in a structured schema, JSON or YAML, where each requirement has an ID, a machine-readable evidence type (log-query, config-check, test-result-reference, or manual-attestation), an acceptance predicate, and a tier applicability field. At CI/CD deploy time for Critical and High-tier artifacts: automated checks run against the artifact's REM (logging confirmed, no-train setting confirmed via admin-console API, secrets scanner confirms zero findings, tool allowlist confirmed against deployed agent configuration, kill-switch test result within defined age, membership-inference test result within defined age for fine-tuning workloads); checks that pass write a signed attestation to the REM record; checks that fail block the deploy for Critical-tier artifacts and emit a warning with auto-routing to IM-Software for High-tier.

B) Automated REM-evidence validation from runtime signals. Subscribe the REM validation pipeline to ML-Software monitoring (logging completeness signal, drift detection signal for decision-affecting models), IM-Software incident records (post-incident reviews touching a pack requirement auto-flag relevant REM rows for re-validation), and SM inventory change events (a tier upgrade auto-triggers a full REM re-validation run under the new tier's requirements depth). Human review is reserved for novel requirement types not yet in the structured schema, accepted-gap escalations, and artifact-specific clauses outside the standard archetype deltas.

C) Standards contribution. Contribute the machine-readable requirement schema to OpenSSF AI; submit practitioner input on requirement categories and evidence conditions to the OWASP SAMM AI extensions (Building function, Security Requirements stream); submit practitioner commentary grounded in REM experience to NIST AI RMF Playbook MEASURE and MANAGE function requirement language; and submit concrete, testable AI software security requirements as candidate clause language to ISO/IEC 27090 or equivalent AI security standards successor work. Target: at least two substantive contributions per year, legally vetted and anonymized.

Outcome Metrics (L3).

Success Criteria.

• Machine-readable pack schema published; ≥80% of Critical-tier REM requirements have CI/CD attestation at deploy time.
• ≥70% of REM evidence rows auto-validated; human review reserved for exceptions and novel clauses.
• Zero Critical-tier artifacts deploying to production with a failing REM check; CI/CD gate confirmed enforcing.
• Pack and REM schema published under permissive license with tracked external adoption.
• ≥2 substantive industry-standard contributions per year.

Common Pitfalls

Level 1. - The base pack is authored with 40+ requirements at L1, reviewers cannot complete a REM in three business days and begin skipping rows, producing REMs that are structurally complete but evidentially hollow. - Per-archetype deltas are written but never wired into the intake process, every artifact gets the base pack only; agent tool-scope requirements and RAG retrieval-source provenance requirements are missed on every intake for those archetypes. - Gap-accepted rows lack expiry dates and named owners, the backlog grows silently until an audit surfaces a Critical-tier gap that has been "accepted" for 18 months with no action. - Material-change trigger is not defined, model-family swaps, new tool additions, and RAG corpus expansions ship to production without triggering a REM re-review; the REM drifts from the actual artifact within weeks of production landing.

Level 2. - Quantitative conditions are set but never verified against actual system state, logging retention is specified as "12 months" in the pack but is never confirmed against actual log retention settings; the SLA exists on paper only. - REM re-validation is scheduled quarterly for Critical-tier but samples only what engineers self-report, admin-console state, CI/CD telemetry, IR findings, and ML monitoring outputs are never cross-referenced. - Critical-tier accepted-gap escalation exists in policy but no escalation has ever reached the program sponsor, the threshold is written but the social and tooling mechanism to invoke it is absent. - Per-tier differentiation is documented in the pack overlay but not enforced at intake, Low-tier artifacts receive the same review depth as Critical-tier because the intake routing logic was never built.

Level 3. - Machine-readable pack schema is published but the organization stops maintaining the public version, the external artifact becomes stale while the internal version evolves; external adopters build on outdated requirements that conflict with the current internal standard. - CI/CD attestation covers deploy-time config checks but not post-deploy drift, a kill-switch that passes at deploy time is disabled six weeks later with no detection. - Standards contributions are submitted to working groups with no active AI software security track, they appear in the contribution log but have no path to adoption and no measurable industry impact. - Automated REM validation reports pass/fail counts to the program dashboard but never feeds failures back to the pack, repeatedly failing checks stay in the pack, generating noise and eroding trust in the gate.

Practice Maturity Questions

Level 1. 1. Is there a published, versioned AI/HAI Software Requirements Pack containing a base set of 20 or fewer requirements plus seven per-archetype deltas, with every requirement tagged to at least one TA-Software archetype threat and one PC-Software priority-compliance item, and are reviewers selecting from the pack rather than drafting requirements per artifact at intake? Evidence: Pack document with ID-tagged requirements, quarterly refresh record, and named pack owner. 2. Do 100% of new AI/HAI software artifacts approved in the last 90 days have a completed REM on file, with every applicable requirement marked Met, Met-with-compensating-control, Gap-accepted, or Not-applicable, each Met row citing specific verifiable evidence, each Gap-accepted row naming a compensating control, owner, and re-review date, and material-change triggers defined? Evidence: SM intake tickets with attached REM artifacts; gap register with owner and expiry fields populated. 3. Is the pack on a quarterly refresh cadence with a named owner, and are SA, DR, IR, and ST practices citing REM rows rather than independently re-deriving requirements from scratch? Evidence: Quarterly refresh records; cross-references from DR, IR, and ST artifacts back to REM row IDs.

Level 2. 1. Do 100% of pack requirements carry a quantitative or binary evidence condition, with every SLA and every binary state specified, and has all qualitative "reasonable" and "appropriate" language been removed from the pack? Evidence: Pack document with no instances of qualitative acceptance language. 2. Are ≥95% of Critical-tier REMs re-validated against observed reality (admin-console, CI/CD, IR findings, ML monitoring) in the last 90 days, with validation deltas routed to IM-Software and no Critical-tier accepted gap aging beyond 60 days without documented escalation to the program sponsor? Evidence: Validation log with timestamps; gap register with escalation records. 3. Does 100% of Critical-tier artifacts carry a full EU AI Act Art. 26 deployer-duty checklist in their REM with verifiable evidence, and is the per-tier pack overlay enforced at SM intake, with Critical-tier artifacts receiving full depth and Low-tier artifacts receiving base pack only? Evidence: Critical-tier REM appendices; SM intake routing log showing tier-differentiated processing.

Level 3. 1. Is the AI/HAI Software Requirements Pack expressed in a machine-readable schema enforced via CI/CD attestation at deploy time, with ≥80% of Critical-tier requirements having automated checks, zero Critical-tier artifacts deploying with a failing REM check, and the schema published under a permissive license with tracked external adoption? Evidence: CI/CD pipeline attestation log; zero-failure production deploy record; external adoption tracking. 2. Are ≥70% of REM evidence rows auto-validated via CI/CD signals, runtime monitoring, and admin-console API ingestion, with automation error-rate monitored and human review reserved for exceptions and novel clauses? Evidence: Validation telemetry showing auto-vs-manual split; false-positive and false-negative rate tracking. 3. Does the program contribute at least two substantive artifacts per year to recognized standards bodies (OpenSSF AI, OWASP SAMM AI, NIST AI RMF Playbook, ISO AI security standards work), with contributions publicly documented and traceable to adoption? Evidence: Contribution log with public links to accepted or in-progress submissions.

19. Secure Architecture (SA)

Practice Overview

Objective: Publish the reference architectures for safely building each AI/HAI software archetype the organization ships, so teams have a vetted green path that already implements SR-Software requirements and contains the threats identified by TA-Software.

Description: SA-Software ships a catalog of reference patterns, one per AI/HAI software archetype, showing how to place the data boundary, enforce identity, route traffic, log activity, isolate tenants, and contain agentic behavior for software the org builds. Each pattern covers scope, data boundary, identity and auth, traffic path, logging, controls mapped to SR requirements, and threats mitigated, tagged to HAI TTPs (EA/AGH/TM/RA) and MITRE ATLAS mitigation IDs. The catalog is accompanied by an anti-pattern list derived from real incidents and from first-party post-incident reviews. Teams use the reference pattern as the starting point; deviations require design review. At L2, patterns are encoded as IaC and cover multi-region, multi-tenant, and agent-platform complexity calibrated to SM L2's tier-treatment matrix. At L3, patterns are published as open artifacts adopted by the industry and MITRE ATLAS mitigation library entries are proposed from pattern controls.

Context: Without reference patterns, every engineering team inventing an LLM-integrated feature, a tool-using agent, or a RAG pipeline makes the same architectural missteps, LLM provider API keys embedded in client code, agents granted tool scopes broader than any human permission set, RAG pipelines over unclassified corpora, fine-tunings on regulated data without privacy sign-off, customer-facing inference with no output filter. The downstream cost is design reviews that repeat the same finding set and incidents that replay avoidable anti-patterns. SA-Software makes the secure path the default path, not by blocking engineering, but by publishing a pre-vetted architecture for each archetype so teams reach for the pattern first.

Maturity Level 1

Objective: Publish reference architectures per AI/HAI software archetype and an anti-pattern catalog derived from real incidents; link each pattern to SR-Software requirements and TA-Software threats.

Activities.

A) Publish reference architectures per AI/HAI software archetype. Publish one pattern per archetype the org actually ships. Each pattern is concise (target three pages), includes a labeled data-flow diagram, and covers a consistent skeleton. Scope: what the pattern covers and explicitly does not. Data boundary: where org data meets LLM/model calls, what crosses, what is blocked, DLP inspection points, per-tenant context isolation. Identity and auth: secrets-vault-backed credentials for LLM provider APIs, no hardcoded keys, service-principal model for internal service-to-service calls, per-user vs. per-service token lifecycle. Traffic path: egress through monitored network, allowlist of LLM provider domains, API-gateway placement, rate-limit and abuse-detection layer. Logging: prompt, completion, tool-call, and admin-audit events with retention and exportability meeting the longest applicable regulation. Controls mapped to SR requirements: explicit row-by-row mapping with gaps acknowledged. Threats mitigated: which TA-Software archetype threats the pattern addresses, which remain residual, HAI TTP tags (EA/AGH/TM/RA), and applicable MITRE ATLAS mitigation IDs. All seven archetype reference patterns ship at L1. The LLM-integrated app pattern places an API gateway in front of all LLM provider calls, enforces prompt-template versioning in source control, applies an output filter before responses leave the service boundary, enforces per-customer context isolation, uses secrets-vault-backed API keys, and maintains an egress allowlist; threats mitigated include prompt injection (AGH, ATLAS TA0008 Defense Evasion mitigated by output filter), training-data leakage via inference, and cross-tenant data leakage. The AI agent pattern requires an explicit tool allowlist, per-tool argument schema validation, per-tool scope minimization, a human-in-the-loop gate for destructive and customer-affecting tool calls, a process-level kill-switch, session-bounded multi-turn memory, and full tool-call logging; threats mitigated include EA via tool-scope minimization and allowlist (ATLAS TA0007 Privilege Escalation), TM via argument schema validation, AGH via memory bounds and HITL gate, and RA via session bounds and kill-switch (ATLAS TA0006 Persistence mitigated by session bounds). The RAG pipeline pattern enforces retrieval-source provenance tracked per chunk, per-source classification labels at index time, injection-defense treating retrieved content as untrusted, and per-tenant retrieval isolation; threats mitigated include retrieval poisoning and AGH via retrieved-content injection (ATLAS TA0006 Persistence mitigated by source provenance). The fine-tuning/training workload pattern requires training-data provenance recording, a data-classification pre-flight check blocking regulated data without explicit legal sign-off, training-job isolation with no production network access, model-card auto-population from the job run, and eval-suite gating before model promotion. The eval harness pattern enforces isolated data flows, reproducible runs with pinned data and model versions, regression corpora for jailbreaks and prompt injections in source control, and eval-result attestation signed and stored in the model registry as a prerequisite for model promotion. The model-serving service pattern requires versioned deployment behind a stable API, canary deployment for new model versions, per-model-version logging, a tested rollback playbook, and model-version pinning. The classical ML pattern requires model provenance tracking, artifact storage in a registry with access control, authenticated inference endpoints, per-request logging, and a drift monitoring baseline.

B) Publish the anti-pattern catalog. Name, describe, and prohibit AI/HAI software architectural patterns that reliably produce incidents. Each entry includes: description, why it is dangerous, real-incident flavor (industry or first-party), and the reference pattern element that replaces it. The L1 set covers: prompt-injection-trusting agent (AGH, agent passes retrieved content or user input to the model without structural separation); over-broad tool scope (EA, agent declared with tool access far wider than any single task requires); system-prompt-leaking persona (AGH, system prompt contains credentials or customer PII the output filter does not prevent leaking); long-session agent without memory bounds (RA, agent accumulates multi-turn context across sessions with no budget or session boundary); fine-tune on user data without opt-out (training workload ingests production user data without legal basis or consent, triggering GDPR Art. 17 obligations); silent model-family swap with no eval gate (output-integrity regression enabled by absence of eval harness on model version change); RAG over unclassified corpus (AGH and data leakage via unclassified retrieval over cross-tenant boundary); LLM API key embedded in client code (credential exfiltration enabling abuse of the org's API quota); secrets in prompts (any output-extraction attack becomes a credential-extraction attack); and output-integrity-critical decisions with no human gate (no HITL for consequential decisions, violating GDPR Art. 22 safeguards and EU AI Act Art. 14).

C) Integrate patterns into the intake/inventory flow and establish the deviation-review path. SM inventory records link to the applicable reference pattern(s) at intake. Teams choosing an archetype see the reference pattern and declare "using pattern" or "deviating from pattern." Deviations require a lightweight design review (DR-Software L1) with a named architect reviewer and a documented rationale stored with the artifact's inventory record. Patterns are reviewed and change-logged quarterly; repeat deviations in the same direction signal the need to update the pattern rather than continue approving exceptions. New archetypes that do not fit an existing pattern trigger a pattern-authoring sprint within 30 days of the first intake.

Outcome Metrics (L1).

Success Criteria.

• Seven reference patterns published, one per archetype, each with a labeled data-flow diagram, scope declaration, data-boundary definition, identity and auth model, traffic path, logging spec, and row-by-row mapping to SR-Software requirements and TA-Software threats with HAI TTP tags and applicable MITRE ATLAS mitigation IDs.
• Anti-pattern catalog published with at least 10 entries, linked from the AI Acceptable Use Policy, the SM intake gate, and EG-Software training.
• Deviation-review path operational with a named architect-reviewer population and ≤5 business day SLA.
• ≥85% of active AI/HAI software artifacts in the SM inventory classified as "on pattern" or "deviation with review"; no silent deviations.
• 100% of LLM-integrated app and agent artifacts with LLM provider credentials flowing through a secrets vault, confirmed by CI secrets-scanning.

Maturity Level 2

Objective: Extend reference patterns to multi-region, multi-tenant, and agent-platform complexity calibrated to SM L2's tier-treatment matrix; encode patterns as IaC with conformance test suites; update the anti-pattern catalog from IM-Software incidents.

Activities.

A) Tier-conditional pattern extensions. Publish extended pattern variants calibrated to SM L2's tier-treatment matrix. The Critical-tier overlay adds per-tenant isolation enforced at the data boundary and retrieval store, EU/US/sector data-residency variants with residency enforcement and cross-region data-flow legal basis under GDPR Art. 44–49, kill-switch IaC with infrastructure-level process termination tested quarterly, EU AI Act Art. 9 and Art. 15 controls explicitly mapped in the pattern, and a technical-documentation artifact template auto-populated from the IaC module to support Art. 11 documentation duties. The High-tier overlay includes monitoring and logging IaC modules pre-wired with prompt/completion, tool-call, and admin-audit log pipelines plus alert routing to the SIEM. The multi-region pattern covers region pinning at the API gateway, cross-region failover with residency preservation, and GDPR international-transfer mechanism selection as a required decision gate in the IaC module. The multi-tenant SaaS pattern covers per-tenant namespace isolation at the prompt and context layer, per-tenant embedding-store partitioning, per-tenant API key scoping, and a tenant-isolation conformance test wired into CI. The agent-platform pattern for multi-agent systems covers an orchestrator-control-plane design with an explicit inter-agent trust model, tool-scope inheritance rules prohibiting a sub-agent from inheriting broader scope than its parent, agent-to-agent authentication, orchestrator kill-switch propagation to all child agents, and consolidated multi-agent session logging.

B) Patterns-as-IaC with conformance test suites. Encode all Critical and High-tier pattern variants as forkable IaC modules, Terraform, Pulumi, CloudFormation, or equivalent, so teams fork rather than handcraft; deviations surface at plan or apply time. Each IaC module ships with a conformance test suite: automated checks that the deployed artifact matches the pattern's controls (secrets in vault, egress allowlist applied, logging pipeline wired, HITL gate present for agent archetypes, per-tenant isolation enforced for multi-tenant patterns). IaC modules are version-pinned; module updates trigger a drift-detection pass against all deployed instances. A module change log is maintained; teams consuming a module are notified of updates requiring remediation.

C) Incident-informed anti-pattern catalog refresh. Every IM-Software incident is classified to an anti-pattern (existing or new); classification is recorded in the IM finding. The catalog is refreshed monthly from IM-Software findings; new anti-patterns are surfaced to teams at intake time rather than stored in a reference document. Quarterly review: if three or more artifacts have deviated from a pattern in the same direction, the pattern is queued for update rather than continued exception approval. Anti-patterns originating from Critical-tier incidents are escalated to the SM working group for a pattern-update sprint within 30 days.

Outcome Metrics (L2).

Success Criteria.

• Five tier-conditional extended patterns published (Critical overlay, High overlay, multi-region, multi-tenant, agent-platform), each encoded as a forkable IaC module with a conformance test suite.
• ≥80% of Critical and High-tier AI/HAI software artifacts running on IaC-encoded patterns with plan-time deviation flagging.
• Anti-pattern catalog updated from ≥3 real IM-Software incidents in the last 12 months; new entries surfaced at intake time.
• Conformance test coverage at 100% of IaC-encoded artifact deployments.
• 100% of Critical-tier artifacts with EU AI Act Art. 9 and Art. 15 controls explicitly mapped in the pattern documentation.

Maturity Level 3

Objective: Publish reference patterns as open industry artifacts; contribute pattern-derived mitigations to MITRE ATLAS; engage regulators and standards bodies on architecture norms for AI/HAI software.

Activities.

A) Publish reference patterns as open artifacts. Publish patterns under Apache 2.0 or equivalent via OWASP SAMM AI, OpenSSF AI, CSA AI Safety Initiative, or equivalent body; sector-specific variants through relevant sector bodies (FS-ISAC, H-ISAC, sector AI working groups). Maintain the public repository as the upstream source; internal use aligns with the external version; internal deviations are documented with rationale and fed back as upstream proposed changes rather than silent forks. Track pattern adoption telemetry: GitHub forks, citations in published work, documented adopters. New archetypes or overlays developed internally are proposed for inclusion in the external catalog within 90 days of internal publication.

B) Contribute to MITRE ATLAS mitigation library. For each control in the reference patterns that corresponds to a threat technique in the ATLAS taxonomy, propose or validate a mitigation entry in the ATLAS mitigation library (AML.M00xx). Priority contributions align to SA's primary ATLAS tactics: TA0006 Persistence (model-version pinning, prompt-template versioning, lineage tracking), TA0007 Privilege Escalation (tool-scope minimization, allowlist enforcement, per-tenant isolation), TA0008 Defense Evasion (output filter, injection defense, model-card transparency, conformance testing). Target at least two AML.M00xx entries proposed or validated per year, contributions traceable to specific SA-Software pattern controls. Participate in the ATLAS practitioner community to align SA-Software control vocabulary with ATLAS technique taxonomy.

C) Engage regulators and standards bodies on architecture norms. Participate actively in EU AI Act implementing-act consultations where architecture standards for high-risk AI systems (Annex III use cases) are under discussion; submit SA-Software patterns as evidence of state-of-the-art architectural practice under Art. 9. Contribute to ISO/IEC 42001 AIMS community guidance on architecture documentation for AI Management Systems. Engage NIST AI RMF Playbook successor editions with SA-Software pattern mappings to GOVERN, MAP, MEASURE, and MANAGE. Engage sector regulators, FINRA/SEC model risk, HHS/FDA AI-enabled device guidance, NYDFS Part 500, with sector-relevant pattern variants; seek inclusion in sector architecture guidance documents.

Outcome Metrics (L3).

Success Criteria.

• ≥5 reference patterns published as open artifacts under a recognized open license via at least one industry body (OWASP, OpenSSF AI, CSA, or equivalent).
• ≥2 patterns externally cited or forked by recognized industry or sector bodies.
• ≥2 MITRE ATLAS AML.M00xx mitigation entries proposed or validated, traceable to SA-Software pattern controls, aligned to TA0006, TA0007, and TA0008.
• Internal practice 100% aligned to the published external version; all deviations proposed as upstream contributions, none silently forked.
• At least one documented regulatory or standards-body reference to SA-Software patterns in implementing-act, guidance, or standards text.

Common Pitfalls

Level 1. - Patterns are written but not linked from the SM inventory record or the intake gate, teams skip them because they are hard to find, not because they disagree with them. - The agent reference pattern omits the human-in-the-loop gate for destructive and customer-affecting tool calls, the most consequential control is the most commonly excluded. - Anti-patterns are theoretical and not tied to real incidents or to the specific pattern element that replaces them, developers do not recognize the hazard when they encounter it. - Deviations are approved individually but the repeat-deviation signal is never wired, patterns never update because no one aggregates the pattern-update trigger.

Level 2. - IaC patterns are forked once and then hand-edited at each deployment, drift is immediate and the IaC substrate provides no baseline enforcement; conformance tests are skipped because they block the fastest path to production. - Tier-conditional patterns exist in documents but the IaC modules do not enforce the tier-specific controls, the Critical overlay exists on paper; deployed Critical-tier artifacts lack per-tenant isolation or kill-switch IaC. - Multi-region pattern covers residency in the diagram but does not include the GDPR international-transfer mechanism selection step, teams deploy cross-region data flows without a legal basis. - Agent-platform pattern for multi-agent systems includes orchestrator kill-switch propagation in the diagram but the IaC module does not wire it, child agents run after the orchestrator is stopped.

Level 3. - Externally contributed patterns diverge from internal practice, what is published reflects what the org once did; external adopters discover the discrepancy during implementation and trust erodes. - ATLAS contribution targets are treated as a compliance checkbox, entries are proposed but never followed through to publication because internal legal or security review creates indefinite delay. - Regulatory engagement is declaratory ("we participated in the consultation") rather than substantive ("our pattern text was incorporated into the guidance"), the program cannot demonstrate that engagement produced outcomes. - Industry contributions are conference presentations and blog posts; no technical artifacts actually land in MITRE, OWASP, NIST, or OpenSSF, external recognition is aspirational.

Practice Maturity Questions

Level 1. 1. Are seven reference patterns published, one per archetype (LLM-integrated app, agent, RAG, fine-tuning/training, eval harness, model-serving service, classical ML), each with a labeled data-flow diagram, data-boundary definition, identity and auth model, logging spec, and explicit row-by-row mapping to SR-Software requirements and TA-Software threats with HAI TTP tags and applicable MITRE ATLAS mitigation IDs, accessible within one click of the SM inventory record? Evidence: Pattern catalog with seven versioned documents; SM inventory record containing direct links. 2. Are 100% of LLM-integrated app and agent artifacts verified via CI secrets-scanning (not only policy declaration) to route LLM provider credentials through a secrets vault, and is the anti-pattern catalog linked from the AI Acceptable Use Policy, the SM intake gate, and EG-Software training, with each entry tied to a real incident? Evidence: CI secrets-scan results with zero findings; anti-pattern catalog linked from AUP, intake gate, and EG training curriculum. 3. Is a repeat-deviation signal operational, such that three deviations in the same direction for the same archetype automatically queue a pattern-update review with SA ownership, and are ≥85% of active AI/HAI software artifacts in the SM inventory classified as "on pattern" or "deviation with review" with no silent deviations? Evidence: Pattern metadata showing on-pattern or deviation-with-review status; deviation aggregation report from the last quarter.

Level 2. 1. Are the five tier-conditional extended patterns (Critical overlay, High overlay, multi-region, multi-tenant, agent-platform) published as forkable IaC modules with conformance test suites, and are ≥80% of Critical and High-tier AI/HAI software artifacts running on IaC-encoded patterns as confirmed by the IaC and SM inventory registries? Evidence: IaC module repository with five variant directories; conformance test run history; SM inventory showing tier-to-pattern alignment. 2. Has the anti-pattern catalog been updated from ≥3 real IM-Software incidents in the last 12 months, with new entries surfaced at intake time rather than stored only in a reference document, and does conformance testing cover 100% of IaC-encoded artifact deployments with findings tracked to resolution? Evidence: Anti-pattern change log with IM incident references; intake gate showing current anti-pattern catalog version; CI/CD conformance test coverage report. 3. Are 100% of Critical-tier artifacts carrying explicit EU AI Act Art. 9 and Art. 15 control mappings in the pattern documentation, and is the tier-treatment matrix from SM L2 reflected in the pattern variants? Evidence: Critical-tier pattern documents with Art. 9/Art. 15 mapping sections; SM intake routing log showing tier-differentiated pattern assignment.

Level 3. 1. Have ≥5 reference patterns been published as open artifacts under a recognized open license via at least one industry body, and have ≥2 been cited or forked by recognized industry or sector bodies, with documented adoption evidence and internal practice aligned to the published version? Evidence: External repository with license file; citation or fork count; internal-vs-external pattern diff audit with no unexplained deviations. 2. Have ≥2 MITRE ATLAS AML.M00xx mitigation entries been proposed or validated, traceable to specific SA-Software pattern controls aligned to ATLAS primary tactics TA0006, TA0007, and TA0008, and is there an active ATLAS practitioner engagement cadence? Evidence: ATLAS contribution log with PR or submission references; meeting records from ATLAS practitioner community. 3. Is there at least one documented reference to SA-Software patterns in a regulatory implementing-act, sector guidance document, or published standards text, and is the regulatory engagement calendar maintained with active items, target timelines, and evidence of substantive participation? Evidence: Regulatory engagement log with document references and citation extracts; engagement calendar with active items.

20. Design Review (DR)

Practice Overview

Objective: Operate the design checkpoint between intake approval and build-out for every non-trivial AI/HAI software artifact, confirming the proposed design follows the applicable SA reference pattern, covers the SR requirements pack, and documents residual risks before engineering begins.

Description: DR-Software is the single moment where architecture (SA-Software), requirements (SR-Software), and threats (TA-Software) meet a specific planned build. The review runs before the engineering team begins build-out, catching deviations when they cost hours to correct, not sprints. A two-lane model routes Low / Medium-tier artifacts to an async fast-lane (target ≤2 business days) and High / Critical-tier or deviation cases to a full-lane architect review (target ≤5 business days). Every review produces a written decision (approve / approve-with-conditions / send-back) stored against the SM inventory record. Loop-back signals ensure the review process improves SA patterns and SR packs over time rather than accumulating silent technical debt.

Context: Without a design checkpoint, AI/HAI software builds proceed without a verified data boundary, without a confirmed tool allowlist, and without an output-filter placement decision. SA reference patterns and SR requirements packs exist, but teams skip them under sprint pressure, deviate without recording rationale, or fail to find the applicable pattern before build begins. DR-Software enforces the handoff between "design approved" and "build begins," making deviations visible and deliberate. EU AI Act Art. 9 risk management requires documented pre-deployment decisions for high-risk AI systems; the DR decision record is that documentation.

Maturity Level 1

Objective: Run a per-archetype design checkpoint for every AI/HAI software artifact before build-out, producing a written decision traceable to SA pattern, SR requirements, and TA threat snapshot.

Activities.

A) Publish the per-archetype AI/HAI Software Design Checklist. One checklist per SM-Software archetype, derived from the applicable SA-Software reference pattern and keyed to the SR-Software base pack and archetype delta. The seven checklists share a common spine, pattern adherence (using the SA reference pattern or documented deviation with rationale), data boundary (which data classes reach the model or retrieval index, DLP inspection points, output-filter placement), identity (SSO-backed human access, vault-backed API keys, no hardcoded credentials), logging (prompt, completion, tool-call, admin-audit, and identity events per the SR base pack; retention meeting the longest applicable regulation), failure modes (fallback or degraded-mode behavior for LLM provider outage, model deprecation, and rate-limit events), disclosure (EU AI Act Art. 50 user-facing disclosure for customer-visible AI interaction; Art. 26 deployer-duty evidence trail), and residual risk (explicit list with named owner, rationale, and expiry date), plus archetype-specific additions. The AI agent checklist adds: tool allowlist (every tool explicitly declared), per-tool scope minimization (parameter range restricted to minimum required), human-in-the-loop gate specification (synchronous approval for destructive actions and external-network calls, with timeout and fallback), kill-switch design (emergency-halt mechanism, test plan defined), session memory bounds (multi-turn memory bounded by session and token budget), and tool-call logging (full args and return value captured). For the agent archetype, reviewers verify the four HAI TTPs: EA (tool allowlist and scope minimization present), AGH (session memory bounds and HITL gate present), TM (argument schema validation per tool), and RA (session termination condition defined). The RAG checklist adds retrieval-source allow-list, provenance requirements, injection-defense structure, and per-tenant retrieval isolation. The fine-tune / training checklist adds training-data provenance, no-train assertion (vendor admin-console setting confirmed, not DPA text alone), training-job isolation from production network, data minimization, and eval gating. The model-serving checklist adds model-version pinning, canary deployment plan, rollback playbook, and rate-limit and abuse-detection coverage.

B) Triage and route reviews by risk tier and deviation status. The two-lane model is driven by the SM tier assignment and the deviation flag. Fast-lane (Low / Medium tier, on-pattern): async checklist review, target SLA ≤2 business days; output is one structured decision record stored against the SM inventory record. Full-lane (High / Critical tier, or any pattern deviation, agent archetype, or regulated-data involvement): 45–60 minute architect review with the engineering team walking the SA reference pattern section-by-section, target SLA ≤5 business days; output is a written decision record with the residual-risk list reviewed by a named architect. Before SM L2 tiers are established, agent archetypes and artifacts processing regulated data default to full-lane; all others default to fast-lane with override available on reviewer judgment. Every decision record, both lanes, carries: decision (approve / approve-with-conditions / send-back), checklist completed with evidence pointers, deviations listed with rationale, residual risks with named owner and expiry, reviewer name and date, and links to the SM inventory record, TA threat snapshot, and SR REM.

C) Close the loop with SA-Software, SR-Software, and IM-Software. Design review is a learning surface for the program. Three deviations in the same direction for the same archetype auto-queue a pattern-update review with SA-Software ownership, recurring deviations signal the pattern is miscalibrated, not that engineering teams are wrong. An SR requirement repeatedly waived with a compensating control auto-queues an SR pack-revision review. Every IM-Software incident re-examines the DR decision record that approved the affected artifact: was the issue visible at design time, and which checklist item would have caught it? The answer updates the checklist and feeds the next archetype review cycle.

Outcome Metrics (L1).

Success Criteria.

• Per-archetype design checklists published, versioned, and traceable to the applicable SA reference pattern, SR requirements pack, and TA threat snapshot.
• Two-lane review model operational with published SLAs (≤2 BD fast-lane, ≤5 BD full-lane) and named lead reviewers per archetype trained on EG-Software L1.
• ≥95% of AI/HAI software artifacts going to production in the last 90 days carry a completed DR decision record before build-out begins.
• SA pattern-update and SR pack-update triggers wired so recurring deviations and waived requirements feed back; every IM-Software incident re-examines the DR record that approved the affected artifact.

Maturity Level 2

Objective: Upgrade Critical-tier reviews to scenario-based walkthroughs driven by TA-Software per-artifact threat models, detect design drift for High and Critical artifacts on a published cadence, and coordinate joint DR-Software / DR-Vendors reviews for Critical-tier first-party artifacts integrating with vendor AI.

Activities.

A) Scenario-based reviews for Critical and High-tier artifacts. For every Critical-tier artifact, the full-lane checklist walkthrough is replaced by a scenario walkthrough. The reviewer sources 3–5 specific threat scenarios from the TA-Software per-artifact deep threat model and the TA-Software archetype library. Scenarios must be specific to this artifact's tool list, retrieval sources, data classes, and output-integrity-critical paths, not generic archetype scenarios. Each scenario is walked as: "If an adversary does X, does the proposed design have a control that prevents or detects it? Where? What is the residual risk?" The DR decision record maps each scenario to a design control or an accepted residual risk with a named owner and expiry. Scenario sources include the TA-Software per-artifact deep threat model, anonymized IM-Software incidents from the same archetype, MITRE ATLAS technique candidates (ATLAS TA0006 Persistence, TA0007 Privilege Escalation, and TA0008 Defense Evasion mitigations verified as present; TA0001 Reconnaissance and TA0003 Initial Access scenarios added where design decisions affect the initial-access surface), and OWASP LLM / Agentic Top 10 entries relevant to the archetype. For High-tier artifacts, the standard full-lane review is augmented with at least one scenario from the TA archetype library.

B) Cross-org joint reviews for Critical-tier vendor AI integrations. When a Critical-tier first-party artifact integrates with a vendor AI service, an agent calling an external LLM API, a RAG pipeline over a vendor-hosted embedding service, a fine-tune job using a vendor-hosted training API, DR-Software coordinates a joint review with DR-Vendors. The DR-Software reviewer and DR-Vendors reviewer attend the same session; the handoff boundary (which controls are the org's responsibility vs. the vendor's) is explicitly documented in both DR records. DR-Software covers the first-party artifact's design; DR-Vendors covers the vendor integration; residual risks spanning both are noted in both records with shared ownership and a single named resolution owner. Where the vendor integration is new and no DR-Vendors record exists, DR-Software flags the gap and holds the artifact's Sanctioned status until DR-Vendors completes.

C) Design-drift detection. The live production artifact is compared against its approved DR design at a published cadence. Critical-tier: quarterly drift check, examining code-repository changes since the last DR that affect SA-pattern controls, model-registry model version or fine-tune lineage changes, deploy-event configuration changes, CI/CD job parameters, and IaC state (Terraform / Pulumi drift against the approved IaC module from SA-Software L2). High-tier: annual drift check using the same sources. Material drift, new tool added to an agent, new data class flowing into RAG or fine-tune, customer-exposure switched on, model family changed, new retrieval source added, new region, automatically re-opens the DR record and routes back through the appropriate lane. Each drift check produces a written artifact: the diff between approved design and live configuration, each delta classified as material or non-material, with material deltas tracked to DR re-review or accepted residual.

Outcome Metrics (L2).

Success Criteria.

• 100% of Critical-tier DR reviews conducted as scenario-based walkthroughs with the decision tied to how the design handles each scenario.
• Design-drift detection operating quarterly for Critical and annually for High; 100% of material drifts re-routed to DR.
• Joint DR-Software / DR-Vendors review records on file for 100% of Critical-tier first-party artifacts with vendor AI integrations.
• IR-stage design surprises measurably fewer than at L1 over consecutive quarters.

Maturity Level 3

Objective: Operate continuous design attestation via automated SA-pattern-compliance scans, automate drift-triggered DR exception tickets, and contribute review rubrics and scenario templates to OpenSSF AI, CSA, and OWASP SAMM AI.

Activities.

A) Continuous design attestation via automated SA-pattern-compliance scans. Critical-tier artifacts produce a daily attestation signal covering: code-repo scan (SA reference pattern controls present and unmodified in the deployed codebase), model-registry check (model version and fine-tune lineage within the bounds approved at DR), IaC drift check (deployed configuration within tolerance of the approved IaC module from SA-Software L2), and logging-completeness check (ML-Software signal that required prompt/completion/tool-call/admin-audit event types are flowing at expected volume). Deviations from the approved design automatically open a DR-exception ticket in IM-Software, triaged within 3 business days. Attestation artifacts are machine-readable and regulator-consumable, EU AI Act Art. 9 risk-management evidence and ISO/IEC 42001 AIMS operational records are produced without manual assembly. Human reviewers handle novel architectures that do not fit existing attestation rules, accepted exceptions with documented rationale, and escalations from the IM-Software backlog.

B) Contribute review rubrics and scenario templates to industry. Publish under Apache 2.0 or equivalent through OpenSSF AI, CSA AI Safety Initiative, or OWASP SAMM AI extensions: per-archetype AI/HAI software design review rubrics (tier-assignment criteria, checklist items with evidence pointers, scenario-selection guidance keyed to ATLAS tactics), scenario template libraries (scenario format, per-archetype examples, debrief rubric for calibration exercises), and a pattern-evolution framework (how external signals, ATLAS updates, ISAC advisories, IM incidents, feed DR checklist and scenario updates on a quarterly cadence). Internal rubrics and templates remain aligned to the published external versions; internal deviations are proposed as upstream changes, not silently forked. Adoption is tracked by citations, forks, and direct acknowledgment from peer organizations or standards bodies.

C) Pattern evolution driven by external and internal signals. A quarterly pattern-evolution review combines external signals (MITRE ATLAS technique additions and refinements for TA0006 Persistence, TA0007 Privilege Escalation, and TA0008 Defense Evasion; TA0001 Reconnaissance scenarios where design decisions affect the initial-access surface; sector ISAC AI-specific advisories; OWASP LLM / Agentic Top 10 revisions) with internal signals (IM-Software incident patterns by archetype, ML-Software telemetry anomalies, ST-Software red-team findings) to produce structured checklist and scenario library updates. Updates are change-logged with signal provenance; downstream DR records for in-flight reviews are notified of pattern changes affecting their archetype. Where a new ATLAS technique or IM incident reveals a checklist gap, the gap is propagated to SA-Software and SR-Software to maintain the full traceability chain from threat to requirement to design review.

Outcome Metrics (L3).

Success Criteria.

• Daily attestation operating for ≥90% of Critical-tier artifacts; DR-exception tickets opened on deviation and triaged within 3 business days.
• ≥2 externally contributed review artifacts per year, per-archetype rubrics, scenario templates, or pattern-evolution frameworks, with documented adoption.
• Review backlog for non-exception work inside ≤7 days; attestation has absorbed the routine review volume.
• Quarterly pattern-evolution cadence traceable to external (MITRE ATLAS TA0006/TA0007/TA0008, sector ISACs) and internal (IM-Software, ML-Software, ST-Software) signals with a versioned change log.

Common Pitfalls

Level 1. - Design review runs after the engineering team has already built the feature, the checkpoint loses leverage because rework cost is already sunk; the review becomes a retrospective, not a gate. - Checklists are identical across archetypes, the agent checklist does not include HITL gate, tool allowlist, or kill-switch items because it was copied from the LLM-integrated app checklist. - Fast-lane becomes the default for everything, agent archetypes and regulated-data artifacts slip through with a 15-minute async check rather than the full-lane architect session they require. - Approve-with-conditions is issued but conditions have no named owner and no expiry, conditions sit unresolved at go-live with no enforcement path.

Level 2. - "Scenario-based" review is the same checklist read aloud in a meeting, same items, different format; the scenario-to-design-control mapping is never actually performed. - Scenario library is not refreshed quarterly, scenarios pulled from a 12-month-old TA snapshot do not reflect the current per-artifact deep model or recent IM-Software incidents. - Design-drift detection runs on a schedule but findings dead-end in a spreadsheet, no DR-exception ticket is opened; the approved design remains fiction while the live artifact has diverged. - Joint DR-Software / DR-Vendors reviews never happen because the coordination channel with DR-Vendors was never established, Critical-tier agents calling vendor AI APIs have no handoff boundary documentation on file.

Level 3. - Attestation signals show green across all Critical artifacts but underlying checks cover only logging settings, tool allowlist state, model-version pinning, and HITL gate wiring are not checked; attestation is cosmetic. - Externally published rubrics diverge from internal practice, the published artifact reflects how the org reviewed artifacts 18 months ago; peer adopters find inconsistencies when comparing the rubric to actual DR records. - Exception queue overwhelms reviewers because attestation thresholds are too sensitive, every prompt-template commit opens a DR-exception ticket; reviewers suppress the signal source rather than tune the sensitivity threshold. - Industry contributions are conference talks and blog posts, no technical artifacts (rubrics, scenario templates, pattern-evolution frameworks) land in OWASP / OpenSSF / CSA with documented adoption.

Practice Maturity Questions

Level 1. 1. Is there a published, versioned per-archetype AI/HAI Software Design Checklist, one per SM-Software archetype (LLM-integrated app, agent, RAG, fine-tune/training, eval harness, model-serving service, classical ML), traceable to the applicable SA reference pattern, SR requirements pack, and TA threat snapshot, with the agent checklist covering tool allowlist, per-tool scope minimization, HITL gate specification, kill-switch design, and tool-call logging? Evidence: Checklist document with version history; traceability matrix linking each item to an SA pattern control and SR requirement; agent-specific checklist section signed off by the named lead reviewer. 2. Do ≥95% of AI/HAI software artifacts going to production in the last 90 days carry a completed DR decision record (approve / approve-with-conditions / send-back) before build-out begins, with a two-lane routing model (fast-lane ≤2 BD, full-lane ≤5 BD), named lead reviewers trained on EG-Software L1, and a residual-risk list with named owner and expiry in every record? Evidence: SM inventory query showing last-90-days production entries with DR decision record IDs linked; review SLA telemetry report; sample of 5 decision records showing residual-risk section populated. 3. Are recurring pattern deviations and repeatedly-waived SR requirements automatically queuing SA pattern-update and SR pack-update reviews, and does every IM-Software incident trigger a re-examination of the DR record that approved the affected artifact? Evidence: SA pattern-update queue entries with triggering deviation counts; SR pack-update tickets linked to waiver patterns; IM incident post-mortems with a DR-record re-examination section completed.

Level 2. 1. Are 100% of Critical-tier DR reviews conducted as scenario-based walkthroughs, with 3–5 specific threat scenarios sourced from TA-Software per-artifact deep models and anonymized IM-Software incidents, with the DR decision tied explicitly to how the proposed design handles each scenario rather than checklist conformance alone? Evidence: Critical-tier DR decision records from the last 90 days, each showing a scenario-to-design-control mapping table and a decision statement tied to scenario outcomes. 2. Is design-drift detection running quarterly for Critical-tier and annually for High-tier, using code-repository changes, model-registry events, deploy-event configs, CI/CD parameters, and IaC state, with 100% of material drifts automatically re-routed to DR for a new review? Evidence: Drift-detection run log with cadence dates; material-drift classification report showing re-routed artifacts; DR queue entries with drift-triggered source tag. 3. Are joint DR-Software / DR-Vendors review records on file for 100% of Critical-tier first-party artifacts integrating with vendor AI services, with an explicit handoff boundary and shared residual-risk ownership documented in both DR records? Evidence: Cross-reference report of Critical-tier artifacts with vendor AI integrations; matching DR-Software and DR-Vendors decision records; handoff-boundary section in each record.

Level 3. 1. Are ≥90% of Critical-tier AI/HAI software artifacts producing a daily automated SA-pattern-compliance attestation signal, checking code-repo control presence, model-registry bounds, IaC drift, and logging completeness, with deviations auto-opening DR-exception tickets triaged within 3 business days? Evidence: Attestation telemetry dashboard showing daily signal per Critical artifact; DR-exception ticket queue with open/triage timestamps; sample attestation artifact in machine-readable format. 2. Has the program contributed ≥2 substantive review artifacts per year (per-archetype rubrics, scenario templates, pattern-evolution frameworks) to OpenSSF AI, CSA AI Safety Initiative, or OWASP SAMM AI, with documented adoption and internal practice aligned to the published versions? Evidence: Contribution log with external publication links and adoption indicators; comparison document showing internal checklist aligned to the published version. 3. Is there a quarterly pattern-evolution review driven by external signals (MITRE ATLAS updates for TA0006/TA0007/TA0008 and TA0001, sector ISAC advisories) and internal signals (IM-Software incidents, ML-Software telemetry, ST-Software findings), with a versioned change log and notification to in-flight DR reviews affected by pattern changes? Evidence: Quarterly pattern-evolution review minutes with signal-source citations; versioned checklist change log; in-flight DR review notification records for the most recent pattern update.

21. Implementation Review (IR)

Practice Overview

Objective: Verify, at go-live and on a recurring cadence, that the actual code and configuration of AI/HAI software the organization builds matches the design approved at DR, and that it stays there as the artifact evolves.

Description: IR-Software is the configuration and code check for first-party AI/HAI software artifacts, the moment a reviewer opens the codebase, the deployed configuration, the model registry, and the CI/CD pipeline and confirms that what is running matches the DR decision record. At L1 the review runs at go-live, at least annually, and on material change (model swap, new tool added to an agent, new data class flowing into RAG or fine-tune, customer-exposure switched on). At L2, IR consumes code-repo webhooks, model-registry events, deploy-event telemetry, IaC scan tooling, and vendor admin APIs to detect configuration drift continuously for High and Critical-tier artifacts. Findings are severity-tagged and SLA-bound per the SM L2 tier-treatment matrix; they feed IM-Software for tracking and resolution. No-train assertions, retention settings, tool allowlists, and kill-switch functionality are probed recurrently, not trusted from design text alone.

Context: The gap between the approved design and the running system is the primary source of silent security exposure in AI/HAI software. An agent's tool allowlist is correct in the DR record but widened at deploy time by a CI/CD environment variable. A fine-tune workload's no-train assertion appears in the SR REM but the vendor admin-console setting was reset by a product update. A kill-switch is documented in the SA pattern but the implementation test was skipped during the go-live crunch. IR-Software closes these gaps by making the implementation check systematic, evidence-based, and recurring, not a one-time pre-launch checkbox or a scramble when an incident reveals a configuration regression.

Maturity Level 1

Objective: Run per-archetype implementation reviews at go-live, annually, and on material change, verifying code matches the SA pattern, config matches the DR decision, and the SR REM evidence is current.

Activities.

A) Publish the per-archetype implementation review checklist. One checklist per SM-Software archetype, focused on the configuration and code points where production reality most commonly drifts from the approved design. Each item is a yes/no with a required evidence artifact (screenshot, config export, code-review finding, test result, log sample). The common spine across all archetypes covers: code matches SA pattern (structural controls present and unmodified in the deployed codebase, output filter wired, secrets routed through vault, SSO enforcement present, egress allowlist applied); config matches DR decision (deployed environment variables, model-provider settings, feature flags, and IaC state match the DR decision record; deviations flagged); SR REM evidence is current (a stratified sample of REM rows verified against current observable reality, logging confirmed via log-export test, no-train setting confirmed in vendor admin console not from DPA text alone, secrets-scan result current with zero findings, kill-switch confirmed to function with a test record on file); logging actually producing the events the design promised (pull a sample of prompt/completion/tool-call/admin-audit events and confirm they are present with correct format and retention policy); kill-switches and circuit breakers actually work when triggered (execute the kill-switch test and record the result). The AI agent checklist adds verification that the tool allowlist is actually enforced in deployed code, not only declared in the DR checklist, and that per-tool argument schema validation is present; the HITL gate fires for declared trigger conditions (tested with an out-of-scope action); kill-switch stops all tool invocation within the specified SLA; session memory bounds are enforced; and tool-call logging captures full args and return values. ATLAS drift sources checked for agent artifacts: TA0006 Persistence (session memory bounds still enforced), TA0007 Privilege Escalation (tool allowlist and per-tool scope not widened since DR), TA0008 Defense Evasion (tool-call logging and kill-switch not disabled). The fine-tune / training checklist confirms no-train assertion via vendor admin-console API state, not DPA text alone, and verifies training-data provenance records match the SR REM for all datasets used in the last training run.

B) Perform reviews at the right moments. Three triggers at L1: go-live (before the artifact enters production, verify the as-built code and configuration against the DR-approved design; no production cutover with a blocker finding open); annual (every active AI/HAI software artifact reviewed at least annually, scheduled from the SM inventory with a last-IR-date field linked to a review-due alert); material-change (any of the following triggers an ad-hoc review before the change ships, model version or model family swap, new tool added to an agent's allowlist, new data class flowing into RAG corpus or fine-tune dataset, customer-exposure switched on for a previously internal feature, new retrieval source, new region or data-residency zone, agent scope or permission model changed). The material-change trigger is wired to the same signal sources as SM inventory material-change events. Reviews are evidence-based, screenshots, config exports, or code-review findings stored with the IR record. Target timebox: 30–90 minutes per artifact depending on archetype complexity.

C) Track findings to closure. Every review produces zero or more findings. Each finding carries a severity (Critical / High / Medium / Low, calibrated to the SM L2 tier-treatment matrix's IM SLA column; at L1 use a consistent judgment rubric pending SM L2 formalization), a named owner, an SLA (Critical blocker resolved before production cutover or rollback required; High ≤7 days; Medium ≤30 days; Low ≤90 days or accepted residual), and an evidence artifact linked before closure. Findings feed IM-Software as issues for tracking and aging, and loop back to SR-Software where a finding reveals that an REM row's cited evidence was inaccurate, the REM row is updated before the finding is closed.

Outcome Metrics (L1).

Success Criteria.

• Per-archetype IR checklists published, owned, and linked from the SM inventory record and the DR decision record.
• Go-live, annual, and material-change review triggers wired to the SM inventory; 100% of new AI/HAI software artifacts in the last 90 days have a go-live IR record.
• ≥90% of active AI/HAI software artifacts carry a current-year IR record.
• All Critical / blocker findings resolved before production cutover; High findings closed within 7 days with evidence linked.
• Findings-aging dashboard reviewed at least monthly by the program sponsor.

Maturity Level 2

Objective: Detect configuration drift continuously for Critical and High-tier artifacts via code-repo webhooks, model-registry events, IaC scan tooling, and vendor admin APIs; probe no-train and retention settings recurrently; calibrate IR cadence per SM L2 risk tier.

Activities.

A) Continuous drift detection from code-repo, model-registry, IaC, and deploy telemetry. Wire the following signal sources to an automated drift-detection pipeline for Critical and High-tier artifacts. Code-repo webhooks: commits to files touching SA-pattern-control logic (output filter, tool allowlist, session bounds, vault bindings, HITL gate, kill-switch, egress config, prompt template) trigger an automated diff against the DR-approved baseline; material deviations open an IR finding automatically. Model-registry events: model version promotions, fine-tune lineage changes, and model deprecation events trigger an IR re-review gate; a model swap without a corresponding DR material-change review is a Critical finding. IaC scan tooling: Terraform / Pulumi / CloudFormation plan-vs-apply drift scans run on each deploy for Critical and High artifacts; configuration deviations from the approved IaC module open IR findings. Deploy-event telemetry: environment variable changes, feature flag changes, and secrets-rotation events captured at deploy time; changes to SA-pattern-control variables since the last IR open findings automatically. CI/CD job parameter telemetry: build-job parameters compared against the DR-approved specification; deviations flagged. Detection latency targets: Critical-tier ≤7 days from change event to finding opened; High-tier ≤30 days.

B) Vendor admin API probing for no-train and retention settings. No-train and retention settings are probed recurrently via vendor admin APIs for Critical and High-tier artifacts, not trusted from DPA text or one-time admin-console screenshots. OpenAI: Org Settings API confirming data controls training data sharing is false for all applicable API keys and retention period matches the DPA. Anthropic: Organization admin settings API confirming model training usage terms reflect the no-train commitment. Amazon Bedrock: AWS Service Control Policy and Bedrock model invocation logging config confirming no model fine-tuning on customer data paths and CloudTrail logging active for all Bedrock API calls. Google Vertex AI / Gemini: Google Cloud Organization Policy constraints confirming data logging settings and no training-data usage opt-in active. Other vendors: equivalent admin API or authenticated endpoint where available; UI-based verification with screenshot evidence where APIs are not available. Probing cadence: Critical-tier monthly, High-tier quarterly. Delta from the previous probe opens an IR finding with severity matching the data-class impact. These probes explicitly verify EA (tool scope not widened), AGH (no new retrieval sources without DR approval), TM (argument validation settings unchanged), and RA (session-bound settings unchanged) controls for agent artifacts.

C) Tool-scope boundary testing for agent artifacts. For every Critical and High-tier agent artifact, tool-scope boundaries are tested at each IR cycle, not assumed from the tool allowlist declaration. Each declared tool is called with an out-of-scope argument (a file path outside the declared directory scope, a record ID outside the declared customer scope, an action flagged as destructive without a HITL gate); rejection is verified and the test input, expected behavior, and actual behavior are documented. Invocation of a tool not on the allowlist is attempted from within the agent session; the call is verified blocked at the allowlist-enforcement layer, not only rejected downstream. The HITL gate condition for a declared destructive action is triggered; the gate is verified to fire synchronously and the timeout-and-fallback behavior verified against the DR specification. The kill-switch is triggered; the agent process and all active tool invocations are verified stopped within the specified SLA; the test date and result are recorded. Tool-scope test evidence stored with the IR record; failures are Critical findings for Critical-tier agents and High findings for High-tier agents.

D) Tier-calibrated IR cadence. Publish and enforce per the SM L2 tier-treatment matrix: Critical (go-live + semi-annual + material-change-triggered + continuous drift detection), High (go-live + annual + material-change-triggered), Medium (go-live + annual), Low (go-live + re-review on material change). Every artifact in the SM inventory carries a last-IR-date and next-IR-due field; Critical-tier artifacts with no IR in the last 180 days are escalated to the program sponsor.

Outcome Metrics (L2).

Success Criteria.

• ≥90% of Critical-tier artifacts under continuous drift detection; median detection latency ≤7 days.
• No-train and retention settings verified via vendor admin APIs for ≥80% of Critical/High-tier artifacts on a monthly (Critical) and quarterly (High) probing cadence.
• 100% of Critical/High-tier agent artifacts with tool-scope boundary tests and kill-switch test on record in the current IR cycle.
• Tier-cadence adherence ≥95%; Critical-tier findings aged per the SM L2 tier-treatment matrix SLAs.

Maturity Level 3

Objective: Operate continuous configuration attestation for Critical-tier artifacts with a daily signal confirming pattern compliance and evidence freshness, auto-open IM tickets on drift, and contribute per-archetype configuration baseline schemas to OpenSSF AI, OWASP SAMM AI, and CSA AI Safety Initiative.

Activities.

A) Daily attestation signal for Critical-tier artifacts. Each Critical-tier AI/HAI software artifact produces a daily composite attestation signal covering three dimensions: pattern compliance (automated SA-pattern-compliance scan confirming key controls are present and active in the deployed codebase, output filter status, tool allowlist enforcement presence, vault binding status, HITL gate logic, egress allowlist configuration, session memory bound enforcement, using code-repo and IaC scan tooling from IR L2 on a daily schedule with machine-readable output); evidence freshness (the SR REM's evidence citations checked for staleness against defined freshness windows: secrets scan ≤7 days, kill-switch test ≤90 days, no-train API probe ≤30 days for Critical / ≤90 days for High, tool-scope boundary test ≤180 days; stale evidence opens a finding automatically); configuration within tolerance (deployed configuration checked against the DR-approved baseline per defined tolerances, model patch versions within the same major version tolerated; model family changes not tolerated without DR re-review). Attestation artifacts are machine-readable, signed, and stored in the SM inventory record; they are regulator-consumable for EU AI Act Art. 9 risk-management evidence and ISO/IEC 42001 AIMS operational records. Drift auto-opens an IM-Software ticket carrying the drift dimension, the specific control that failed tolerance, and a link to the DR decision record.

B) Contribute per-archetype configuration baseline schemas. Publish per-archetype IR configuration baseline schemas, defining what correct implementation looks like for each AI/HAI software archetype at each SM tier, to OpenSSF AI (reference attestation schema compatible with SLSA and in-toto supply-chain attestation frameworks), OWASP SAMM AI extensions (Verification function, Implementation Review stream, with practitioner-level checklist items and evidence-type definitions), and CSA AI Safety Initiative AI Controls Matrix (per-archetype configuration controls mapped to CSA control categories). Internal practice remains aligned to the published external versions; internal-only deviations are proposed as upstream changes. Adoption is tracked by citations, forks, and direct acknowledgment from peer organizations or inclusion in external tooling or assessment frameworks.

C) Automated drift-to-IM escalation and SLA enforcement. All IR findings, whether from daily attestation or periodic reviews, flow into IM-Software automatically with severity and SLA pre-populated from the SM L2 tier-treatment matrix. The IM-Software SLA clock starts when the finding is opened; overdue Critical findings escalate to the program sponsor automatically at 50% and 100% of the SLA window. Post-incident reviews in IM-Software that touch a configuration or code control automatically re-examine the IR record for the affected artifact, was the drift detectable earlier, and what attestation rule would have caught it? The answer updates the attestation rule and the IR checklist.

Outcome Metrics (L3).

Success Criteria.

• Daily attestation operating for ≥90% of Critical-tier artifacts across all three dimensions (pattern compliance, evidence freshness, configuration tolerance); deviations auto-opening IM tickets within 1 hour.
• Zero stale-evidence violations for Critical-tier REMs; High-tier stale-evidence rate trending down.
• Per-archetype configuration baseline schemas published to OpenSSF AI, OWASP SAMM AI, or CSA AI Safety Initiative with documented external adoption.
• IR reviewer-hours per Critical artifact per year trending down over two consecutive quarters.

Common Pitfalls

Level 1. - IR treated as a one-time go-live formality, no annual re-review and no material-change trigger; configuration drift accumulates silently for quarters until an audit or incident surfaces it. - Reviewers take the DR decision record at face value without opening the codebase, the tool allowlist is declared in the checklist but never verified in deployed code; the allowlist enforcement logic was never wired. - No-train and retention settings verified from DPA text alone without opening the vendor admin console, the setting can be reset by a vendor product update and the team does not know. - Kill-switch documented in the DR record but never tested, the IR checklist has a kill-switch checkbox that is checked without an actual test execution and a recorded result.

Level 2. - Drift-detection pipeline ingests code-repo events but generates no findings on deltas, the pipeline exists but automated finding creation was never configured; drift detection is manual in practice. - Vendor admin API probing is configured once at onboarding and never re-run, a no-train setting reset by a vendor product update is undetected for months. - Tool-scope boundary testing is documented as "verified at go-live" but never repeated, agent tool scope may have been widened at a subsequent deploy; the go-live test is the only record. - Drift findings from automated detection dead-end in an alert dashboard rather than auto-opening IM tickets, findings age without owners.

Level 3. - Daily attestation signals show green across all Critical artifacts but underlying checks cover only logging volume, tool allowlist enforcement, HITL gate wiring, and model-version tolerance are not checked; attestation is cosmetic. - Configuration baseline schemas published externally diverge from internal practice, what is published reflects the L1 checklist; internal practice has advanced to L2 tooling and L3 vendor API probing; external adopters build on a stale baseline. - Attestation-exception queue overwhelms the team because configuration tolerance thresholds are too tight, every dependency version bump triggers a deviation; reviewers suppress the signal source rather than tune the tolerance rules. - Post-incident IR feedback loop exists in policy but never fires in practice, IM post-incident reviews do not include the IR-record re-examination step; attestation rules never update from incident learning.

Practice Maturity Questions

Level 1. 1. Is there a published, per-archetype IR checklist, one per SM-Software archetype (LLM-integrated app, agent, RAG, fine-tune/training, eval harness, model-serving service, classical ML), covering code-matches-pattern verification, config-matches-DR verification, SR REM evidence currency check, logging-event production verification, and kill-switch test execution, with the agent checklist verifying tool allowlist enforcement in deployed code, per-tool scope, HITL gate function, and tool-call logging? Evidence: Published checklists with version history; agent-specific checklist section showing allowlist-enforcement verification step distinct from DR checklist conformance; sample IR record with screenshot evidence attached. 2. Do 100% of new AI/HAI software artifacts going to production in the last 90 days carry a go-live IR record, and do ≥90% of all active artifacts carry a current-year IR record, with material-change triggers wired to SM inventory events, Critical / blocker findings resolved before production, and High findings closed within 7 days with evidence linked? Evidence: SM inventory query for last-90-days production entries with IR record IDs; annual review calendar with last-IR dates; findings backlog report showing zero open blockers at go-live and High-finding closure times. 3. Are findings severity-tagged and tracked in IM-Software with named owners and SLA-bound closure dates, and does every IR finding that reveals stale or inaccurate REM evidence trigger an SR REM row update before the finding is closed? Evidence: IM-Software backlog export showing severity tags and SLA fields populated; REM update log showing IR-triggered row updates; findings-aging dashboard reviewed by the program sponsor within the last 30 days.

Level 2. 1. Are ≥90% of Critical-tier AI/HAI software artifacts under continuous drift detection, via code-repo webhooks, model-registry events, IaC scan tooling, deploy-event telemetry, and CI/CD parameter monitoring, with median detection latency ≤7 days and automated finding creation on material deviations? Evidence: Drift-detection telemetry report showing per-artifact signal coverage; detection-latency histogram for Critical-tier; sample auto-generated IR finding linked to a code-repo webhook event. 2. Are no-train and retention settings verified via vendor admin APIs on a monthly (Critical) and quarterly (High) probing cadence for ≥80% of Critical/High-tier artifacts, not from DPA text alone, with deltas from the previous probe opening IR findings with severity matching the data-class impact? Evidence: Vendor API probing log with cadence dates per artifact; delta-detection finding showing an admin-console change detected between probes; coverage report showing percentage of Critical/High artifacts covered. 3. Are 100% of Critical/High-tier agent artifacts covered by tool-scope boundary tests in the current IR cycle, confirming out-of-scope argument rejection, non-allowlisted tool-call blocking, HITL gate trigger, and kill-switch SLA, and is tier-cadence adherence ≥95% with Critical-tier findings aged per the SM L2 tier-treatment matrix SLAs? Evidence: Tool-scope boundary test records per agent artifact (input, expected behavior, actual behavior, date); kill-switch test records with halt-time measured; tier-cadence adherence report from the IR schedule.

Level 3. 1. Are ≥90% of Critical-tier AI/HAI software artifacts producing a daily attestation signal across all three dimensions (pattern compliance, evidence freshness, configuration tolerance), with deviations auto-opening IM-Software tickets within 1 hour and zero stale-evidence violations for Critical-tier REMs? Evidence: Attestation telemetry dashboard showing daily signal per Critical artifact for the last 30 days; IM-Software ticket creation log with timestamps within 1 hour of attestation findings; Critical-tier REM evidence freshness report with zero stale entries. 2. Has the program published per-archetype configuration baseline schemas to OpenSSF AI, OWASP SAMM AI, or CSA AI Safety Initiative, with documented adoption and internal practice aligned to the published versions, and is IR reviewer-hours per Critical artifact per year trending down over two consecutive quarters? Evidence: External publication links with adoption indicators (forks, citations, inclusion in external tooling); comparison document showing internal checklist version aligned to the published baseline schema; reviewer time tracking report showing QoQ decline. 3. Is the post-incident IR feedback loop operational, IM-Software post-incident reviews include a mandatory IR-record re-examination step, and ≥1 attestation rule update is produced per material incident, ensuring incident learning continuously improves attestation coverage? Evidence: Sample post-incident review showing IR-record re-examination section completed; attestation rule change log entries linked to incident review IDs; trend showing attestation rule count increasing with incident volume.

22. Security Testing (ST)

Practice Overview

Objective: Prove that every AI/HAI software artifact the organization builds behaves correctly under adversarial conditions, by running a foundational per-archetype test battery in CI, maintaining versioned regression corpora, and escalating to scheduled red-team and continuous adversarial testing at higher maturity levels.

Description: ST-Software exercises the AI/HAI software the organization ships, LLM-integrated applications, autonomous agents, RAG pipelines, fine-tuning and training workloads, evaluation harnesses, model-serving services, and classical ML models, against a battery of AI-specific test classes tied directly to the threats in the TA-Software library and the requirements in the SR-Software pack. At L1, every archetype has a published test battery (prompt-injection probes, tool-scope boundary tests, data-egress canaries, no-train verification, logging-completeness tests, kill-switch tests, output-integrity regressions, training-data leakage tests, model-extraction probes) plus four versioned regression corpora (jailbreak, prompt-injection, agent goal-hijack, tool-misuse) running on every PR via CI. L2 adds per-tier scheduled red-team exercises using TA L2 per-artifact deep threat models and cross-archetype composition tests. L3 operates continuous automated adversarial testing and contributes findings to MITRE ATLAS, AVID, and OWASP LLM / Agentic Top 10.

Context: Classic CI test suites exercise the happy path and leave the adversarial path untested. An LLM-integrated feature passes all unit tests and then leaks its system prompt on the first prompt-injection attempt. An agent passes its integration tests and then invokes tools outside its declared scope when fed a crafted retrieval response. A fine-tuned model ships to production and regurgitates a canary string from the training corpus. These failures are invisible to classic testing because classic testing was not designed to enumerate AI-specific failure modes, prompt injection (ATLAS TA0001/TA0003), ML model access probing (TA0004), attack staging (TA0012), and data exfiltration via inference (TA0013). ST-Software closes this gap by making AI-specific tests a first-class CI citizen and connecting them directly to the TA threat library so test coverage tracks threat coverage, not just code coverage.

Maturity Level 1

Objective: Establish a foundational per-archetype test battery and regression corpora that run in CI on every PR, and verify that every AI/HAI software artifact reaches production with a passed go-live battery on record.

Activities.

A) Publish the foundational per-archetype test battery. One test battery per AI/HAI software archetype targeting the top archetype threats from TA-Software and the archetype-specific SR requirements. L1 target: ≤8 named test classes per archetype. Each test class specifies inputs, expected output, pass/fail criteria, an evidence artifact (log snippet, trace ID, CI run link), and the TA threat plus SR requirement it maps to. The LLM-integrated application battery covers: prompt-injection probe corpus in regression CI (direct injection targeting system-prompt extraction, instruction-override, role-manipulation, and prefix-injection, ATLAS TA0001/TA0003; AGH); training-data leakage canary (unique canary string inserted into a fine-tune or RAG corpus, prefix-completion probes via the inference endpoint, asserted not recoverable, ATLAS TA0013); output-integrity regression (golden test set of 20–50 structured-output prompts with expected schemas, alerting on schema or content drift, gating model-version promotions, ATLAS TA0004); rate-limit and abuse-detection test (verify the rate-limit layer enforces the cap and degraded-mode fallback produces a logged, non-misleading response); and logging-completeness test (verify required log fields appear in the org-side log store within the retention SLA). The autonomous AI agent battery adds: tool-scope boundary test (call each declared tool with an out-of-scope argument and verify the tool layer rejects and logs the rejection, ATLAS TA0004; EA / TM); tool-argument-smuggling corpus in regression CI (path traversal fragments, SQL meta characters, JSON-injection strings, TM; OWASP LLM07); multi-turn agent goal-hijack probe corpus (alternate-goal instructions seeded into retrieval path or tool-response payload, ATLAS TA0001/TA0003; AGH); recursive-invocation guardrail test (verify recursion budget enforced before cost or blast-radius exceeds the declared SLA, TM / RA); kill-switch test (agent process halts within declared SLA; in-flight tool invocations not silently completed after kill signal, RA); HITL-gate test (destructive or external-network tool invocation requires explicit approval event in the log, EA / OWASP LLM08); and session-bound memory test (cross-session memory does not persist regulated data past the declared session boundary, RA). The RAG pipeline battery covers retrieval-source provenance, classification-label-respecting retrieval, injection-defense over retrieved content (seeded corpus with prompt-injection instructions verifying the artifact does not follow them, AGH; OWASP LLM01 indirect), and per-tenant retrieval isolation. The fine-tune / training battery covers training-data classification pre-flight, no-regulated-PII-without-sign-off, model-card auto-population, and eval-suite gating (a model cannot be promoted without a passing eval-suite result, ATLAS TA0004). The model-serving battery covers version-pinning, canary-deployment, and rollback tests. The classical ML battery covers drift-detection threshold and retraining-trigger tests.

B) Build and maintain regression corpora in CI. Four versioned regression corpora in source control, running on every PR via CI. Jailbreak corpus (30–100 inputs targeting role-override, persona-switch, authority-claim, and encoding-bypass patterns; run against LLM-integrated app and agent archetypes; failure blocks merge for Critical/High-tier). Prompt-injection corpus (30–100 direct and indirect inputs covering system-prompt extraction, instruction override, injected-document payloads, tool-response payloads, and multi-turn history injection; run against LLM-integrated, agent, and RAG archetypes). Agent goal-hijack corpus (20–60 multi-turn sequences designed to redirect a tool-using agent's goal via retrieval-path content, tool responses, or long-context accumulation; run against agent archetypes). Tool-misuse corpus (20–60 argument-smuggling and out-of-scope invocation payloads per tool type, file-path tools, HTTP tools, database-query tools, API-call tools; run against agent archetypes). Each corpus entry carries a threat tag (HAI TTP + ATLAS tactic ID), OWASP reference, source, and date added. Corpus refresh cadence: monthly minimum from internal observations (IR findings, IM incidents), external public corpora (OWASP LLM Top 10 examples, HackAPrompt dataset, ATLAS technique examples), and jailbreak research repositories. CI run token-spend budget-capped; failing runs are a blocking CI check for Critical/High-tier and a non-blocking warning for Medium/Low.

C) Operate the go-live battery and wire test failures to IM. Every AI/HAI software artifact must pass its archetype battery before receiving Sanctioned status in the SM inventory. Go-live triggers: pre-production (all applicable archetype tests must pass before the artifact is promoted); post-model-update (any model-family swap, model-version promotion, or new fine-tune triggers a re-run of the output-integrity regression and the full archetype battery, Critical-tier within 7 days, all tiers within 14 days); post-incident (any IM-Software incident involving the artifact triggers a re-run of the relevant battery subset before the incident is closed); quarterly (all active AI/HAI software artifacts re-run their battery; results reviewed by the named test-battery owner). All test failures route to IM-Software within one business day with a severity tag derived from the ST severity rubric (Blocker / Critical / High / Medium / Low). Named battery owner per archetype is a named role, not a shared-team responsibility.

Outcome Metrics (L1).

Success Criteria.

• Per-archetype foundational test battery published for all seven archetypes, linked from the SM inventory record and DR/IR artifacts.
• Four regression corpora published in source control, running in CI on every PR for Critical/High-tier artifacts, with a named corpus owner and a monthly refresh cadence.
• 100% of AI/HAI software artifacts reaching production in the last 90 days have a passed go-live battery on record.
• All test failures routed to IM with a 1-day handoff SLA and named owner.
• CI automation covers ≥60% of battery items; TA-Software archetype threat coverage ≥80%.

Maturity Level 2

Objective: Calibrate test depth per risk tier using the SM L2 tier-treatment matrix, run scheduled red-team exercises per tier using TA L2 deep threat models, and test cross-archetype compositions for Critical-tier artifacts.

Activities.

A) Tier-calibrated test battery and CI corpus depth. Publish a per-tier test treatment aligned to SM L2's tier-treatment matrix. Critical tier: full archetype battery at go-live with executive sign-off on results; all four corpora running on every PR with a Critical-specific corpus separately tuned to the artifact's tool set, retrieval sources, and data classes from the TA L2 per-artifact threat model; model-update re-run of the full battery within 7 days of any model-family swap; daily output-integrity regression against the production golden test set with drift firing an IM alert; logging-completeness verified quarterly with findings routed to IM within 1 business day. High tier: full archetype battery in CI; all four corpora on merge; model-update re-run within 14 days; weekly output-integrity regression; semi-annual logging-completeness verification. Medium tier: subset battery (top-4 threat classes) in CI; jailbreak and prompt-injection corpus on merge; subset battery model-update re-run within 30 days; monthly output-integrity regression; annual logging-completeness verification. Low tier: spot-check (3 test classes) at go-live; jailbreak corpus on merge; output-integrity regression at next quarterly; logging-completeness verified at go-live. For Critical-tier artifacts, the CI regression corpus is separated into a PR-blocking Critical-specific corpus (30–100 entries tuned to the artifact's threat model) and a broader corpus running on merge.

B) Scheduled per-tier red-team exercises using TA L2 threat models. Red-team cadence by tier: Critical (quarterly, 4 per year, scope derived from TA L2 per-artifact deep threat model, covering prompt-injection chains, indirect prompt injection via RAG retrieval, agent tool abuse, jailbreak regression, data-egress canaries, multi-turn AGH probes, cross-tenant isolation probes, and for agent archetypes explicit EA boundary testing, TM argument-smuggling variants, and RA long-session drift probes); High (semi-annual, 2 per year, scope from TA L2 artifact deltas, covering the top-5 threats from the per-artifact model); Medium/Low (ad-hoc before major model changes or scope expansions, archetype snapshot driving scope). Each exercise follows the AI Security Testing Methodology: written rules of engagement, test plan reviewed with the artifact owner, execution log, structured findings report (severity, root cause, ATLAS tactic ID, SR requirement traced, remediation pairing). Cross-archetype composition tests for Critical-tier: agent plus RAG (indirect injection via RAG path into agent goal-hijack chain); fine-tune plus model-serving (membership-inference probe against the served fine-tuned model; canary extraction from production completions); multi-agent orchestration (sub-agent goal-hijack via orchestrator response; scope inheritance tests confirming sub-agent cannot exceed parent scope).

C) Red-team findings to corpus pipeline. Every Critical or High-severity red-team finding produces: a new corpus entry (input, expected safe output, threat tag, ATLAS tactic ID, date, source reference) committed to the relevant regression corpus within 30 days; an IM-Software finding with severity tag and the named artifact owner as assignee; and a TA-Software library-gap ticket if the finding was not in the archetype library, tracked with a named owner and a 30-day close SLA for Critical-tier gaps. This pipeline ensures every quarterly red-team exercise produces durable CI coverage for the findings it surfaces.

Outcome Metrics (L2).

Success Criteria.

• Quarterly red-team for 100% of Critical-tier artifacts; semi-annual for 100% of High-tier; scope tied to TA L2 per-artifact deep threat models.
• Critical-tier regression corpora (all four) running on every PR for all Critical-tier artifacts; per-tier calibration enforced.
• ≥90% of Critical/High-severity red-team findings converted to corpus entries within 30 days.
• Cross-archetype composition tests documented and run for all Critical-tier artifacts with composite archetypes.
• Per-tier SLA adherence for testing activities ≥90%.

Maturity Level 3

Objective: Operate continuous automated adversarial testing for Critical-tier artifacts, publish regression corpora and test patterns as open artifacts, and contribute discovered TTPs to MITRE ATLAS, AVID, and OWASP LLM / Agentic Top 10.

Activities.

A) Continuous automated adversarial testing harness. Deploy an AI-vs-AI automated red-team scaffolding that runs daily against all Critical-tier AI/HAI software artifacts. Prompt-injection generator: produces novel direct and indirect prompt-injection inputs using mutation of the regression corpus, template-based variation, and model-assisted jailbreak-ladder generation; runs against LLM-integrated app, agent, and RAG archetypes (ATLAS TA0001/TA0003; AGH). Indirect-injection seeder: generates poisoned-document payloads designed to redirect agent goals; seeds a test corpus environment and exercises the retrieval-to-prompt-to-action pipeline (ATLAS TA0003; AGH). Tool-misuse generator: generates argument-smuggling variants for each declared tool in each Critical-tier agent; probes argument validation boundaries and unexpected tool combinations (ATLAS TA0004; TM / EA). Output-integrity monitor: runs the golden test set daily against production endpoints; any drift triggers a P1 alert to IM-Software and a re-run of the full regression corpus (ATLAS TA0013). Findings are triaged by a named ST owner at least weekly. Novel TTPs, patterns not in the TA library, are fed into the TA L3 auto-proposal pipeline within 14 days. High-severity automated findings route to IM-Software within 24 hours.

B) Contribute findings to industry. Contribute anonymized, legally-vetted findings to MITRE ATLAS (new technique observations, novel prompt-injection variants, agent-loop attack patterns, new retrieval-poisoning mechanics, following ATLAS evidence-and-provenance requirements; target ≥4 contributions per year), AI Vulnerability Database (AVID) (structured disclosure submissions for novel vulnerabilities in own-built AI/HAI software or its upstream dependencies, with coordinated disclosure where third-party components are involved), OWASP LLM Top 10 / Agentic Top 10 (real-world telemetry evidence during revision cycles; target ≥2 substantive submissions per revision cycle), and external benchmarks (AISI Inspect evaluation benchmarks, HELM safety evaluations, and sector ISAC AI red-team exercises with anonymized findings).

C) Publish regression corpora and test patterns as open artifacts. Publish anonymized versions of the four regression corpora (jailbreak, prompt-injection, AGH, tool-misuse) under an open license, scrubbed of org-specific tool names, data classes, and artifact identifiers. The internal corpora are a superset of the published versions with org-specific entries not shared externally. Maintain the published versions upstream; internal updates that belong upstream are proposed as contributions, not silently retained. Host or co-host at least one industry red-team benchmark per year (OWASP AI chapter, ATLAS practitioner table, or sector ISAC AI working group); collect cross-org detection-benchmark improvement data from participants.

Outcome Metrics (L3).

Success Criteria.

• ≥80% of Critical-tier AI/HAI software artifacts under continuous automated adversarial testing with daily probe execution; novel TTPs triaged into the TA library within 14 days; high-severity findings routed to IM within 24 hours.
• ≥4 industry contributions per year to MITRE ATLAS, AVID, or OWASP LLM / Agentic Top 10.
• ≥4 open regression corpora published under a permissive license and maintained upstream.
• ≥1 industry-shared exercise hosted per year plus ≥2 participated; cross-org detection-benchmark improvement documented.

Common Pitfalls

Level 1. - Test battery reduced to a logging-completeness check and a golden-path assertion, no adversarial probes (prompt injection, tool-scope boundary, kill-switch) are actually exercised. - Regression corpora committed to source control but not wired into CI, they exist but run only when a reviewer manually triggers them; coverage erodes after every sprint. - Go-live battery runs once pre-production but is never re-run after model-version updates, test coverage erodes as model versions change and configurations drift. - Test failures logged in a spreadsheet separate from IM, no SLA enforcement, no aging visibility, no named owner; the same failure recurs across multiple PRs undetected.

Level 2. - Red-team scope defined as "prompt-injection probes" but indirect-prompt-injection via RAG retrieval, argument smuggling, multi-turn AGH chains, and cross-tenant isolation tests are excluded, the top threat classes for agent plus RAG compositions go untested. - Per-tier calibration documented in the tier-treatment matrix but CI pipeline applies the same corpus to all tiers, Critical artifacts run the same tests as Low; differentiation exists on paper only. - Red-team findings route to IM but the finding-to-corpus pipeline is never executed, 12 months of Critical/High findings sit in IM as closed tickets with no corpus entries; the same vulnerabilities are re-discovered at the next exercise. - Cross-archetype composition tests scoped but not executed because no engineer owns agent-plus-RAG testing, composition-specific failure modes are in the threat model but not in any test.

Level 3. - Continuous harness runs prompt-injection probes that the artifact's content filter trivially blocks, coverage metric looks good but the probes do not exercise the real threat surface; novel jailbreak techniques are not generated. - Industry contributions are legal-vetted case-study summaries rather than actionable, reproducible technique descriptions, ATLAS reviewers cannot map them to a technique ID; AVID submissions lack reproducibility notes. - Open corpora published once and then not maintained, external organizations build on a stale version while the internal corpus has 60 new entries; discrepancies surface at community exercises. - New-TTP ingestion from automated probes to the TA library is manual and quarterly, by the time a novel technique reaches SR and SA updates and is reflected in controls, the technique is already exploited in the wild.

Practice Maturity Questions

Level 1. 1. Is a per-archetype foundational test battery published for all seven AI/HAI software archetypes, with each test class tied to a TA-Software archetype threat (HAI TTP + ATLAS tactic ID) and an SR-Software requirement, defined inputs/outputs/pass-fail criteria, and an evidence artifact, and are 100% of new AI/HAI software artifacts required to pass the battery before production Sanctioned status is issued? Evidence: Published battery documents per archetype with TA-threat and SR-requirement traceability table; SM inventory showing Sanctioned entries with a passed go-live battery record linked; sample test run with evidence artifact attached. 2. Are four regression corpora (jailbreak, prompt-injection, AGH, tool-misuse) versioned in source control, running in CI on every PR for Critical/High-tier artifacts, with a named corpus owner, a monthly refresh cadence from internal and external sources, and a CI token-spend budget cap, and are ≥95% of Critical/High-tier PR merges verified to have run and passed the corpus? Evidence: Source-control repository showing four corpus directories with version history and corpus owner in CODEOWNERS; CI telemetry report showing corpus run results per PR for the last 30 days; monthly corpus refresh commit log. 3. Are all test failures routed to IM-Software within 1 business day with a severity tag and named owner, and does TA-Software archetype threat coverage by the test battery and corpus reach ≥80% by end of year one? Evidence: IM-Software query for ST-originated issues with creation timestamps within 24 hours of test failure; threat-coverage matrix mapping TA archetype threats to battery test classes and corpus entries showing ≥80% coverage ratio.

Level 2. 1. Are 100% of Critical-tier AI/HAI software artifacts red-teamed at least quarterly, and 100% of High-tier semi-annually, with scope derived from TA-Software L2 per-artifact deep threat models, covering prompt-injection chains, indirect-prompt-injection via RAG, agent tool abuse (EA/TM), multi-turn AGH probes, data-egress canaries, and cross-tenant isolation, with findings routed to IM and remediation tracked? Evidence: ST records showing red-team exercise dates per Critical and High-tier artifact for the last 12 months; red-team report for the most recent Critical-tier exercise showing scope sourced from the TA L2 per-artifact model; IM-Software findings linked from the report. 2. Is per-tier corpus calibration enforced in CI (Critical-tier: all four corpora on every PR plus daily output-integrity regression; Low-tier: jailbreak corpus on merge), and are ≥90% of Critical/High-severity red-team findings converted to corpus entries within 30 days? Evidence: CI pipeline configuration showing per-tier corpus routing; CI telemetry confirming daily output-integrity run for Critical-tier artifacts; finding-to-corpus pipeline telemetry showing conversion rate and lead times. 3. Are cross-archetype composition tests (agent plus RAG, fine-tune plus model-serving, multi-agent orchestration) documented and executed for all Critical-tier composite artifacts, and is per-tier SLA adherence for testing activities ≥90%? Evidence: Composition test plans per Critical-tier composite artifact; execution logs with pass/fail results; per-tier SLA adherence report from program telemetry for the last two quarters.

Level 3. 1. Are ≥80% of Critical-tier AI/HAI software artifacts under continuous automated adversarial testing with daily probe execution, with novel TTPs triaged into the TA-Software library within 14 days and high-severity automated findings routed to IM within 24 hours? Evidence: ST harness telemetry showing daily probe execution per Critical artifact; harness-to-TA-library pipeline log with lead time per novel TTP; IM-Software high-severity finding creation timestamps within 24 hours of automated detection. 2. Has the program contributed ≥4 anonymized, legally-vetted findings per year to MITRE ATLAS, AVID, or OWASP LLM / Agentic Top 10, with at least one accepted as a new or refined technique, and are ≥4 open regression corpora published under a permissive license and maintained upstream? Evidence: Contribution log with external submission links and acceptance confirmation from ATLAS, AVID, or OWASP; open-source repository links for the four published corpora with commit history showing active maintenance; legal review records for each submission. 3. Has the program hosted at least 1 industry-shared red-team exercise per year and participated in ≥2 additional cross-org exercises, with documented cross-org detection-benchmark improvement data from participants? Evidence: Exercise log with hosted and participated entries for the last 12 months; post-exercise report showing detection-benchmark data collected from participants; testimonials or co-published results from at least one cross-org partner.

23. Environment Hardening (EH)

Practice Overview

Objective: Harden the compute, build, model-supply-chain, engineering-endpoint, and data-flow envelopes in which AI/HAI software the organization builds is developed, trained, and served, so each artifact runs in a least-privilege, observable perimeter and unsanctioned AI development is detectable before it reaches production.

Description: EH-Software tunes the organization's existing perimeter, identity, and pipeline controls for the specific surfaces AI/HAI software development creates. Five envelope dimensions are in scope: the compute/runtime envelope (per-artifact service accounts, secrets vault for LLM provider keys, egress allowlists, resource limits, per-tenant isolation); the build-time envelope (CI secrets scanning, signed builds, SLSA-style provenance for model artifacts, AI SDK supply-chain scanning); the model-supply-chain envelope (model registry SSO + MFA, signed-model verification, deny-listing of known-poisoned upstream weights); the engineering-endpoint envelope (SSO + MFA on LLM provider consoles, AI-tuned DLP for bulk embeddings / prompt-completion exports / model-weight exfiltration, managed-browser policy preventing unsanctioned consumer LLM use); and the data-flow envelope (PII redaction at the prompt/completion logging layer, classification labels propagated through ETL, fine-tuning, and inference pipelines). Agent-runtime sandboxing, kill-switch wiring, and rate-limit / circuit-breaker defaults are baseline controls for agent and model-serving archetypes.

Context: Engineering teams adopting AI move faster than platform and security can follow. LLM API keys end up hardcoded in Jupyter notebooks committed to the monorepo. Fine-tuning jobs run against training datasets they have no business touching, with no egress constraints. Agents ship with service accounts that have org-wide read access "because that's what the demo used." Model-serving endpoints lack version pinning, so when the provider silently swaps the underlying model family the artifact's behavior regresses without anyone noticing. EH-Software closes these gaps by tuning controls the organization already operates, secrets scanning in CI, model-registry access control, egress allowlists, SSO enforcement on LLM provider consoles, DLP on engineer workstations, for the specific surfaces AI/HAI software development creates. The HAI TTPs EA, AGH, TM, and RA are mitigated here at the perimeter level: EA via least-privilege IAM and egress allowlists; AGH via agent-runtime sandboxing and kill-switch wiring; TM via tool-scope enforcement at the runtime envelope and rate-limit / circuit-breaker defaults; RA via session-bounded execution and runtime resource limits.

Maturity Level 1

Objective: Harden the compute/runtime, build-time, model-supply-chain, engineering-endpoint, and data-flow envelopes for every AI/HAI software artifact in the SM inventory so each artifact runs in a least-privilege, observable perimeter with AI-specific exfiltration paths controlled.

Activities.

A) Harden the compute/runtime envelope per artifact. Every AI/HAI software artifact registered in the SM inventory, LLM-integrated app, AI agent, RAG pipeline, fine-tuning/training workload, eval harness, model-serving service, classical ML model, runs under a named, dedicated service account (IAM role, Kubernetes service account, workload identity, or equivalent) with no shared credentials across artifacts. IAM is least-privilege: the service account has access only to the specific LLM provider endpoint, vector-store namespace, model-registry path, and secrets-vault path the artifact requires; wildcard policies and inherited org-level reads are prohibited. All LLM provider API keys (OpenAI, Anthropic, Bedrock, Vertex, self-hosted OSS endpoints, HuggingFace tokens) live in the secrets vault (Vault, AWS Secrets Manager, GCP Secret Manager, Azure Key Vault); a CI secrets-scanning check (truffleHog, gitleaks, detect-secrets) blocks any PR containing a plaintext key. Egress is allowlisted at the service-account or network-namespace level to declared LLM provider domains, vector-store endpoints, and model-registry endpoints; unexpected outbound traffic to AI provider domains from unregistered services raises a shadow-AI discovery alert. Multi-tenant artifacts enforce per-tenant namespacing at the prompt/context layer and at the vector store. Agent and model-serving artifacts expose a kill-switch that is wired, documented, and quarterly-tested. GPU/CPU/memory limits cap inference and training workloads; rate-limit and circuit-breaker defaults are applied at the inference gateway to prevent cost-abuse and token-spend runaway.

B) Harden the build-time and model-supply-chain envelopes. Secrets scanning runs on every PR and every main-branch commit as a blocking check. Build artifacts (container images, Python packages, model weights) are signed at build time; downstream deployment gates reject unsigned artifacts. Model artifacts promoted to the model registry carry an SLSA-style provenance attestation listing training-data sources, training-job identity, eval-suite result reference, and build-system identity; unsigned promotions are blocked at the registry policy layer. AI SDK dependencies (openai, anthropic, langchain, llama_index, transformers, vllm) are version-pinned across all manifest files; a software-composition-analysis tool runs in CI and blocks Critical/High-tier artifact merges on critical findings. Training and fine-tuning workloads are configured for reproducibility from pinned data, pinned model versions, and pinned hyperparameters; reproducibility is a gate for model promotion in the eval harness. Known-poisoned upstream weights (advisories from HuggingFace, model-vendor security feeds) populate a deny-list enforced at the registry ingestion gate.

C) Harden the engineering-endpoint and data-flow envelopes. Model registry, LLM provider admin consoles, CI/CD systems, and code repositories all require SSO/SAML/OIDC with MFA; local-account access to AI provider admin consoles is disabled for org-domain identities. AI-tuned DLP on engineering endpoints recognizes the patterns specific to AI/HAI software development: bulk vector-embedding exports, CSV/JSON bulk exports of prompt/completion logs, uploads of model-checkpoint file extensions (.bin, .safetensors, .gguf, .pt, .ckpt) to external destinations, and bulk training-dataset transfers to unmanaged storage. A managed-browser policy prevents engineers from pasting source code, internal documents, or customer data into unsanctioned consumer LLM services during managed work sessions; browser-extension AI coding assistants are governed by the PC-Software sanctioned-assistant policy. Prompt/completion logging pipelines pass through a PII redaction layer before logs reach the long-term log store; regulated data classes (PII, PHI, PCI) are redacted, tokenized, or routed to a separately controlled log tier with stricter access controls, Privacy/Legal review the redaction configuration quarterly. Classification labels (public, internal, confidential, regulated) are attached to datasets at ingest and propagated through ETL, fine-tuning, and inference pipelines; any pipeline stage that drops or downgrades classification labels without explicit authorization is a blocking finding.

Outcome Metrics (L1).

Success Criteria.

• 100% of production AI/HAI software artifacts run under named, dedicated, least-privilege service accounts; confirmed by quarterly SM inventory × IAM audit reconciliation.
• CI secrets-scanning enforced as a blocking check on every PR; zero LLM provider API keys in source code, CI environment variables, or container image layers.
• Model registry requires SSO + MFA; 100% of Critical/High-tier model promotions carry a signed SLSA-style provenance attestation; unauthenticated promotions blocked.
• AI-tuned DLP rules active on ≥95% of managed engineering endpoints; managed-browser policy enforced.
• PII redaction layer active on prompt/completion logging pipelines for all artifacts processing regulated data; classification labels propagated end-to-end through ETL, fine-tuning, and inference pipelines.

Maturity Level 2

Objective: Calibrate hardening depth per SM-Software L2 risk tier, Critical-tier artifacts receive per-artifact SASE egress, zero-trust AI access, infrastructure-layer per-tenant isolation, and enhanced DLP; Low-tier artifacts stay on baseline L1 controls.

Activities.

A) Tier-conditional hardening calibration. Publish and enforce a tier-treatment matrix aligned to the SM-Software L2 risk-tier rubric. Critical: per-artifact workload-identity service account with no standing IAM permissions, per-artifact SASE egress rule with content-inspection where permissible, just-in-time model-registry access (≤4-hour time-limited, approval-gated), dedicated vault path with ≤30-day key rotation, dedicated private link or VPC endpoint to the LLM provider with no public egress, enhanced DLP with content inspection and bulk-embedding / model-weight transfer alerts, infrastructure-layer per-tenant isolation (dedicated namespace, VPC endpoint, or per-tenant encryption key), and PII redaction with a separate regulated-data log tier reviewed quarterly. High: per-artifact service account scoped to declared resources, per-artifact egress allowlist, SSO + MFA with signed-provenance requirements on model registry, ≤90-day key rotation, private link preferred, standard AI-specific DLP, application-layer per-tenant isolation. Medium: per-artifact service account, per-service egress allowlist, SSO + MFA, standard DLP, application-layer isolation. Low: baseline L1 controls. Each SM inventory artifact record carries its tier's hardening status; gaps between required and actual controls become open IM findings.

B) SASE egress governance and zero-trust AI access. For Critical-tier artifacts replace per-service egress rules with per-workload-identity SASE policies; each artifact's service account has its own outbound-traffic policy so an incident in one artifact's context does not contaminate another's allowlist. Enable API-content inspection on outbound calls to LLM provider endpoints where the DPA and provider terms permit; alert on bulk-export patterns that exceed the declared operational profile. Eliminate standing access for Critical-tier: no permanent write/promote permissions on the model registry, access is just-in-time, time-limited to ≤4 hours, scoped to a specific model artifact, and approval-gated; no standing access to LLM provider admin consoles with billing or key-management permissions. Fine-tuning and training-pipeline execution requires the submitter identity logged, the training-data classification pre-flight check passed, and a privacy sign-off reference for any regulated data in scope, all enforced at pipeline submission time, not by honor system.

C) Infrastructure-layer per-tenant isolation and enhanced DLP. Multi-tenant Critical-tier artifacts enforce tenant boundaries at the infrastructure layer (dedicated namespace or VPC per tenant, or a separate encryption key per tenant in shared infrastructure); the boundary is verified by IR-Software implementation review and ST-Software isolation tests. Enhanced DLP for Critical-tier development includes content inspection on engineer-workstation outbound traffic to LLM provider domains (prompt/completion payloads above size threshold trigger review), model-weight exfiltration detection on .bin / .safetensors / .gguf / .pt / .ckpt uploads to external destinations, and embedding-export detection for bulk vector files going to undeclared targets. Classification-label propagation is enforced through every ETL, fine-tuning, and inference pipeline stage; a stage that drops or downgrades a classification label without explicit policy authorization blocks the pipeline.

Outcome Metrics (L2).

Success Criteria.

• 100% of Critical-tier artifacts under per-artifact SASE egress with just-in-time model-registry and LLM provider admin access.
• ≥90% of Critical-tier multi-tenant artifacts with infrastructure-layer per-tenant isolation confirmed by IR review and ST isolation test.
• Enhanced DLP policies for AI-specific exfiltration patterns active on engineering endpoints for Critical-tier development; FP rate monitored monthly and trending down.
• Tier-hardening matrix published and enforced at provisioning; SM inventory artifact records show hardening status per tier; gaps tracked as open IM findings.

Maturity Level 3

Objective: Express all EH controls as IaC; drive adaptive policy tightening from ML-Software detections and IM-Software incidents; auto-provision tier-appropriate hardening for new artifacts; contribute hardening baselines to CIS, CSA, and sector ISACs.

Activities.

A) Hardening-as-code. Express every EH control as a version-controlled, parameterized IaC module: a runtime-envelope module for service-account creation, IAM, egress allowlists, resource limits, and vault paths; a build-time module for secrets-scanning, SCA, signed-build configuration, and SLSA provenance attestation encoded as reusable GitHub Actions / GitLab CI templates; a model-supply-chain module for registry access policy, version-pinning enforcement, and signed-artifact verification; an engineering-endpoint module for DLP rule sets, browser policy, and managed-endpoint AI-tool allowlists expressed as configuration-as-code for MDM/EDR and CASB platforms; a data-flow module for PII redaction, classification-label propagation, and regulated-data log-tier access. Modules are version-pinned; updates notify consuming artifact teams with a required-remediation flag. A drift-detection pipeline runs hourly against all deployed artifact configurations; low-risk drift (configuration noise) is auto-remediated; high-risk drift (service-account over-permissioning, egress-allowlist expansion, MFA disabled on model registry) raises a human-review alert within 2 business days and opens an IM-Software finding.

B) Adaptive policy tightening from ML and IM signals. Wire ML-Software detection signals (abuse-pattern token-spend spikes, shadow-AI egress, prompt-injection volume) and IM-Software incident patterns to a human-approved adaptive-tightening pipeline. An abuse spike on a specific service account produces a rate-limit tightening proposal; a shadow-AI egress detection produces an egress-narrowing proposal and an SM inventory shadow-AI alert; a post-incident review that identifies a hardening gap produces a baseline-update proposal; a Critical-tier incident involving a misconfigured service account produces a zero-trust upgrade proposal for the affected tier. Proposals are human-reviewed by a security platform engineer before deploy; the change log is machine-readable; downstream artifact teams are notified within 24 hours of a tightening change affecting their artifact's hardening profile. Hardening changes that reflect a new threat pattern are fed back to TA-Software as a candidate new threat entry and to SR-Software as a candidate new requirement, the adaptive loop is bidirectional. Auto-provisioning fires on SM inventory registration: when a new AI/HAI software artifact is registered, the IaC automation provisions its tier-appropriate hardening profile within 24 hours.

C) Contribute AI/HAI software hardening baselines to industry. Contribute anonymized EH-Software hardening baseline modules to CIS benchmarks for AI workloads (service-account least-privilege, model-registry access control, AI SDK supply-chain scanning), to the CSA AI Safety Initiative (compute/runtime and build-time envelope controls), and to sector ISACs (FS-ISAC, H-ISAC, IT-ISAC AI working groups). Target ≥2 substantive contributions per year; maintained upstream; internal practice aligned with the published external version.

Outcome Metrics (L3).

Success Criteria.

• ≥90% of EH controls expressed as authoritative IaC; drift detected continuously; ≥70% of low-risk drift auto-remediated; high-risk drift human-reviewed within 2 business days.
• Adaptive-policy pipeline operational with ML-Software and IM-Software signal sources; every change traceable; downstream teams notified within 24 hours.
• New artifacts auto-provisioned with tier-appropriate hardening within 24 hours of SM registration.
• ≥2 industry baseline contributions per year with documented adoption.

Common Pitfalls

Level 1. - Service-account policy is "per-team" rather than per-artifact, one account covers a dozen microservices including AI features and high-trust services, and an overprivileged AI feature inherits trust from its neighbors until an incident exposes the conflation. - CI secrets-scanning runs as an informational check that produces a report nobody reads, engineers merge PRs with hardcoded API keys because the scan is not wired as blocking. - Egress allowlist covers LLM provider domains for one service but not for fine-tuning and training workloads that run under a separate pipeline identity, training jobs exfiltrate embeddings to unmonitored destinations. - PII redaction layer is documented in the design but never deployed, prompt/completion logs for a customer-facing LLM feature contain customer names and emails in clear-text, discovered 8 months later during an IR review.

Level 2. - Per-artifact SASE egress rules created for Critical-tier artifacts but implemented at the service level rather than the workload-identity level, multiple artifacts share an egress rule and an incident in one creates noise across all. - Just-in-time access for model-registry write declared in policy but the tooling to enforce it is not implemented, engineers continue using standing service tokens; the JIT policy exists on paper only. - Infrastructure-layer per-tenant isolation documented in architecture but only application-layer isolation is deployed, tenant A's embeddings sit in a shared vector store with application-layer key filtering that a SQL injection at the app layer bypasses. - Tier-hardening matrix exists but is not gated at provisioning, engineers self-provision Critical-tier artifacts with Low-tier baseline controls because the provisioning tooling does not enforce tier.

Level 3. - IaC coverage declared at ≥90% but the registry counts artifacts that have an IaC stub rather than artifacts whose IaC is the authoritative deployed source, drift accumulates between stub and live config. - Adaptive-policy pipeline wired to ML detections but not to IM post-incident outputs, hardening gaps surfaced by IR reviews never convert to tightening proposals. - Industry baselines published once and not maintained upstream, internal practice advances while the published baseline ages, and external adopters find the public version conflicts with the program's current guidance. - Drift auto-remediation closes findings without recording the root cause, the program loses the learning, and downstream teams observe unexpected configuration resets they cannot trace.

Practice Maturity Questions

Level 1. 1. Does every AI/HAI software artifact in the SM inventory (all seven archetypes) run under a named, dedicated, least-privilege service account, confirmed by a quarterly IAM audit, with all LLM provider API keys and model-registry credentials in a secrets vault and CI secrets-scanning enforced as a blocking PR check with zero current findings? Evidence: SM inventory × IAM audit reconciliation; CI secrets-scan telemetry showing zero findings on last 30 days of PRs. 2. Does the model registry require SSO + MFA with 100% of Critical/High-tier promotions gated by a signed SLSA-style provenance attestation, and is an egress allowlist scoped to declared LLM provider and model-registry domains in place per service account with unexpected AI-provider egress from unregistered services raising a shadow-AI discovery alert? Evidence: model-registry policy export, signed-provenance attestation log, egress allowlist policy export, shadow-AI alert sample. 3. Are AI-tuned DLP rules (bulk embeddings, prompt/completion exports, model-weight extensions) active on ≥95% of managed engineering endpoints, and are prompt/completion logging pipelines for regulated-data-processing artifacts running a PII redaction layer with Privacy/Legal sign-off documented where clear-text regulated data is retained? Evidence: DLP management console coverage report; redaction-pipeline configuration; Privacy/Legal sign-off records.

Level 2. 1. Are 100% of Critical-tier AI/HAI software artifacts under per-artifact SASE egress rules at the workload-identity level, with just-in-time access (≤4-hour time-limited, approval-gated) enforced for model-registry write permissions and LLM provider admin console access, and standing service tokens deprecated for Critical-tier? Evidence: SASE policy registry × SM inventory; IAM audit telemetry showing zero standing-write grants on Critical-tier; JIT-access approval log. 2. Are ≥90% of Critical-tier multi-tenant AI/HAI software artifacts enforcing per-tenant isolation at the infrastructure layer (dedicated namespace, VPC endpoint, or per-tenant encryption key), confirmed by IR-Software implementation reviews and ST-Software isolation tests, with a hardening tier-treatment matrix published and enforced at provisioning? Evidence: IR findings × SA pattern conformance; ST isolation-test results; published tier-treatment matrix and provisioning-gate configuration. 3. Are enhanced DLP policies (content inspection on outbound AI-provider calls, bulk-embedding and model-weight-transfer alerts) active for Critical-tier artifact development on engineering endpoints, with false-positive rates monitored monthly and trending down? Evidence: DLP management console policy export; FP rate trend chart across last 6 months; monthly tuning-review records.

Level 3. 1. Are ≥90% of EH controls expressed as authoritative IaC (not stubs) in a version-controlled registry, with drift detected continuously, ≥70% of low-risk drift auto-remediated, and a machine-readable change log visible to downstream network and identity teams? Evidence: IaC registry inventory; drift-detection telemetry; auto-remediation rate; change-log export consumable by downstream teams. 2. Is the adaptive-policy pipeline operational with ML-Software detections and IM-Software incidents generating human-approved tightening proposals on a tracked cadence, every change traceable to a source signal, and downstream artifact teams notified within 24 hours? Evidence: adaptive-policy change log with ML/IM source references; human-approval records; downstream-team notification log. 3. Does the program contribute ≥2 AI/HAI software hardening baselines per year to CIS AI workloads, CSA AI Safety Initiative, or sector ISACs with documented adoption, and are new AI/HAI software artifacts auto-provisioned with tier-appropriate hardening within 24 hours of SM inventory registration? Evidence: contribution log with upstream adoption references; auto-provisioning telemetry tied to SM registration events.

24. Issue Management (IM)

Practice Overview

Objective: Run a single unified backlog and a single tier-calibrated incident playbook for every AI/HAI software issue the organization ships, findings from TA snapshots, SR gaps, DR conditions, IR drifts, ST failures, ML detections, and external advisories, with named owners, tier-aware SLAs, AI-specific containment plays, and regulatory SLA tracking that never misses a notification window because of organizational diffusion.

Description: IM-Software is the clearinghouse for everything the other Software-domain practices produce. Every TA threat-snapshot row that carries residual risk, every SR REM accepted gap with an owner and expiry, every DR approve-with-conditions item, every IR drift finding, every ST CI corpus failure or red-team finding, every ML detection that fires, and every external advisory (MITRE ATLAS updates, OWASP LLM updates, AVID entries, model-vendor advisories) flows into a single, prioritized backlog with named owners, tier-calibrated SLAs, and an unambiguous playbook. The playbook contains AI-specific containment plays, kill-switch execution for rogue agents, model rollback for training-data-leakage and output-integrity incidents, RAG-source disable for retrieval-poisoning events, tool-revoke for tool-scope violations, prompt-template rollback for prompt-injection surfaces, and egress-block for shadow-AI emergence. Pre-established escalation paths cover CTO, Legal/Privacy, Communications, executive sponsor, and regulator routing. Every Critical/blocker incident receives a post-incident review whose outputs feed back to SA (pattern update), SR (requirements-pack update), EG (training content), and ML (detection update).

Context: Without a unified backlog, AI/HAI software issues scatter across product Jira projects, security queues, legal trackers, privacy dashboards, and ML-platform alert channels. TA residual risks age without remediation owners. SR gaps renew silently past their expiry dates. An ML detection fires on a Friday and routes to nobody because the on-call rotation does not cover AI-specific alerts. An agent executes a rogue tool invocation and the first human to know is a customer reporting unexpected behavior two days later. The GDPR Art. 33 72-hour clock starts at the moment the organization becomes aware of a personal data breach, not when the responsible team processes the notification. EU AI Act Art. 73 imposes serious-incident reporting obligations on Annex III high-risk systems. HIPAA breach-notification windows and NYDFS Part 500 72-hour notification apply when AI/HAI software processes the regulated data classes. IM-Software closes these gaps with a single backlog, one triage rubric, AI-specific incident classes named in advance, and a regulatory SLA tracker that escalates automatically as clocks approach expiry.

Maturity Level 1

Objective: Operate a single unified AI/HAI software issue backlog with a standard triage rubric, an AI-specific incident playbook covering the six primary HAI incident classes, and regulatory SLA tracking for GDPR Art. 33, EU AI Act Art. 73, HIPAA, and sector-specific obligations.

Activities.

A) Stand up the AI/HAI software issue backlog and triage rubric. One backlog with standardized metadata per issue: source (TA / SR / DR / IR / ST / ML / external), affected artifact(s) linked to SM inventory with archetype and tier, severity (Critical / High / Medium / Low anchored to AI-specific axes), named owner from the SM inventory with escalation path to program sponsor, SLA target, evidence link to the originating artifact, and a regulatory flag indicating whether the issue carries a notification obligation (GDPR Art. 33 clock started, EU AI Act Art. 73 clock started, HIPAA breach-notification triggered, sector-specific). The AI-specific severity rubric: Critical means active exfiltration through an AI/HAI artifact, agent executing tool invocations with real-world damage, kill-switch failure in a production agent, personal-data breach triggering GDPR Art. 33, or regulated-data exposure in training or inference; High means confirmed control failure with potential impact (output-integrity regression on a decision-affecting artifact, tool-scope violation without confirmed damage, HITL-gate bypass attempt in production, RAG retrieval-source poisoning confirmed but not yet serving customers); Medium covers confirmed gaps in non-production or compensating-control-protected production artifacts and SR REM accepted-gap renewals past expiry; Low captures informational items and Low-tier logging gaps. Published SLAs: Critical acknowledge ≤4h / contain ≤48h / root-cause ≤30d; High ack ≤24h / contain ≤7d / root-cause ≤45d; Medium ack ≤48h / remediate ≤14d; Low ack ≤5 business days / remediate ≤30d. Triage cadence: daily for Critical and new High; weekly for Medium; monthly aging review for the full backlog.

B) Publish the AI-specific incident playbook. Publish playbook entries for the six primary AI/HAI software incident classes; each entry names trigger conditions, pre-assigned roles (deployer-duty owner, AppSec on-call, Privacy/Legal, executive sponsor escalation path, CTO and Communications routing for Critical-tier customer impact), step-by-step containment, artifacts to collect, evidence-capture instructions for the deployer-duty record, closure criteria, and SLA targets. The six classes are: prompt-injection containment (disable affected input path or LLM integration via feature flag, assess scope of affected sessions and completions, rollback prompt template to last known-good, route to Privacy/Legal if PII exfiltrated to start the GDPR Art. 33 clock); agent rogue-action containment (execute kill-switch for the affected session and document invocation, revoke the offending tool from the agent's allowlist, assess full session tool-call log for prior actions, reverse reversible actions under human review, notify affected customer or internal user); training-data-leakage containment (roll back the affected model version to previous production, replay the eval harness against the rolled-back version, identify the training run that introduced the vulnerable fine-tune, quarantine the training dataset pending data-provenance review, update the training-data provenance record in the REM); silent model-family-swap rollback (pin the API call to the approved model version, re-run the eval harness to assess output-integrity regression, rollback to pinned-version traffic and notify the provider, update the SR REM row for model-version pinning, trigger IR re-review); RAG retrieval-source poisoning containment (disable the affected retrieval source from the RAG query allowlist, initiate re-indexing without the poisoned source, assess which users received completions influenced by the poisoned retrieval, notify users where the completion materially affected a decision with GDPR Art. 22 implications); shadow-AI emergence containment (block egress from the identified service to the LLM provider domain, identify the service and owning team from the SM inventory or open a new record, route the artifact through SM intake via the amnesty path if appropriate, assess whether customer or regulated data transited the LLM provider endpoint, evaluate GDPR Art. 33 and EU AI Act Art. 73 obligations if data transited).

C) Track regulatory SLAs and run post-incident reviews. The regulatory SLA tracker is live with named obligations and automated escalation as deadlines approach. GDPR Art. 33: 72-hour supervisory-authority notification after the controller becomes aware of a personal-data breach; clock starts on the first internal alert that constitutes awareness; named owner Privacy/Legal; daily-at-minimum status updates required until the notification is filed or the clock expires. EU AI Act Art. 73: serious-incident reporting for Annex III high-risk systems on the timeline set by the implementing act; named owner Privacy/Legal plus executive sponsor; escalation immediate on any Annex III-classified artifact incident. HIPAA: 60-day discovery-to-notification ceiling for covered entities and business associates, individual notification without unreasonable delay; named owner Privacy/Legal; flag any AI/HAI software incident involving PHI immediately. NYDFS Part 500: 72-hour notification to the Superintendent for material cybersecurity events affecting covered entities; named owner CISO plus Privacy/Legal. PCI-DSS, FINRA / SEC model-risk, and sector-specific obligations carry named owners per the organization's compliance program. Every Critical or blocker incident receives a post-incident review within 14 days of containment covering what happened (root cause, initiation path, controls that failed or were absent), what caught it (which detection, source, or external report surfaced it first; was this the expected detection path or a gap), what did not catch it (controls that should have detected or prevented this but did not), and update outputs to SA (pattern-update request if an architectural gap was exploited), SR (requirements-pack update if a missing or vague requirement was exploited), EG (training-content update if the incident indicates a literacy gap), and ML (detection-update request, new detection, tuned query, or evidence that an existing detection can be sharpened). Post-incident review outputs are tracked as IM issues of type "improvement" and age against the same process metric cadence as other issues.

Outcome Metrics (L1).

Success Criteria.

• Single AI/HAI software issue backlog operational with standardized metadata; AI-specific severity rubric published.
• Six AI-specific playbook entries (prompt-injection, agent rogue-action, training-data-leakage, silent model-swap, RAG poisoning, shadow-AI emergence) published with pre-assigned roles, containment plays, evidence-capture steps, and SLA targets; each exercised in at least one tabletop in the last 12 months.
• Regulatory SLA tracker live covering GDPR Art. 33, EU AI Act Art. 73, HIPAA, NYDFS Part 500, and sector-specific obligations; 100% adherence in the last 90 days.
• Post-incident review loop wired to SA, SR, EG, and ML; every Critical/blocker incident produces a review within 14 days with named update outputs per downstream practice.
• Program-sponsor dashboard refreshed monthly showing backlog aging, SLA adherence, and post-incident learning outputs.

Maturity Level 2

Objective: Calibrate incident response depth per SM-Software L2 risk tier; operate dedicated 24/7 on-call coverage and pre-staged escalation for Critical-tier artifacts; auto-flow post-incident review outputs to SA/SR/EG/ML practice backlogs; activate cross-domain coordination when a Software-domain incident implicates Vendors, Infrastructure, or Processes.

Activities.

A) Tier-calibrated incident playbook and on-call. Extend L1 playbook entries with tier-specific activation criteria and on-call coverage. Critical tier: full IM activation, CISO or delegate plus Privacy/Legal plus engineering deployer-duty owner plus executive sponsor notification plus CTO and Communications routing; ≤1 hour acknowledgement; ≤4 hours containment-action initiated; 24/7 on-call coverage with a named AI/HAI software incident responder in each on-call rotation; pre-staged communication templates (internal, customer-facing, regulatory) loaded and reviewed quarterly. High tier: scoped response team, AppSec lead plus Privacy/Legal if regulated data involved plus deployer-duty owner; ≤4 hours acknowledgement; ≤24 hours containment-action initiated; business-hours on-call with after-hours escalation path defined. Medium tier: standard response; ≤1 business day acknowledgement; queue-based triage. Low tier: tracked in queue with aggregated weekly handling. Critical-tier on-call rotation is documented per week with named individuals, coverage-handoff protocol, and an on-call briefing that includes the current Critical-tier artifact list, their kill-switch invocation paths, and their active detection set.

B) Post-incident review auto-flow integration. Wire IM post-incident review outputs to downstream practice backlogs via a defined integration. SA-Software pattern-update requests auto-create architecture-backlog tickets with the IM incident reference linked. SR-Software requirements-pack update requests auto-create pack-backlog tickets with the requirements-pack version and failing requirement row linked. EG-Software training-content update requests auto-create training-backlog tickets with the affected population segment and incident summary linked. ML-Software detection-update requests auto-create detection-registry update tickets with the detection name, current query, and proposed change linked. SLA for downstream updates: Critical-tier post-incident review outputs must be accepted or rejected by the downstream practice owner within 14 days; accepted updates are treated as High-severity issues in the receiving practice's backlog. The program sponsor reviews post-incident review quality quarterly, are update outputs substantive (concrete change to a pattern, pack, curriculum, or detection) or nominal (a note saying "consider reviewing")?

C) Cross-domain coordination protocol. Publish a cross-domain coordination protocol that activates when a Software-domain AI/HAI incident implicates another domain. Software → Vendors: a production artifact's completions are impaired by a model-family swap at the inference provider; activates the Vendors-domain IM playbook entry for vendor material change alongside the Software-domain rollback play; named cross-domain IC from the Software side. Software → Infrastructure: a model-serving misconfiguration exposes the training pipeline or model registry; activates Infrastructure-domain EH and IM alongside Software-domain containment; named Infrastructure-domain IM contact on file. Software → Processes: a production agent makes unauthorized writes to a business-process workflow; activates the Processes-domain business-continuity coordinator alongside the Software-domain agent-rogue-action play; named Processes-domain contact on file. Cross-domain activations share a single status board, a single IC from the primary impacted domain, coordinated remediation tracking, and a joint post-incident review spanning all affected domains. Tier-movement in the SM-Software inventory auto-triggers IM configuration updates: a Medium→Critical re-tier updates on-call path, playbook variant, and SLA targets within 14 days.

Outcome Metrics (L2).

Success Criteria.

• Critical-tier MTTA ≤1 hour; MTTC ≤4 hours; 24/7 on-call coverage with a documented rotation including a current Critical-tier artifact briefing.
• Post-incident review auto-flow integration live; 100% of Critical-tier review outputs auto-routed; ≥90% downstream practice owners responding within 14 days.
• Cross-domain coordination protocol published and used for 100% of multi-domain AI/HAI software incidents; named cross-domain contacts verified quarterly.
• Tier-movement in SM-Software inventory auto-triggers IM configuration updates within 14 days for Critical re-tiers.

Maturity Level 3

Objective: Contribute incident patterns and playbook templates to sector ISACs, MITRE ATLAS, and AVID; execute pre-authorized automated containment for defined low-severity high-confidence detections; benchmark MTTR against industry peers and link deltas to investment proposals.

Activities.

A) Industry-coordinated incident sharing and contribution. Participate in sector-ISAC AI incident-sharing programs (FS-ISAC AI working group, H-ISAC, IT-ISAC, sector-specific). Consume ISAC AI incident feeds and integrate relevant advisories into the IM-Software external-advisory source. Contribute anonymized incident classification (incident type, ATLAS tactic tag, HAI-TTP tag, containment play used, MTTR achieved) on a per-incident-class basis; target ≥4 ISAC contributions per year. Contribute to AI-incident taxonomy standards: CSA AI Safety Initiative (severity-anchor definitions, playbook template schemas), AVID (≥2 entries per year for novel incident classes discovered in production), OpenSSF AI (runbook schema for pre-authorized containment actions). Contribute to MITRE ATLAS TA0014 Impact documentation, submit incident-derived technique observations or mitigation entries for Impact-tactic techniques in the AML.T0048 through AML.T0053 range and successors; target ≥1 ATLAS contribution per year for IM-primary tactics.

B) Pre-authorized automated runbook decisioning. Define and publish a pre-authorization policy for automated containment actions, the set of actions that may execute without human approval when a detection fires at a defined confidence threshold. Pre-authorized actions include kill-switch execution for Low-tier or Medium-tier agent artifacts when an AGH detection fires above 95% confidence, egress-block for shadow-AI emergence on non-Critical-tier services (first-time detection of new LLM provider domain from an unregistered service), retrieval-source disable for a RAG pipeline when an injection-defense detection fires with a specific flagged source document ID, and tool-revoke for pre-defined "auto-revocable" tool categories on agents when a tool-scope-violation detection fires. Pre-authorized actions for Critical-tier artifacts require human confirmation within 15 minutes; the action fires after that window if no confirmation arrives (timer-based fallback) with executive notification at fire time. All pre-authorized actions produce a full audit-log entry in the IM backlog, a human-review ticket auto-created at execution, and notification to the artifact's deployer-duty owner. Pre-authorization policy is reviewed quarterly by Privacy/Legal and the executive sponsor; any automated action producing an unexpected outcome triggers an out-of-cycle review.

C) MTTR benchmarking. Establish MTTR benchmarks from ISAC AI incident data exchanges, BSIMM-style observational data on AI/HAI incident response at comparable organizations, MITRE ATLAS practitioner community data, and peer roundtables (CISO and AI-safety practitioner communities). Publish a quarterly MTTR benchmark brief to the program sponsor: MTTR per incident class vs. benchmark (prompt-injection, agent rogue-action, training-data-leakage, shadow-AI emergence, RAG poisoning, model-swap rollback); MTTR per tier vs. benchmark; delta trend (improving, stable, degrading); and an investment driver, where MTTR is above benchmark, root-cause mapped to a specific practice gap (missing detection, unclear playbook, on-call latency) with a budget-linked improvement proposal.

Outcome Metrics (L3).

Success Criteria.

• ≥4 ISAC contributions per year, ≥2 AVID entries per year, ≥1 ATLAS TA0014 contribution per year; all maintained and tracked for external adoption.
• ≥3 pre-authorized automated containment actions live, vetted, producing 100% audit records plus human-review tickets; pre-authorization policy reviewed quarterly with zero unauthorized executions.
• Quarterly MTTR benchmark brief published; Critical-tier MTTR at or below benchmark for ≥3 of 6 incident classes; deltas linked to investment proposals.

Common Pitfalls

Level 1. - "Single backlog" exists in name only, ST failures stay in the CI dashboard, ML alerts route to a Slack channel, TA residual risks live in a spreadsheet; coverage stalls near 40% and the ≥95% target is never achieved. - Triage rubric severity anchors are generic CVSS-analog scoring without AI-specific axes, an agent executing a rogue tool invocation with real-world damage is triaged Medium because the rubric does not capture kill-switch failure or unauthorized account modification scope. - GDPR Art. 33 72-hour clock is tracked informally, a prompt-injection incident involving PII lands on a Friday evening, the clock starts, no named owner confirms the start time, the SLA slips before anyone documents the awareness event. - Post-incident reviews are completed but outputs file into a document nobody downstream reads, SA, SR, EG, and ML never update; the same incident class recurs with the same root cause six months later.

Level 2. - Critical-tier activation criteria are vague, incidents that qualify for full-team plus executive activation stay in the standard queue until the deployer-duty owner escalates; the ≤1-hour MTTA SLA is already missed by the time the right people engage. - Post-incident review auto-flow integration is wired but downstream practice backlogs treat the auto-created tickets as nominal, the SR team closes the ticket as "acknowledged" without updating the requirements pack; the feedback loop produces no change. - Cross-domain coordination protocol exists on paper but no IC is pre-designated, the first cross-domain incident produces ownership confusion, with Vendors-domain IM and Software-domain IM each waiting for the other to take the IC role. - 24/7 on-call coverage is implemented but the on-call briefing is stale, new Critical-tier artifacts are not in the briefing, and on-call responders do not know the kill-switch path for recently tiered artifacts.

Level 3. - ISAC participation limited to consuming feeds, contributions are absent, the organization is labeled a free-rider, and influence over AI incident taxonomy standards diminishes as the feed quality degrades without reciprocal intelligence. - Pre-authorized automated containment fires on a Critical-tier artifact because the confidence threshold was set too loosely, a false positive executes a kill-switch on a production agent handling customer sessions because the pre-authorization policy had no Critical-tier exception check. - MTTR benchmark brief cites benchmarks from organizations with fundamentally different AI/HAI portfolio scale or risk profiles, "we are at benchmark" is true but the benchmark set was chosen to flatter rather than stretch. - Automated containment produces audit records that are technically complete but lack the narrative context needed for post-incident root-cause review, humans reviewing automated-action logs cannot reconstruct what the detection saw or why the threshold triggered.

Practice Maturity Questions

Level 1. 1. Is a single AI/HAI software issue backlog operating with standardized metadata (source, affected artifact linked to SM inventory, severity rubric anchored to AI-specific axes, owner, SLA, regulatory flag, evidence link) capturing ≥95% of issues from all source practices (TA, SR, DR, IR, ST, ML, external)? Evidence: backlog audit cross-referenced against per-practice source queues for the last 90 days. 2. Is the AI-specific incident playbook published with the six named incident classes (prompt-injection, agent rogue-action, training-data-leakage, silent model-swap, RAG poisoning, shadow-AI emergence), each with pre-assigned roles, containment plays (kill-switch, model rollback, tool-revoke, retrieval-source disable, egress-block, prompt-template rollback), evidence-capture steps, and SLA targets, and has each class been exercised in at least one tabletop in the last 12 months? Evidence: published playbook; tabletop exercise records covering all six classes. 3. Is the regulatory SLA tracker live covering GDPR Art. 33 (72h), EU AI Act Art. 73, HIPAA (60d), NYDFS Part 500 (72h), and sector-specific obligations with 100% adherence in the last 90 days, and does every Critical/blocker incident produce a post-incident review within 14 days with named update outputs to SA, SR, EG, and ML? Evidence: SLA tracker export; post-incident review records with downstream-practice update tickets.

Level 2. 1. Is a tier-calibrated incident playbook operational with Critical-tier MTTA ≤1 hour and MTTC ≤4 hours, 24/7 on-call coverage with a documented rotation including a current Critical-tier artifact briefing, and tier-movement in the SM-Software inventory automatically triggering IM configuration updates within 14 days for Critical re-tiers? Evidence: IM telemetry showing MTTA/MTTC distributions; on-call rotation registry with briefing; auto-update log for tier-movement events. 2. Is a post-incident review auto-flow integration live routing Critical-tier review outputs to SA/SR/EG/ML practice backlogs with ≥90% of downstream practice owners responding within 14 days, and is sponsor review of output quality occurring quarterly? Evidence: integration telemetry; downstream backlog aging report; quarterly sponsor review notes. 3. Is a cross-domain coordination protocol published and used for 100% of multi-domain AI/HAI software incidents with named cross-domain contacts for Vendors, Infrastructure, and Processes verified quarterly, a single IC from the primary impacted domain, and joint post-incident reviews spanning all affected domains? Evidence: published protocol; cross-domain contact registry with quarterly verification log; joint post-incident review records.

Level 3. 1. Does the program contribute ≥4 anonymized AI incident-classification entries per year to sector ISACs, ≥2 entries per year to AVID, and ≥1 contribution per year to MITRE ATLAS TA0014 Impact tactic documentation with all contributions maintained current, legally vetted, and tracked for external adoption? Evidence: contribution log with submission dates and upstream adoption references. 2. Are ≥3 pre-authorized automated containment actions live (kill-switch, egress-block, retrieval-source disable, or tool-revoke classes), vetted by Privacy/Legal and the executive sponsor, producing 100% audit records plus human-review tickets, with quarterly policy review and zero unauthorized executions? Evidence: pre-authorization policy; automation execution log with audit records and human-review tickets; quarterly review minutes. 3. Is a quarterly MTTR benchmark brief published to the sponsor comparing MTTR per incident class and per tier against ISAC-sourced and peer-sourced benchmarks with Critical-tier MTTR at or below benchmark for ≥3 of 6 incident classes and deltas above benchmark linked to specific practice gaps and investment proposals? Evidence: quarterly benchmark briefs for the last 12 months with benchmark sources and investment-proposal references.

25. Monitoring & Logging (ML)

Practice Overview

Objective: Establish the logging baseline per AI/HAI software archetype, operate a small high-signal detection set targeted at the top threats from TA-Software, and produce the evidence trail that proves EU AI Act deployer duties, GDPR processor obligations, and ISO/IEC 42001 AIMS requirements on demand inside a published SLA.

Description: ML-Software captures the signals produced by every AI/HAI software artifact the organization builds, LLM-integrated applications, autonomous agents, RAG pipelines, fine-tuning and training workloads, evaluation harnesses, model-serving services, and classical ML models in product surfaces. For each archetype it specifies the exact events to capture (prompt/completion, tool-call, retrieval, fine-tune run, eval scores, model-version transitions, agent-loop steps, admin-audit, identity), the retention window required to satisfy the longest applicable regulation (EU AI Act Art. 12 high-risk-system logs ≥6 months; GDPR Art. 30 records-of-processing per data-class and processing purpose; HIPAA PHI ≥6 years; sector-specific where applicable), and the export path that supports auditor review. On top of the logging baseline it operates a bounded, purposeful detection set, each detection tied to a TA-Software archetype threat, each with a named owner, a defined query, an SLA, and a tuning record. The full corpus is the primary evidence artifact for PC-Software's compliance map: EU AI Act Art. 26 deployer duties, GDPR Art. 28 processor obligations, and ISO/IEC 42001 AIMS operational evidence.

Context: Logging AI/HAI software is not the same as logging classic web services. A prompt/completion event requires request-id correlation, model version, classification label, and latency alongside text content or a hash. A tool-call event from an agent must carry the tool name, arguments, return value, principal, and success/fail status, not just an HTTP status code. Training-job events must capture data-source lineage and eval-gate results to support model-promotion audit. None of this exists by default in standard APM or SIEM tooling unless someone has explicitly instrumented the archetype's event schema. ML-Software makes that schema explicit, per archetype, from day one, so the organization is not reconstructing an evidence trail from incomplete telemetry the first time a regulator or incident demands it. ML-Software is also the upstream feed for IM-Software: detections route directly to the unified backlog, and post-incident review outputs return to ML as detection-update requests.

Maturity Level 1

Objective: Establish the per-archetype logging baseline, operate a small high-signal detection set targeting the top TA-Software threats and HAI TTPs, and produce an on-demand evidence trail satisfying EU AI Act Art. 12, GDPR Art. 30, and ISO/IEC 42001 AIMS within a published SLA.

Activities.

A) Establish the per-archetype logging baseline. Define and instrument the minimum event schema for each archetype in the SM-Software inventory. Every event carries a request-id / correlation-id, principal (user or service account), timestamp, archetype tag, and the archetype-specific fields. LLM-integrated app: prompt event (text or hash, classification label, retrieval sources if RAG-backed, model + version), completion event (text or hash, model + version, input/output tokens, latency, error code), guardrail-decision event, rate-limit / abuse-detection event. AI agent: all LLM-app events plus tool-call event (tool name, arguments or argument hash for sensitive parameters, return value or hash, principal, success/fail, latency), agent-goal vs. action delta event, HITL-gate event (gate triggered, approver identity, decision, timestamp), kill-switch event (trigger source, timestamp, agent session scope, invoking principal), and agent-loop step event capturing each reflection/iteration in long-running sessions. RAG pipeline: retrieval event (sources retrieved with document IDs and classification labels, source provenance, tenant-id, query-id correlated to prompt event), injection-defense decision event. Fine-tuning / training workload: training-job event (job-id, data-source list with classification labels and consent-basis flags, model-output identifier, final eval-results summary, training-job duration), model-promotion event. Eval harness: eval-run event (corpus version, model version, result, regression-delta vs. baseline). Model-serving service: version-deployment event, canary-decision event, rollback event, model-version transition event. Classical ML: inference event (model-id, feature-set version, prediction, confidence, principal), drift-detection event. Admin-audit events across all archetypes capture config changes, secret rotations, IAM changes, model-registry changes (promotion / deprecation / deletion), tool-list changes for agents, kill-switch arm/disarm events, and on-call hand-off records. Identity events capture SSO sign-ins to model registry consoles, LLM provider admin consoles, CI/CD, and code repositories, plus service-principal token use against LLM provider APIs. Retention meets or exceeds the longest applicable requirement; where multiple windows apply the longest governs. Export path (JSON or structured CSV) tested at least annually; on-demand pull SLA ≤24 hours for evidence requests. Admin-audit and deployer-duty evidence tiers use write-once or append-only storage with access-control separation between application teams and log-store administrators. PII scrubbing applied per SR-Software data-boundary requirements before logs reach long-term storage.

B) Operate a small high-signal detection set. L1 target ≤12 detections, each tied to a TA-Software archetype threat and to at least one HAI TTP tag (EA / AGH / TM / RA) or ATLAS tactic, each with named owner, detection query, SLA (time-to-IM-ticket), and last-tuned date. Core detections: AGH detection (multi-turn agent goal drift, agent goal declared at session start vs. tool invocations across turns, delta above threshold); prompt-injection success detection (output exfiltration patterns matching known exfiltration signatures, credential patterns, or canary strings on outbound completions); tool-scope-violation detection (TM/EA, agent attempted a tool invocation with a tool or argument outside the published allowlist); training-data-leakage canary detection (canary string injected into the training corpus at a known position emitted verbatim in a completion); shadow-AI emergence detection (new outbound flow to an unsanctioned LLM provider domain from a service not in the SM-Software inventory); kill-switch-not-triggered detection (incident-state flag active for an agent artifact with no kill-switch event within the declared response SLA); HITL-gate-bypass attempt detection (HITL-gate event shows repeated decline followed by an attempt to invoke the same destructive tool in the same session without a new gate event); silent model-family-swap detection (model-version field on completions diverges from the approved model version); RAG retrieval-source poisoning detection (injection-defense decision event flags retrieved content with a specific source); rate-spike / token-spend anomaly detection; admin-key anomaly detection; no-train setting change detection. Each detection routes to the IM-Software backlog on fire; median detection-to-ticket time ≤1 hour for Critical-tier artifacts; false-positive rate tracked per detection with monthly tuning review.

C) Produce and drill the deployer-duty evidence trail. ML-Software is the primary evidence source for PC-Software's compliance map. Wire the log store to the compliance requirements. EU AI Act Art. 26 deployer duties and Art. 12 high-risk-system logging: for every artifact assessed as Annex III high-risk or carrying a customer-facing decision-affecting output, confirm that prompt/completion, tool-call, and admin-audit events are captured and retained at the required window; produce a deployer-duty evidence view (log record + retention attestation + export test result) for each such artifact. GDPR Art. 28 processor obligations and Art. 30 records of processing: for every artifact processing personal data, the prompt/completion log entries with principal identity, data-class tag, and purpose label constitute the operational records; link log-store retention policy to the Art. 30 record for each artifact. ISO/IEC 42001 AIMS: training-job events, model-promotion events, eval-run events, and admin-audit events constitute the AIMS operational records; identify gaps and open IM-Software findings for any archetype not yet emitting these events. Quarterly deployer-duty drill: pull the deployer-duty evidence package for one randomly selected production AI/HAI software artifact per archetype within the published SLA (≤24 hours from request to assembled package); record drill results; gaps route to IM-Software.

Outcome Metrics (L1).

Success Criteria.

• Per-archetype logging baseline published and instrumented for ≥90% of production AI/HAI software artifacts; PII scrubbing applied before long-term storage.
• ≤12-detection high-signal set live, each with owner, detection query, SLA, ATLAS-tactic or HAI-TTP tag, and monthly tuning record; FP rate tracked per detection.
• Retention meets the longest applicable regulatory window for every production artifact; export path tested at least annually.
• EU AI Act Art. 26 / Art. 12, GDPR Art. 28 / Art. 30, and ISO/IEC 42001 AIMS evidence-trail wiring documented; quarterly deployer-duty drill executed inside the ≤24-hour SLA.

Maturity Level 2

Objective: Calibrate logging depth and detection set to the SM-Software L2 risk-tier rubric; integrate ML feeds into the SIEM for cross-artifact correlation; operate a quarterly detection-tuning loop fed by IM-Software post-incident reviews and ST-Software findings; establish anomaly-detection baselines for Critical and High-tier artifacts.

Activities.

A) Tier-calibrated logging depth. Apply the SM-Software L2 tier-treatment matrix to logging configuration. Critical: full prompt text and completion text (not hashes) retained for the longest regulatory window; full tool-call argument and return corpora retained; admin-audit and identity events at maximum fidelity; all archetype detections tuned to the artifact; per-tenant isolation enforced at the log store. High: full prompt/completion text retained; tool-call events at full fidelity; standard admin-audit and identity events; core detections active. Medium: prompt/completion hashes retained for the regulatory window; standard admin-audit; shadow-AI emergence plus baseline detections active. Low: baseline logging schema only; shadow-AI emergence detection only. For every Critical-tier artifact the ML log store is the primary source for PC-Software's compliance evidence bundle, completing inside the PC L2 staleness threshold (≤30 days).

B) SIEM integration and cross-artifact correlation. Ingest all tier-appropriate ML log feeds into the SIEM. Author and maintain at least three cross-artifact correlation rules: multi-artifact AGH correlation (same principal appears in HITL-gate-bypass attempts on two or more agent artifacts in the same session window, fires a unified incident); training-to-inference leakage chain (a canary-string detection in a completion correlates to a training-data-leakage canary detection from the same model-id's training-job event, escalates to Critical regardless of artifact tier); shadow-AI emergence plus identity pivot (a shadow-AI emergence detection correlates to an unusual SSO sign-in to an LLM provider admin console from the same service-principal in the same time window). Cross-artifact correlation alerts route to IM-Software at the tier of the highest-tier artifact involved with links to component-artifact findings to preserve triage context.

C) Detection tuning loop and anomaly baselines. Operate a quarterly detection review cycle. IM-Software post-incident reviews that touch a logging or detection gap generate detection-update requests (new detection, tuned query, or retired false-positive rule). ST-Software CI corpus failures (prompt-injection regression, data-egress canary, kill-switch test) that are not caught by the current detection set generate detection-gap findings routed to ML-Software. External advisory updates (MITRE ATLAS new techniques, OWASP LLM Top 10 updates, AVID advisories) are assessed quarterly; each applicable update either adds a candidate detection or updates an existing detection's query. Monthly anomaly-baseline refresh for Critical and High-tier artifacts: normal behavior baseline (prompt volume, tool-call patterns, completion-latency distribution, tool-call argument distributions, time-of-day patterns) refreshed from the previous 30-day window; anomaly threshold auto-tunes to maintain target FP rate. Detections that have not fired a true positive in 90 days or exceed a 20% FP rate are reviewed for retirement at the quarterly cycle. Retention-tier calibration reconciles with SM inventory tier changes within 14 days (Critical re-tier) or 30 days (other tiers).

Outcome Metrics (L2).

Success Criteria.

• Tier-calibrated logging depth applied to 100% of SM inventory; Critical-tier full corpus retention confirmed; calibration auto-updated on re-tier within 14 days.
• SIEM integration live with ≥3 cross-artifact correlation rules active.
• Quarterly detection tuning loop operating from IM and ST feedback with ≥1 net change per cycle.
• ≥90% of Critical/High-tier artifacts with anomaly-detection baselines refreshed monthly; FP rate tracked and trending down.
• ML logging-baseline validation element fresh (≤30 days) for all Critical-tier artifacts in PC-Software compliance evidence bundles.

Maturity Level 3

Objective: Express detections as code deployed through CI/CD; apply ML-driven anomaly detection on prompt/completion and tool-call corpora; contribute anonymized detection signatures and telemetry schemas to OpenTelemetry AI, MITRE ATLAS, OWASP LLM/Agentic, and sector ISACs.

Activities.

A) Detection-as-code. Every detection in the set is a version-controlled, tested artifact in source control with detection query plus metadata (owner, SLA, ATLAS-tactic tag, HAI-TTP tag, FP threshold, last-test-result). A detection CI/CD pipeline triggers a test suite (unit tests over synthetic log data, integration tests against a log replay environment) before production deployment. Detection deployment runs through the same change-management pipeline as AI/HAI software; detection changes are reviewed, not applied ad hoc in the SIEM console. Detection coverage is automatically checked on SM inventory change events: when a new archetype is registered or an artifact is re-tiered to Critical, the automation verifies the required detection set is active and opens a gap finding within 24 hours if not.

B) ML-driven anomaly detection on AI/HAI corpora. Apply unsupervised and semi-supervised anomaly models to the prompt/completion and tool-call corpora for Critical and High-tier artifacts. Prompt-sequence anomaly identifies sessions whose prompt-sequence distribution is a statistical outlier from normal user sessions (attacker-probing signatures, jailbreak-attempt escalation patterns, multi-turn goal-hijack sequences). Completion-distribution anomaly identifies completions whose embedding distribution shifts from baseline on a rolling window (potential output-integrity regression or prompt-injection influence). Tool-call argument anomaly identifies argument combinations that have never appeared in normal sessions and fall outside the declared tool-scope range (novel TM TTP variants). Anomaly model outputs feed the same detection-to-IM-ticket pipeline as rule-based detections; anomaly severity is tagged to the artifact's tier. Anomaly models retrained monthly; retraining produces a new version in the ML-Software model registry with lineage tracking equivalent to production AI/HAI software. Retraining excludes attacker-session logs from past incidents to avoid baseline poisoning.

C) Contribute detection signatures and telemetry schemas. Contribute semantic conventions for AI/HAI event types (prompt/completion spans, tool-call spans, agent-session traces, training-job events) to the OpenTelemetry AI workgroup in OTel-compatible format. For each detection that corresponds to an ATLAS tactic/technique, propose or validate an AML.M00xx mitigation entry (detection-based mitigation type); priority covers TA0001 Reconnaissance (shadow-AI emergence), TA0008 Defense Evasion (HITL-gate bypass), TA0013 Exfiltration (canary detection), and TA0014 Impact (kill-switch coverage). Contribute detection-pattern examples from production telemetry to OWASP LLM Top 10 or Agentic Top 10 community contributions; target at least one detection pattern per cycle. Share anonymized, generalized detection signatures with sector ISACs (FS-ISAC, H-ISAC, IT-ISAC AI working groups); target ≥12 signatures per year; signatures must be implementable by partner organizations without significant adaptation. Target ≥2 telemetry-standard contributions per year and ≥12 ISAC detection signatures per year; all contributions anonymized, legally vetted, and maintained, not point-in-time submissions.

Outcome Metrics (L3).

Success Criteria.

• ≥90% of detections expressed as version-controlled, CI/CD-deployed artifacts; detection coverage auto-verified for 100% of new or re-tiered SM inventory entries within 24 hours.
• ≥90% of Critical/High-tier artifacts running ML-driven anomaly detection on prompt/completion and tool-call corpora with monthly retraining and lineage tracking.
• ≥2 telemetry-standard contributions per year to OpenTelemetry AI workgroup or equivalent; ≥12 anonymized detection signatures per year to sector ISACs; ≥2 ATLAS AML.M00xx mitigation entries proposed or validated.

Common Pitfalls

Level 1. - Logging baseline defined at the archetype level but actual production artifacts never audited against it, gaps accumulate inside the SM inventory without appearing in any backlog. - Tool-call events for agents logged at the HTTP level (method plus status) but not at the argument level, HITL-gate-bypass and tool-scope-violation detections are architecturally impossible without argument-level logging. - Detection set grows without governance, new detections are added at every incident but none are ever retired, and the team spends more time triaging false positives than investigating real signals. - Retention meets GDPR Art. 30 but not EU AI Act Art. 12 high-risk-system windows, evidence requests for Annex III-adjacent artifacts cannot be satisfied because the relevant log tier was retained for only 30 days.

Level 2. - Tier-calibrated logging configured at deployment time but not maintained, when an artifact is re-tiered Medium → Critical the logging depth is not updated, and full corpora are absent when the first Critical-tier incident fires. - SIEM correlation rules built once and never validated, a rule that has not fired in 90 days may be broken (log format changed, query syntax stale) rather than evidence that no correlatable events occurred. - Anomaly baselines established at onboarding and never refreshed, behavioral drift in normal usage makes the baseline stale and FP rates spike over the following quarters. - Detection tuning loop exists on paper but IM and ST feedback never actually feeds into the review cycle, the same false-positive detections remain in the set for years because the quarterly process has no dedicated owner.

Level 3. - Detection-as-code pipeline deployed but detection tests use synthetic data that does not resemble production log patterns, tests pass in CI and detections fail silently in production. - ML-driven anomaly models retrained on the full log corpus including attacker-session logs from past incidents, poisoned baseline; the model learns to treat past attack patterns as normal. - Contributed telemetry schemas published as point-in-time artifacts and then diverge from internal practice, external adopters build against v1.0 while the org operates v1.3 internally and trust erodes. - ISAC detection signatures generalized to the point of uselessness, partner organizations cannot implement them without reconstructing the context removed for anonymization.

Practice Maturity Questions

Level 1. 1. Has a per-archetype logging baseline been published specifying the minimum event schema, fields, retention window, and export path for each AI/HAI software archetype in the SM-Software inventory (LLM-integrated app, agent, RAG, fine-tune/training, eval harness, model-serving service, classical ML), and has compliance of each production artifact been measured against it within the last quarter with ≥90% meeting the baseline? Evidence: published baseline; logging configuration audit cross-referenced against SM inventory. 2. Is a high-signal detection set of ≤12 detections active, each with named owner, detection query, SLA, ATLAS-tactic or HAI-TTP tag, and last-tuned date, covering AGH detection, prompt-injection success, tool-scope violation, training-data-leakage canary, shadow-AI emergence, kill-switch-not-triggered, HITL-gate bypass, silent model-family-swap, RAG retrieval-source poisoning, rate-spike / token-spend anomaly, admin-key anomaly, and no-train setting change, with FP rates tracked per detection and monthly tuning reviews occurring? Evidence: detection registry export; monthly tuning-review records; FP-rate trend per detection. 3. Has the evidence trail for EU AI Act Art. 26 / Art. 12, GDPR Art. 28 / Art. 30, and ISO/IEC 42001 AIMS been wired to the ML-Software log store, and has a quarterly deployer-duty drill confirmed that the evidence package for a randomly selected production artifact can be assembled within the ≤24-hour SLA? Evidence: documented wiring of log store to compliance requirements; quarterly drill records for the last 12 months with assembly times.

Level 2. 1. Is tier-calibrated logging depth applied per the SM-Software L2 tier-treatment matrix (Critical-tier full prompt/completion and tool-call corpora at longest regulatory window; Low-tier baseline only), with this calibration automatically updated within 14 days when an artifact is re-tiered to Critical? Evidence: log-store retention audit × SM inventory tier assignments; re-tier auto-update log. 2. Is the SIEM ingesting ML-Software log feeds with ≥3 cross-artifact correlation rules active (covering multi-artifact AGH, training-to-inference leakage chain, and shadow-AI emergence plus identity pivot), and is a quarterly detection tuning cycle operating from IM-Software post-incident and ST-Software finding inputs? Evidence: SIEM rule registry; correlation-alert sample; quarterly detection change log. 3. Are ≥90% of Critical/High-tier artifacts running anomaly-detection baselines refreshed monthly with FP rates tracked and trending down, and is the ML logging-baseline validation element completing inside the ≤30-day staleness threshold for all Critical-tier artifacts in PC-Software compliance evidence bundles? Evidence: detection telemetry showing baseline-refresh cadence; FP-rate trend; PC-Software compliance evidence bundle freshness report.

Level 3. 1. Are ≥90% of detections expressed as version-controlled, CI/CD-deployed code artifacts with automated test coverage against realistic synthetic log data, and is detection coverage auto-verified for 100% of new or re-tiered SM inventory entries within 24 hours of the inventory change event? Evidence: detection registry × source control; CI test results; automation telemetry for inventory-change events. 2. Are ≥90% of Critical/High-tier artifacts running ML-driven anomaly detection on prompt/completion and tool-call corpora with anomaly models retrained monthly on production log data (excluding attacker-session logs from past incidents), model versions tracked in the ML-Software model registry, and anomaly-model alerts feeding the IM-Software incident backlog through the same detection-to-ticket pipeline as rule-based detections? Evidence: anomaly model registry with monthly retraining records; lineage-tracking export; IM backlog showing anomaly-sourced tickets. 3. Has the program contributed ≥2 telemetry-standard artifacts per year to the OpenTelemetry AI workgroup or equivalent, ≥12 anonymized detection signatures per year to sector ISACs, and ≥2 MITRE ATLAS AML.M00xx detection-mitigation entries proposed or validated, with contributions maintained current and external adoption tracked? Evidence: contribution log with submission dates, upstream adoption references, and maintenance records.

Part IV, Maturity Assessment Workbook

26. How the assessment works

Scope. A single assessment covers all 12 practices in the Software domain, with 3 questions per maturity level per practice, 108 questions total. The assessment measures the organization's ability to secure the AI/HAI software it builds, LLM-integrated applications, autonomous agents, RAG pipelines, fine-tuning and training workloads, evaluation harnesses, model-serving services, and classical ML models integrated into product surfaces.

Cumulative levels. Maturity levels are cumulative. A practice cannot be at Level 2 unless it is at Level 1; it cannot be at Level 3 unless it is at Level 2. Score Level 1 questions before answering Level 2 questions for the same practice. This is not optional, the gate is by design.

Answers. Each question accepts one of three answers: Yes (fully implemented, evidence-backed, sustained over time, requires an artifact, telemetry pull, or process record the assessor has actually seen), Partial (partially implemented, or implemented but not sustained, or evidence is incomplete, counts as half credit), or No (not implemented, or implemented inconsistently to the point that no evidence supports it).

Evidence. Every "Yes" requires a citation in the Evidence box. "It is in the wiki" is not evidence. "Wiki page X dated Y, screenshot of admin console Z, LMS completion export from HR system" is.

Honesty. The assessment is for the program, not for the assessor. A "No" honestly recorded is more useful than a "Yes" that does not survive auditor scrutiny.

Cadence. Run the full assessment at least annually. Run a Level 1 self-check quarterly during the first year of program operation.

Roles. The assessment is led by the AI/HAI Software Assurance program lead (typically the AppSec lead or AI Security lead) working with the cross-functional working group that includes CTO and CISO co-sponsorship, AI/ML engineering leads, MLOps and Platform engineering, and Product owners. The assessor should be independent of day-to-day program operations, a peer assessor from another team or function works well; an external assessor works better. The program lead should not assess their own program. Working group members from Engineering and Product provide evidence artifacts; Legal and Privacy provide compliance-map evidence; ML platform provides model-registry and CI/CD telemetry.

Scope boundary. This assessment covers only Software-domain practices. The same 12 practices applied to the Data, Infrastructure, Vendors, Processes, and Endpoints domains are assessed in their own handbooks. Do not conflate Software-domain answers with answers about what vendors provide, what infrastructure hosts, or what data flows contain.

27. Scoring methodology

Two scoring approaches are supported. Use the simplified scoring for self-assessments and quarterly check-ins. Use the precise scoring for formal audits and external benchmarking.

Simplified scoring (recommended for self-assessment)

For each practice:

Level 1 achieved (all 3 Level 1 questions = Yes): 1.0 point
Level 2 achieved (all 3 Level 2 questions = Yes, AND Level 1 achieved): +1.0 (total 2.0)
Level 3 achieved (all 3 Level 3 questions = Yes, AND Level 2 achieved): +1.0 (total 3.0)

A "Partial" answer counts as half toward the level, but the level is only achieved when all three questions are at full Yes. Partial credit shows up in the precise score.

Precise scoring (recommended for formal audits)

For each practice, with Y = Yes (1.0), P = Partial (0.5), N = No (0):

L1_score = (sum of L1 answers) / 3
L2_score = (sum of L2 answers) / 3 × L1_score
L3_score = (sum of L3 answers) / 3 × L2_score

Practice Score = L1_score + L2_score + L3_score    (max 3.0)

The L2 and L3 multipliers enforce the cumulative rule, a practice cannot earn full L2 credit if L1 is incomplete.

Domain rollup.

Domain Maturity = (sum of all 12 Practice Scores) / 12    (max 3.0)

Maturity bands.

• 0.0 – 0.9, Ad-hoc. No AI/HAI Software Assurance program in operational use; AI/HAI software ships without governance.
• 1.0 – 1.9, Foundational. L1 in place across most practices; inventory and intake gate operational; some L2 progress.
• 2.0 – 2.9, Comprehensive. L2 calibrated by risk tier across most practices; continuous validation for Critical/High artifacts; some L3 contributions.
• 3.0, Industry-Leading. L3 automation, benchmarking, and contribution sustained across all practices.

Worked example, precise scoring

Suppose the TA-Software practice scores as follows:

L1_score = 2.5 / 3 = 0.833
L2_score = (1.5 / 3) × 0.833 = 0.500 × 0.833 = 0.417
L3_score = (0.0 / 3) × 0.417 = 0.0

TA Practice Score = 0.833 + 0.417 + 0.0 = 1.25 / 3.0

Interpretation: the practice is solidly in the Foundational band, with a partial L2 story. The cumulative multiplier correctly suppresses L2 credit because L1 is not complete.

28. The questionnaire

The 108 questions follow. Each question has the same workbook layout: question text, answer field, evidence box, and notes box. Practice and level headings are repeated so the workbook is usable as a printout or as a standalone assessment instrument.

28.1 Strategy & Metrics (SM)

SM Level 1.

Q-SM-L1-1. Is there a published AI/HAI Software Assurance program charter with a named executive sponsor (CISO co-sponsored by CTO / Head of Engineering / Chief AI Officer), a cross-functional working group (Security, Engineering, Data/ML, Privacy/Legal, Product, Platform/SRE, application-architect reviewer), and clear decision rights for approval, block, exception, and go-live?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SM-L1-2. Does a single AI/HAI software inventory exist, seeded from source-code signals, dependency manifests, CI/CD telemetry, runtime-egress logs, model registries, cloud-spend, and self-attestation, covering all in-scope archetypes (LLM-integrated app, agent, RAG, fine-tune/training, eval harness, model-serving service, classical ML), with ≥90% coverage of discovered artifacts within 12 months?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SM-L1-3. Are the L1 outcome metrics baselined and reported quarterly to the executive sponsor, inventory coverage (≥90%), shadow-AI-in-engineering ratio (≤15% and trending down), AI AUP attestation coverage (≥95% of engineering headcount), AI/HAI artifacts in production with a named owning team (100%), and known data-exposure events from AI/HAI software?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

SM Level 2.

Q-SM-L2-1. Is every AI/HAI software artifact in the inventory assigned a risk tier based on an auditable rubric covering data sensitivity, decision-affecting use (EU AI Act Annex III / GDPR Art. 22), agentic capability, user exposure, training-data posture, production-load-bearing criticality, and concentration?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SM-L2-2. Is there a published tier-treatment matrix driving differential program intensity across PC, TA, SR, SA, DR, IR, ST, EH, ML, and IM, with ≥95% of Critical-tier artifacts receiving full-scope treatment in the last 12 months?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SM-L2-3. Does the quarterly shadow AI scoreboard report per tier and per archetype, with Critical-tier unsanctioned AI in production explicitly tracked at zero, and does tier-movement get logged with rationale and reviewed by the program sponsor?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

SM Level 3.

Q-SM-L3-1. Does inventory and tier assignment auto-update from live build/deploy/runtime signals (CI/CD, model registries, dependency manifests, runtime egress, prompt/completion telemetry, intake, self-attestation) with a published data-quality SLO, and is ≥80% of curation handled automatically with exception-based human review?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SM-L3-2. Does the program publish a semi-annual external-benchmarking brief comparing the program against at least five peer-comparable metrics via OWASP SAMM AI / OpenSSF / MITRE ATLAS / sector ISACs, and does it drive program investment decisions?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SM-L3-3. Does the program contribute at least four substantive, anonymized artifacts per year to the AI/HAI software assurance ecosystem (MITRE ATLAS, OWASP LLM/Agentic Top 10, NIST AI RMF, AVID, OpenSSF AI, sector ISACs), and does the executive/board ROI narrative cite external benchmarks?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.2 Policy & Compliance (PC)

PC Level 1.

Q-PC-L1-1. Have the three priority AI/HAI software engineering policies been published and formally approved, AI Engineering Standards, AI Acceptable Use & Engineering Standards, and AI Software Intake / Go-Live Gate, with archetype-specific controls, data-class restrictions, agentic scope constraints, output-integrity-critical designations, and a deployer-duty owner requirement, and is there a one-page priority compliance map tracing each requirement (EU AI Act Art. 26/50/Annex III/Art. 9/Art. 15, GDPR Art. 22/32/33/44–49, NIST AI RMF, ISO/IEC 42001, SOC 2 CC9.2, sector-specific) to the specific policy that carries it?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-PC-L1-2. Is the go-live gate operational with a per-archetype artifacts checklist, a published intake SLA (triage ≤5 BD; provisional approval ≤10 BD for Low-tier), and an amnesty path, and does ≥85% of AI/HAI software reaching production in the last 12 months have a gate record (100% for Critical/High-tier)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-PC-L1-3. Are ≥95% of engineering headcount covered by a current-year AI AUP acknowledgment, and does every customer-facing or decision-affecting AI/HAI artifact in production have a named deployer-duty owner logged in the SM-Software inventory?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

PC Level 2.

Q-PC-L2-1. Have the three priority policies been extended with tier-specific addenda, and do Critical artifacts carry explicit executive plus privacy-officer sign-off at go-live with a live compliance evidence bundle covering TA snapshot, SR REM, SA pattern confirmation, DR decision, IR attestation, ST evidence, ML logging-baseline, deployer-duty record, and foundation-model provider attestation?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-PC-L2-2. Is a compliance evidence bundle continuously maintained for every Critical/High artifact with staleness inside tier-specific targets, and can a regulatory or auditor inquiry be satisfied with provenance-complete artifacts within 5 business days?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-PC-L2-3. Is an exception register operated with named owners, compensating controls, and expiry dates, reviewed monthly, with Critical-tier missing go-live artifacts treated as blocking findings (no amnesty), and sector-specific evidence bundles (HIPAA / PCI-DSS / FDA / FINRA as applicable) complete for in-scope artifacts?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

PC Level 3.

Q-PC-L3-1. Does a continuous attestation pipeline auto-update evidence bundles from CI/CD events, model-registry promotions, and runtime signals, with attestation currency ≤24 hours latency and ≤3 BD on-demand evidence pack generation, and is ≥99% of Critical/High artifacts continuously attested?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-PC-L3-2. Does the program operate a quarterly, telemetry-driven policy-refresh cycle (ML detection trends + IM incident learnings + regulatory-motion tracker + tier-movement data) with a versioned changelog where 100% of changes are traceable to a named signal or regulatory update, and are EG-Software training materials updated within 30 days of any policy change?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-PC-L3-3. Does the program contribute at least two substantive public comments or standards artifacts per year on AI/HAI software policy topics (EU AI Act implementing guidance, GDPR EDPB AI guidance, NIST AI RMF Playbook, ISO/IEC 42001, sector regulators, or community standards bodies), with documented external recognition?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.3 Education & Guidance (EG)

EG Level 1.

Q-EG-L1-1. Have all engineers building or operating AI/HAI software completed a current-year AI-assurance literacy course covering the seven in-scope archetypes, the four HAI TTPs (EA / AGH / TM / RA) plus prompt injection / training-data leakage / output-integrity regression, the AI AUP rules, and the go-live gate intake process, with ≥95% completion and content updated within 30 days of any policy or archetype change?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EG-L1-2. Has the practitioner population (AppSec reviewers, AI/ML platform engineers, AI-feature architects, red-teamers) completed role-based training covering ATLAS tactics, OWASP LLM / Agentic Top 10, prompt injection patterns, agent goal hijack, tool misuse, training-data poisoning, output-integrity testing, and kill-switch design, with completion gated on intake-approval permissions and calibration drift ≤1 tier step and ≤2 TTP misclassifications per sample for two consecutive quarters?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EG-L1-3. Is a shadow-AI-in-engineering awareness campaign running with at least monthly content, a visible amnesty path linked from the AUP and intake form, and measurable attribution of intake submissions and amnesty disclosures to campaign channels, with disclosures rising in Q1–Q2 after launch then declining as the sanctioned-archetype program grows?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

EG Level 2.

Q-EG-L2-1. Is there a scenario library of ≥30 anonymized real intake cases powering practitioner training across the org's in-scope archetypes, with paired calibration exercises showing Critical-tier drift ≤1 tier step and ≤1 TTP misclassification per sample for two consecutive quarters?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EG-L2-2. Have product-line-specific engineering tracks (covering the relevant archetypes and SA reference patterns for mobile, web, ML platform, and backend as applicable) been delivered to ≥1 practitioner per Critical/High-tier artifact, with team-level training coverage tracked in the SM-Software inventory?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EG-L2-3. Are shadow-AI campaigns running on a seasonal, behavior-driven cadence with pre-set behavior targets and post-campaign measurement, with ≥70% of campaigns hitting their target, and is ≥80% of training content updated in the last 90 days?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

EG Level 3.

Q-EG-L3-1. Has the practitioner curriculum, anonymized scenario library, and reviewer rubric been published externally (CSA, OpenSSF AI, OWASP AI, or sector ISAC) with documented adoption, citations, forks, or direct acknowledgment, and do contributions loop back into internal content within 30 days?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EG-L3-2. Is a monthly live calibration cadence operating (anonymized intake from the live queue, independent reviewer scoring, drift reported to sponsor), with calibration results feeding the scenario library within 30 days, and do ≥50% of Critical-tier reviewers hold an external AI-assurance or AI-engineering credential where one exists?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EG-L3-3. Does the program contribute ≥2 substantive artifacts per year to industry AI-engineering certification or curriculum working groups, and ≥1 MITRE ATLAS TTP contribution or confirmation per year where novel observations exist in own-built AI/HAI software?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.4 Threat Assessment (TA)

TA Level 1.

Q-TA-L1-1. Are published, versioned threat models in place for all seven AI/HAI software archetypes (LLM-integrated app, agent, RAG, fine-tune/training, eval harness, model-serving service, classical ML), each mapping archetype-specific threats to HAI TTPs (EA/AGH/TM/RA), ATLAS tactic IDs, OWASP LLM/Agentic Top 10 references, and PC-Software compliance items, with a named library steward and a documented quarterly refresh cadence?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-TA-L1-2. Does every AI/HAI software artifact entering the SM inventory receive a threat snapshot (delivered within one business day of intake) that documents the applicable archetype(s), artifact-specific deltas (tool list, retrieval sources, data classes, output-integrity-critical paths), top-5 threats with HAI TTP tags and ATLAS tactic IDs, and gaps for SR/SA follow-up, with 100% of newly Sanctioned artifacts carrying a snapshot in the last 90 days?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-TA-L1-3. Is there a published shadow-AI-in-engineering threat view, reviewed by the program sponsor in the last 12 months, that documents entry vectors, elevated threat scenarios for unreviewed AI/HAI software artifacts, and the specific detections (from SM discovery sources) used to surface them?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

TA Level 2.

Q-TA-L2-1. Does every Critical-tier AI/HAI software artifact have a current-year per-artifact deep threat model (not an archetype snapshot) covering artifact-specific attack trees, an abuse-case catalog, deployer-duty mapping, and a full ATLAS tactic walk with technique-level specificity, with a semi-annual refresh cadence and change-driven updates on model swap, new tools, or scope change?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-TA-L2-2. Is external AI-security threat intel (MITRE ATLAS updates, AVID, OWASP LLM/Agentic Top 10 revisions, sector ISACs, academic adversarial-ML venues) integrated with a quarterly triage cadence and a documented change-log, with intel-to-library update ≤30 days on Critical-impact items?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-TA-L2-3. Does the program run a quarterly red-team-the-library exercise that probes an in-scope AI/HAI software artifact using only library threats and surfaces misses as library gaps, with every gap carrying a named owner and an expiry date, Critical gaps closing within 30 days, and the gap rate trending down quarter over quarter?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

TA Level 3.

Q-TA-L3-1. Does the threat library auto-update from telemetry (ML-Software detections, IM-Software incidents) and external feeds (ATLAS, AVID, OWASP, academic) via a human-curated auto-proposal pipeline, with ≥60% of changes auto-proposed, a ≤14-day lead time from signal to update, and a machine-readable change-log consumed by downstream SR and ST practices?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-TA-L3-2. Does the program contribute at least four substantive, evidence-backed technical artifacts per year to MITRE ATLAS / AVID / OWASP LLM/Agentic Top 10, with at least two externally recognized in published advisory or standard revisions?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-TA-L3-3. Are anonymized archetype threat models published under a permissive license with tracked peer-org adoption, and does the program host or co-host at least one industry tabletop per year (ATLAS practitioner table, OWASP AI chapter, or sector ISAC AI working group) tied to the library?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.5 Security Requirements (SR)

SR Level 1.

Q-SR-L1-1. Is there a published, versioned AI/HAI Software Requirements Pack containing a base set (≤20 requirements) plus seven per-archetype deltas, with every requirement tagged to at least one TA-Software archetype threat and one PC-Software priority-compliance item, and are reviewers selecting from the pack rather than drafting requirements per artifact at intake?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SR-L1-2. Do 100% of new AI/HAI software artifacts approved in the last 90 days have a completed Requirements-Evidence Map (REM) on file, with every applicable requirement marked Met / Met-with-compensating-control / Gap-accepted / Not-applicable, each Met row citing specific verifiable evidence, each Gap-accepted row naming a compensating control with owner and re-review date, and material-change triggers defined for when the REM must be re-reviewed?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SR-L1-3. Is the pack on a quarterly refresh cadence with a named owner, and are SA, DR, IR, and ST practices citing REM rows rather than independently re-deriving requirements from scratch?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

SR Level 2.

Q-SR-L2-1. Do 100% of pack requirements carry a quantitative or binary evidence condition, with every SLA (retention days, kill-switch response time, drift-detection threshold, secrets-scan result) and binary state (no-train toggle confirmed, tool allowlist verified, EU AI Act Art. 26 clause present) specified, and has all qualitative "reasonable" and "appropriate" language been removed from the pack?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SR-L2-2. Are ≥95% of Critical-tier REMs re-validated against observed reality (admin console, CI/CD, IR findings, ML monitoring) in the last 90 days, with validation deltas routed to IM-Software and no Critical-tier accepted gap aging beyond 60 days without documented escalation to the program sponsor?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SR-L2-3. Does 100% of Critical-tier artifacts carry a full EU AI Act Art. 26 deployer-duty checklist in the REM with verifiable evidence (not vendor assertion alone), and is the per-tier pack overlay enforced at SM intake, with Critical-tier artifacts receiving full depth and Low-tier artifacts receiving base pack only?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

SR Level 3.

Q-SR-L3-1. Is the AI/HAI Software Requirements Pack expressed in a machine-readable schema and enforced via CI/CD attestation at deploy time, with ≥80% of Critical-tier requirements having automated checks, zero Critical-tier artifacts deploying to production with a failing REM check, and the schema published under a permissive license with tracked external adoption?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SR-L3-2. Are ≥70% of REM evidence rows auto-validated via CI/CD signals, runtime monitoring (ML-Software), and admin-console API ingestion, with automation error-rate monitored and human review reserved for exceptions, novel clauses, and accepted-gap escalations?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SR-L3-3. Does the program contribute at least two substantive artifacts per year (machine-readable requirement schema, REM schema, model requirement clauses) to recognized standards bodies (OpenSSF AI, OWASP SAMM AI, NIST AI RMF Playbook, ISO AI security standards work), with contributions publicly documented and traceable to adoption?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.6 Secure Architecture (SA)

SA Level 1.

Q-SA-L1-1. Are seven reference patterns published, one per archetype (LLM-integrated app, agent, RAG, fine-tune/training, eval harness, model-serving service, classical ML), each with a labeled data-flow diagram, data-boundary definition, identity and auth model, logging spec, and explicit row-by-row mapping to SR-Software requirements and TA-Software threats with HAI TTP tags and applicable MITRE ATLAS mitigation IDs, accessible within one click of the SM inventory record?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SA-L1-2. Are 100% of LLM-integrated app and agent artifacts verified (via CI secrets-scanning, not only policy declaration) to route LLM provider credentials through a secrets vault, and is the anti-pattern catalog linked from the AI Acceptable Use Policy, the SM intake gate, and EG-Software training, with each entry tied to the real incident that generated it?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SA-L1-3. Is a repeat-deviation signal operational, such that three deviations in the same direction for the same archetype automatically queue a pattern-update review with SA ownership, and are ≥85% of active AI/HAI software artifacts in the SM inventory classified as "on pattern" or "deviation with review" with no silent deviations?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

SA Level 2.

Q-SA-L2-1. Are the five tier-conditional extended patterns (Critical overlay, High overlay, multi-region, multi-tenant, agent-platform) published as forkable IaC modules with conformance test suites, and are ≥80% of Critical and High-tier AI/HAI software artifacts running on IaC-encoded patterns as confirmed by the IaC and SM inventory registries?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SA-L2-2. Has the anti-pattern catalog been updated from ≥3 real IM-Software incidents in the last 12 months, with new entries surfaced at intake time rather than stored only in a reference document, and is conformance testing covering 100% of IaC-encoded artifact deployments with findings tracked to resolution?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SA-L2-3. Are 100% of Critical-tier artifacts carrying explicit EU AI Act Art. 9 and Art. 15 control mappings in the pattern documentation, and is the tier-treatment matrix from SM L2 reflected in the pattern variants (Critical artifacts get the Critical overlay, High artifacts get the High overlay, Medium/Low follow the base pattern)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

SA Level 3.

Q-SA-L3-1. Have ≥5 reference patterns been published as open artifacts under a recognized open license via at least one industry body, and have ≥2 of those patterns been cited or forked by recognized industry or sector bodies, with documented adoption evidence and internal practice aligned to the published version?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SA-L3-2. Have ≥2 MITRE ATLAS AML.M00xx mitigation entries been proposed or validated, traceable to specific SA-Software pattern controls aligned to ATLAS primary tactics TA0006 Persistence, TA0007 Privilege Escalation, and TA0008 Defense Evasion, and is there an active ATLAS practitioner engagement cadence?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SA-L3-3. Is there at least one documented reference to SA-Software patterns in a regulatory implementing-act, sector guidance document, or published standards text, and is the regulatory engagement calendar maintained with active items, target timelines, and evidence of substantive (not declaratory) participation?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.7 Design Review (DR)

DR Level 1.

Q-DR-L1-1. Is there a published, versioned per-archetype AI/HAI Software Design Checklist, one per SM-Software archetype, traceable to the applicable SA reference pattern, SR requirements pack, and TA threat snapshot, with the agent checklist covering tool allowlist, per-tool scope minimization, human-in-the-loop gate specification, kill-switch design, and tool-call logging?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-DR-L1-2. Do ≥95% of AI/HAI software artifacts going to production in the last 90 days carry a completed DR decision record (approve / approve-with-conditions / send-back) before build-out begins, with a two-lane routing model (fast-lane ≤2 BD, full-lane ≤5 BD), named lead reviewers per archetype trained on EG-Software L1, and a residual-risk list with named owner and expiry in every record?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-DR-L1-3. Are recurring pattern deviations and repeatedly-waived SR requirements automatically queuing SA pattern-update and SR pack-update reviews, and does every IM-Software incident trigger a re-examination of the DR record that approved the affected artifact?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

DR Level 2.

Q-DR-L2-1. Are 100% of Critical-tier DR reviews conducted as scenario-based walkthroughs, with 3–5 specific threat scenarios sourced from TA-Software per-artifact deep models and anonymized IM-Software incidents, with the DR decision tied explicitly to how the proposed design handles each scenario rather than checklist conformance alone?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-DR-L2-2. Is design-drift detection running quarterly for Critical-tier and annually for High-tier, using code-repository changes, model-registry events, deploy-event configs, CI/CD parameters, and IaC state, with 100% of material drifts automatically re-routed to DR for a new review?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-DR-L2-3. Are joint DR-Software / DR-Vendors review records on file for 100% of Critical-tier first-party artifacts integrating with vendor AI services, with an explicit handoff boundary and shared residual-risk ownership documented in both DR records?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

DR Level 3.

Q-DR-L3-1. Are ≥90% of Critical-tier AI/HAI software artifacts producing a daily automated SA-pattern-compliance attestation signal, checking code-repo control presence, model-registry bounds, IaC drift, and logging completeness, with deviations auto-opening DR-exception tickets triaged within 3 business days?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-DR-L3-2. Has the program contributed ≥2 substantive review artifacts per year (per-archetype rubrics, scenario templates, pattern-evolution frameworks) to OpenSSF AI, CSA AI Safety Initiative, or OWASP SAMM AI, with documented adoption and internal practice aligned to the published versions?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-DR-L3-3. Is there a quarterly pattern-evolution review driven by external signals (MITRE ATLAS updates for TA0006/TA0007/TA0008 and TA0001, sector ISAC advisories) and internal signals (IM-Software incidents, ML-Software telemetry, ST-Software findings), with a versioned change log and notification to in-flight DR reviews affected by pattern changes?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.8 Implementation Review (IR)

IR Level 1.

Q-IR-L1-1. Is there a published, per-archetype IR checklist, one per SM-Software archetype, covering code-matches-pattern verification, config-matches-DR verification, SR REM evidence currency check, logging-event production verification, and kill-switch test execution, with the agent checklist verifying tool allowlist enforcement in deployed code, per-tool scope, HITL gate function, and tool-call logging?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IR-L1-2. Do 100% of new AI/HAI software artifacts going to production in the last 90 days carry a go-live IR record, and do ≥90% of all active artifacts carry a current-year IR record, with material-change triggers wired to SM inventory events, Critical / blocker findings resolved before production, and High findings closed within 7 days with evidence linked?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IR-L1-3. Are findings severity-tagged and tracked in IM-Software with named owners and SLA-bound closure dates, and does every IR finding that reveals stale or inaccurate REM evidence trigger an SR REM row update before the finding is closed?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

IR Level 2.

Q-IR-L2-1. Are ≥90% of Critical-tier AI/HAI software artifacts under continuous drift detection, via code-repo webhooks, model-registry events, IaC scan tooling, deploy-event telemetry, and CI/CD parameter monitoring, with median detection latency ≤7 days and automated finding creation on material deviations?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IR-L2-2. Are no-train and retention settings verified via vendor admin APIs (OpenAI / Anthropic / Bedrock / Vertex / equivalent) on a monthly (Critical) and quarterly (High) probing cadence, not from DPA text alone, covering ≥80% of Critical/High-tier artifacts, with deltas from the previous probe opening IR findings with severity matching the data-class impact?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IR-L2-3. Are 100% of Critical/High-tier agent artifacts covered by tool-scope boundary tests in the current IR cycle, confirming out-of-scope argument rejection, non-allowlisted tool-call blocking, HITL gate trigger, and kill-switch SLA, and is tier-cadence adherence ≥95% with Critical-tier findings aged per the SM L2 tier-treatment matrix SLAs?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

IR Level 3.

Q-IR-L3-1. Are ≥90% of Critical-tier AI/HAI software artifacts producing a daily attestation signal across all three dimensions (pattern compliance, evidence freshness, configuration tolerance), with deviations auto-opening IM-Software tickets within 1 hour and zero stale-evidence violations for Critical-tier REMs?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IR-L3-2. Has the program published per-archetype configuration baseline schemas to OpenSSF AI, OWASP SAMM AI, or CSA AI Safety Initiative, with documented adoption and internal practice aligned to the published versions, and is IR reviewer-hours per Critical artifact per year trending down over two consecutive quarters?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IR-L3-3. Is the post-incident IR feedback loop operational, with IM-Software post-incident reviews including a mandatory IR-record re-examination step, and ≥1 attestation rule update produced per material incident, ensuring incident learning continuously improves the attestation coverage?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.9 Security Testing (ST)

ST Level 1.

Q-ST-L1-1. Is a per-archetype foundational test battery published for all seven AI/HAI software archetypes, with each test class tied to a TA-Software archetype threat (HAI TTP + ATLAS tactic ID) and an SR-Software requirement, with defined inputs/outputs/pass-fail criteria, and an evidence artifact, and are 100% of new AI/HAI software artifacts required to pass the battery before production Sanctioned status is issued?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ST-L1-2. Are four regression corpora (jailbreak, prompt-injection, AGH, tool-misuse) versioned in source control, running in CI on every PR for Critical/High-tier artifacts, with a named corpus owner, a monthly refresh cadence from internal and external sources, and a CI token-spend budget cap, and are ≥95% of Critical/High-tier PR merges verified to have run and passed the corpus?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ST-L1-3. Are all test failures routed to IM-Software within 1 business day with a severity tag and named owner, and does TA-Software archetype threat coverage by the test battery and corpus reach ≥80% by end of year one?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

ST Level 2.

Q-ST-L2-1. Are 100% of Critical-tier AI/HAI software artifacts red-teamed at least quarterly, and 100% of High-tier semi-annually, with scope derived from TA-Software L2 per-artifact deep threat models, covering prompt-injection chains, indirect-prompt-injection via RAG, agent tool abuse, multi-turn AGH probes, data-egress canaries, and cross-tenant isolation, with findings routed to IM and remediation tracked?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ST-L2-2. Is per-tier corpus calibration enforced in CI (Critical-tier: all 4 corpora on every PR plus daily output-integrity regression; Low-tier: jailbreak corpus on merge), and are ≥90% of Critical/High-severity red-team findings converted to corpus entries within 30 days?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ST-L2-3. Are cross-archetype composition tests (agent + RAG, fine-tune + model-serving, multi-agent orchestration) documented and executed for all Critical-tier composite artifacts, and is per-tier SLA adherence for testing activities ≥90%?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

ST Level 3.

Q-ST-L3-1. Are ≥80% of Critical-tier AI/HAI software artifacts under continuous automated adversarial testing with daily probe execution, with novel TTPs triaged into the TA-Software library within 14 days and high-severity automated findings routed to IM within 24 hours?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ST-L3-2. Has the program contributed ≥4 anonymized, legally-vetted findings per year to MITRE ATLAS, AVID, or OWASP LLM/Agentic Top 10, with at least one accepted as a new or refined technique, and are ≥4 open regression corpora published under a permissive license and maintained upstream?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ST-L3-3. Has the program hosted at least 1 industry-shared red-team exercise per year and participated in ≥2 additional cross-org exercises, with documented cross-org detection-benchmark improvement data from participants?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.10 Environment Hardening (EH)

EH Level 1.

Q-EH-L1-1. Does every AI/HAI software artifact in the SM inventory (across all seven archetypes) run under a named, dedicated, least-privilege service account, confirmed by a quarterly IAM audit reconciliation, and are all LLM provider API keys and model-registry credentials managed via a secrets vault with CI secrets-scanning enforced as a blocking PR check with zero current findings?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EH-L1-2. Does the model registry require SSO + MFA for all access, with 100% of Critical/High-tier model promotions gated by a signed SLSA-style provenance attestation, and is there an egress allowlist for each AI/HAI service account scoped to declared LLM provider and model registry domains, with unexpected AI-provider egress from unregistered services triggering a shadow-AI discovery alert?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EH-L1-3. Are DLP rules tuned for AI-specific exfiltration patterns (bulk embeddings, prompt/completion bulk exports, model-weight exfiltration) active on ≥95% of managed engineering endpoints, and are prompt/completion logging pipelines for artifacts processing regulated data configured with a PII redaction layer, with Privacy/Legal sign-off documented where clear-text regulated data is retained?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

EH Level 2.

Q-EH-L2-1. Are 100% of Critical-tier AI/HAI software artifacts under per-artifact SASE egress rules (not per-service) at the workload-identity level, and is just-in-time access (≤4-hour time-limited, approval-gated) enforced for model registry write permissions and LLM provider admin console access, with standing service tokens deprecated for Critical-tier?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EH-L2-2. Are ≥90% of Critical-tier multi-tenant AI/HAI software artifacts enforcing per-tenant isolation at the infrastructure layer (dedicated namespace, VPC endpoint, or per-tenant encryption key), confirmed by IR-Software implementation reviews and ST-Software isolation tests, and is a hardening tier-treatment matrix published and enforced at provisioning with gaps tracked as open IM findings?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EH-L2-3. Are enhanced DLP policies for AI-specific exfiltration patterns (content inspection on outbound AI-provider calls, bulk-embedding and model-weight-transfer alerts) active for Critical-tier artifact development on engineering endpoints, with false-positive rates actively monitored and trending down through monthly review cadences?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

EH Level 3.

Q-EH-L3-1. Are ≥90% of EH controls expressed as authoritative IaC (not stubs) in a version-controlled IaC registry, with drift detected continuously and ≥70% of low-risk drift auto-remediated, and a machine-readable change log visible to downstream network and identity teams, with high-risk drift human-reviewed within 2 business days?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EH-L3-2. Is the adaptive-policy pipeline operational, with ML-Software detections and IM-Software incidents generating human-approved policy-tightening proposals on a tracked cadence, every change traceable to a source signal, and downstream artifact teams notified within 24 hours of a tightening change affecting their artifact's hardening profile?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EH-L3-3. Does the program contribute ≥2 AI/HAI software hardening baselines per year to industry bodies (CIS AI workloads, CSA AI Safety Initiative, sector ISACs) with documented adoption, and are new AI/HAI software artifacts auto-provisioned with their tier-appropriate hardening profile within 24 hours of SM inventory registration?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.11 Issue Management (IM)

IM Level 1.

Q-IM-L1-1. Is there a single AI/HAI software issue backlog with standardized metadata (source, affected artifact linked to SM inventory, severity rubric anchored to AI-specific axes, active exfiltration / agent damage / kill-switch failure / regulated-data breach for Critical; confirmed control failure with potential impact for High; and so on, owner, SLA, regulatory flag, evidence link) capturing ≥95% of issues from all source practices (TA, SR, DR, IR, ST, ML, external)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IM-L1-2. Is the AI/HAI software incident playbook published with ≥6 named AI-specific incident classes (prompt-injection, agent rogue-action, training-data-leakage, silent model-swap, RAG poisoning, shadow-AI emergence), each with pre-assigned roles, containment plays (kill-switch, model rollback, tool-revoke, retrieval-source disable, egress-block), evidence-capture steps, and SLA targets, and has each class been exercised in at least one tabletop in the last 12 months?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IM-L1-3. Is the regulatory SLA tracker live covering GDPR Art. 33 (72h), EU AI Act Art. 73, HIPAA (60d), NYDFS Part 500 (72h), and applicable sector-specific obligations, with 100% adherence in the last 90 days, and does every Critical/blocker incident produce a post-incident review within 14 days with named update outputs flowing to SA, SR, EG, and ML?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

IM Level 2.

Q-IM-L2-1. Is a tier-calibrated incident playbook operational with Critical-tier MTTA ≤1 hour and MTTC ≤4 hours, 24/7 on-call coverage with a documented rotation including a current Critical-tier artifact briefing, and tier-movement in the SM-Software inventory automatically triggering IM configuration updates (on-call path, playbook variant, SLA targets) within 14 days of a Critical re-tier event?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IM-L2-2. Is a post-incident review auto-flow integration live, routing Critical-tier review outputs to SA/SR/EG/ML practice backlogs, with ≥90% of downstream practice owners responding within 14 days, and the sponsor reviewing output quality quarterly to distinguish substantive changes from nominal acknowledgements?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IM-L2-3. Is a cross-domain coordination protocol published and used for 100% of multi-domain AI/HAI software incidents, with named cross-domain contacts for Vendors, Infrastructure, and Processes domains verified quarterly, a single Incident Commander from the primary impacted domain, and joint post-incident reviews spanning all affected domains?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

IM Level 3.

Q-IM-L3-1. Does the program contribute ≥4 anonymized AI incident-classification entries per year to sector ISACs, ≥2 entries per year to AVID, and ≥1 contribution per year to MITRE ATLAS TA0014 Impact tactic documentation, with all contributions maintained current, legally vetted, and tracked for external adoption?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IM-L3-2. Are ≥3 pre-authorized automated containment actions live (kill-switch, egress-block, retrieval-source disable, or tool-revoke classes), vetted by Privacy/Legal and the executive sponsor, producing 100% audit records plus human-review tickets on execution, with the pre-authorization policy reviewed quarterly and any unexpected outcome triggering an out-of-cycle review?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IM-L3-3. Is a quarterly MTTR benchmark brief published to the sponsor, comparing the program's MTTR per incident class and per tier against ISAC-sourced and peer-sourced benchmarks, with Critical-tier MTTR at or below benchmark for ≥3 of 6 incident classes and deltas above benchmark linked to specific practice gaps and investment proposals?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.12 Monitoring & Logging (ML)

ML Level 1.

Q-ML-L1-1. Has a per-archetype logging baseline been published specifying the minimum event schema, fields, retention window, and export path for each AI/HAI software archetype in the SM-Software inventory (LLM-integrated app, agent, RAG, fine-tune/training, eval harness, model-serving service, classical ML), and has compliance of each production artifact been measured against it within the last quarter?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ML-L1-2. Is a high-signal detection set of ≤12 detections active, each with a named owner, detection query, SLA, ATLAS-tactic or HAI-TTP tag, and last-tuned date, including AGH detection, prompt-injection success, tool-scope-violation, training-data-leakage canary, shadow-AI emergence, kill-switch-not-triggered, and HITL-gate-bypass, with false-positive rates tracked per detection and monthly tuning reviews occurring?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ML-L1-3. Has the evidence trail for EU AI Act Art. 12, GDPR Art. 30, and ISO/IEC 42001 AIMS been wired to the ML-Software log store, and has a quarterly deployer-duty drill confirmed that the evidence package for a randomly selected production artifact can be assembled within the ≤24-hour SLA?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

ML Level 2.

Q-ML-L2-1. Is tier-calibrated logging depth applied per the SM-Software L2 tier-treatment matrix, Critical-tier artifacts retaining full prompt/completion and tool-call corpora at the longest regulatory window, Low-tier artifacts receiving baseline only, and is this calibration automatically updated when an artifact is re-tiered?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ML-L2-2. Is the SIEM ingesting ML-Software log feeds with ≥3 cross-artifact correlation rules active (covering at minimum multi-artifact AGH, training-to-inference leakage chain, and shadow-AI emergence plus identity pivot), and is a quarterly detection tuning cycle operating from IM-Software post-incident and ST-Software finding inputs?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ML-L2-3. Are ≥90% of Critical/High-tier artifacts running anomaly-detection baselines with behavioral profiles refreshed monthly and FP rates tracked and trending down, and is the ML logging-baseline validation element completing inside the ≤30-day staleness threshold for all Critical-tier artifacts in PC-Software compliance evidence bundles?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

ML Level 3.

Q-ML-L3-1. Are ≥90% of detections expressed as version-controlled, CI/CD-deployed code artifacts with automated test coverage against realistic synthetic log data, and is detection coverage auto-verified for 100% of new or re-tiered SM inventory entries within 24 hours of the inventory change event?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ML-L3-2. Are ≥90% of Critical/High-tier artifacts running ML-driven anomaly detection on prompt/completion and tool-call corpora, with anomaly models retrained monthly on production log data, model versions tracked in the ML-Software model registry, and anomaly-model alerts feeding the IM-Software incident backlog through the same detection-to-ticket pipeline as rule-based detections?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ML-L3-3. Has the program contributed ≥2 telemetry-standard artifacts per year to the OpenTelemetry AI workgroup or equivalent and ≥12 anonymized detection signatures per year to sector ISACs, and has it proposed or validated ≥2 MITRE ATLAS AML.M00xx detection-mitigation entries, with contributions maintained current and external adoption tracked?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

29. Practice-level rollup

After completing all 108 questions, fill in the table below. For each practice, count Yes (Y), Partial (P), and No (N) answers per level. Compute the precise score as described in Section 27: L1_score = (Y + 0.5P) / 3; L2_score = (Y + 0.5P) / 3 × L1_score; L3_score = (Y + 0.5P) / 3 × L2_score; Practice Score = L1_score + L2_score + L3_score.

Worked example

The assessment team answers for TA-Software: L1 Q1 = Y, L1 Q2 = Y, L1 Q3 = P; L2 Q1 = Y, L2 Q2 = P, L2 Q3 = N; L3 all N.

L1_score = (1.0 + 1.0 + 0.5) / 3 = 0.833
L2_score = (1.0 + 0.5 + 0.0) / 3 × 0.833 = 0.417
L3_score = 0.0

TA Practice Score = 0.833 + 0.417 + 0.0 = 1.25 / 3.0

Interpretation: TA-Software scored L1 = 0.83, L2 = 0.42, L3 = 0.0, yielding a practice maturity of 1.25, solidly Foundational with partial L2. The archetype library and per-intake snapshot gate are working; the shadow-AI threat view is incomplete; the external intel triage cadence is partial and the red-team-the-library exercise is not yet operational. Roadmap priority: complete the shadow-AI threat view (closes the L1 Partial), operationalize external intel triage (L2 Q2), launch the red-team-the-library cadence (L2 Q3). L3 work is premature.

Notes column for assessor. Use the space below to record per-practice observations: which questions were hardest to answer, where evidence was thin, where Partial answers cluster, and what the most actionable next step is.

SM: _________ PC: _________ EG: _________ TA: _________ SR: _________ SA: _________ DR: _________ IR: _________ ST: _________ EH: _________ IM: _________ ML: _________

30. Domain-level rollup

Domain Maturity = (sum of all 12 Practice Scores) / 12 = ____ / 3.0

Maturity band achieved: ☐ Ad-hoc (0.0–0.9) ☐ Foundational (1.0–1.9) ☐ Comprehensive (2.0–2.9) ☐ Industry-Leading (3.0)

Per-Business-Function rollup

A domain is mature when all four Business Functions are at the same band. A domain whose Operations function trails the others has built and verified well but cannot run the program in production. A domain whose Verification function trails ships AI/HAI software without proof. The most common pattern in early-stage programs is Governance ahead of Building, and both ahead of Verification and Operations, because policies are easier to write than tests are to run and monitoring baselines are to maintain.

Worked example, domain-level rollup

The following shows a plausible result for an organization 18 months into its Software-domain program.

Domain Maturity = 15.17 / 12 = 1.26 / 3.0

Band: Foundational. This organization has crossed L1 across most practices but has not yet closed L2 for any practice. The Operations function is weakest (EH 0.92, ML 0.83 both sub-Foundational), which is typical. The program has a visible inventory and published policies, but the hardening controls and logging baselines are not consistently measured.

Per-Business-Function summary for this example:

The imbalance is clear: Governance is at 1.67 while Operations is at 0.97. The program has published good policies but has not yet instrumented its own AI/HAI software in production. The roadmap should front-load EH L1 and ML L1 before deepening Governance to L2.

Strengths

Gaps

Highest-priority remediation areas (top 5)

31. Improvement roadmap template

Use this template to convert assessment findings into a 12-month roadmap. Each entry names a target gap, the practice and level it addresses, the owner, the success metric, and the deadline.

A 12-month roadmap for the Software domain follows four natural quarters. The sequencing mirrors the dependency graph in HAIAMM v3.0 §9: Governance must be in place before Building can operate; Building must be in place before Verification can produce meaningful results; Operations depends on all three preceding functions.

Quarter 1 (months 1–3). Stabilize L1 across the four Business Functions. Priority practices: SM L1, PC L1, EG L1, TA L1.

Quarter 1 focus: make every AI/HAI software artifact in production visible, named, and governed. The inventory, the charter, the three policies, and the archetype threat library must all exist at L1 before engineering teams can self-serve on intake. A shadow-AI-in-engineering discovery sweep should run within the first 30 days so the inventory is seeded from signals rather than declared from memory.

Quarter 2 (months 4–6). Complete remaining L1 practices. Priority practices: SR L1, SA L1, DR L1, IR L1, ST L1, EH L1, IM L1, ML L1; SM L2 risk-tier rubric.

Quarter 2 focus: close the Building and Operations L1 gaps, and begin the L2 calibration work starting with the SM risk-tier rubric, which is the prerequisite every other practice needs to move to L2.

Quarter 3 (months 7–9). Operationalize L2 across the Governance and Building functions. Priority practices: PC L2 evidence bundles, TA L2 per-artifact deep models, SA L2 IaC-encoded patterns, DR L2 scenario-based walkthroughs, SR L2 quantitative requirements.

Quarter 3 focus: the tier rubric now exists, use it. Evidence bundles for Critical/High artifacts should be assembling automatically. Deep threat models for Critical-tier replace archetype snapshots. Design reviews for Critical-tier move to scenario-based walkthroughs. The SR pack sheds all qualitative language.

Quarter 4 (months 10–12). Complete L2 across all 12 practices and prepare L3 entries for selected practices. Priority practices: IR L2 continuous drift detection, ST L2 red-team cadence, EH L2 SASE per-workload egress, ML L2 anomaly detection, IM L2 tier-calibrated playbook; begin L3 scope decisions for SM, TA, and EG.

Quarter 4 focus: close the Verification and Operations L2 gaps. Drift detection, red-team cadence, anomaly baselines, and tier-calibrated incident response are the four load-bearing L2 capabilities that most programs defer because they require engineering investment. The L3 scope decisions for SM, TA, and EG can be made now even if the automation work begins in year 2.

Reassessment date (12 months from this assessment): ____

When the next annual assessment runs, compare practice scores to this baseline. The expected trajectory for a program executing this roadmap is: Domain maturity moves from the Foundational band toward the low end of the Comprehensive band (1.6 to 2.0). The Operations function moves from Ad-hoc to Foundational, the most common single-year jump possible. The Governance function moves from mid-Foundational to low-Comprehensive. The largest score gains come from practices where the Q1–Q2 L1 foundation was weakest: typically IR, EH, and ML.

Part V, Reference

32. Glossary

AGH, Agent Goal Hijack. One of the four HAI-specific TTPs. The agent's benign goal is redirected via content injected through a trusted-looking path (retrieved document, tool response, multi-turn history).

AI Engineering Standards Policy. The first of the three priority AI/HAI software policies. Specifies the minimum required controls per archetype (TA threat snapshot, SR REM, SA reference-pattern adherence, IR readiness attestation, ST evidence, ML logging-baseline), the data classes permitted per archetype and deployment context, agentic scope constraints, output-integrity-critical designations, and model/provider version-logging at go-live.

AI Acceptable Use & Engineering Standards Policy. The second priority policy. Enumerates sanctioned LLM SDKs and providers, lists actions requiring approval, prohibits actions without explicit sign-off (fine-tuning on customer PII without privacy approval, agents acting on customer accounts without DR approval, automated decisions with legal/significant effect without Art. 22 safeguards), imposes a disclosure obligation to the SM-Software inventory, and requires attestation at hire and annually.

AI Software Intake / Go-Live Gate Policy. The third priority policy. Makes intake mandatory before production deployment for all in-scope archetypes, lists required go-live artifacts by archetype, exposes an amnesty path for previously ungated production artifacts, and names the go-live decision authority.

AI/HAI software archetype. One of seven categories of AI/HAI software the organization builds: LLM-integrated app, AI agent, RAG pipeline, fine-tuning/training workload, eval harness, model-serving service, classical ML. Threat libraries, requirements, reference architectures, design checklists, and tests are archetype-keyed.

AI/HAI software inventory. The single source of truth for all AI/HAI software artifacts the organization builds, sanctioned or not, owned by the program lead. Seeded from source-code, dependency-manifest, CI/CD, runtime-egress, model-registry, and cloud-spend signals plus self-attestation under amnesty.

Amnesty path. A documented path for engineers to disclose previously undisclosed AI/HAI software in production without penalty, routed to retroactive intake. Specific to the L1 program; replaced by blocking enforcement at L2 for Critical-tier missing go-live artifacts.

Anti-pattern catalog. A catalog of AI/HAI software architectural patterns known to cause incidents, prompt-injection-trusting agent, over-broad tool scope, system-prompt-leaking persona, long-session agent without memory bounds, fine-tune on user data without opt-out, silent model-family swap with no eval gate, RAG over unclassified corpus, LLM API key embedded in client code, secrets in prompts, output-integrity-critical decisions with no human gate. Each anti-pattern is named, described, prohibited, and paired with the reference pattern element that replaces it.

Archetype snapshot. A category-level threat model produced once per archetype at L1 and adapted per intake to produce a per-intake threat snapshot in minutes.

Compliance evidence bundle. A continuously-assembled view of compliance evidence per Critical/High AI/HAI software artifact, covering TA snapshot, SR REM, SA pattern confirmation, DR decision, IR attestation, ST evidence, ML logging-baseline confirmation, deployer-duty record, and foundation-model provider attestation. Becomes the source for on-demand evidence packs at L3.

Critical / High / Medium / Low. The four risk tiers introduced at SM L2. Driven by data sensitivity, decision-affecting use (GDPR Art. 22 / EU AI Act Annex III), agentic capability, user exposure, training-data posture, production-load-bearing role, and concentration. Each tier carries different intake depth, review cadence, hardening depth, logging depth, monitoring depth, and re-review triggers.

Deployer. The legal role assigned to the organization that uses or ships an AI system, under EU AI Act terminology. In the Software domain, the organization that builds and ships the AI/HAI software is the deployer. The Software-domain program is the operational apparatus for meeting deployer duties (Article 26) for AI systems the organization builds.

Deployer-duty owner. A named human-oversight owner for each customer-facing or decision-affecting AI/HAI software artifact in production, logged in the SM-Software inventory and accountable for EU AI Act Art. 26 obligations.

EA, Excessive Agency. One of the four HAI-specific TTPs. The AI or agent has more capability than its use case requires.

FRIA. Fundamental Rights Impact Assessment, required for certain high-risk AI systems under the EU AI Act.

HAA, HAIAMM AI Attack Taxonomy. Canonical catalog of high-impact attacks against AI systems with severity, prerequisites, indicators, and cross-references to MITRE ATLAS.

HAI TTPs (EA, AGH, TM, RA). Four AI-specific threat-tactic categories carried throughout HAIAMM v3.0: Excessive Agency, Agent Goal Hijack, Tool Misuse, Rogue Agents.

HITL gate. Human-in-the-loop gate. A synchronous approval gate implemented in the agent loop for each tool in a destructive, external-network, or customer-account-affecting category.

Intake gate. The single intake gate through which all AI/HAI software artifacts must pass before production deployment. Operates a published SLA, fast-track for sanctioned archetypes with no regulated data, and the amnesty path for previously undisclosed production artifacts.

Kill-switch. Emergency-halt mechanism for an AI agent or model-serving service. Wired, documented, and quarterly-tested at L1; tested with measured halt-time at L2; pre-authorized for automated execution under defined conditions at L3.

Priority compliance map. A one-page artifact that ties each priority regulatory or standards requirement to the specific organizational policy that carries it.

RA, Rogue Agents. One of the four HAI-specific TTPs. Autonomous agents drift from intended behavior across long sessions, reflective loops, or multi-agent miscoordination.

REM, Requirements-Evidence Map. A per-artifact map that records, for each applicable requirement in the AI/HAI Software Requirements Pack, whether the requirement is Met, Met-with-compensating-control, Gap-accepted, or Not-applicable, with a citation to evidence.

Reference pattern. A vetted "green path" architecture pattern published per AI/HAI software archetype. Engineering teams reach for the pattern first; deviations require design review.

Risk-tier rubric. A short table introduced at SM L2 that derives a deterministic risk tier (Critical / High / Medium / Low) from auditable inputs (data sensitivity, decision-affecting use, agentic capability, user exposure, training-data posture, production-load-bearing role, concentration). Drives the differential intensity of every downstream practice's L2 and L3 work.

Sanctioned / Provisional / Under review / Prohibited. AI/HAI software approval statuses logged in the SM inventory. Sanctioned artifacts have passed the go-live gate; Provisional are approved with conditions; Under review are in intake; Prohibited are explicitly blocked.

Shadow AI in engineering. AI/HAI software adopted, built, or shipped outside the program's visibility, attribution, and governance. The Software-domain program's primary L1 outcome is to make shadow AI in engineering visible, attributable, and trending down.

Shadow-AI-in-engineering ratio. Unsanctioned AI/HAI software artifacts in production divided by total AI/HAI software artifacts. A primary outcome metric of the L1 program. Reported quarterly and trending down. Reported per tier at L2; Critical-tier shadow AI tracked at zero.

Shadow AI scoreboard. A quarterly artifact delivered to the executive sponsor summarizing inventory state, new discoveries, shadow-AI-in-engineering ratio trend, AUP attestation, and top exposure risks. Becomes tier-aware at L2.

SLSA-style provenance attestation. Supply-chain Levels for Software Artifacts. A signed attestation listing training-data sources, training-job identity, eval-suite result reference, and build-system identity, attached to model artifacts at registry promotion.

Tier-treatment matrix. A published matrix introduced at SM L2 specifying what each tier (Critical, High, Medium, Low) receives from each downstream practice (PC, TA, SR, SA, DR, IR, ST, EH, ML, IM).

TM, Tool Misuse. One of the four HAI-specific TTPs. Tools available to the AI or agent are invoked for attacker purposes, argument smuggling, unexpected combinations, crafted parameters, recursive invocation.

Tool allowlist. The explicit list of tools an AI agent may invoke, with per-tool scope (parameter types, rate, data class). At L1 the allowlist exists; at L3 it is runtime-enforced as a formal specification.

33. Reference frameworks

This handbook is one of six domain handbooks that, together with a master handbook, constitute HAIAMM v3.0. The frameworks named here are referenced throughout. They are listed by name only; consult the issuing body's current published version when running an assessment.

Maturity-model lineage.

• OWASP SAMM (Software Assurance Maturity Model). HAIAMM borrows SAMM's lifecycle shape (Governance, Building, Verification, Operations) and practice-per-function structure. SAMM covers classic AppSec maturity; HAIAMM extends into AI-specific territory rather than replacing it. The Software domain in particular sits closely to SAMM and is intended to complement it.
• BSIMM (Building Security In Maturity Model). HAIAMM borrows the observational "this is what organizations actually do" posture at higher maturity levels.

AI-governance frameworks (complementary).

• NIST AI RMF 1.0 + Playbook. The risk-management-framework counterpart to HAIAMM's maturity-model shape. An organization uses NIST AI RMF to decide what to govern, and HAIAMM to measure how mature its ability to govern is. The Playbook gives concrete actions for each function. NIST AI RMF GOVERN aligns to HAIAMM's Governance function; MAP to Building; MEASURE to Verification and Operations; MANAGE to Operations.
• ISO/IEC 42001 (AI Management System). A management-system standard for AI. HAIAMM practices supply the operational evidence an ISO 42001 AIMS requires.
• ISO/IEC 27001 / 27002. General Information Security Management System and controls. HAIAMM practices produce evidence that maps to relevant Annex A controls.

Regulations applicable to AI/HAI software the organization builds.

• EU AI Act. Article 26 (deployer duties), Article 50 (transparency), Annex III (high-risk systems), Article 9 (risk management), Article 15 (accuracy/robustness/cybersecurity), Article 12 (logging), Article 73 (serious-incident reporting), Article 11 (technical documentation).
• GDPR. Article 22 (automated decision-making), Article 32 (security of processing), Article 33 (breach notification), Articles 44–49 (international transfers), Article 30 (records of processing), Article 28 (processor obligations at the foundation-model boundary).
• SOC 2. CC9.2 (vendor management at the foundation-model-provider boundary) and trust services criteria applicable to AI/HAI software services.
• HIPAA. Business Associate Agreement requirements; subcontractor agreements; safeguards for PHI in inference and training.
• PCI-DSS 12.8. Service-provider management; written agreements; ongoing monitoring.
• FINRA / SEC. Third-party and internal model-risk management guidance.
• HHS / FDA. AI-enabled medical device guidance for org-built clinical AI software.
• NYDFS Part 500. Third-party service-provider security policy; material cybersecurity event notification.
• OCC third-party risk guidance. Banking sector third-party risk lifecycle.

Threat taxonomies.

• MITRE ATLAS (Adversarial Threat Landscape for AI Systems). Canonical adversarial-ML reference. HAIAMM's TA practice consumes ATLAS technique IDs and contributes back at L3. ATLAS is used throughout this handbook in TA archetype threat models, SR requirements traceability, SA reference patterns (with AML.M00xx mitigation entries), and ST test batteries (with AML.TXXXX technique tags).
• OWASP Top 10 for LLM Applications. Threat reference for LLM-integrated AI systems. Consumed by TA-Software; reviewed in EG-Software practitioner curriculum.
• OWASP Top 10 for Agentic AI. Threat reference for agent-based systems. Consumed by TA-Software.
• AI Vulnerability Database (AVID). Catalog of disclosed AI-specific vulnerabilities. Consumed by TA-Software; contributed to at L3.

Industry communities.

• CSA AI Safety Initiative / AI Controls Matrix. Cross-organization AI controls work. HAIAMM contributes to controls matrix at L3.
• OpenSSF AI. Open Source Security Foundation working group on AI. HAIAMM contributes reference patterns, requirements schemas, telemetry standards, and detection signatures at L3.
• OpenTelemetry AI workgroup. Semantic conventions for AI/HAI event types. ML-Software L3 contributes prompt/completion spans, tool-call spans, agent-session traces, and training-job event conventions.
• Sector ISACs. FS-ISAC (financial services), H-ISAC (health), IT-ISAC (IT), and others provide intelligence sharing for AI software incidents.

HAIAMM canonical companions.

• HAIAMM-v3.0-Framing.md, model master document; canonical definitions for the 12 practices, 6 domains, 3 maturity levels, cell template, dependency graph, through-lines, and authoring rules.
• AI-Attack-Taxonomy.md (HAA), high-impact AI attacks catalog cross-referenced to MITRE ATLAS.
• Cloud-Threat-Taxonomy.md (HCT), cloud-infrastructure threats organized under BadCode / BadAction / BadPrincipal / BadPermissions.
• Cloud-Controls-Taxonomy.md (HCC), controls cross-mapped to ATLAS mitigations.
• AI-Threat-Assessment-Methodology.md, organization-wide AI threat assessment methodology.
• Threat-Modeling-Methodology.md, per-system threat modeling methodology.
• AI-Security-Testing-Methodology.md, per-archetype test battery methodology.

Threat-tactic categories specific to HAIAMM (reproduced for reference).

• EA, Excessive Agency. The AI or agent has more capability than its use case requires.
• AGH, Agent Goal Hijack. The agent's benign goal is redirected by content injected along a trusted-looking path.
• TM, Tool Misuse. Tools available to the AI or agent are invoked for attacker purposes.
• RA, Rogue Agents. Autonomous agents drift from intended behavior across long sessions, reflective loops, or multi-agent miscoordination.

34. Change log

End of HAIAMM v3.0 Software Domain Handbook.