# Security Testing (ST)
## Software Domain - HAIAMM v3.0

---

### Practice Overview

**Objective:** Prove that every AI/HAI software artifact the organization **builds** behaves correctly under adversarial conditions, by running a foundational per-archetype test battery in CI, maintaining versioned regression corpora, and escalating to scheduled red-team and continuous adversarial testing at higher maturity levels.

**Description:** ST-Software exercises the AI/HAI software the organization *ships*, LLM-integrated applications, autonomous agents, RAG pipelines, fine-tuning and training workloads, evaluation harnesses, model-serving services, and classical ML models, against a battery of AI-specific test classes tied directly to the threats in the TA-Software library and the requirements in the SR-Software pack. At L1, every archetype has a published test battery (prompt-injection probes, tool-scope boundary tests, data-egress canaries, logging-completeness tests, kill-switch tests, output-integrity regressions) plus versioned regression corpora (jailbreak corpus, prompt-injection corpus, agent goal-hijack corpus, tool-misuse corpus) running on every PR via CI. L2 adds per-tier scheduled red-team exercises using TA L2's deep threat models, and cross-archetype composition tests. L3 operates continuous automated adversarial testing (AI-vs-AI red-team scaffolding) and contributes findings to MITRE ATLAS, AVID, and OWASP LLM / Agentic Top 10.

**Context:** Classic CI test suites exercise the happy path and leave the adversarial path untested. An LLM-integrated feature passes all unit tests and then leaks its system prompt on the first prompt-injection attempt. An agent passes its integration tests and then invokes tools outside its declared scope when fed a crafted retrieval response. A fine-tuned model ships to production and regurgitates a canary string from the training corpus. These failures are invisible to classic testing because classic testing was not designed to enumerate AI-specific failure modes, prompt injection (ATLAS TA0001/TA0003), ML model access probing (TA0004), attack staging (TA0012), and data exfiltration via inference (TA0013). ST-Software closes this gap by making AI-specific tests a first-class CI citizen and by connecting them directly to the TA threat library so test coverage tracks threat coverage, not just code coverage.

---

## Maturity Level 1
### Objective: Establish a foundational per-archetype test battery and regression corpora that run in CI on every PR, and verify that every AI/HAI software artifact reaches production with a passed go-live battery on record

At this level, the organization gives every AI/HAI software artifact a documented, automated-where-possible test battery drawn from the TA-Software archetype threat library and the SR-Software requirements pack, so every production landing is backed by observable test evidence, not only design assurance.

#### Dependencies

- **TA-Software L1 (required):** tests target specific threats from the archetype threat library; without the library, test scope is invented per intake rather than inherited.
- **SR-Software L1 (required):** requirements pack defines what a passing test means; tests answer "does this artifact meet its SR requirements?" not a free-form question.
- **SA-Software L1 (required):** reference patterns define the control points (tool allowlist, HITL gate, output filter, secrets vault) that tests exercise.
- **IR-Software L1 (required):** implementation reviews confirm the artifact is configured as designed before ST probes run; testing an unconfigured artifact produces noise.
- **Supports / unblocks:** IM-Software L1 (test failures become issues), ML-Software L1 (detections validated by shadow-AI and logging-completeness tests), EH-Software L1 (hardening informed by test findings).

#### Desired Outcomes

- Every AI/HAI software artifact reaching production has passed a documented per-archetype test battery; the battery result is on file and linked from the SM inventory record.
- Regression corpora (jailbreak, prompt-injection, agent goal-hijack, tool-misuse) run on every PR via CI and catch AI-specific regressions before merge.
- Test failures become IM issues with named owners within one business day, not reports in a drive.
- The battery is automated enough to re-run on demand (post-model-update, post-incident, on cadence) without a bespoke effort per run.
- Test coverage tracks threat coverage: every threat in the TA-Software library for a given archetype has at least one corresponding test in the battery or corpus.

#### Activities

**A) Publish the foundational per-archetype test battery**

Ship one test battery per AI/HAI software archetype. Each battery targets the top archetype threats from TA-Software and the archetype-specific SR requirements. L1 target: ≤8 named test classes per archetype. Each test class has: inputs, expected output, pass/fail criteria, evidence artifact (log snippet / trace ID / CI run link), and the TA threat and SR requirement it maps to.

Per-archetype battery (using SM-Software L1 archetypes):

**LLM-integrated application or feature:**
- *Prompt-injection probe corpus (regression CI):* direct prompt-injection inputs covering system-prompt extraction, instruction-override, role-manipulation, and prefix-injection techniques (ATLAS TA0001/TA0003; AGH); run on every PR against the versioned corpus; failure blocks merge for Critical/High-tier artifacts.
- *Training-data leakage canary:* insert a unique canary string into a fine-tune or RAG corpus; run prefix-completion probes via the inference endpoint; assert the canary is not recoverable (ATLAS TA0013; training-data leakage / OWASP LLM10 partial).
- *Output-integrity regression:* golden test set of 20–50 structured-output prompts with expected schemas; alert on schema drift or content drift exceeding a defined threshold; gate model-version promotions (model-swap regression; ATLAS TA0004).
- *Rate-limit / abuse-detection test:* send inputs at and above the declared rate limit; assert that the rate-limit layer enforces the cap and that abuse-detection fires; verify degraded-mode fallback produces a logged, non-misleading response.
- *Logging-completeness test:* for a known prompt/completion pair, verify that the required log fields (session ID, timestamp, user identifier, prompt hash, completion hash, model version) appear in the org-side log store within the retention SLA.

**Autonomous AI agent:**
- *Tool-scope boundary test:* call each declared tool with an out-of-scope argument (path traversal, cross-user ID, out-of-range parameter); verify the tool layer rejects the call and logs the rejection (ATLAS TA0004; EA / TM).
- *Tool-argument-smuggling corpus (regression CI):* versioned corpus of argument-smuggling payloads (path traversal fragments, SQL meta characters, JSON-injection strings, user-ID substitution patterns); run on every PR; assert rejection at validation layer (TM; OWASP LLM07).
- *Multi-turn agent goal-hijack (AGH) probe corpus:* seed the agent's retrieval path or tool-response payload with alternate-goal instructions; verify the agent does not change its declared goal; run on every PR (ATLAS TA0001/TA0003; AGH).
- *Recursive-invocation guardrail test:* craft input that causes the agent to attempt a recursive or looping tool invocation; verify the recursion budget is enforced before cost or blast-radius exceeds the declared SLA (TM / RA).
- *Kill-switch test:* trigger the kill-switch path under a simulated load condition; verify the agent process halts within the declared SLA (≤5 minutes); log the halt event; assert that in-flight tool invocations are not silently completed after kill signal (RA).
- *HITL-gate test:* invoke a tool in the "destructive / external-network / customer-account-affecting" category; verify the human-confirmation gate fires and that the action cannot complete without an explicit approval event in the log (EA / OWASP LLM08).
- *Session-bound memory test:* run a long session (100+ turns); verify that cross-session memory does not persist regulated or customer-specific data past the declared session boundary; assert the per-session token budget is enforced (RA).

**RAG / retrieval-augmented generation pipeline:**
- *Retrieval-source provenance test:* query the pipeline with a known-origin document in the corpus; verify the retrieved chunk carries the correct source identifier and classification label through to the model response log (retrieval poisoning; ATLAS TA0003).
- *Classification-label-respecting retrieval test:* with a query that would match a document classified "Confidential" and a query context that is "Public," assert the confidential chunk is not returned or is filtered before reaching the prompt (cross-classification leakage; EA).
- *Injection-defense test over retrieved content:* seed the corpus with a document containing prompt-injection instructions; verify the agent or app does not follow those instructions; assert the provenance label marks the chunk as "retrieved / untrusted" (AGH; OWASP LLM01 indirect).
- *Per-tenant retrieval isolation test:* with a multi-tenant deployment, send a query from tenant A that would match a document indexed under tenant B's namespace; verify zero results or a namespacing error, not tenant B's data, is returned (EA; cross-tenant leakage).

**Fine-tuning or model-training workload:**
- *Training-data classification pre-flight test:* before any fine-tuning job starts, run a data-classification check against the proposed training dataset; assert that regulated data classes (PII, PHI, PCI, customer source code) fail the pre-flight unless a documented privacy sign-off is on file (training-data leakage; ATLAS TA0002).
- *No-regulated-PII-without-sign-off test:* attempt to queue a fine-tuning job with a synthetic dataset containing a known PII pattern (e.g., SSN-format strings); assert the job is blocked at intake without a valid privacy-officer approval artifact.
- *Model-card auto-population test:* on completion of a fine-tuning job, assert that the model registry record carries: training data sources, data-class classification, eval-suite results, and the date/identity of the approving reviewer; fail the promotion gate if any field is empty.
- *Eval-suite gating test:* confirm that a model cannot be promoted from the training registry to the serving registry without a passing eval-suite result on the defined eval harness; attempt a promotion without the eval artifact and assert it is blocked (output-integrity; ATLAS TA0004).

**Evaluation / red-team harness:**
- *Isolation test:* assert that eval data does not flow to training pipelines; verify via data-flow check that the eval harness's input datasets and annotated outputs are not accessible from the training workload's artifact store.
- *Reproducibility test:* run the same eval suite twice against the same pinned model version and assert the results are within the declared variance threshold; fail if results differ beyond threshold (output-integrity regression).
- *Regression-corpus completeness test:* assert that the eval harness includes test cases from the TA-Software archetype library for the artifact under evaluation; fail if the coverage ratio falls below the declared target (e.g., all top-5 archetype threats covered).

**Model-serving service:**
- *Version-pinning test:* assert that the serving endpoint returns the declared model version in the response metadata; attempt to serve a request without a pinned version specified and assert the request is rejected or routed to the default-pinned version, not an arbitrary latest.
- *Canary-deployment test:* assert that a new model version is deployed to a canary slice before full rollout; verify that rollout automation checks the canary eval result before promoting to full traffic.
- *Rollback test:* trigger the rollback playbook for a canary deployment; verify the serving endpoint reverts to the previous pinned version within the declared rollback SLA; assert the rollback event is logged.

**Classical ML model in a product surface:**
- *Drift-detection threshold test:* inject a synthetic distribution-shift into the model's input stream; assert the drift-detection signal fires within the declared detection window.
- *Retraining-trigger test:* verify that when drift-detection fires, the defined retraining or human-review trigger is activated and produces an IM ticket with the correct severity tag.

**B) Build and maintain regression corpora in CI**

Maintain four versioned regression corpora in source control, running on every PR via CI. Each corpus is a collection of structured test fixtures: input, expected safe output pattern, threat tag (HAI TTP + ATLAS tactic ID), OWASP reference, source, date added.

- **Jailbreak corpus**, 30–100 jailbreak inputs targeting role-override, persona-switch, authority-claim, and encoding-bypass patterns; run against all LLM-integrated app and agent archetypes; failure blocks merge for Critical/High-tier.
- **Prompt-injection corpus**, 30–100 direct and indirect prompt-injection inputs (direct: system-prompt extraction, instruction override; indirect: injected-document payloads, tool-response payloads, multi-turn history injection); run against all LLM-integrated, agent, and RAG archetypes.
- **Agent goal-hijack (AGH) corpus**, 20–60 multi-turn sequences designed to redirect a tool-using agent's declared goal via retrieval-path content, tool responses, or long-context accumulation; run against agent archetypes.
- **Tool-misuse corpus**, 20–60 argument-smuggling and out-of-scope invocation payloads per tool type (file-path tools, HTTP tools, database-query tools, API-call tools); run against agent archetypes.

Corpus management:
- Versioned in source control; corpus changes go through PR review with a named corpus owner.
- Corpus refresh cadence: monthly minimum from three sources, internal observations (IR findings, IM incidents, red-team results), external public corpora (OWASP LLM Top 10 examples, HackAPrompt dataset, ATLAS technique examples), and jailbreak research repositories.
- CI run budget-capped (token spend limit per run); failing runs are a blocking CI check for Critical/High-tier; non-blocking warning for Medium/Low.
- New PR adding or modifying AI/HAI software that changes archetype triggers a corpus completeness check before merge.

**C) Operate the go-live battery and wire test failures to IM**

Every AI/HAI software artifact must pass its archetype battery before receiving Sanctioned status in the SM inventory. Go-live triggers:
- **Pre-production:** all applicable archetype tests must pass before the artifact is promoted to production; the go-live test record is linked from the SM inventory and the PC intake artifact.
- **Post-model-update:** any model-family swap, model-version promotion, or new fine-tune triggers a re-run of the output-integrity regression and the full archetype battery for the serving layer within 14 days (Critical-tier: within 7 days).
- **Post-incident:** any IM-Software incident involving the artifact triggers a re-run of the relevant battery subset before the incident is closed.
- **Quarterly:** all active AI/HAI software artifacts re-run their battery; results reviewed by the named test-battery owner.

All test failures route to IM-Software within one business day with a severity tag derived from the ST severity rubric (Blocker / Critical / High / Medium / Low). Named battery owner per archetype; battery ownership is a named role, not a shared-team responsibility.

#### Outcome Metrics (L1)

| Metric | Baseline | L1 Target | Source |
|---|---|---|---|
| % of AI/HAI software artifacts reaching production with a passed go-live battery on record | measure | ≥90% within 12 months; 100% for Critical/High-tier | SM inventory × test-run registry |
| Regression corpora published (jailbreak, prompt-injection, AGH, tool-misuse) | 0 / 4 | 4 / 4 | Corpus registry |
| % PR merges for Critical/High-tier artifacts that ran the regression corpus and passed | measure | ≥95% | CI telemetry |
| % archetype threat library entries covered by at least one test or corpus entry | measure | ≥80% by end of year 1 | TA library × test metadata |
| % test failures routed to IM within 1 business day | measure | 100% | Test → IM handoff metrics |

#### Process Metrics (leading)

- Battery owner named per archetype; quarterly re-run scheduled in advance.
- Test-automation coverage, target ≥60% of battery items running in CI without human intervention at L1.
- Corpus refresh cadence honored monthly; corpus owner reviews incoming sources.
- New-archetype lead time, from "first intake in new software category" to "battery published" ≤30 days.

#### Effectiveness Metrics (business value)

- Pre-production catch rate, test failures that uncovered a misconfiguration or control gap before production landing (prompt injection in new agent, tool-scope overshoot in RAG, system-prompt leakage in new LLM-integrated feature).
- Model-swap regression catch rate, output-integrity regressions caught by the battery before the model version shipped to full traffic.
- Reduced IM incident volume, artifacts with a full battery pass-rate in CI have a measurably lower IM incident rate than those without (tracked as a rolling 12-month comparison).

#### Success Criteria

- Per-archetype foundational test battery published for all seven archetypes (LLM-integrated app, agent, RAG, fine-tune/training, eval harness, model-serving service, classical ML), linked from the SM inventory record and the DR/IR artifacts.
- Four regression corpora (jailbreak, prompt-injection, AGH, tool-misuse) published in source control, running in CI on every PR for Critical/High-tier artifacts, with a named corpus owner and a monthly refresh cadence.
- 100% of AI/HAI software artifacts reaching production in the last 90 days have a passed go-live battery on record.
- All test failures routed to IM with a 1-day handoff SLA and named owner.
- Named battery owner per archetype; CI automation covers ≥60% of battery items.

---

## Maturity Level 2
### Objective: Calibrate test depth per risk tier using the SM L2 tier-treatment matrix, run scheduled red-team exercises per tier using TA L2 deep threat models, and test cross-archetype compositions for Critical-tier artifacts

At this level, testing stops treating all AI/HAI software artifacts the same. Per-tier calibration drives what each artifact receives from the test program. Critical-tier artifacts are red-teamed quarterly; High-tier semi-annually. TA L2 deep threat models replace archetype snapshots as the scenario library for red-team exercises. Cross-archetype compositions (e.g., a customer-facing agent that also does RAG and calls a fine-tuned model) receive combined test suites that exercise composition-specific failure modes.

#### Dependencies

- **ST-Software L1 (required):** per-archetype batteries and regression corpora must be operational before per-tier calibration is meaningful.
- **SM-Software L2 (required):** the risk-tier rubric and tier-treatment matrix determine which artifacts receive full-scope testing (Critical/High) vs. a subset (Medium/Low); without tier assignments, L2 calibration has no substrate.
- **TA-Software L2 (required):** per-artifact deep threat models for Critical-tier artifacts shape red-team scope; archetype-only snapshots are insufficient for L2 red-team exercises.
- **Supports / unblocks:** IM-Software L2 (tier-calibrated incident handling relies on tier-calibrated test signals), ML-Software L2 (detections tuned per-artifact threat model depend on L2 test findings to validate coverage).

#### Desired Outcomes

- Test depth is visibly differentiated: Critical-tier artifacts get the full battery plus quarterly red-team and cross-archetype composition tests; Low-tier get the base CI corpus and a spot-check.
- Red-team findings are scenario-based (from TA L2 per-artifact threat models), not free-form; findings trace to specific ATLAS tactics and to SR requirements.
- The regression corpora grow from red-team findings: every critical or high-severity red-team finding produces a new corpus entry that runs in CI within 30 days.
- Cross-archetype composition failure modes (agent + RAG interaction, fine-tune + serving-service interaction, eval harness → training feedback loop) are in scope and have documented test coverage.
- Per-tier SLA adherence for testing activities is tracked and reported to the program sponsor.

#### Activities

**A) Tier-calibrated test battery and CI corpus depth**

Publish a per-tier test treatment aligned to SM L2's tier-treatment matrix:

| Treatment | Critical | High | Medium | Low |
|---|---|---|---|---|
| Go-live battery | Full archetype battery, all test classes, executive sign-off on results | Full archetype battery in CI | Subset battery (top-4 threat classes) in CI | Spot-check (3 test classes) |
| Regression corpus | All 4 corpora running on every PR; Critical corpus separately tuned | All 4 corpora on merge | Jailbreak + prompt-injection corpus on merge | Jailbreak corpus on merge |
| Model-update re-run | Full battery within 7 days of any model-family swap or version promotion | Full battery within 14 days | Subset battery within 30 days | Output-integrity regression at next quarterly |
| Output-integrity | Daily regression run against production golden test set; drift fires an IM alert | Weekly regression run | Monthly regression run | Quarterly regression run |
| Logging-completeness | Verified quarterly by ST; findings route to IM within 1 BD | Verified semi-annually | Verified annually | Verified at go-live |

For Critical and High-tier artifacts, the CI regression corpus is separated into:
- A Critical-tier corpus running on every PR (30–100 entries, tuned to the artifact's specific tool set, retrieval sources, and data classes from the TA L2 per-artifact threat model).
- A broader tier-appropriate corpus running on merge.

**B) Scheduled per-tier red-team exercises using TA L2 threat models**

Red-team cadence by tier:
- **Critical:** quarterly (4 per year); scope derived from TA L2 per-artifact deep threat model; covers prompt-injection chains, indirect prompt injection via RAG retrieval, agent tool abuse, jailbreak regression, data-egress canaries, multi-turn AGH probes, cross-tenant isolation probes.
- **High:** semi-annual (2 per year); scope from TA L2 artifact deltas; covers the top-5 threats from the per-artifact model.
- **Medium/Low:** ad-hoc (before major model changes or scope expansions); archetype snapshot drives scope.

Each red-team exercise follows the AI Security Testing Methodology §9: written rules of engagement, test plan reviewed with artifact owner, execution log, structured findings report (severity / root cause / ATLAS tactic ID / SR requirement traced), remediation pairings. Red-team findings at Critical or High severity produce corpus entries within 30 days.

Cross-archetype composition tests for Critical-tier:
- Agent + RAG composition: indirect-injection via RAG path into agent goal-hijack chain.
- Fine-tune + model-serving: membership-inference probe against the served fine-tuned model; canary extraction from production completions.
- Multi-agent orchestration: sub-agent goal-hijack via orchestrator response; scope inheritance tests (sub-agent cannot exceed parent scope).

**C) Red-team findings → corpus pipeline**

Every Critical or High-severity red-team finding produces:
1. A new corpus entry (input, expected safe output, threat tag, ATLAS tactic ID, date, source reference) committed to the relevant regression corpus within 30 days.
2. An IM-Software finding with severity tag and the named artifact owner as assignee.
3. A TA-Software library-gap ticket if the finding was not in the archetype library, the gap is tracked with a named owner and a 30-day close SLA for Critical-tier gaps.

#### Outcome Metrics (L2)

| Metric | Baseline | L2 Target | Source |
|---|---|---|---|
| % Critical-tier artifacts red-teamed in last 90 days | measure | 100% | ST records |
| % High-tier artifacts red-teamed in last 180 days | measure | 100% | ST records |
| Regression corpus growth rate, Critical-tier corpora | measure | ≥1 new entry per month from red-team or incident findings | Corpus change-log |
| % red-team findings (Critical/High severity) converted to corpus entries within 30 days | measure | ≥90% | Finding → corpus pipeline telemetry |
| Per-tier SLA adherence for testing activities (go-live battery, model-update re-run, red-team cadence) | measure | ≥90% per tier | Program telemetry |

#### Process Metrics (leading)

- Red-team schedule on calendar; no Critical-tier artifact skips a quarterly exercise.
- Corpus review cadence, monthly.
- Cross-archetype composition test plan published for each Critical-tier artifact with composition; reviewed by named architect.
- Finding → TA library-gap pipeline: Critical gaps closed within 30 days; High within 60 days.

#### Effectiveness Metrics (business value)

- Pre-production catch rate for Critical-tier increases as red-team exercises catch issues before model-update deploys.
- Regression corpus catches model-update regressions early, model-swap regressions that would have reached production users, detected by CI corpus before full traffic rollout.
- Incident rate for Critical-tier artifacts with full-scope testing is measurably lower than for those without (rolling 12-month comparison).

#### Success Criteria

- Quarterly red-team for 100% of Critical-tier artifacts; semi-annual for 100% of High-tier; scope tied to TA L2 per-artifact deep threat models.
- Critical-tier regression corpora (all 4) running on every PR for all Critical-tier artifacts; per-tier calibration enforced.
- ≥90% of Critical/High-severity red-team findings converted to corpus entries within 30 days.
- Cross-archetype composition tests documented and run for all Critical-tier artifacts with composite archetypes.
- Per-tier SLA adherence for testing activities ≥90%.

---

## Maturity Level 3
### Objective: Operate continuous automated adversarial testing for Critical-tier artifacts, publish regression corpora and findings as open artifacts, and contribute discovered TTPs to MITRE ATLAS, AVID, and OWASP LLM / Agentic Top 10

At this level, testing runs continuously rather than periodically. An automated adversarial testing harness, AI-vs-AI red-team scaffolding, probes Critical-tier artifacts daily using generated jailbreak ladders, indirect-injection seeders, and tool-misuse generators. Novel findings are triaged into the TA library weekly. Anonymized regression corpora, test patterns, and discovered TTPs are contributed to MITRE ATLAS, AVID, OWASP LLM/Agentic Top 10, and external benchmarks (HELM, AISI Inspect evaluations, sector ISAC AI working groups).

#### Dependencies

- **ST-Software L2 (required):** per-tier red-team function, regression corpora, and finding → corpus pipeline must be operational.
- **TA-Software L3 (required):** automated TTP-ingestion pipeline from ST telemetry feeds the TA L3 auto-update loop.
- **ML-Software L2+ (required):** detections instrumented to catch automated-probe activity so the harness and the monitoring pipeline are calibrated against each other.
- **SM-Software L3 (alignment):** automated inventory signals feed the automated test harness with the current Critical-tier artifact list and their tier metadata.

#### Desired Outcomes

- Critical-tier adversarial posture is measured daily, not quarterly.
- Novel TTPs discovered by the automated harness reach the TA library within 14 days, the org's defenses update faster than most peers' quarterly red-team cycles.
- Program-originated findings appear in MITRE ATLAS, AVID, and OWASP revisions, the org is a net contributor to the AI software security ecosystem.
- Published regression corpora and test patterns are adopted by peer organizations, demonstrating industry-leading testing maturity.

#### Activities

**A) Continuous automated adversarial testing harness**

Deploy an AI-vs-AI automated red-team scaffolding that runs daily against all Critical-tier AI/HAI software artifacts:
- **Prompt-injection generator:** produces novel direct and indirect prompt-injection inputs using mutation of the regression corpus, template-based variation, and model-assisted jailbreak-ladder generation; runs against LLM-integrated app, agent, and RAG archetypes.
- **Indirect-injection seeder:** generates poisoned-document payloads designed to redirect agent goals; seeds a test corpus environment and exercises the retrieval → prompt → action pipeline.
- **Tool-misuse generator:** generates argument-smuggling variants for each declared tool in each Critical-tier agent; probes argument validation boundaries and unexpected tool combinations.
- **Output-integrity monitor:** runs the golden test set daily against production endpoints; any drift triggers a P1 alert to IM-Software and a re-run of the full regression corpus.

Findings triaged by a named ST owner at least weekly. New TTPs (patterns not in the TA library) fed into the TA L3 auto-proposal pipeline within 14 days. High-severity automated findings route to IM-Software within 24 hours.

**B) Contribute findings to industry**

Contribute anonymized, legally-vetted findings to:
- **MITRE ATLAS**, new technique observations (novel prompt-injection variants, agent-loop attack patterns, new retrieval-poisoning mechanics); submissions follow ATLAS evidence-and-provenance requirements; target ≥4 contributions per year.
- **AI Vulnerability Database (AVID)**, structured disclosure submissions for novel vulnerabilities in own-built AI/HAI software or its upstream dependencies (coordinated disclosure where third-party components are involved).
- **OWASP LLM Top 10 / Agentic Top 10**, real-world telemetry evidence during revision cycles; target ≥2 substantive submissions per revision cycle.
- **External benchmarks**, participate in AISI Inspect evaluation benchmarks, HELM safety evaluations, and sector ISAC AI red-team exercises with anonymized findings.

**C) Publish regression corpora and test patterns as open artifacts**

- Publish anonymized versions of the four regression corpora (jailbreak, prompt-injection, AGH, tool-misuse) under an open license; scrubbed of org-specific tool names, data classes, and artifact identifiers.
- Maintain the published versions upstream; internal corpora are a superset of the published versions with org-specific entries not shared externally.
- Host or co-host at least one industry red-team benchmark per year (OWASP AI chapter, ATLAS practitioner table, sector ISAC AI working group); collect cross-org detection-benchmark improvement data.

#### Outcome Metrics (L3)

| Metric | Baseline | L3 Target | Source |
|---|---|---|---|
| % Critical-tier artifacts under continuous automated adversarial testing (daily probe execution) | measure | ≥80% | ST harness telemetry |
| New-TTP ingestion lead time (automated finding to TA library entry) | measure | ≤14 days | Harness → TA pipeline telemetry |
| Industry contributions per year (MITRE ATLAS / AVID / OWASP) | 0 | ≥4 | Contribution log |
| Open regression corpus published and maintained upstream | 0 | ≥4 corpora published | External repository |
| Industry-shared exercises per year | 0 | ≥1 hosted + ≥2 participated | Exercise log |

#### Process Metrics (leading)

- Continuous harness health, % Critical-tier artifacts producing a fresh automated probe result within the last 24 hours; on-call paged when a harness feed goes stale >24 hours.
- New-TTP triage cadence, automated probe findings reviewed weekly by named ST owner; novel patterns forwarded to TA library within 14 days.
- Industry-contribution pipeline, at least one anonymized finding in-preparation, in-legal-review, or submitted at any time.
- Industry-exercise calendar, next hosted or co-hosted exercise scheduled at least 60 days in advance.

#### Effectiveness Metrics (business value)

- Mean time to detect novel AI-specific attack techniques decreases as the continuous harness catches model-update regressions within hours, not sprint cycles.
- Program-originated TTPs recognized in MITRE ATLAS or AVID demonstrate external validation of testing rigor.
- Cross-org exercise participants cite improved detection benchmarks from shared corpora, measurable uplift in peer organizations' defenses.
- Critical-tier incidents attributable to prompt injection, tool-scope abuse, or data-egress drop as the continuous harness closes the gap between quarterly red-team cycles.

#### Success Criteria

- ≥80% of Critical-tier AI/HAI software artifacts under continuous automated adversarial testing with daily probe execution.
- New-TTP ingestion lead time ≤14 days; automated findings triaged weekly by named owner.
- ≥4 industry contributions per year to MITRE ATLAS / AVID / OWASP LLM / Agentic Top 10.
- ≥4 open regression corpora published under a permissive license and maintained upstream.
- ≥1 industry-shared exercise hosted per year plus ≥2 participated; cross-org detection-benchmark improvement documented.

---

## Key Success Indicators

**Level 1:**
- Per-archetype foundational test battery published for all seven archetypes (LLM-integrated app, agent, RAG, fine-tune/training, eval harness, model-serving service, classical ML), each test class tied to a TA-Software archetype threat and an SR-Software requirement.
- Four regression corpora (jailbreak, prompt-injection, AGH, tool-misuse) in source control, running on every PR for Critical/High-tier artifacts, with named corpus owner and monthly refresh cadence.
- 100% of AI/HAI software artifacts reaching production in the last 90 days have a passed go-live battery on record linked from the SM inventory.
- All test failures routed to IM within 1 business day; CI automation covers ≥60% of battery items.
- TA-Software archetype threat coverage by test battery ≥80%.

**Level 2:**
- 100% of Critical-tier artifacts red-teamed quarterly; 100% of High-tier semi-annually; scope tied to TA L2 per-artifact deep threat models.
- Per-tier calibration enforced: Critical-tier gets all 4 corpora on every PR plus daily output-integrity regression; Low-tier gets jailbreak corpus on merge.
- ≥90% of Critical/High-severity red-team findings converted to corpus entries within 30 days.
- Cross-archetype composition tests documented and run for all Critical-tier composite artifacts.

**Level 3:**
- ≥80% of Critical-tier artifacts under continuous automated adversarial testing with daily probe execution; novel TTPs reaching TA library within 14 days.
- ≥4 industry contributions per year to MITRE ATLAS / AVID / OWASP LLM / Agentic Top 10; ≥4 open regression corpora published and maintained.
- ≥1 hosted + ≥2 participated industry exercises per year with documented cross-org detection-benchmark improvement.

---

## Common Pitfalls

**Level 1:**
- ❌ Test battery reduced to a logging-completeness check and a golden-path assertion, no behavioral adversarial probes (prompt injection, tool-scope boundary, kill-switch) actually exercised.
- ❌ Regression corpora committed to source control but not wired into CI, they exist but run only when a reviewer manually triggers them; coverage erodes after every sprint.
- ❌ Go-live battery runs once pre-production but is never re-run after model-version updates, test coverage erodes as model versions change and configurations drift.
- ❌ Test failures logged in a spreadsheet separate from IM, no SLA enforcement, no aging visibility, no named owner; the same failure recurs across multiple PRs undetected.
- ❌ Corpus refresh skipped for three months because "nothing new happened", public jailbreak research and ATLAS technique additions pile up unread; the corpus is frozen at launch while the external threat landscape advances.
- ❌ Test battery conflates agent and LLM-integrated app, agent-specific tests (kill-switch, HITL gate, tool-scope boundary) are absent from the agent battery because the same checklist was applied to both archetypes.
- ❌ Training-data canary planted at fine-tune time but never probed via the serving endpoint, the leakage test is half-implemented; the canary is there but extraction is never attempted.

**Level 2:**
- ❌ Red-team scope defined as "prompt-injection probes" but indirect-prompt-injection via RAG retrieval, argument smuggling, multi-turn AGH chains, and cross-tenant isolation tests are excluded, the top threat classes for agent + RAG compositions go untested.
- ❌ Corpus growth declared at ≥1 per month but entries are variations on the same payload format, the corpus does not cover the new jailbreak techniques published since L2 launch; model-update regressions pass the corpus and fail in production.
- ❌ Per-tier calibration documented in the tier-treatment matrix but CI pipeline applies the same corpus to all tiers, Critical artifacts run the same tests as Low; differentiation exists on paper only.
- ❌ Red-team findings route to IM but the finding → corpus pipeline is never executed, 12 months of Critical/High findings sit in IM as closed tickets with no corpus entries; the same vulnerabilities are re-discovered at the next red-team exercise.
- ❌ Cross-archetype composition tests scoped but not executed because "no engineer owns agent-plus-RAG testing", composition-specific failure modes (indirect injection via RAG into agent goal-hijack chain) are in the threat model but not in any test.

**Level 3:**
- ❌ Continuous harness runs prompt-injection probes that the artifact's content filter trivially blocks, coverage metric looks good but the probes are not exercising the real threat surface; novel jailbreak techniques are not generated.
- ❌ Industry contributions are legal-vetted case-study summaries rather than actionable, reproducible technique descriptions, ATLAS reviewers cannot map them to a technique ID; AVID submissions lack reproducibility notes.
- ❌ Open corpora published once and then not maintained, external organizations build on a stale version while the internal corpus has 60 new entries; discrepancies surface at community exercises and damage the program's credibility.
- ❌ New-TTP ingestion from automated probes to the TA library is manual and quarterly, by the time a novel technique reaches SR and SA updates and is reflected in controls, the technique is already being exploited in the wild.
- ❌ Hosted industry exercise becomes a capabilities showcase rather than a detection-benchmarking session, no measurable improvement data is collected from participants; the "≥1 hosted per year" metric is met without producing any cross-org security uplift.

---

## Practice Maturity Questions

**Level 1:**
1. Is a per-archetype foundational test battery published for all seven AI/HAI software archetypes, with each test class tied to a TA-Software archetype threat (HAI TTP + ATLAS tactic ID) and an SR-Software requirement, defined inputs/outputs/pass-fail criteria, and an evidence artifact, and are 100% of new AI/HAI software artifacts required to pass the battery before production Sanctioned status is issued?
2. Are four regression corpora (jailbreak, prompt-injection, AGH, tool-misuse) versioned in source control, running in CI on every PR for Critical/High-tier artifacts, with a named corpus owner, a monthly refresh cadence from internal + external sources, and a CI token-spend budget cap, and are ≥95% of Critical/High-tier PR merges verified to have run and passed the corpus?
3. Are all test failures routed to IM-Software within 1 business day with a severity tag and named owner, and does TA-Software archetype threat coverage by the test battery and corpus reach ≥80% by end of year one?

**Level 2:**
1. Are 100% of Critical-tier AI/HAI software artifacts red-teamed at least quarterly, and 100% of High-tier semi-annually, with scope derived from TA-Software L2 per-artifact deep threat models, covering prompt-injection chains, indirect-prompt-injection via RAG, agent tool abuse, multi-turn AGH probes, data-egress canaries, and cross-tenant isolation, with findings routed to IM and remediation tracked?
2. Is per-tier corpus calibration enforced in CI (Critical-tier: all 4 corpora on every PR + daily output-integrity regression; Low-tier: jailbreak corpus on merge), and are ≥90% of Critical/High-severity red-team findings converted to corpus entries within 30 days?
3. Are cross-archetype composition tests (agent + RAG, fine-tune + model-serving, multi-agent orchestration) documented and executed for all Critical-tier composite artifacts, and is per-tier SLA adherence for testing activities ≥90%?

**Level 3:**
1. Are ≥80% of Critical-tier AI/HAI software artifacts under continuous automated adversarial testing with daily probe execution, with novel TTPs triaged into the TA-Software library within 14 days and high-severity automated findings routed to IM within 24 hours?
2. Has the program contributed ≥4 anonymized, legally-vetted findings per year to MITRE ATLAS, AVID, or OWASP LLM/Agentic Top 10, with at least one accepted as a new or refined technique, and are ≥4 open regression corpora published under a permissive license and maintained upstream?
3. Has the program hosted at least 1 industry-shared red-team exercise per year and participated in ≥2 additional cross-org exercises, with documented cross-org detection-benchmark improvement data from participants?

---

**Document Version:** HAIAMM v3.0
**Practice:** Security Testing (ST)
**Domain:** Software
**Last Updated:** 2026-05-13
**Author:** Verifhai