# Security Testing (ST)
## Infrastructure Domain - HAIAMM v3.0

---

### Practice Overview

**Objective:** Prove that every AI/HAI infrastructure component the organization **operates** behaves correctly under adversarial conditions, by running a foundational per-archetype test battery in CI and on a defined cadence, maintaining versioned regression corpora, and escalating to scheduled red-team and continuous adversarial testing at higher maturity levels.

**Description:** ST-Infrastructure exercises the AI/HAI infrastructure the organization *runs*, inference endpoints, model registries, GPU / accelerator fleets, orchestrator control planes, vector-store infrastructure, AI-specific CI/CD pipelines, and feature stores, against a battery of AI-specific test classes tied directly to the threats in the TA-Infrastructure library and the requirements in the SR-Infrastructure pack. At L1, every archetype has a published test battery (model-extraction probes, cross-tenant isolation tests, GPU residual-state-clearing tests, workflow-signing enforcement tests, retrieval-extraction probes, pipeline-signing tests, feature-poisoning detection tests) plus versioned regression corpora running on a defined cadence. L2 adds per-tier scheduled red-team exercises using TA-Infrastructure L2 deep threat models, and cross-archetype composition tests (inference endpoint + GPU fleet residual-state composition). L3 operates continuous automated adversarial testing and contributes findings to MITRE ATLAS, AVID, CNCF AI infrastructure advisories, and OWASP LLM / Agentic Top 10 infrastructure patterns.

**Context:** Classic infrastructure testing exercises availability and performance, leaving AI-specific adversarial paths untested. An inference endpoint passes all health checks and then leaks embeddings at scale through an extraction-pattern query campaign. A GPU fleet passes scheduling tests and then allows job B to read job A's residual GPU memory. A model registry passes its access-control tests and then accepts an unsigned model artifact because the signing enforcement flag was reset by a platform update. These failures are invisible to classic infrastructure testing because classic testing was not designed for AI-specific infrastructure failure modes, model extraction (ATLAS TA0004), cross-tenant retrieval extraction (ATLAS TA0013), ML attack staging via the pipeline (ATLAS TA0012), and initial access through the inference endpoint surface (ATLAS TA0001). ST-Infrastructure closes this gap by making AI-specific infrastructure tests a first-class testing citizen and by connecting them directly to the TA threat library so test coverage tracks threat coverage, not just availability coverage.

---

## Maturity Level 1
### Objective: Establish a foundational per-archetype test battery and regression corpora that run on a defined cadence, and verify that every AI/HAI infrastructure component reaches production with a passed go-live battery on record

At this level, the organization gives every AI/HAI infrastructure component a documented, automated-where-possible test battery drawn from the TA-Infrastructure archetype threat library and the SR-Infrastructure requirements pack, so every production landing is backed by observable test evidence, not only design assurance.

#### Dependencies

- **TA-Infrastructure L1 (required):** tests target specific threats from the archetype threat library; without the library, test scope is invented per intake rather than inherited.
- **SR-Infrastructure L1 (required):** requirements pack defines what a passing test means; tests answer "does this component meet its SR requirements?" not a free-form question.
- **SA-Infrastructure L1 (required):** reference patterns define the control points (workload identity, isolation policy, signing enforcement, rate-limit layer) that tests exercise.
- **IR-Infrastructure L1 (required):** implementation reviews confirm the component is configured as designed before ST probes run; testing a misconfigured component produces noise.
- **Supports / unblocks:** IM-Infrastructure L1 (test failures become issues), ML-Infrastructure L1 (detections validated by logging-completeness tests), EH-Infrastructure L1 (hardening informed by test findings).

#### Desired Outcomes

- Every AI/HAI infrastructure component reaching production has passed a documented per-archetype test battery; the battery result is on file and linked from the SM-Infrastructure inventory record.
- Regression corpora (model-extraction corpus, cross-tenant isolation probe corpus, GPU residual-state corpus, workflow-injection corpus, retrieval-extraction corpus, pipeline-signing corpus) run on a defined cadence and catch AI-specific regressions before components go live.
- Test failures become IM issues with named owners within one business day, not reports in a drive.
- The battery is automated enough to re-run on demand (post-configuration-update, post-incident, on cadence) without a bespoke effort per run.
- Test coverage tracks threat coverage: every threat in the TA-Infrastructure library for a given archetype has at least one corresponding test in the battery or corpus.

#### Activities

**A) Publish the foundational per-archetype test battery**

Ship one test battery per AI/HAI infrastructure archetype. Each battery targets the top archetype threats from TA-Infrastructure and the archetype-specific SR requirements. Each test class has: inputs, expected output, pass/fail criteria, evidence artifact (probe result / log snippet / CI run link), and the TA threat and SR requirement it maps to.

**Inference endpoint / model-serving cluster:**
- *Model-extraction probe* (rate-limit holds against extraction-pattern queries): send a sequence of prefix-completion queries at the rate-limit ceiling; assert the rate-limit layer enforces the cap before query volume reaches extraction-pattern scale; verify the inference endpoint logs the rate-limit event (ATLAS TA0004 ML Model Access / TA0013 Exfiltration).
- *Cross-tenant isolation test*: with a multi-tenant deployment, send a request with tenant A's credentials that references tenant B's model endpoint, model version, or model artifact; verify rejection and confirm the rejection is logged with tenant context (EA; cross-tenant leakage).
- *DoS / prompt-flood resilience test*: send inputs above the declared burst limit; assert the rate-limit and abuse-detection layer enforces the cap; verify degraded-mode fallback produces a logged, non-misleading response and does not cascade to other tenants.
- *Model-swap detection test* (canary signal flips when model version changes silently): assert the serving endpoint returns the declared model version in response metadata; attempt to promote a new model version without a canary signal and verify the canary slot flips before full traffic promotion.
- *Logging-completeness test*: for a known inference request, verify required log fields (tenant ID, session ID, timestamp, model version, prompt hash, completion hash) appear in the org-side log store within the retention SLA (ATLAS TA0013 Exfiltration, completeness is the defense evidence).

**Model registry:**
- *Unauthorized-upload probe*: attempt to push a model artifact without promotion rights; verify the registry rejects the upload and logs the attempt (ATLAS TA0004 ML Model Access).
- *Signed-artifact enforcement test*: attempt to push an unsigned model artifact with valid promotion credentials; verify the registry rejects the artifact and logs the policy violation.
- *Lineage-required test*: attempt to promote a model artifact without a complete lineage record (missing training data version, missing eval suite results); verify the promotion gate blocks it.
- *Rollback-abuse probe*: attempt to roll back to a retired model version that has a known critical finding in the IM backlog without an explicit overriding authorization; verify the registry requires authorization and logs the rollback event.

**GPU / accelerator fleet:**
- *Residual-state-clearing test*: run job A on a GPU node (job A writes a known pattern to GPU memory); then run job B on the same GPU node; verify job B cannot read job A's memory pattern (test record includes job IDs, node ID, and confirmation of inaccessibility) (ATLAS TA0012 ML Attack Staging, residual state is an attack staging vector).
- *Classification-aware scheduler test*: attempt to schedule a Critical-tier or regulated workload on a node designated as shared; verify the scheduler rejects or redirects the placement.
- *Credential-isolation test*: confirm that job A's credentials (IAM role, service account token, API key) cannot be used by job B running on the same node after job A completes; verify via probe using job A's credential identity after job A's completion.

**Orchestrator / control plane:**
- *Workflow-signing test* (unsigned workflow rejected): attempt to submit an unsigned workflow definition to the production orchestrator; verify rejection and confirm the rejection is logged (ATLAS TA0012 ML Attack Staging, unsigned workflows are a staging vector).
- *Per-step principal test*: confirm each workflow step runs under its own minimal-scope identity; run a step that attempts to invoke a resource outside its declared principal scope and verify rejection.
- *Control-plane API auth test*: send an unauthenticated request to the control-plane API; verify 401/403 response and confirm the attempt is logged.
- *Agent-state tampering test*: inject a crafted payload into a workflow step's input that attempts to modify the orchestrator's in-flight state; verify the state isolation boundary prevents the modification (AGH; ATLAS TA0012).

**Vector-store infrastructure:**
- *Cross-tenant retrieval probe*: send a query from tenant A that would match embeddings indexed under tenant B's namespace; verify zero results or a namespacing error, not tenant B's data (ATLAS TA0013 Exfiltration; EA).
- *Embedding-extraction at scale test*: send a sequence of nearest-neighbor queries designed to reconstruct a significant portion of a tenant's embedding space; verify the rate-limit and query-scope bounds prevent extraction at scale before the embedding space is recoverable (ATLAS TA0013 Exfiltration).
- *Retrieval-policy-bypass probe*: attempt to retrieve a document classified at a level above the requesting principal's clearance through a crafted query; verify the classification-label-respecting retrieval policy blocks or filters the result (EA; cross-classification leakage).

**AI-specific CI/CD:**
- *Pipeline-signing test*: attempt to trigger a pipeline run using an unsigned pipeline definition; verify rejection and confirm the rejection is logged (ATLAS TA0012 ML Attack Staging, unsigned pipelines are a staging vector).
- *SLSA-provenance verification test*: pull a recently promoted model artifact; verify a valid SLSA provenance attestation is attached and passes verification.
- *Eval-gate enforcement test*: attempt to promote a model artifact through the pipeline without a passing eval attestation; verify the gate blocks the promotion and logs the blocked event.
- *Secrets-leak probe*: scan a recent pipeline run's log output and artifact storage for credential patterns (API key patterns, token formats, private key headers); assert zero findings (ATLAS TA0001 Reconnaissance, leaked secrets reduce the attack surface needed).

**Feature store / online serving cache:**
- *Feature-poisoning detection test*: inject a synthetic anomalous feature value into the online serving cache; verify the offline/online skew alert fires within the declared detection window.
- *Offline/online-skew alert test*: create a controlled skew between the offline feature store and the online serving cache; verify the skew-monitoring alert fires within the declared detection window and produces an IM ticket.
- *Access-control test*: attempt to read features with an identity that has no declared read rights; verify rejection and confirm the attempt is logged.

**B) Build and maintain regression corpora**

Maintain six versioned regression corpora, running on a defined cadence. Each corpus is a collection of structured test fixtures: input, expected safe outcome, threat tag (HAI TTP + ATLAS tactic ID), source, date added.

- **Model-extraction corpus**, 20–60 prefix-completion and nearest-neighbor query patterns designed to probe for model extraction; run against inference endpoint and vector-store archetypes on a monthly cadence; Critical-tier: run before any model version promotion.
- **Cross-tenant-isolation probe corpus**, 20–60 cross-tenant query patterns across inference endpoint, vector-store, and GPU fleet archetypes; run on a monthly cadence; failure blocks production promotion for Critical-tier components.
- **GPU-residual-state corpus**, 15–40 job-pair test configurations documenting memory patterns that should be inaccessible after job completion; run quarterly or after any GPU fleet scheduling policy change.
- **Workflow-injection corpus**, 20–50 crafted workflow step payloads and unsigned workflow definitions designed to probe signing enforcement, per-step principal enforcement, and state-isolation; run against orchestrator archetypes on a monthly cadence.
- **Retrieval-extraction corpus**, 20–60 query sequences designed to probe for cross-tenant retrieval, embedding extraction at scale, and classification-label bypass; run against vector-store archetypes on a monthly cadence.
- **Pipeline-signing corpus**, 15–40 pipeline trigger attempts with unsigned definitions, missing provenance, and bypass variations; run against AI-CI/CD archetypes on a monthly cadence.

Corpus management:
- Versioned in source control; corpus changes go through review with a named corpus owner.
- Corpus refresh cadence: monthly minimum from three sources, internal observations (IR findings, IM incidents, red-team results), external public corpora (ATLAS technique examples, OWASP LLM / Agentic Top 10 infrastructure examples), and MLOps security research.
- New AI/HAI infrastructure component provisioning triggers a corpus completeness check against the component's archetype battery.

**C) Operate the go-live battery and wire test failures to IM**

Every AI/HAI infrastructure component must pass its archetype battery before receiving Sanctioned status in the SM-Infrastructure inventory. Go-live triggers:
- **Pre-production:** all applicable archetype tests must pass before the component is promoted to production; the go-live test record is linked from the SM-Infrastructure inventory and the PC intake artifact.
- **Post-configuration-update:** any GPU scheduling policy change, signing policy change, rate-limit policy change, or isolation policy change triggers a re-run of the relevant battery subset within 7 days for Critical-tier components and within 14 days for High-tier.
- **Post-incident:** any IM-Infrastructure incident involving the component triggers a re-run of the relevant battery subset before the incident is closed.
- **Quarterly:** all active AI/HAI infrastructure components re-run their battery; results reviewed by the named test-battery owner.

All test failures route to IM-Infrastructure within one business day with a severity tag. Named battery owner per archetype; battery ownership is a named role, not a shared-team responsibility.

#### Outcome Metrics (L1)

| Metric | Baseline | L1 Target | Source |
|---|---|---|---|
| % of AI/HAI infrastructure components reaching production with a passed go-live battery on record | measure | ≥90% within 12 months; 100% for Critical/High-tier | SM inventory × test-run registry |
| Regression corpora published (model-extraction, cross-tenant-isolation, GPU-residual-state, workflow-injection, retrieval-extraction, pipeline-signing) | 0 / 6 | 6 / 6 | Corpus registry |
| % archetype threat library entries covered by at least one test or corpus entry | measure | ≥80% by end of year 1 | TA library × test metadata |
| % test failures routed to IM within 1 business day | measure | 100% | Test → IM handoff metrics |
| Quarterly battery re-runs completed for all active components | measure | ≥90% per quarter | Battery run registry |

#### Process Metrics (leading)

- Battery owner named per archetype; quarterly re-run scheduled in advance.
- Test-automation coverage, target ≥60% of battery items running without human intervention at L1.
- Corpus refresh cadence honored monthly; corpus owner reviews incoming sources.
- New-archetype lead time, from "first intake in new infrastructure category" to "battery published" ≤30 days.

#### Effectiveness Metrics (business value)

- Pre-production catch rate, test failures that uncovered a misconfiguration or control gap before production landing (GPU residual-state-clearing misconfigured, cross-tenant isolation not enforced, unsigned pipeline accepted).
- Configuration-change regression catch rate, adversarial probes catching control regressions introduced by infrastructure configuration changes before they reach production traffic.
- Reduced IM incident volume, components with a full battery pass-rate have a measurably lower IM incident rate than those without.

#### Success Criteria

- Per-archetype foundational test battery published for all seven archetypes (inference endpoint, model registry, GPU fleet, orchestrator, vector-store, AI-CI/CD, feature store), linked from the SM-Infrastructure inventory record and the DR/IR artifacts.
- Six regression corpora published in source control, running on a monthly-or-better cadence for Critical/High-tier components, with a named corpus owner and a monthly refresh cadence.
- 100% of AI/HAI infrastructure components reaching production in the last 90 days have a passed go-live battery on record.
- All test failures routed to IM with a 1-day handoff SLA and named owner.
- Named battery owner per archetype; automation covers ≥60% of battery items.

---

## Maturity Level 2
### Objective: Calibrate test depth per risk tier using the SM-Infrastructure L2 tier-treatment matrix, run scheduled per-tier red-team exercises using TA-Infrastructure L2 deep threat models, and test cross-archetype compositions, particularly inference endpoint and GPU fleet residual-state composition, for Critical-tier components

At this level, testing stops treating all AI/HAI infrastructure components the same. Per-tier calibration drives what each component receives from the test program. Critical-tier components are red-teamed quarterly; High-tier semi-annually. TA-Infrastructure L2 deep threat models replace archetype snapshots as the scenario library. Cross-archetype composition tests (e.g., inference endpoint + GPU fleet residual-state composition; orchestrator + vector-store retrieval-injection chain) receive combined test suites that exercise composition-specific failure modes.

#### Dependencies

- **ST-Infrastructure L1 (required):** per-archetype batteries and regression corpora must be operational before per-tier calibration is meaningful.
- **SM-Infrastructure L2 (required):** the risk-tier rubric (Critical / High / Medium / Low) and tier-treatment matrix determine which components receive full-scope testing vs. a subset; without tier assignments, L2 calibration has no substrate.
- **TA-Infrastructure L2 (required):** per-component deep threat models for Critical-tier components shape red-team scope; archetype-only snapshots are insufficient for L2 red-team exercises.
- **Supports / unblocks:** IM-Infrastructure L2 (tier-calibrated incident handling relies on tier-calibrated test signals), ML-Infrastructure L2 (detections tuned per-component threat model depend on L2 test findings to validate coverage).

#### Desired Outcomes

- Test depth is visibly differentiated: Critical-tier components get the full battery plus quarterly red-team and cross-archetype composition tests; Low-tier get the base corpus and a spot-check.
- Red-team findings are scenario-based (from TA-Infrastructure L2 per-component deep threat models), not free-form; findings trace to specific ATLAS tactics and to SR-Infrastructure requirements.
- The regression corpora grow from red-team findings: every critical or high-severity red-team finding produces a new corpus entry that runs on cadence within 30 days.
- Cross-archetype composition failure modes (inference endpoint + GPU fleet residual-state, orchestrator + vector-store retrieval-injection, AI-CI/CD pipeline + model registry tampering) are in scope and have documented test coverage.

#### Activities

**A) Tier-calibrated test battery and corpus depth**

Publish a per-tier test treatment aligned to SM-Infrastructure L2's tier-treatment matrix:

| Treatment | Critical | High | Medium | Low |
|---|---|---|---|---|
| Go-live battery | Full archetype battery, all test classes, executive sign-off on results | Full archetype battery | Subset battery (top-4 threat classes) | Spot-check (3 test classes) |
| Regression corpus | All 6 corpora on monthly cadence; Critical corpora separately tuned per component | All 6 corpora on monthly cadence | Model-extraction + cross-tenant-isolation on quarterly cadence | Model-extraction corpus on quarterly cadence |
| Post-config-update re-run | Full battery within 7 days of any policy change | Full battery within 14 days | Subset battery within 30 days | Relevant corpus subset at next quarterly |
| Logging-completeness | Verified quarterly by ST; findings route to IM within 1 BD | Verified semi-annually | Verified annually | Verified at go-live |

For Critical-tier components, the regression corpus is maintained as a component-specific tuned corpus (probe inputs specific to this component's tenant population, model versions, query patterns, and GPU fleet configuration) in addition to the archetype-level corpus.

**B) Scheduled per-tier red-team exercises using TA-Infrastructure L2 threat models**

Red-team cadence by tier:
- **Critical:** quarterly (4 per year); scope derived from TA-Infrastructure L2 per-component deep threat model; covers model-extraction campaigns against inference endpoint, cross-tenant retrieval extraction against vector-store, GPU residual-state composition with inference endpoint, workflow-injection against orchestrator, pipeline-tampering against AI-CI/CD, and feature-poisoning chains.
- **High:** semi-annual (2 per year); scope from TA-Infrastructure L2 component deltas; covers the top-5 threats from the per-component model.
- **Medium / Low:** ad-hoc (before major configuration changes or tenant expansions); archetype snapshot drives scope.

Cross-archetype composition tests for Critical-tier:
- **Inference endpoint + GPU fleet residual-state composition:** does a model served from a GPU node retain extractable state from a prior tenant's job? Run job A from tenant A on a GPU node, then run inference from tenant B on the same GPU node; probe for evidence of tenant A's job residual state in tenant B's completions (ATLAS TA0004 / TA0012).
- **Orchestrator + vector-store retrieval-injection chain:** craft a retrieval response from the vector store that contains workflow-injection instructions; verify the orchestrator does not execute the injected instructions (AGH; ATLAS TA0001 / TA0012).
- **AI-CI/CD + model registry tampering:** craft a pipeline run that attempts to replace a signed model artifact in the registry with an unsigned or modified artifact; verify the registry and pipeline signing enforcement prevent the substitution (ATLAS TA0012 ML Attack Staging).

Each red-team exercise follows a written rules of engagement, test plan reviewed with component owner, execution log, and structured findings report (severity / root cause / ATLAS tactic ID / SR requirement traced). Red-team findings at Critical or High severity produce corpus entries within 30 days.

**C) Red-team findings corpus pipeline**

Every Critical or High-severity red-team finding produces:
1. A new corpus entry (input, expected safe outcome, threat tag, ATLAS tactic ID, date, source reference) committed to the relevant regression corpus within 30 days.
2. An IM-Infrastructure finding with severity tag and the named component owner as assignee.
3. A TA-Infrastructure library-gap ticket if the finding was not in the archetype library, the gap is tracked with a named owner and a 30-day close SLA for Critical-tier gaps.

#### Outcome Metrics (L2)

| Metric | Baseline | L2 Target | Source |
|---|---|---|---|
| % Critical-tier components red-teamed in last 90 days | measure | 100% | ST records |
| % High-tier components red-teamed in last 180 days | measure | 100% | ST records |
| Regression corpus growth rate, Critical-tier corpora | measure | ≥1 new entry per month from red-team or incident findings | Corpus change-log |
| % red-team findings (Critical/High severity) converted to corpus entries within 30 days | measure | ≥90% | Finding → corpus pipeline telemetry |
| Per-tier SLA adherence for testing activities | measure | ≥90% per tier | Program telemetry |

#### Process Metrics (leading)

- Red-team schedule on calendar; no Critical-tier component skips a quarterly exercise.
- Corpus review cadence, monthly.
- Cross-archetype composition test plan published for each Critical-tier component with composition dependencies; reviewed by named architect.
- Finding → TA library-gap pipeline: Critical gaps closed within 30 days; High within 60 days.

#### Effectiveness Metrics (business value)

- Pre-production catch rate for Critical-tier increases as red-team exercises catch issues before configuration-change deploys.
- Corpus catches configuration-change regressions early, policy changes that would have reached production, detected before full rollout.
- Incident rate for Critical-tier components with full-scope testing is measurably lower than for those without.

#### Success Criteria

- Quarterly red-team for 100% of Critical-tier components; semi-annual for 100% of High-tier; scope tied to TA-Infrastructure L2 per-component deep threat models.
- All 6 regression corpora running on monthly cadence for all Critical-tier components; per-tier calibration enforced.
- ≥90% of Critical/High-severity red-team findings converted to corpus entries within 30 days.
- Cross-archetype composition tests documented and run for all Critical-tier components with composition dependencies (inference endpoint + GPU fleet, orchestrator + vector-store, AI-CI/CD + model registry).
- Per-tier SLA adherence for testing activities ≥90%.

---

## Maturity Level 3
### Objective: Operate continuous automated adversarial testing for Critical-tier infrastructure components, publish regression corpora and findings as open artifacts, and contribute discovered techniques to MITRE ATLAS, AVID, CNCF AI infrastructure advisories, and OWASP LLM / Agentic Top 10 infrastructure patterns

At this level, testing runs continuously rather than periodically. An automated adversarial testing harness probes Critical-tier infrastructure components daily using generated model-extraction ladder sequences, cross-tenant isolation probes, GPU residual-state composition probes, and pipeline-tampering generators. Novel findings are triaged into the TA-Infrastructure library weekly. Anonymized regression corpora, test patterns, and discovered techniques are contributed to MITRE ATLAS, AVID, CNCF AI infrastructure advisories, and OWASP LLM / Agentic Top 10 infrastructure patterns.

#### Dependencies

- **ST-Infrastructure L2 (required):** per-tier red-team function, regression corpora, and finding → corpus pipeline must be operational.
- **TA-Infrastructure L3 (required):** automated technique-ingestion pipeline from ST telemetry feeds the TA L3 auto-update loop.
- **ML-Infrastructure L2+ (required):** detections instrumented to catch automated-probe activity so the harness and the monitoring pipeline are calibrated against each other.
- **SM-Infrastructure L3 (alignment):** automated inventory signals feed the automated test harness with the current Critical-tier component list and their tier metadata.

#### Desired Outcomes

- Critical-tier adversarial posture is measured daily, not quarterly.
- Novel techniques discovered by the automated harness reach the TA-Infrastructure library within 14 days.
- Program-originated findings appear in MITRE ATLAS, AVID, CNCF AI advisories, and OWASP revisions, the org is a net contributor to the AI infrastructure security ecosystem.
- Published regression corpora and test patterns are adopted by peer organizations.

#### Activities

**A) Continuous automated adversarial testing harness**

Deploy an automated testing harness that runs daily against all Critical-tier AI/HAI infrastructure components:
- **Model-extraction generator:** produces novel model-extraction query sequences using mutation of the regression corpus, query-pattern variation, and rate-limit boundary probing; runs against inference endpoint and vector-store archetypes (ATLAS TA0004 / TA0013).
- **Cross-tenant isolation prober:** generates cross-tenant access patterns targeting namespace boundaries, IAM scope boundaries, and API key scope limits; runs against inference endpoint, vector-store, and GPU fleet archetypes (ATLAS TA0013; EA).
- **GPU residual-state prober:** generates job-pair compositions for residual-state leakage testing on GPU fleet archetypes; surfaces new memory-pattern leakage paths (ATLAS TA0012).
- **Pipeline-tampering generator:** generates unsigned pipeline trigger attempts, missing-provenance promotion attempts, and eval-gate bypass variations; runs against AI-CI/CD and model registry archetypes (ATLAS TA0012).
- **Retrieval-extraction seeder:** generates query sequences designed to reconstruct tenant embedding spaces at scale; runs against vector-store archetypes (ATLAS TA0013).

Findings triaged by a named ST owner at least weekly. New techniques (patterns not in the TA-Infrastructure library) fed into the TA L3 auto-proposal pipeline within 14 days. High-severity automated findings route to IM-Infrastructure within 24 hours.

**B) Contribute findings to industry**

Contribute anonymized, legally-vetted findings to:
- **MITRE ATLAS**, new technique observations (novel model-extraction patterns, GPU residual-state leakage mechanics, pipeline-tampering paths, cross-tenant retrieval extraction methods); target ≥4 contributions per year.
- **AI Vulnerability Database (AVID)**, structured disclosure submissions for novel vulnerabilities in AI/HAI infrastructure components or their upstream dependencies (coordinated disclosure where third-party components are involved).
- **CNCF AI Working Group / CNCF TAG Security**, AI infrastructure security advisories and configuration guidance contributions for inference serving, model registry, GPU scheduling, and MLOps pipeline security; target ≥2 substantive contributions per year.
- **OWASP LLM / Agentic Top 10 infrastructure patterns**, real-world telemetry evidence during revision cycles; target ≥2 substantive submissions per revision cycle.

**C) Publish regression corpora and test patterns as open artifacts**

- Publish anonymized versions of the six regression corpora (model-extraction, cross-tenant-isolation, GPU-residual-state, workflow-injection, retrieval-extraction, pipeline-signing) under an open license; scrubbed of org-specific component names, tenant identifiers, and model versions.
- Maintain the published versions upstream; internal corpora are a superset of the published versions with org-specific entries not shared externally.
- Host or co-host at least one industry red-team benchmark per year (CNCF AI security working group, ATLAS practitioner table, sector ISAC AI working group); collect cross-org detection-benchmark improvement data.

#### Outcome Metrics (L3)

| Metric | Baseline | L3 Target | Source |
|---|---|---|---|
| % Critical-tier components under continuous automated adversarial testing (daily probe execution) | measure | ≥80% | ST harness telemetry |
| New-technique ingestion lead time (automated finding to TA-Infrastructure library entry) | measure | ≤14 days | Harness → TA pipeline telemetry |
| Industry contributions per year (MITRE ATLAS / AVID / CNCF AI / OWASP) | 0 | ≥4 | Contribution log |
| Open regression corpora published and maintained upstream | 0 | ≥6 corpora published | External repository |
| Industry-shared exercises per year | 0 | ≥1 hosted + ≥2 participated | Exercise log |

#### Process Metrics (leading)

- Continuous harness health, % Critical-tier components producing a fresh automated probe result within the last 24 hours; on-call paged when a harness feed goes stale >24 hours.
- New-technique triage cadence, automated probe findings reviewed weekly by named ST owner; novel patterns forwarded to TA-Infrastructure library within 14 days.
- Industry-contribution pipeline, at least one anonymized finding in-preparation, in-legal-review, or submitted at any time.
- Industry-exercise calendar, next hosted or co-hosted exercise scheduled at least 60 days in advance.

#### Effectiveness Metrics (business value)

- Mean time to detect novel AI-infrastructure-specific attack techniques decreases as the continuous harness catches configuration-change regressions within hours, not quarterly red-team cycles.
- Program-originated techniques recognized in MITRE ATLAS, AVID, or CNCF AI advisories demonstrate external validation of testing rigor.
- Cross-org exercise participants cite improved detection benchmarks from shared corpora, measurable uplift in peer organizations' infrastructure defenses.
- Critical-tier incidents attributable to model extraction, cross-tenant isolation failure, GPU residual-state leakage, or pipeline tampering drop as the continuous harness closes the gap between quarterly red-team cycles.

#### Success Criteria

- ≥80% of Critical-tier AI/HAI infrastructure components under continuous automated adversarial testing with daily probe execution.
- New-technique ingestion lead time ≤14 days; automated findings triaged weekly by named owner.
- ≥4 industry contributions per year to MITRE ATLAS / AVID / CNCF AI / OWASP.
- All 6 open regression corpora published under a permissive license and maintained upstream.
- ≥1 industry-shared exercise hosted per year plus ≥2 participated; cross-org detection-benchmark improvement documented.

---

## Key Success Indicators

**Level 1:**
- Per-archetype foundational test battery published for all seven archetypes (inference endpoint, model registry, GPU fleet, orchestrator, vector-store, AI-CI/CD, feature store), each test class tied to a TA-Infrastructure archetype threat (HAI TTP + ATLAS tactic ID) and an SR-Infrastructure requirement.
- Six regression corpora (model-extraction, cross-tenant-isolation, GPU-residual-state, workflow-injection, retrieval-extraction, pipeline-signing) in source control, running on a monthly-or-better cadence for Critical/High-tier components, with named corpus owner and monthly refresh cadence.
- 100% of AI/HAI infrastructure components reaching production in the last 90 days have a passed go-live battery on record linked from the SM-Infrastructure inventory.
- All test failures routed to IM within 1 business day; automation covers ≥60% of battery items.
- TA-Infrastructure archetype threat coverage by test battery ≥80%.

**Level 2:**
- 100% of Critical-tier components red-teamed quarterly; 100% of High-tier semi-annually; scope tied to TA-Infrastructure L2 per-component deep threat models.
- Per-tier calibration enforced: Critical-tier gets all 6 corpora on monthly cadence plus quarterly red-team; Low-tier gets model-extraction corpus on quarterly cadence.
- ≥90% of Critical/High-severity red-team findings converted to corpus entries within 30 days.
- Cross-archetype composition tests documented and run for all Critical-tier components with composition dependencies (inference endpoint + GPU fleet, orchestrator + vector-store, AI-CI/CD + model registry).

**Level 3:**
- ≥80% of Critical-tier components under continuous automated adversarial testing with daily probe execution; novel techniques reaching TA-Infrastructure library within 14 days.
- ≥4 industry contributions per year to MITRE ATLAS / AVID / CNCF AI / OWASP LLM / Agentic Top 10; all 6 open regression corpora published and maintained.
- ≥1 hosted + ≥2 participated industry exercises per year with documented cross-org detection-benchmark improvement.

---

## Common Pitfalls

**Level 1:**
- ❌ Test battery reduced to a logging-completeness check and a rate-limit availability probe, no adversarial probes (model-extraction, cross-tenant isolation, GPU residual-state-clearing) actually exercised.
- ❌ Regression corpora committed to source control but not wired to a cadenced run, they exist but run only when a reviewer manually triggers them; coverage erodes after every sprint.
- ❌ Go-live battery runs once pre-production but is never re-run after configuration policy changes, test coverage erodes as GPU scheduling policies, signing enforcement settings, and rate-limit configs change.
- ❌ Test failures logged in a spreadsheet separate from IM, no SLA enforcement, no aging visibility, no named owner; the same failure recurs across multiple configuration changes undetected.
- ❌ Corpus refresh skipped for three months because "nothing new happened", ATLAS technique additions and CNCF AI advisories pile up unread; the corpus is frozen at launch while the external threat landscape advances.
- ❌ Test battery conflates inference endpoint and GPU fleet archetypes, GPU-specific tests (residual-state-clearing, classification-aware scheduling, credential isolation) are absent from the GPU fleet battery because the same checklist was applied to both archetypes.

**Level 2:**
- ❌ Red-team scope defined as "availability probes and rate-limit tests" but model-extraction campaigns, cross-tenant retrieval extraction, GPU residual-state composition, and pipeline-tampering are excluded, the top threat classes for Critical-tier AI infrastructure go untested.
- ❌ Corpus growth declared at ≥1 per month but entries are variations on the same query format, the corpus does not cover the new extraction techniques published since L2 launch; configuration-change regressions pass the corpus and fail in production.
- ❌ Per-tier calibration documented in the tier-treatment matrix but the test runner applies the same corpus to all tiers, Critical components run the same tests as Low; differentiation exists on paper only.
- ❌ Red-team findings route to IM but the finding → corpus pipeline is never executed, 12 months of Critical/High findings sit in IM as closed tickets with no corpus entries; the same vulnerabilities are re-discovered at the next red-team exercise.
- ❌ Cross-archetype composition tests scoped but not executed because "no engineer owns inference-endpoint-plus-GPU testing", composition-specific failure modes (GPU residual-state leakage into inference endpoint serving) are in the threat model but not in any test.

**Level 3:**
- ❌ Continuous harness runs extraction probes that the component's rate-limit trivially blocks, coverage metric looks good but the probes are not exercising novel extraction techniques; new attack patterns are not generated.
- ❌ Industry contributions are legally-vetted case-study summaries rather than actionable, reproducible technique descriptions, ATLAS reviewers cannot map them to a technique ID; CNCF advisories lack reproducibility notes.
- ❌ Open corpora published once and then not maintained, external organizations build on a stale version while the internal corpus has 40 new entries; discrepancies surface at community exercises and damage the program's credibility.
- ❌ New-technique ingestion from automated probes to the TA-Infrastructure library is manual and quarterly, by the time a novel technique reaches SR and SA updates and is reflected in controls, the technique is already being exploited in the wild.
- ❌ Hosted industry exercise becomes a capabilities showcase rather than a detection-benchmarking session, no measurable improvement data is collected from participants; the "≥1 hosted per year" metric is met without producing any cross-org security uplift.

---

## Practice Maturity Questions

**Level 1:**
1. Is a per-archetype foundational test battery published for all seven AI/HAI infrastructure archetypes, with each test class tied to a TA-Infrastructure archetype threat (HAI TTP + ATLAS tactic ID) and an SR-Infrastructure requirement, defined inputs/outputs/pass-fail criteria, and an evidence artifact, and are 100% of new AI/HAI infrastructure components required to pass the battery before production Sanctioned status is issued?
2. Are six regression corpora (model-extraction, cross-tenant-isolation, GPU-residual-state, workflow-injection, retrieval-extraction, pipeline-signing) versioned in source control, running on a monthly-or-better cadence for Critical/High-tier components, with a named corpus owner and a monthly refresh cadence from internal + external sources, and are ≥90% of Critical/High-tier component changes verified to have triggered the relevant corpus run?
3. Are all test failures routed to IM-Infrastructure within 1 business day with a severity tag and named owner, and does TA-Infrastructure archetype threat coverage by the test battery and corpus reach ≥80% by end of year one?

**Level 2:**
1. Are 100% of Critical-tier AI/HAI infrastructure components red-teamed at least quarterly, and 100% of High-tier semi-annually, with scope derived from TA-Infrastructure L2 per-component deep threat models, covering model-extraction campaigns, cross-tenant retrieval extraction, GPU residual-state composition, pipeline-tampering, workflow-injection chains, and feature-poisoning detection, with findings routed to IM and remediation tracked?
2. Is per-tier corpus calibration enforced (Critical-tier: all 6 corpora on monthly cadence; Low-tier: model-extraction corpus on quarterly cadence), and are ≥90% of Critical/High-severity red-team findings converted to corpus entries within 30 days?
3. Are cross-archetype composition tests (inference endpoint + GPU fleet residual-state, orchestrator + vector-store retrieval-injection, AI-CI/CD + model registry tampering) documented and executed for all Critical-tier composite components, and is per-tier SLA adherence for testing activities ≥90%?

**Level 3:**
1. Are ≥80% of Critical-tier AI/HAI infrastructure components under continuous automated adversarial testing with daily probe execution, using model-extraction generators, cross-tenant isolation probers, GPU residual-state probers, pipeline-tampering generators, and retrieval-extraction seeders, with novel techniques triaged into the TA-Infrastructure library within 14 days and high-severity automated findings routed to IM within 24 hours?
2. Has the program contributed ≥4 anonymized, legally-vetted findings per year to MITRE ATLAS, AVID, CNCF AI Working Group, or OWASP LLM / Agentic Top 10 infrastructure patterns, with at least one accepted as a new or refined technique, and are all 6 open regression corpora published under a permissive license and maintained upstream?
3. Has the program hosted at least 1 industry-shared red-team exercise per year and participated in ≥2 additional cross-org exercises, with documented cross-org detection-benchmark improvement data from participants?

---

**Document Version:** HAIAMM v3.0
**Practice:** Security Testing (ST)
**Domain:** Infrastructure
**Last Updated:** 2026-05-14
**Author:** Verifhai