Security Testing (ST)

Infrastructure Domain - HAIAMM v3.0

Practice Overview

Objective: Prove that every AI/HAI infrastructure component the organization operates behaves correctly under adversarial conditions, by running a foundational per-archetype test battery in CI and on a defined cadence, maintaining versioned regression corpora, and escalating to scheduled red-team and continuous adversarial testing at higher maturity levels.

Description: ST-Infrastructure exercises the AI/HAI infrastructure the organization runs, inference endpoints, model registries, GPU / accelerator fleets, orchestrator control planes, vector-store infrastructure, AI-specific CI/CD pipelines, and feature stores, against a battery of AI-specific test classes tied directly to the threats in the TA-Infrastructure library and the requirements in the SR-Infrastructure pack. At L1, every archetype has a published test battery (model-extraction probes, cross-tenant isolation tests, GPU residual-state-clearing tests, workflow-signing enforcement tests, retrieval-extraction probes, pipeline-signing tests, feature-poisoning detection tests) plus versioned regression corpora running on a defined cadence. L2 adds per-tier scheduled red-team exercises using TA-Infrastructure L2 deep threat models, and cross-archetype composition tests (inference endpoint + GPU fleet residual-state composition). L3 operates continuous automated adversarial testing and contributes findings to MITRE ATLAS, AVID, CNCF AI infrastructure advisories, and OWASP LLM / Agentic Top 10 infrastructure patterns.

Context: Classic infrastructure testing exercises availability and performance, leaving AI-specific adversarial paths untested. An inference endpoint passes all health checks and then leaks embeddings at scale through an extraction-pattern query campaign. A GPU fleet passes scheduling tests and then allows job B to read job A's residual GPU memory. A model registry passes its access-control tests and then accepts an unsigned model artifact because the signing enforcement flag was reset by a platform update. These failures are invisible to classic infrastructure testing because classic testing was not designed for AI-specific infrastructure failure modes, model extraction (ATLAS TA0004), cross-tenant retrieval extraction (ATLAS TA0013), ML attack staging via the pipeline (ATLAS TA0012), and initial access through the inference endpoint surface (ATLAS TA0001). ST-Infrastructure closes this gap by making AI-specific infrastructure tests a first-class testing citizen and by connecting them directly to the TA threat library so test coverage tracks threat coverage, not just availability coverage.

Maturity Level 1

Objective: Establish a foundational per-archetype test battery and regression corpora that run on a defined cadence, and verify that every AI/HAI infrastructure component reaches production with a passed go-live battery on record

At this level, the organization gives every AI/HAI infrastructure component a documented, automated-where-possible test battery drawn from the TA-Infrastructure archetype threat library and the SR-Infrastructure requirements pack, so every production landing is backed by observable test evidence, not only design assurance.

Dependencies

• TA-Infrastructure L1 (required): tests target specific threats from the archetype threat library; without the library, test scope is invented per intake rather than inherited.
• SR-Infrastructure L1 (required): requirements pack defines what a passing test means; tests answer "does this component meet its SR requirements?" not a free-form question.
• SA-Infrastructure L1 (required): reference patterns define the control points (workload identity, isolation policy, signing enforcement, rate-limit layer) that tests exercise.
• IR-Infrastructure L1 (required): implementation reviews confirm the component is configured as designed before ST probes run; testing a misconfigured component produces noise.
• Supports / unblocks: IM-Infrastructure L1 (test failures become issues), ML-Infrastructure L1 (detections validated by logging-completeness tests), EH-Infrastructure L1 (hardening informed by test findings).

Desired Outcomes

• Every AI/HAI infrastructure component reaching production has passed a documented per-archetype test battery; the battery result is on file and linked from the SM-Infrastructure inventory record.
• Regression corpora (model-extraction corpus, cross-tenant isolation probe corpus, GPU residual-state corpus, workflow-injection corpus, retrieval-extraction corpus, pipeline-signing corpus) run on a defined cadence and catch AI-specific regressions before components go live.
• Test failures become IM issues with named owners within one business day, not reports in a drive.
• The battery is automated enough to re-run on demand (post-configuration-update, post-incident, on cadence) without a bespoke effort per run.
• Test coverage tracks threat coverage: every threat in the TA-Infrastructure library for a given archetype has at least one corresponding test in the battery or corpus.

Activities

A) Publish the foundational per-archetype test battery

Ship one test battery per AI/HAI infrastructure archetype. Each battery targets the top archetype threats from TA-Infrastructure and the archetype-specific SR requirements. Each test class has: inputs, expected output, pass/fail criteria, evidence artifact (probe result / log snippet / CI run link), and the TA threat and SR requirement it maps to.

Inference endpoint / model-serving cluster: - Model-extraction probe (rate-limit holds against extraction-pattern queries): send a sequence of prefix-completion queries at the rate-limit ceiling; assert the rate-limit layer enforces the cap before query volume reaches extraction-pattern scale; verify the inference endpoint logs the rate-limit event (ATLAS TA0004 ML Model Access / TA0013 Exfiltration). - Cross-tenant isolation test: with a multi-tenant deployment, send a request with tenant A's credentials that references tenant B's model endpoint, model version, or model artifact; verify rejection and confirm the rejection is logged with tenant context (EA; cross-tenant leakage). - DoS / prompt-flood resilience test: send inputs above the declared burst limit; assert the rate-limit and abuse-detection layer enforces the cap; verify degraded-mode fallback produces a logged, non-misleading response and does not cascade to other tenants. - Model-swap detection test (canary signal flips when model version changes silently): assert the serving endpoint returns the declared model version in response metadata; attempt to promote a new model version without a canary signal and verify the canary slot flips before full traffic promotion. - Logging-completeness test: for a known inference request, verify required log fields (tenant ID, session ID, timestamp, model version, prompt hash, completion hash) appear in the org-side log store within the retention SLA (ATLAS TA0013 Exfiltration, completeness is the defense evidence).

Model registry: - Unauthorized-upload probe: attempt to push a model artifact without promotion rights; verify the registry rejects the upload and logs the attempt (ATLAS TA0004 ML Model Access). - Signed-artifact enforcement test: attempt to push an unsigned model artifact with valid promotion credentials; verify the registry rejects the artifact and logs the policy violation. - Lineage-required test: attempt to promote a model artifact without a complete lineage record (missing training data version, missing eval suite results); verify the promotion gate blocks it. - Rollback-abuse probe: attempt to roll back to a retired model version that has a known critical finding in the IM backlog without an explicit overriding authorization; verify the registry requires authorization and logs the rollback event.

GPU / accelerator fleet: - Residual-state-clearing test: run job A on a GPU node (job A writes a known pattern to GPU memory); then run job B on the same GPU node; verify job B cannot read job A's memory pattern (test record includes job IDs, node ID, and confirmation of inaccessibility) (ATLAS TA0012 ML Attack Staging, residual state is an attack staging vector). - Classification-aware scheduler test: attempt to schedule a Critical-tier or regulated workload on a node designated as shared; verify the scheduler rejects or redirects the placement. - Credential-isolation test: confirm that job A's credentials (IAM role, service account token, API key) cannot be used by job B running on the same node after job A completes; verify via probe using job A's credential identity after job A's completion.

Orchestrator / control plane: - Workflow-signing test (unsigned workflow rejected): attempt to submit an unsigned workflow definition to the production orchestrator; verify rejection and confirm the rejection is logged (ATLAS TA0012 ML Attack Staging, unsigned workflows are a staging vector). - Per-step principal test: confirm each workflow step runs under its own minimal-scope identity; run a step that attempts to invoke a resource outside its declared principal scope and verify rejection. - Control-plane API auth test: send an unauthenticated request to the control-plane API; verify 401/403 response and confirm the attempt is logged. - Agent-state tampering test: inject a crafted payload into a workflow step's input that attempts to modify the orchestrator's in-flight state; verify the state isolation boundary prevents the modification (AGH; ATLAS TA0012).

Vector-store infrastructure: - Cross-tenant retrieval probe: send a query from tenant A that would match embeddings indexed under tenant B's namespace; verify zero results or a namespacing error, not tenant B's data (ATLAS TA0013 Exfiltration; EA). - Embedding-extraction at scale test: send a sequence of nearest-neighbor queries designed to reconstruct a significant portion of a tenant's embedding space; verify the rate-limit and query-scope bounds prevent extraction at scale before the embedding space is recoverable (ATLAS TA0013 Exfiltration). - Retrieval-policy-bypass probe: attempt to retrieve a document classified at a level above the requesting principal's clearance through a crafted query; verify the classification-label-respecting retrieval policy blocks or filters the result (EA; cross-classification leakage).

AI-specific CI/CD: - Pipeline-signing test: attempt to trigger a pipeline run using an unsigned pipeline definition; verify rejection and confirm the rejection is logged (ATLAS TA0012 ML Attack Staging, unsigned pipelines are a staging vector). - SLSA-provenance verification test: pull a recently promoted model artifact; verify a valid SLSA provenance attestation is attached and passes verification. - Eval-gate enforcement test: attempt to promote a model artifact through the pipeline without a passing eval attestation; verify the gate blocks the promotion and logs the blocked event. - Secrets-leak probe: scan a recent pipeline run's log output and artifact storage for credential patterns (API key patterns, token formats, private key headers); assert zero findings (ATLAS TA0001 Reconnaissance, leaked secrets reduce the attack surface needed).

Feature store / online serving cache: - Feature-poisoning detection test: inject a synthetic anomalous feature value into the online serving cache; verify the offline/online skew alert fires within the declared detection window. - Offline/online-skew alert test: create a controlled skew between the offline feature store and the online serving cache; verify the skew-monitoring alert fires within the declared detection window and produces an IM ticket. - Access-control test: attempt to read features with an identity that has no declared read rights; verify rejection and confirm the attempt is logged.

B) Build and maintain regression corpora

Maintain six versioned regression corpora, running on a defined cadence. Each corpus is a collection of structured test fixtures: input, expected safe outcome, threat tag (HAI TTP + ATLAS tactic ID), source, date added.

• Model-extraction corpus, 20–60 prefix-completion and nearest-neighbor query patterns designed to probe for model extraction; run against inference endpoint and vector-store archetypes on a monthly cadence; Critical-tier: run before any model version promotion.
• Cross-tenant-isolation probe corpus, 20–60 cross-tenant query patterns across inference endpoint, vector-store, and GPU fleet archetypes; run on a monthly cadence; failure blocks production promotion for Critical-tier components.
• GPU-residual-state corpus, 15–40 job-pair test configurations documenting memory patterns that should be inaccessible after job completion; run quarterly or after any GPU fleet scheduling policy change.
• Workflow-injection corpus, 20–50 crafted workflow step payloads and unsigned workflow definitions designed to probe signing enforcement, per-step principal enforcement, and state-isolation; run against orchestrator archetypes on a monthly cadence.
• Retrieval-extraction corpus, 20–60 query sequences designed to probe for cross-tenant retrieval, embedding extraction at scale, and classification-label bypass; run against vector-store archetypes on a monthly cadence.
• Pipeline-signing corpus, 15–40 pipeline trigger attempts with unsigned definitions, missing provenance, and bypass variations; run against AI-CI/CD archetypes on a monthly cadence.

Corpus management: - Versioned in source control; corpus changes go through review with a named corpus owner. - Corpus refresh cadence: monthly minimum from three sources, internal observations (IR findings, IM incidents, red-team results), external public corpora (ATLAS technique examples, OWASP LLM / Agentic Top 10 infrastructure examples), and MLOps security research. - New AI/HAI infrastructure component provisioning triggers a corpus completeness check against the component's archetype battery.

C) Operate the go-live battery and wire test failures to IM

Every AI/HAI infrastructure component must pass its archetype battery before receiving Sanctioned status in the SM-Infrastructure inventory. Go-live triggers: - Pre-production: all applicable archetype tests must pass before the component is promoted to production; the go-live test record is linked from the SM-Infrastructure inventory and the PC intake artifact. - Post-configuration-update: any GPU scheduling policy change, signing policy change, rate-limit policy change, or isolation policy change triggers a re-run of the relevant battery subset within 7 days for Critical-tier components and within 14 days for High-tier. - Post-incident: any IM-Infrastructure incident involving the component triggers a re-run of the relevant battery subset before the incident is closed. - Quarterly: all active AI/HAI infrastructure components re-run their battery; results reviewed by the named test-battery owner.

All test failures route to IM-Infrastructure within one business day with a severity tag. Named battery owner per archetype; battery ownership is a named role, not a shared-team responsibility.

Outcome Metrics (L1)

Process Metrics (leading)

• Battery owner named per archetype; quarterly re-run scheduled in advance.
• Test-automation coverage, target ≥60% of battery items running without human intervention at L1.
• Corpus refresh cadence honored monthly; corpus owner reviews incoming sources.
• New-archetype lead time, from "first intake in new infrastructure category" to "battery published" ≤30 days.

Effectiveness Metrics (business value)

• Pre-production catch rate, test failures that uncovered a misconfiguration or control gap before production landing (GPU residual-state-clearing misconfigured, cross-tenant isolation not enforced, unsigned pipeline accepted).
• Configuration-change regression catch rate, adversarial probes catching control regressions introduced by infrastructure configuration changes before they reach production traffic.
• Reduced IM incident volume, components with a full battery pass-rate have a measurably lower IM incident rate than those without.

Success Criteria

• Per-archetype foundational test battery published for all seven archetypes (inference endpoint, model registry, GPU fleet, orchestrator, vector-store, AI-CI/CD, feature store), linked from the SM-Infrastructure inventory record and the DR/IR artifacts.
• Six regression corpora published in source control, running on a monthly-or-better cadence for Critical/High-tier components, with a named corpus owner and a monthly refresh cadence.
• 100% of AI/HAI infrastructure components reaching production in the last 90 days have a passed go-live battery on record.
• All test failures routed to IM with a 1-day handoff SLA and named owner.
• Named battery owner per archetype; automation covers ≥60% of battery items.

Maturity Level 2

Objective: Calibrate test depth per risk tier using the SM-Infrastructure L2 tier-treatment matrix, run scheduled per-tier red-team exercises using TA-Infrastructure L2 deep threat models, and test cross-archetype compositions, particularly inference endpoint and GPU fleet residual-state composition, for Critical-tier components

At this level, testing stops treating all AI/HAI infrastructure components the same. Per-tier calibration drives what each component receives from the test program. Critical-tier components are red-teamed quarterly; High-tier semi-annually. TA-Infrastructure L2 deep threat models replace archetype snapshots as the scenario library. Cross-archetype composition tests (e.g., inference endpoint + GPU fleet residual-state composition; orchestrator + vector-store retrieval-injection chain) receive combined test suites that exercise composition-specific failure modes.

Dependencies

• ST-Infrastructure L1 (required): per-archetype batteries and regression corpora must be operational before per-tier calibration is meaningful.
• SM-Infrastructure L2 (required): the risk-tier rubric (Critical / High / Medium / Low) and tier-treatment matrix determine which components receive full-scope testing vs. a subset; without tier assignments, L2 calibration has no substrate.
• TA-Infrastructure L2 (required): per-component deep threat models for Critical-tier components shape red-team scope; archetype-only snapshots are insufficient for L2 red-team exercises.
• Supports / unblocks: IM-Infrastructure L2 (tier-calibrated incident handling relies on tier-calibrated test signals), ML-Infrastructure L2 (detections tuned per-component threat model depend on L2 test findings to validate coverage).

Desired Outcomes

• Test depth is visibly differentiated: Critical-tier components get the full battery plus quarterly red-team and cross-archetype composition tests; Low-tier get the base corpus and a spot-check.
• Red-team findings are scenario-based (from TA-Infrastructure L2 per-component deep threat models), not free-form; findings trace to specific ATLAS tactics and to SR-Infrastructure requirements.
• The regression corpora grow from red-team findings: every critical or high-severity red-team finding produces a new corpus entry that runs on cadence within 30 days.
• Cross-archetype composition failure modes (inference endpoint + GPU fleet residual-state, orchestrator + vector-store retrieval-injection, AI-CI/CD pipeline + model registry tampering) are in scope and have documented test coverage.

Activities

A) Tier-calibrated test battery and corpus depth

Publish a per-tier test treatment aligned to SM-Infrastructure L2's tier-treatment matrix:

For Critical-tier components, the regression corpus is maintained as a component-specific tuned corpus (probe inputs specific to this component's tenant population, model versions, query patterns, and GPU fleet configuration) in addition to the archetype-level corpus.

B) Scheduled per-tier red-team exercises using TA-Infrastructure L2 threat models

Red-team cadence by tier: - Critical: quarterly (4 per year); scope derived from TA-Infrastructure L2 per-component deep threat model; covers model-extraction campaigns against inference endpoint, cross-tenant retrieval extraction against vector-store, GPU residual-state composition with inference endpoint, workflow-injection against orchestrator, pipeline-tampering against AI-CI/CD, and feature-poisoning chains. - High: semi-annual (2 per year); scope from TA-Infrastructure L2 component deltas; covers the top-5 threats from the per-component model. - Medium / Low: ad-hoc (before major configuration changes or tenant expansions); archetype snapshot drives scope.

Cross-archetype composition tests for Critical-tier: - Inference endpoint + GPU fleet residual-state composition: does a model served from a GPU node retain extractable state from a prior tenant's job? Run job A from tenant A on a GPU node, then run inference from tenant B on the same GPU node; probe for evidence of tenant A's job residual state in tenant B's completions (ATLAS TA0004 / TA0012). - Orchestrator + vector-store retrieval-injection chain: craft a retrieval response from the vector store that contains workflow-injection instructions; verify the orchestrator does not execute the injected instructions (AGH; ATLAS TA0001 / TA0012). - AI-CI/CD + model registry tampering: craft a pipeline run that attempts to replace a signed model artifact in the registry with an unsigned or modified artifact; verify the registry and pipeline signing enforcement prevent the substitution (ATLAS TA0012 ML Attack Staging).

Each red-team exercise follows a written rules of engagement, test plan reviewed with component owner, execution log, and structured findings report (severity / root cause / ATLAS tactic ID / SR requirement traced). Red-team findings at Critical or High severity produce corpus entries within 30 days.

C) Red-team findings corpus pipeline

Every Critical or High-severity red-team finding produces: 1. A new corpus entry (input, expected safe outcome, threat tag, ATLAS tactic ID, date, source reference) committed to the relevant regression corpus within 30 days. 2. An IM-Infrastructure finding with severity tag and the named component owner as assignee. 3. A TA-Infrastructure library-gap ticket if the finding was not in the archetype library, the gap is tracked with a named owner and a 30-day close SLA for Critical-tier gaps.

Outcome Metrics (L2)

Process Metrics (leading)

• Red-team schedule on calendar; no Critical-tier component skips a quarterly exercise.
• Corpus review cadence, monthly.
• Cross-archetype composition test plan published for each Critical-tier component with composition dependencies; reviewed by named architect.
• Finding → TA library-gap pipeline: Critical gaps closed within 30 days; High within 60 days.

Effectiveness Metrics (business value)

• Pre-production catch rate for Critical-tier increases as red-team exercises catch issues before configuration-change deploys.
• Corpus catches configuration-change regressions early, policy changes that would have reached production, detected before full rollout.
• Incident rate for Critical-tier components with full-scope testing is measurably lower than for those without.

Success Criteria

• Quarterly red-team for 100% of Critical-tier components; semi-annual for 100% of High-tier; scope tied to TA-Infrastructure L2 per-component deep threat models.
• All 6 regression corpora running on monthly cadence for all Critical-tier components; per-tier calibration enforced.
• ≥90% of Critical/High-severity red-team findings converted to corpus entries within 30 days.
• Cross-archetype composition tests documented and run for all Critical-tier components with composition dependencies (inference endpoint + GPU fleet, orchestrator + vector-store, AI-CI/CD + model registry).
• Per-tier SLA adherence for testing activities ≥90%.

Maturity Level 3

Objective: Operate continuous automated adversarial testing for Critical-tier infrastructure components, publish regression corpora and findings as open artifacts, and contribute discovered techniques to MITRE ATLAS, AVID, CNCF AI infrastructure advisories, and OWASP LLM / Agentic Top 10 infrastructure patterns

At this level, testing runs continuously rather than periodically. An automated adversarial testing harness probes Critical-tier infrastructure components daily using generated model-extraction ladder sequences, cross-tenant isolation probes, GPU residual-state composition probes, and pipeline-tampering generators. Novel findings are triaged into the TA-Infrastructure library weekly. Anonymized regression corpora, test patterns, and discovered techniques are contributed to MITRE ATLAS, AVID, CNCF AI infrastructure advisories, and OWASP LLM / Agentic Top 10 infrastructure patterns.

Dependencies

• ST-Infrastructure L2 (required): per-tier red-team function, regression corpora, and finding → corpus pipeline must be operational.
• TA-Infrastructure L3 (required): automated technique-ingestion pipeline from ST telemetry feeds the TA L3 auto-update loop.
• ML-Infrastructure L2+ (required): detections instrumented to catch automated-probe activity so the harness and the monitoring pipeline are calibrated against each other.
• SM-Infrastructure L3 (alignment): automated inventory signals feed the automated test harness with the current Critical-tier component list and their tier metadata.

Desired Outcomes

• Critical-tier adversarial posture is measured daily, not quarterly.
• Novel techniques discovered by the automated harness reach the TA-Infrastructure library within 14 days.
• Program-originated findings appear in MITRE ATLAS, AVID, CNCF AI advisories, and OWASP revisions, the org is a net contributor to the AI infrastructure security ecosystem.
• Published regression corpora and test patterns are adopted by peer organizations.

Activities

A) Continuous automated adversarial testing harness

Deploy an automated testing harness that runs daily against all Critical-tier AI/HAI infrastructure components: - Model-extraction generator: produces novel model-extraction query sequences using mutation of the regression corpus, query-pattern variation, and rate-limit boundary probing; runs against inference endpoint and vector-store archetypes (ATLAS TA0004 / TA0013). - Cross-tenant isolation prober: generates cross-tenant access patterns targeting namespace boundaries, IAM scope boundaries, and API key scope limits; runs against inference endpoint, vector-store, and GPU fleet archetypes (ATLAS TA0013; EA). - GPU residual-state prober: generates job-pair compositions for residual-state leakage testing on GPU fleet archetypes; surfaces new memory-pattern leakage paths (ATLAS TA0012). - Pipeline-tampering generator: generates unsigned pipeline trigger attempts, missing-provenance promotion attempts, and eval-gate bypass variations; runs against AI-CI/CD and model registry archetypes (ATLAS TA0012). - Retrieval-extraction seeder: generates query sequences designed to reconstruct tenant embedding spaces at scale; runs against vector-store archetypes (ATLAS TA0013).

Findings triaged by a named ST owner at least weekly. New techniques (patterns not in the TA-Infrastructure library) fed into the TA L3 auto-proposal pipeline within 14 days. High-severity automated findings route to IM-Infrastructure within 24 hours.

B) Contribute findings to industry

Contribute anonymized, legally-vetted findings to: - MITRE ATLAS, new technique observations (novel model-extraction patterns, GPU residual-state leakage mechanics, pipeline-tampering paths, cross-tenant retrieval extraction methods); target ≥4 contributions per year. - AI Vulnerability Database (AVID), structured disclosure submissions for novel vulnerabilities in AI/HAI infrastructure components or their upstream dependencies (coordinated disclosure where third-party components are involved). - CNCF AI Working Group / CNCF TAG Security, AI infrastructure security advisories and configuration guidance contributions for inference serving, model registry, GPU scheduling, and MLOps pipeline security; target ≥2 substantive contributions per year. - OWASP LLM / Agentic Top 10 infrastructure patterns, real-world telemetry evidence during revision cycles; target ≥2 substantive submissions per revision cycle.

C) Publish regression corpora and test patterns as open artifacts

• Publish anonymized versions of the six regression corpora (model-extraction, cross-tenant-isolation, GPU-residual-state, workflow-injection, retrieval-extraction, pipeline-signing) under an open license; scrubbed of org-specific component names, tenant identifiers, and model versions.
• Maintain the published versions upstream; internal corpora are a superset of the published versions with org-specific entries not shared externally.
• Host or co-host at least one industry red-team benchmark per year (CNCF AI security working group, ATLAS practitioner table, sector ISAC AI working group); collect cross-org detection-benchmark improvement data.

Outcome Metrics (L3)

Process Metrics (leading)

• Continuous harness health, % Critical-tier components producing a fresh automated probe result within the last 24 hours; on-call paged when a harness feed goes stale >24 hours.
• New-technique triage cadence, automated probe findings reviewed weekly by named ST owner; novel patterns forwarded to TA-Infrastructure library within 14 days.
• Industry-contribution pipeline, at least one anonymized finding in-preparation, in-legal-review, or submitted at any time.
• Industry-exercise calendar, next hosted or co-hosted exercise scheduled at least 60 days in advance.

Effectiveness Metrics (business value)

• Mean time to detect novel AI-infrastructure-specific attack techniques decreases as the continuous harness catches configuration-change regressions within hours, not quarterly red-team cycles.
• Program-originated techniques recognized in MITRE ATLAS, AVID, or CNCF AI advisories demonstrate external validation of testing rigor.
• Cross-org exercise participants cite improved detection benchmarks from shared corpora, measurable uplift in peer organizations' infrastructure defenses.
• Critical-tier incidents attributable to model extraction, cross-tenant isolation failure, GPU residual-state leakage, or pipeline tampering drop as the continuous harness closes the gap between quarterly red-team cycles.

Success Criteria

• ≥80% of Critical-tier AI/HAI infrastructure components under continuous automated adversarial testing with daily probe execution.
• New-technique ingestion lead time ≤14 days; automated findings triaged weekly by named owner.
• ≥4 industry contributions per year to MITRE ATLAS / AVID / CNCF AI / OWASP.
• All 6 open regression corpora published under a permissive license and maintained upstream.
• ≥1 industry-shared exercise hosted per year plus ≥2 participated; cross-org detection-benchmark improvement documented.

Key Success Indicators

Level 1: - Per-archetype foundational test battery published for all seven archetypes (inference endpoint, model registry, GPU fleet, orchestrator, vector-store, AI-CI/CD, feature store), each test class tied to a TA-Infrastructure archetype threat (HAI TTP + ATLAS tactic ID) and an SR-Infrastructure requirement. - Six regression corpora (model-extraction, cross-tenant-isolation, GPU-residual-state, workflow-injection, retrieval-extraction, pipeline-signing) in source control, running on a monthly-or-better cadence for Critical/High-tier components, with named corpus owner and monthly refresh cadence. - 100% of AI/HAI infrastructure components reaching production in the last 90 days have a passed go-live battery on record linked from the SM-Infrastructure inventory. - All test failures routed to IM within 1 business day; automation covers ≥60% of battery items. - TA-Infrastructure archetype threat coverage by test battery ≥80%.

Level 2: - 100% of Critical-tier components red-teamed quarterly; 100% of High-tier semi-annually; scope tied to TA-Infrastructure L2 per-component deep threat models. - Per-tier calibration enforced: Critical-tier gets all 6 corpora on monthly cadence plus quarterly red-team; Low-tier gets model-extraction corpus on quarterly cadence. - ≥90% of Critical/High-severity red-team findings converted to corpus entries within 30 days. - Cross-archetype composition tests documented and run for all Critical-tier components with composition dependencies (inference endpoint + GPU fleet, orchestrator + vector-store, AI-CI/CD + model registry).

Level 3: - ≥80% of Critical-tier components under continuous automated adversarial testing with daily probe execution; novel techniques reaching TA-Infrastructure library within 14 days. - ≥4 industry contributions per year to MITRE ATLAS / AVID / CNCF AI / OWASP LLM / Agentic Top 10; all 6 open regression corpora published and maintained. - ≥1 hosted + ≥2 participated industry exercises per year with documented cross-org detection-benchmark improvement.

Common Pitfalls

Level 1: - ❌ Test battery reduced to a logging-completeness check and a rate-limit availability probe, no adversarial probes (model-extraction, cross-tenant isolation, GPU residual-state-clearing) actually exercised. - ❌ Regression corpora committed to source control but not wired to a cadenced run, they exist but run only when a reviewer manually triggers them; coverage erodes after every sprint. - ❌ Go-live battery runs once pre-production but is never re-run after configuration policy changes, test coverage erodes as GPU scheduling policies, signing enforcement settings, and rate-limit configs change. - ❌ Test failures logged in a spreadsheet separate from IM, no SLA enforcement, no aging visibility, no named owner; the same failure recurs across multiple configuration changes undetected. - ❌ Corpus refresh skipped for three months because "nothing new happened", ATLAS technique additions and CNCF AI advisories pile up unread; the corpus is frozen at launch while the external threat landscape advances. - ❌ Test battery conflates inference endpoint and GPU fleet archetypes, GPU-specific tests (residual-state-clearing, classification-aware scheduling, credential isolation) are absent from the GPU fleet battery because the same checklist was applied to both archetypes.

Level 2: - ❌ Red-team scope defined as "availability probes and rate-limit tests" but model-extraction campaigns, cross-tenant retrieval extraction, GPU residual-state composition, and pipeline-tampering are excluded, the top threat classes for Critical-tier AI infrastructure go untested. - ❌ Corpus growth declared at ≥1 per month but entries are variations on the same query format, the corpus does not cover the new extraction techniques published since L2 launch; configuration-change regressions pass the corpus and fail in production. - ❌ Per-tier calibration documented in the tier-treatment matrix but the test runner applies the same corpus to all tiers, Critical components run the same tests as Low; differentiation exists on paper only. - ❌ Red-team findings route to IM but the finding → corpus pipeline is never executed, 12 months of Critical/High findings sit in IM as closed tickets with no corpus entries; the same vulnerabilities are re-discovered at the next red-team exercise. - ❌ Cross-archetype composition tests scoped but not executed because "no engineer owns inference-endpoint-plus-GPU testing", composition-specific failure modes (GPU residual-state leakage into inference endpoint serving) are in the threat model but not in any test.

Level 3: - ❌ Continuous harness runs extraction probes that the component's rate-limit trivially blocks, coverage metric looks good but the probes are not exercising novel extraction techniques; new attack patterns are not generated. - ❌ Industry contributions are legally-vetted case-study summaries rather than actionable, reproducible technique descriptions, ATLAS reviewers cannot map them to a technique ID; CNCF advisories lack reproducibility notes. - ❌ Open corpora published once and then not maintained, external organizations build on a stale version while the internal corpus has 40 new entries; discrepancies surface at community exercises and damage the program's credibility. - ❌ New-technique ingestion from automated probes to the TA-Infrastructure library is manual and quarterly, by the time a novel technique reaches SR and SA updates and is reflected in controls, the technique is already being exploited in the wild. - ❌ Hosted industry exercise becomes a capabilities showcase rather than a detection-benchmarking session, no measurable improvement data is collected from participants; the "≥1 hosted per year" metric is met without producing any cross-org security uplift.

Practice Maturity Questions

Level 1: 1. Is a per-archetype foundational test battery published for all seven AI/HAI infrastructure archetypes, with each test class tied to a TA-Infrastructure archetype threat (HAI TTP + ATLAS tactic ID) and an SR-Infrastructure requirement, defined inputs/outputs/pass-fail criteria, and an evidence artifact, and are 100% of new AI/HAI infrastructure components required to pass the battery before production Sanctioned status is issued? 2. Are six regression corpora (model-extraction, cross-tenant-isolation, GPU-residual-state, workflow-injection, retrieval-extraction, pipeline-signing) versioned in source control, running on a monthly-or-better cadence for Critical/High-tier components, with a named corpus owner and a monthly refresh cadence from internal + external sources, and are ≥90% of Critical/High-tier component changes verified to have triggered the relevant corpus run? 3. Are all test failures routed to IM-Infrastructure within 1 business day with a severity tag and named owner, and does TA-Infrastructure archetype threat coverage by the test battery and corpus reach ≥80% by end of year one?

Level 2: 1. Are 100% of Critical-tier AI/HAI infrastructure components red-teamed at least quarterly, and 100% of High-tier semi-annually, with scope derived from TA-Infrastructure L2 per-component deep threat models, covering model-extraction campaigns, cross-tenant retrieval extraction, GPU residual-state composition, pipeline-tampering, workflow-injection chains, and feature-poisoning detection, with findings routed to IM and remediation tracked? 2. Is per-tier corpus calibration enforced (Critical-tier: all 6 corpora on monthly cadence; Low-tier: model-extraction corpus on quarterly cadence), and are ≥90% of Critical/High-severity red-team findings converted to corpus entries within 30 days? 3. Are cross-archetype composition tests (inference endpoint + GPU fleet residual-state, orchestrator + vector-store retrieval-injection, AI-CI/CD + model registry tampering) documented and executed for all Critical-tier composite components, and is per-tier SLA adherence for testing activities ≥90%?

Level 3: 1. Are ≥80% of Critical-tier AI/HAI infrastructure components under continuous automated adversarial testing with daily probe execution, using model-extraction generators, cross-tenant isolation probers, GPU residual-state probers, pipeline-tampering generators, and retrieval-extraction seeders, with novel techniques triaged into the TA-Infrastructure library within 14 days and high-severity automated findings routed to IM within 24 hours? 2. Has the program contributed ≥4 anonymized, legally-vetted findings per year to MITRE ATLAS, AVID, CNCF AI Working Group, or OWASP LLM / Agentic Top 10 infrastructure patterns, with at least one accepted as a new or refined technique, and are all 6 open regression corpora published under a permissive license and maintained upstream? 3. Has the program hosted at least 1 industry-shared red-team exercise per year and participated in ≥2 additional cross-org exercises, with documented cross-org detection-benchmark improvement data from participants?

Document Version: HAIAMM v3.0 Practice: Security Testing (ST) Domain: Infrastructure Last Updated: 2026-05-14 Author: Verifhai