Security Testing (ST)

Data Domain - HAIAMM v3.0

Quick-Load Slice (L1 essentials)

If you are constructing a per-archetype test battery and need only L1 essentials, load this section plus the per-archetype block(s) that apply to your asset.

L1 archetype list (same as TA-Data)

Training corpus | Inference input stream | Retrieval store | Prompt/completion log corpus | Embedding store | Fine-tuning dataset | Evaluation/test set.

L1 regression corpora (6)

poison-detection | retrieval-extraction | retrieval-poisoning | embedding-inversion | PII-redaction-edge | DSAR-query.

L1 test class schema (every test row must carry all columns)

test class | inputs | pass/fail criteria | evidence artifact | TA-Data threat (HAI TTP + ATLAS tactic + ATLAS technique) | SR-Data requirement | regression corpus | CI gate | severity on fail.

L1 re-run triggers

Pre-prod | post-corpus-update (Critical ≤7 days, others ≤14 days) | post-incident | quarterly.

L1 routing SLA

All test failures route to IM-Data within 1 business day with severity tag and named owner.

Practice Overview

Objective: Prove that every AI/HAI data flow the organization operates behaves correctly under adversarial conditions, by running a foundational per-archetype test battery in CI, maintaining versioned regression corpora, and escalating to scheduled red-team and continuous adversarial testing at higher maturity levels.

Description: ST-Data exercises the data flowing into and out of AI/HAI systems, training corpora, inference inputs, retrieval stores, prompt/completion logs, embeddings, fine-tuning datasets, and evaluation/test sets, against a battery of AI-specific test classes tied directly to the threats in the TA-Data library and the requirements in the SR-Data pack. At L1, every archetype has a published test battery with per-archetype probes (poison-detection scans, PII-redaction-edge canaries, retrieval-extraction probes, retrieval-poisoning probes, per-tenant-isolation tests, inversion probes, opt-out enforcement tests, eval-isolation tests) plus versioned regression corpora (poison-detection corpus, retrieval-extraction corpus, retrieval-poisoning corpus, embedding-inversion corpus, PII-redaction-edge corpus, DSAR-query corpus) running in CI. L2 adds per-tier red-team exercises using TA-Data L2 deep threat models and cross-archetype tests. L3 operates continuous automated adversarial testing and contributes findings to MITRE ATLAS, AVID, OWASP LLM, and NIST AI RMF Data.

Context: Classic data-pipeline test suites exercise the happy path and leave the adversarial path untested. A training corpus passes all validation checks and then contains a poisoned subset that degrades model behavior (ATLAS TA0012 ML Attack Staging, AML.T0019 Poison Training Data). An inference-input stream passes integration tests and then forwards PII to an LLM provider with no-train not verified (ATLAS TA0013 Exfiltration). A retrieval store passes unit tests and then yields a neighboring tenant's documents under a crafted query (ATLAS TA0004 ML Model Access). An embedding store returns raw vectors to an unauthenticated caller who uses them to reconstruct training records (ATLAS TA0013). These failures are invisible to classic testing because classic testing was not designed for data-specific AI attack surfaces. ST-Data closes this gap by making data-specific adversarial tests a first-class CI citizen, connecting them directly to the TA-Data threat library and ATLAS data-attack tactics (TA0001 Reconnaissance, TA0004 ML Model Access, TA0012 ML Attack Staging, TA0013 Exfiltration), and ensuring test coverage tracks threat coverage, not just schema coverage.

Maturity Level 1

Objective: Establish a foundational per-archetype test battery and regression corpora that run in CI, and verify that every AI/HAI data flow reaches production with a passed go-live battery on record

At this level, the organization gives every AI/HAI data flow a documented, automated-where-possible test battery drawn from the TA-Data archetype threat library and the SR-Data requirements pack, so every production landing is backed by observable test evidence, not only design assurance.

Dependencies

• TA-Data L1 (required): tests target specific threats from the archetype threat library; without the library, test scope is invented per intake rather than inherited.
• SR-Data L1 (required): requirements pack defines what a passing test means; tests answer "does this data flow meet its SR-Data requirements?" not a free-form question.
• SA-Data L1 (required): reference patterns define the control points (PII-redaction edge, per-tenant isolation boundary, inversion-defense access control) that tests exercise.
• IR-Data L1 (required): implementation reviews confirm the data flow is configured as designed before ST-Data probes run; testing a misconfigured flow produces noise.
• Supports / unblocks: IM-Data L1 (test failures become issues), ML-Data L1 (detections validated by logging-completeness tests).

Desired Outcomes

• Every AI/HAI data flow reaching production has passed a documented per-archetype test battery; the battery result is on file and linked from the SM-Data inventory record.
• Regression corpora (poison-detection, retrieval-extraction, retrieval-poisoning, embedding-inversion, PII-redaction-edge, DSAR-query) run in CI and catch data-specific regressions before deployment.
• Test failures become IM-Data issues with named owners within one business day, not reports in a drive.
• The battery is automated enough to re-run on demand (post-corpus-update, post-incident, on cadence) without a bespoke effort per run.
• Test coverage tracks threat coverage: every threat in the TA-Data library for a given archetype has at least one corresponding test in the battery or corpus.

Activities

A) Publish the foundational per-archetype test battery

Ship one test battery per AI/HAI data archetype. Each battery targets the top archetype threats from TA-Data and the archetype-specific SR-Data requirements. Each test class has: inputs, expected output, pass/fail criteria, evidence artifact (log snippet / scan output / CI run link), and the TA-Data threat and SR-Data requirement it maps to.

Training corpus / training dataset: - Poison-detection scan (regression CI): run a structured poison-detection scan against the corpus using a versioned poison-detection corpus of known-hostile record patterns (label-flipping signatures, backdoor-trigger phrases, systematic mislabeling artifacts); assert zero poison-pattern matches above threshold; failure blocks promotion (ATLAS TA0012 / AML.T0019). - Classification-completeness scan: scan a sample of corpus records against the SM-Data classification taxonomy; assert all records carry correct classification labels and that the corpus does not contain data classes not approved in the DR-Data record. - Consent-basis sample-verify: for a random sample of records, verify the link to the GDPR Art. 6 lawful-basis record (or Art. 9 for special-category data) is present and the basis is active; failure is a Critical finding. - DPIA evidence check: if the DR-Data record flagged a DPIA trigger, verify the DPIA completion artifact is linked and current before the corpus enters any training pipeline.

Inference input stream: - PII-redaction-edge canary test (regression CI): inject a canary PII record (synthetic SSN-format, synthetic card-number-format) into the inference-input test path; verify the payload reaching the LLM provider API has the PII redacted and the canary value does not appear in the outbound request log; run on every PR (ATLAS TA0013 / AML.T0025). - No-train probe: query the vendor admin API for the LLM provider handling this stream; assert the no-train setting is active; failure is a Critical finding (SR-Data no-train requirement). - Classification-gated routing test: inject a test record classified as a regulated data class (PHI-format, PCI-format); assert the routing logic does not forward it to a vendor where no-train has not been confirmed; run on every PR.

Retrieval store: - Retrieval-extraction probe corpus (regression CI): craft a set of queries designed to extract broad swaths of the corpus (prefix-completion queries, empty-string queries, wildcard queries); assert that no query returns more than the declared per-query result cap; run on every PR (ATLAS TA0004 / TA0013). - Retrieval-poisoning probe: insert a hostile document (containing prompt-injection instructions) into a test index; execute a retrieval query that would match the hostile document; assert the injection-defense structure blocks the injected instructions from influencing a downstream model response; clean up test document after the run (ATLAS TA0012 / AML.T0019). - Per-tenant retrieval isolation test: with a multi-tenant deployment, send a query from Tenant A that matches a document indexed under Tenant B's namespace; assert zero results or a namespacing error is returned, not Tenant B's document; run on every PR (EA). - Classification-label-respecting test: send a query from a context classified as "Public" that would match a document classified as "Confidential"; assert the confidential document is not returned or is filtered before reaching the prompt.

Prompt/completion log corpus: - Redaction-at-logging test: inject a canary PII value in a test prompt; verify the log record in the log store shows the redacted form, not the raw PII value; run on every PR. - Retention-expiry test: insert a test log record with a timestamp older than the declared retention window; execute the retention-enforcement job; assert the test record is deleted; evidence: query result showing record absence. - Export-control test: attempt to bulk-export the log corpus using a general service account (not a DSAR-authorized account); assert the request is rejected; evidence: API error response. - Audit-log-completeness test: for a known prompt/completion pair, verify that the required log fields (session ID, timestamp, user identifier, prompt hash, completion hash, model version, redaction flag) appear in the log store within the retention SLA.

Embedding store: - Inversion probe corpus (regression CI): run a set of nearest-neighbor queries designed to recover training-record attributes from the embedding space; assert that query results do not reveal attributes beyond the declared query scope; run on every PR (ATLAS TA0013 / AML.T0025). - Nearest-neighbor extraction probe: craft queries that attempt to reconstruct source text from embedding vectors via exhaustive near-neighbor enumeration; assert the response does not include reconstructable source text above a defined similarity threshold. - Per-tenant partitioning test: in a multi-tenant deployment, run a nearest-neighbor query from a Tenant A session against a Tenant B embedding namespace; assert zero results are returned from Tenant B's partition.

Fine-tuning dataset: - Opt-out enforcement test: mark a synthetic subject as opted-out in the opt-out registry; attempt to include a record linked to that subject in a test fine-tuning run; assert the subject's records are excluded from the dataset presented to the fine-tune job; evidence: dataset manifest showing subject exclusion. - Consent-basis verify test: for a sample of records, verify the link to the consent record is present, the consent is current, and the purpose scope covers fine-tuning use; failure is a Critical finding. - DPIA evidence check: same as training corpus, verify the DPIA completion artifact is linked and current before the dataset enters any fine-tuning pipeline.

Evaluation / test set: - Isolation test: verify via data-catalog lineage that the eval dataset's source records do not appear in any training corpus or fine-tuning dataset artifact store; assert zero overlap between eval-set record IDs and training-pipeline record IDs. - Reproducibility test: run the eval battery twice against the same pinned dataset version; assert results are within the declared variance threshold; failure indicates non-determinism in the eval pipeline. - Corpus-completeness test: assert the eval dataset includes test cases from the TA-Data archetype library for the archetype under evaluation; assert coverage ratio meets the declared minimum.

B) Build and maintain regression corpora in CI

Maintain six versioned regression corpora in source control, running in CI. Each corpus is a collection of structured test fixtures: input, expected safe outcome, threat tag (HAI TTP + ATLAS tactic/technique ID), OWASP reference, source, date added.

• Poison-detection corpus, structured poison-pattern fixtures targeting label-flipping signatures, backdoor-trigger phrases, and systematic mislabeling artifacts; run against training-corpus and fine-tuning-dataset archetypes; failure blocks promotion for Critical/High-tier.
• Retrieval-extraction corpus, query fixtures designed to attempt broad-corpus extraction via prefix-completion, wildcard, and empty-string patterns; run against retrieval-store and embedding-store archetypes.
• Retrieval-poisoning corpus, hostile-document fixtures containing prompt-injection instruction payloads; seeded into test indexes; run against retrieval-store archetypes to verify injection-defense.
• Embedding-inversion corpus, nearest-neighbor query fixtures designed to recover training-record attributes; run against embedding-store archetypes.
• PII-redaction-edge corpus, canary PII fixtures in diverse formats (synthetic SSN, card number, email, IBAN) injected into inference-input test paths; run against inference-input-stream archetypes.
• DSAR-query corpus, canary subject fixtures with known attributes; used to verify DSAR-surface accuracy (inclusion before deletion, exclusion after deletion) across all archetypes contributing to the DSAR surface.

Corpus management: versioned in source control; changes go through PR review with a named corpus owner. Refresh cadence: monthly minimum from three sources, internal observations (IR-Data findings, IM-Data incidents, red-team results), external public corpora (OWASP LLM Top 10 examples, ATLAS technique examples, academic poison-attack datasets), and community vulnerability disclosures. CI run budget-capped; failing runs are a blocking CI check for Critical/High-tier.

C) Operate the go-live battery and wire test failures to IM-Data

Every AI/HAI data flow must pass its archetype battery before receiving Sanctioned status in the SM-Data inventory. Go-live triggers: - Pre-production: all applicable archetype tests must pass before the flow enters production. - Post-corpus-update: any new data source added, classification scheme changed, or vendor LLM provider changed triggers a re-run of the applicable battery within 14 days (Critical-tier: within 7 days). - Post-incident: any IM-Data incident involving the data flow triggers a re-run of the relevant battery subset before the incident is closed. - Quarterly: all active AI/HAI data flows re-run their battery; results reviewed by the named test-battery owner.

All test failures route to IM-Data within one business day with a severity tag derived from the ST severity rubric. Named battery owner per archetype; battery ownership is a named role.

Outcome Metrics (L1)

Process Metrics (leading)

• Battery owner named per archetype; quarterly re-run scheduled in advance.
• Test-automation coverage, target ≥60% of battery items running in CI without human intervention at L1.
• Corpus refresh cadence honored monthly; corpus owner reviews incoming sources.
• New-archetype lead time, from "first intake in new data category" to "battery published" ≤30 days.

Effectiveness Metrics (business value)

• Pre-production catch rate, test failures that uncovered a misconfiguration or control gap before production landing (PII leaking through redaction-edge, corpus containing opt-out subjects, retrieval store exposing cross-tenant documents).
• Post-corpus-update regression catch rate, data-specific regressions caught by the battery before the updated corpus or pipeline entered full production.
• Reduced IM-Data incident volume, data flows with a full battery pass-rate in CI have a measurably lower IM-Data incident rate than those without.

Success Criteria

• Per-archetype foundational test battery published for all seven archetypes, linked from the SM-Data inventory record and the DR-Data / IR-Data artifacts.
• Six regression corpora published in source control, running in CI on every PR for Critical/High-tier data flows, with a named corpus owner and a monthly refresh cadence.
• 100% of AI/HAI data flows reaching production in the last 90 days have a passed go-live battery on record.
• All test failures routed to IM-Data with a 1-day handoff SLA and named owner.
• Named battery owner per archetype; CI automation covers ≥60% of battery items.

Maturity Level 2

Objective: Calibrate test depth per risk tier using the SM-Data L2 tier-treatment matrix, run per-tier red-team exercises using TA-Data L2 deep threat models, and test cross-archetype compositions for Critical-tier data flows

At this level, testing stops treating all AI/HAI data flows the same. Per-tier calibration drives what each data flow receives from the test program. Critical-tier data flows are red-teamed quarterly; High-tier semi-annually. TA-Data L2 per-flow deep threat models replace archetype snapshots as the scenario library for red-team exercises. Cross-archetype compositions (e.g., a training corpus that sources records from a retrieval store; an embedding store whose vectors were derived from a fine-tuning dataset; an inference-input stream whose PII-redaction output feeds a prompt/completion log corpus) receive combined test suites that exercise composition-specific failure modes.

Dependencies

• ST-Data L1 (required): per-archetype batteries and regression corpora must be operational before per-tier calibration is meaningful.
• SM-Data L2 (required): the risk-tier rubric and tier-treatment matrix determine which data flows receive full-scope testing (Critical/High) vs. a subset (Medium/Low).
• TA-Data L2 (required): per-flow deep threat models for Critical-tier data flows shape red-team scope.
• Supports / unblocks: IM-Data L2 (tier-calibrated incident handling relies on tier-calibrated test signals), ML-Data L2 (detections tuned per-flow threat model depend on L2 test findings to validate coverage).

Desired Outcomes

• Test depth is visibly differentiated: Critical-tier data flows get the full battery plus quarterly red-team and cross-archetype composition tests; Low-tier gets the base CI corpus and a spot-check.
• Red-team findings are scenario-based (from TA-Data L2 per-flow threat models), not free-form; findings trace to specific ATLAS data-attack tactics and techniques and to SR-Data requirements.
• The regression corpora grow from red-team findings: every Critical or High-severity red-team finding produces a new corpus entry that runs in CI within 30 days.
• Cross-archetype composition failure modes are in scope and have documented test coverage.
• Per-tier SLA adherence for testing activities is tracked and reported to the program sponsor.

Activities

A) Tier-calibrated test battery and CI corpus depth

Publish a per-tier test treatment aligned to SM-Data L2's tier-treatment matrix:

B) Scheduled per-tier red-team exercises using TA-Data L2 threat models

Red-team cadence by tier: - Critical: quarterly (4 per year); scope derived from TA-Data L2 per-flow deep threat model; covers poison-injection attempts, retrieval-extraction probes beyond the CI corpus, cross-tenant isolation probes, embedding-inversion attacks, DSAR-surface enumeration, opt-out bypass attempts, eval-training contamination probes. - High: semi-annual (2 per year); scope from TA-Data L2 flow deltas; covers the top-5 threats from the per-flow model. - Medium/Low: ad-hoc (before major corpus updates or consumer AI artifacts added); archetype snapshot drives scope.

Each red-team exercise follows the AI Security Testing Methodology: written rules of engagement, test plan reviewed with data-flow owner, execution log, structured findings report (severity / root cause / ATLAS tactic + technique ID / SR-Data requirement traced), remediation pairings. Red-team findings at Critical or High severity produce corpus entries within 30 days.

Cross-archetype composition tests for Critical-tier: - Training corpus + evaluation/test set: contamination-prevention probe, verify no eval-set record IDs appear in the training corpus after a corpus refresh; assert eval-set isolation holds after every corpus update. - Embedding store + retrieval store: inversion-via-retrieval probe, use retrieval queries to obtain embeddings from the retrieval store; apply embedding-inversion techniques to the returned vectors; assert source-text reconstruction does not exceed the declared similarity threshold. - Inference input stream + prompt/completion log corpus: PII-pass-through probe, verify a PII canary that entered the inference-input stream does not appear unredacted in the prompt/completion log corpus; assert redaction is applied at both the input and the logging edge. - Fine-tuning dataset + training corpus: data-lineage cross-contamination probe, verify that a record removed from the training corpus (via opt-out or deletion request) is also absent from the fine-tuning dataset derived from it; assert the deletion propagates to all downstream derived datasets.

C) Red-team findings → corpus pipeline

Every Critical or High-severity red-team finding produces: 1. A new corpus entry (input, expected safe outcome, threat tag, ATLAS tactic/technique ID, date, source reference) committed to the relevant regression corpus within 30 days. 2. An IM-Data finding with severity tag and the named data-flow owner as assignee. 3. A TA-Data library-gap ticket if the finding was not in the archetype library, tracked with a named owner and a 30-day close SLA for Critical-tier gaps.

Outcome Metrics (L2)

Process Metrics (leading)

• Red-team schedule on calendar; no Critical-tier data flow skips a quarterly exercise.
• Corpus review cadence, monthly.
• Cross-archetype composition test plan published for each Critical-tier data flow with composition; reviewed by named architect.
• Finding → TA-Data library-gap pipeline: Critical gaps closed within 30 days; High within 60 days.

Effectiveness Metrics (business value)

• Pre-production catch rate for Critical-tier increases as red-team exercises catch issues before corpus updates and pipeline changes deploy.
• Regression corpus catches post-update regressions early, corpus-update regressions that would have reached production detected by CI corpus before full pipeline rollout.
• Incident rate for Critical-tier data flows with full-scope testing is measurably lower than for those without (rolling 12-month comparison).

Success Criteria

• Quarterly red-team for 100% of Critical-tier data flows; semi-annual for 100% of High-tier; scope tied to TA-Data L2 per-flow deep threat models.
• All 6 regression corpora running on every PR for Critical-tier data flows; per-tier calibration enforced.
• ≥90% of Critical/High-severity red-team findings converted to corpus entries within 30 days.
• Cross-archetype composition tests documented and run for all Critical-tier data flows with composite archetype interactions.
• Per-tier SLA adherence for testing activities ≥90%.

Maturity Level 3

Objective: Operate continuous automated adversarial testing for Critical-tier data flows, contribute regression corpora and findings as open artifacts, and contribute discovered data-attack techniques to MITRE ATLAS, AVID, OWASP LLM, and NIST AI RMF Data

At this level, testing runs continuously rather than periodically. An automated adversarial testing harness probes Critical-tier data flows daily using generated poison-pattern variants, retrieval-extraction ladder generators, embedding-inversion probe generators, and PII-redaction-edge mutators. Novel findings are triaged into the TA-Data library weekly. Anonymized regression corpora, test patterns, and discovered data-attack techniques are contributed to MITRE ATLAS, AVID, OWASP LLM, and NIST AI RMF Data.

Dependencies

• ST-Data L2 (required): per-tier red-team function, regression corpora, and finding → corpus pipeline must be operational.
• TA-Data L3 (required): automated TTP-ingestion pipeline from ST-Data telemetry feeds the TA-Data L3 auto-update loop.
• ML-Data L2+ (required): detections instrumented to catch automated-probe activity so the harness and the monitoring pipeline are calibrated against each other.
• SM-Data L3 (alignment): automated inventory signals feed the automated test harness with the current Critical-tier data flow list and their tier metadata.

Desired Outcomes

• Critical-tier data-flow adversarial posture is measured daily, not quarterly.
• Novel data-attack techniques discovered by the automated harness reach the TA-Data library within 14 days.
• Program-originated findings appear in MITRE ATLAS, AVID, and OWASP LLM / NIST AI RMF Data revisions, the org is a net contributor to the AI data security ecosystem.
• Published regression corpora and test patterns are adopted by peer organizations.

Activities

A) Continuous automated adversarial testing harness

Deploy an automated adversarial testing harness that runs daily against all Critical-tier AI/HAI data flows:

• Poison-pattern generator: produces novel poison-pattern variants using mutation of the regression corpus (label-flip pattern variants, backdoor-trigger phrase mutations, template-based variation); runs against training-corpus and fine-tuning-dataset archetypes.
• Retrieval-extraction ladder generator: generates novel retrieval-extraction query sequences (prefix ladders, semantic near-duplicates, query-rate staircase patterns) to probe the per-query cap enforcement; runs against retrieval-store archetypes.
• Embedding-inversion probe generator: generates nearest-neighbor query sequences designed to recover source-text attributes from embedding space; probes the inversion-defense boundary; runs against embedding-store archetypes.
• PII-redaction-edge mutator: generates canary PII variants in novel formats and encoding patterns to probe redaction-edge completeness; runs against inference-input-stream archetypes.

Findings triaged by a named ST-Data owner at least weekly. New data-attack techniques (patterns not in the TA-Data library) fed into the TA-Data L3 auto-proposal pipeline within 14 days. High-severity automated findings route to IM-Data within 24 hours.

B) Contribute findings to industry

Contribute anonymized, legally-vetted findings to: - MITRE ATLAS, new data-attack technique observations (novel poison-pattern mechanics, new retrieval-extraction sequences, embedding-inversion variants not covered by AML.T0019/AML.T0025/AML.T0018); submissions follow ATLAS evidence-and-provenance requirements; target ≥2 contributions per year. - AI Vulnerability Database (AVID), structured disclosure submissions for novel vulnerabilities in own-operated AI data flows or upstream pipeline dependencies (coordinated disclosure where third-party components are involved). - OWASP LLM Top 10 / NIST AI RMF Data, real-world telemetry evidence during revision cycles; target ≥2 substantive submissions per revision cycle. - External benchmarks, participate in AISI Inspect evaluation benchmarks and sector ISAC AI data-security working groups with anonymized findings.

C) Publish regression corpora and test patterns as open artifacts

• Publish anonymized versions of all six regression corpora (poison-detection, retrieval-extraction, retrieval-poisoning, embedding-inversion, PII-redaction-edge, DSAR-query) under an open license; scrubbed of org-specific data classes, flow identifiers, and vendor names.
• Maintain the published versions upstream; internal corpora are a superset of the published versions with org-specific entries not shared externally.
• Host or co-host at least one industry data-security benchmark per year (OWASP AI chapter, ATLAS practitioner table, sector ISAC AI data working group); collect cross-org detection-benchmark improvement data.

Outcome Metrics (L3)

Process Metrics (leading)

• Continuous harness health, % Critical-tier data flows producing a fresh automated probe result within the last 24 hours; on-call paged when a harness feed goes stale >24 hours.
• New-technique triage cadence, automated probe findings reviewed weekly by named ST-Data owner; novel patterns forwarded to TA-Data library within 14 days.
• Industry-contribution pipeline, at least one anonymized finding in-preparation, in-legal-review, or submitted at any time.
• Industry-exercise calendar, next hosted or co-hosted exercise scheduled at least 60 days in advance.

Effectiveness Metrics (business value)

• Mean time to detect novel AI data-attack techniques decreases as the continuous harness catches corpus-update regressions and new attack patterns within hours, not sprint cycles.
• Program-originated data-attack techniques recognized in MITRE ATLAS or AVID demonstrate external validation of testing rigor.
• Cross-org exercise participants cite improved detection benchmarks from shared corpora, measurable uplift in peer organizations' defenses.
• Critical-tier incidents attributable to training-data poisoning, retrieval-extraction, or PII pass-through drop as the continuous harness closes the gap between quarterly red-team cycles.

Success Criteria

• ≥80% of Critical-tier AI/HAI data flows under continuous automated adversarial testing with daily probe execution.
• New data-attack technique ingestion lead time ≤14 days; automated findings triaged weekly by named owner.
• ≥4 industry contributions per year to MITRE ATLAS / AVID / OWASP LLM / NIST AI RMF Data.
• ≥4 open regression corpora published under a permissive license and maintained upstream.
• ≥1 industry-shared exercise hosted per year plus ≥2 participated; cross-org detection-benchmark improvement documented.

Key Success Indicators

Level 1: - Per-archetype foundational test battery published for all seven archetypes (training corpus, inference input stream, retrieval store, prompt/completion log corpus, embedding store, fine-tuning dataset, evaluation/test set), each test class tied to a TA-Data archetype threat (HAI TTP + ATLAS tactic/technique ID) and an SR-Data requirement. - Six regression corpora (poison-detection, retrieval-extraction, retrieval-poisoning, embedding-inversion, PII-redaction-edge, DSAR-query) in source control, running in CI on every PR for Critical/High-tier data flows, with named corpus owner and monthly refresh cadence. - 100% of AI/HAI data flows reaching production in the last 90 days have a passed go-live battery on record linked from the SM-Data inventory. - All test failures routed to IM-Data within 1 business day; CI automation covers ≥60% of battery items. - TA-Data archetype threat coverage by test battery ≥80%.

Level 2: - 100% of Critical-tier data flows red-teamed quarterly; 100% of High-tier semi-annually; scope tied to TA-Data L2 per-flow deep threat models. - Per-tier calibration enforced: Critical-tier gets all 6 corpora on every PR plus monthly no-train probe and quarterly DSAR-query accuracy test; Low-tier gets poison-detection corpus on merge. - ≥90% of Critical/High-severity red-team findings converted to corpus entries within 30 days. - Cross-archetype composition tests documented and run for all Critical-tier data flows with composite archetype interactions (training+eval contamination, embedding+retrieval inversion, PII-input+log-corpus pass-through, fine-tuning+training lineage).

Level 3: - ≥80% of Critical-tier data flows under continuous automated adversarial testing with daily probe execution; novel data-attack techniques reaching TA-Data library within 14 days. - ≥4 industry contributions per year to MITRE ATLAS / AVID / OWASP LLM / NIST AI RMF Data; ≥4 open regression corpora published and maintained. - ≥1 hosted + ≥2 participated industry exercises per year with documented cross-org detection-benchmark improvement.

Common Pitfalls

Level 1: - ❌ Test battery reduced to a schema-validation check and a classification-accuracy assertion, no behavioral adversarial probes (poison-detection scan, PII-redaction-edge canary, per-tenant retrieval isolation, embedding-inversion probe) actually exercised. - ❌ Regression corpora committed to source control but not wired into CI, they exist but run only when a reviewer manually triggers them; coverage erodes after every sprint. - ❌ Go-live battery runs once pre-production but is never re-run after corpus updates, pipeline changes, or vendor LLM provider changes, test coverage erodes as the data flow evolves. - ❌ Test failures logged in a spreadsheet separate from IM-Data, no SLA enforcement, no aging visibility, no named owner; the same failure recurs across multiple deployments undetected. - ❌ Corpus refresh skipped for three months because "nothing new happened", ATLAS technique additions, GDPR enforcement decisions, and new poison-attack research pile up unread; the corpus is frozen at launch. - ❌ Test battery conflates archetypes, the retrieval-store battery does not include per-tenant isolation or retrieval-poisoning tests because the same checklist was applied to both the retrieval store and the embedding store. - ❌ Opt-out enforcement test planted at dataset construction time but never verified before a fine-tuning run, the test confirms the opt-out record exists but never verifies the subject is actually excluded from the dataset presented to the fine-tune job.

Level 2: - ❌ Red-team scope defined as "injection probes" but retrieval-extraction ladders, cross-tenant isolation probes, embedding-inversion attacks, and DSAR-surface enumeration are excluded, the top threat classes for retrieval and embedding archetypes go untested. - ❌ Corpus growth declared at ≥1 per month but entries are variations on the same payload format, the corpus does not cover the new poison-pattern techniques published since L2 launch; corpus-update regressions pass the CI run and fail in production. - ❌ Per-tier calibration documented in the tier-treatment matrix but CI pipeline applies the same corpus to all tiers, Critical data flows run the same tests as Low; differentiation exists on paper only. - ❌ Red-team findings route to IM-Data but the finding → corpus pipeline is never executed, 12 months of Critical/High findings sit in IM-Data as closed tickets with no corpus entries; the same vulnerabilities are re-discovered at the next red-team exercise. - ❌ Cross-archetype composition tests scoped but not executed because "no engineer owns training-corpus + eval-set testing", contamination-prevention failure modes are in the threat model but not in any test.

Level 3: - ❌ Continuous harness runs poison-pattern probes that the pipeline's pre-flight scan trivially blocks, coverage metric looks good but the probes are not exercising the real threat surface; novel poison-pattern variants from recent research are not generated. - ❌ Industry contributions are legally-vetted case-study summaries rather than actionable, reproducible data-attack technique descriptions, ATLAS reviewers cannot map them to a technique ID; AVID submissions lack reproducibility notes. - ❌ Open corpora published once and then not maintained, external organizations build on a stale version while the internal corpus has 60 new entries; discrepancies surface at community exercises and damage the program's credibility. - ❌ New data-attack technique ingestion from automated probes to the TA-Data library is manual and quarterly, by the time a novel technique reaches SR-Data and SA-Data updates and is reflected in controls, the technique is already being exploited in the wild. - ❌ Hosted industry exercise becomes a capabilities showcase rather than a detection-benchmarking session, no measurable improvement data is collected from participants; the "≥1 hosted per year" metric is met without producing any cross-org security uplift.

Practice Maturity Questions

Level 1: 1. Is a per-archetype foundational test battery published for all seven AI/HAI data archetypes, with each test class tied to a TA-Data archetype threat (HAI TTP + ATLAS tactic/technique ID) and an SR-Data requirement, defined inputs/outputs/pass-fail criteria, and an evidence artifact, and are 100% of new AI/HAI data flows required to pass the battery before production Sanctioned status is issued? 2. Are six regression corpora (poison-detection, retrieval-extraction, retrieval-poisoning, embedding-inversion, PII-redaction-edge, DSAR-query) versioned in source control, running in CI on every PR for Critical/High-tier data flows, with a named corpus owner, a monthly refresh cadence from internal + external sources, and a CI token/compute budget cap, and are ≥95% of Critical/High-tier PR merges verified to have run and passed the applicable corpus? 3. Are all test failures routed to IM-Data within 1 business day with a severity tag and named owner, and does TA-Data archetype threat coverage by the test battery and corpus reach ≥80% by end of year one?

Level 2: 1. Are 100% of Critical-tier AI/HAI data flows red-teamed at least quarterly, and 100% of High-tier semi-annually, with scope derived from TA-Data L2 per-flow deep threat models, covering poison-injection attempts, retrieval-extraction probes beyond the CI corpus, cross-tenant isolation probes, embedding-inversion attacks, DSAR-surface enumeration, opt-out bypass attempts, and eval-training contamination probes, with findings routed to IM-Data and remediation tracked? 2. Is per-tier corpus calibration enforced in CI (Critical-tier: all 6 corpora on every PR + monthly no-train probe + quarterly DSAR-query accuracy; Low-tier: poison-detection corpus on merge), and are ≥90% of Critical/High-severity red-team findings converted to corpus entries within 30 days? 3. Are cross-archetype composition tests (training+eval contamination, embedding+retrieval inversion via retrieval, PII-input+log-corpus pass-through, fine-tuning+training lineage propagation) documented and executed for all Critical-tier data flows with composite archetype interactions, and is per-tier SLA adherence for testing activities ≥90%?

Level 3: 1. Are ≥80% of Critical-tier AI/HAI data flows under continuous automated adversarial testing with daily probe execution, using poison-pattern generators, retrieval-extraction ladder generators, embedding-inversion probe generators, and PII-redaction-edge mutators, with novel data-attack techniques triaged into the TA-Data library within 14 days and high-severity automated findings routed to IM-Data within 24 hours? 2. Has the program contributed ≥4 anonymized, legally-vetted findings per year to MITRE ATLAS, AVID, OWASP LLM, or NIST AI RMF Data, with at least one accepted as a new or refined data-attack technique, and are ≥4 open regression corpora published under a permissive license and maintained upstream? 3. Has the program hosted at least 1 industry-shared data-security exercise per year and participated in ≥2 additional cross-org exercises, with documented cross-org detection-benchmark improvement data from participants?

Document Version: HAIAMM v3.0 Practice: Security Testing (ST) Domain: Data Last Updated: 2026-05-13 Author: Verifhai