Implementation Review (IR) - Data Assessment

Assessment questionnaire for measuring maturity. Answer each question honestly based on current, implemented practices.

Implementation Review (IR) - Data Domain

HAIAMM Assessment Questionnaire v3.0

Practice: Implementation Review (IR) Domain: Data Purpose: Assess organizational maturity in verifying that the actually-deployed data flows feeding AI/HAI systems match the design approved at DR-Data, and stay there as pipelines, classification schemes, and consumer AI artifacts evolve. Scoring Model: Evidence + Outcome Metrics (see Scoring Methodology below)

Instructions

Answer each question honestly based on current, implemented practices (not plans or aspirations)
Each question has two components: Evidence (what you did) and Outcome Metrics (how well it worked)
Scoring uses 4 tiers: Fully Mature (1.0), Implemented (0.67), Partial (0.33), Not Implemented (0.0)
Answer progressively, Complete all Level 1 questions before Level 2
Level progression, Achieve ALL questions at lower level before advancing
Baseline first, Record current metric values before setting targets

Scoring Methodology

Score	Label	Criteria
1.0	Fully Mature	Evidence complete + ≥3 outcome metrics meet targets
0.67	Implemented	Evidence complete + 2 outcome metrics meet targets
0.33	Partial	Evidence partially complete + <2 metrics meet targets
0.0	Not Implemented	No evidence of practice

Level Score = Average of the three question scores at that level Overall Score = Weighted average across all levels achieved

Maturity Level 1

Objective: Run per-archetype implementation reviews at go-live, annually, and on material change, verifying deployed data flows match the SA-Data pattern, configuration matches the DR-Data decision, and SR-Data REM evidence is current

At this level, the gap between approved data-flow design and running pipeline is systematically checked at the moments it matters most. Every review produces findings with severity tags, named owners, and SLA-bound resolution dates.

Question 1: Per-Archetype Data IR Checklist

Q1.1: Is there a published, per-archetype IR checklist, one per SM-Data archetype (training corpus, inference input stream, retrieval store, prompt/completion log corpus, embedding store, fine-tuning dataset, evaluation/test set), covering classification-label propagation, lineage-as-designed, consent-basis currency, retention enforcement, encryption-key vault binding, access-control verification, and DSAR-query accuracy test?

Evidence Required: - [ ] Published checklist per SM-Data archetype on file and linked from the SM-Data inventory (training corpus, inference input stream, retrieval store, prompt/completion log corpus, embedding store, fine-tuning dataset, evaluation/test set) - [ ] Inference-input-stream checklist includes a PII-redaction canary test: a canary PII record injected into the test path and the payload reaching the LLM API verified to have the PII redacted; test execution log on file - [ ] Classification-label propagation verified by pulling a sample of records at each pipeline stage and confirming labels match the approved taxonomy and are not silently stripped by transformation steps - [ ] Retention-enforcement verification present: deletion job or retention policy execution log with last successful run timestamp; not just a policy declaration - [ ] DSAR-query accuracy tested: a test DSAR query for a canary subject record executed and verified to return expected data from this archetype's DSAR surface; test result on file - [ ] No-train flag verified via vendor admin API for flows with outbound LLM calls (OpenAI / Anthropic / Bedrock / Vertex admin APIs), not from DPA text or one-time admin-console screenshots; probe result on file with IR record

Outcome Metrics:

Metric	Baseline	Current	Target	Met?	Notes
% AI/HAI data flows with a go-live IR record	___%	___%	100%	☐	SM-Data inventory × IR records
% active AI/HAI data flows with a current-year IR record	___%	___%	≥90%	☐	SM-Data inventory × IR records
Critical / blocker findings open at go-live	___	___	0	☐	Findings backlog
Median closure time for High findings	___ days	___ days	≤7 days	☐	Findings backlog

Metric Collection Guidance: - Go-live IR coverage: count data flows in SM-Data inventory with a linked IR record dated at or before production cutover divided by total flows entering production. Formula: flows_with_golive_IR / total_flows_in_production × 100 - Current-year IR coverage: count flows with an IR record dated within the last 12 months divided by total active flows. Source: SM-Data inventory last-IR-date field - Blocker findings at go-live: count Critical/blocker findings that were open on the go-live date. Source: IM-Data findings history - High finding closure time: median calendar days from finding-opened to evidence-linked finding-closed for High-severity findings. Source: IM-Data backlog timestamps

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No per-archetype checklist published)

Evidence Location: _________

Validation Date: _________

Notes: _______

Question 2: Review Triggers and No-Train Vendor Probing

Q1.2: Do 100% of new AI/HAI data flows going to production in the last 90 days carry a go-live IR record, with material-change triggers wired to SM-Data inventory events, and are no-train flags verified via vendor admin API for ≥80% of flows with outbound LLM calls?

Evidence Required: - [ ] Go-live IR records on file for all data flows entering production in the last 90 days; catalog and pipeline metadata confirming the IR was completed before production cutover - [ ] Material-change trigger wired to SM-Data inventory: new data sources, classification changes, cross-border routing changes, retention policy changes, new consumer AI artifacts, and vendor LLM provider changes generate a review-due alert within 5 business days - [ ] Vendor admin API probe records on file for flows with outbound LLM calls: OpenAI data_controls.training_data_sharing = false; Anthropic no-train terms; Bedrock Service Control Policy; Vertex AI / Gemini Organization Policy, at go-live, at annual review, and on material change - [ ] Lineage-as-designed verified: data-catalog lineage graph (Atlan / Collibra / DataHub / Unity Catalog) confirmed to match the approved flow design; no new sources or consumers added since the last DR without a corresponding IR - [ ] Consent-basis / lawful-basis evidence current: GDPR Art. 6 or Art. 9 lawful-basis record cited in the SR-Data REM confirmed still active; for consent-based processing, a sample of consent records verified - [ ] Annual review calendar populated from the SM-Data inventory with flows nearing review-due date visible at least 30 days in advance

Outcome Metrics:

Metric	Baseline	Current	Target	Met?	Notes
% AI/HAI data flows with a go-live IR record	___%	___%	100%	☐	SM-Data inventory × IR records
% active AI/HAI data flows with a current-year IR record	___%	___%	≥90%	☐	SM-Data inventory × IR records
% material changes to production data flows that trigger an IR before the change ships	___%	___%	100%	☐	SM-Data change events × IR records
% flows with outbound LLM calls where no-train confirmed via vendor admin API (not DPA text alone)	___%	___%	≥80%	☐	Vendor API probe log

Metric Collection Guidance: - Material-change IR trigger rate: cross-reference SM-Data inventory material-change events against IR records created within 5 business days of each event - No-train API coverage: count flows with outbound LLM calls where a vendor admin API probe result is on file (not DPA text only) divided by total flows with outbound LLM calls. Source: vendor API probe log - Lineage accuracy: % of flows where the live catalog lineage graph matches the DR-approved flow design at the time of IR. Source: catalog diff records × DR flow design - Current-year IR coverage: count flows with an IR record dated within the last 365 days. Source: SM-Data inventory last-IR-date

Evidence Location: _________

Validation Date: _________

Notes: _______

Question 3: Findings Tracking and SR-Data REM Loop

Q1.3: Are findings severity-tagged and tracked in IM-Data with named owners and SLA-bound closure dates, and does every IR finding that reveals stale or inaccurate REM evidence trigger an SR-Data REM row update before the finding is closed?

Evidence Required: - [ ] Findings backlog in IM-Data showing all IR findings with severity tag (Critical / High / Medium / Low), named owner (not "the data team"), SLA-bound closure date, and linked after-fix evidence artifact - [ ] SR-Data REM update loop active: IR findings that reveal stale or inaccurate REM evidence have a linked REM row update on file before the finding is closed - [ ] Pipeline drift sources verified at L1: pipeline metadata (Airflow DAG versions, dbt model versions, Fivetran connector versions) reviewed since the last IR and compared against the DR-Data-approved pipeline specification; deviations documented as findings - [ ] Encryption-key vault binding verified: key references in deployed pipeline artifact point to the declared KMS vault, not to environment variables, code, or config files; confirmed via secrets scan in the deployed artifact - [ ] Severity calibration consistent: Critical findings include classification-label gap on a training corpus, PII-redaction step bypassed on an inference input stream, and DSAR-query returning unexpected subjects, not treated as Medium - [ ] Findings-aging dashboard reviewed at least monthly by the program sponsor: meeting record or dashboard screenshot on file

Outcome Metrics:

Metric	Baseline	Current	Target	Met?	Notes
% AI/HAI data flows with a go-live IR record	___%	___%	100%	☐	SM-Data inventory × IR records
% active AI/HAI data flows with a current-year IR record	___%	___%	≥90%	☐	SM-Data inventory × IR records
Critical / blocker findings open at go-live	___	___	0	☐	Findings backlog
Median closure time for High findings	___ days	___ days	≤7 days	☐	Findings backlog

Metric Collection Guidance: - REM update loop rate: count IR findings where the closure is linked to an SR-Data REM row update vs. total IR findings that identified stale or inaccurate REM evidence. Source: IM-Data × SR-Data REM cross-reference - Named-owner coverage: count findings with a named individual as owner divided by total open findings. Source: IM-Data - SLA adherence by severity: % of High findings closed within 7 days; % of Critical findings closed before go-live. Source: IM-Data timestamps - Findings-aging review cadence: confirm monthly review via calendar record or dashboard export. Source: program sponsor review log

Evidence Location: _________

Validation Date: _________

Notes: _______

Maturity Level 2

Objective: Detect data-flow drift continuously for Critical and High-tier flows via catalog webhooks, pipeline-metadata events, lineage-graph signals, classification-scan deltas, and vendor admin API recurrent probes; calibrate IR cadence per SM-Data tier

At this level, implementation review becomes a continuous signal. Catalog-change webhooks, pipeline-metadata events, lineage-graph signals, classification-label-scan deltas, and vendor admin API probes are all wired to automated detection.

Question 4: Continuous Data-Flow Drift Detection

Q2.1: Are ≥90% of Critical-tier AI/HAI data flows under continuous drift detection, via data-catalog change webhooks, pipeline-metadata events, lineage-graph signals, classification-label-scan deltas, vendor admin API recurrent probes, and cross-border-flow routing monitoring, with median detection latency ≤7 days?

Evidence Required: - [ ] Catalog drift detection active: Atlan / Collibra / DataHub / Unity Catalog change webhooks configured for Critical and High-tier flows; classification-label changes, ownership changes, policy-tag changes, and new downstream consumers trigger automated diff against DR-Data-approved baseline with findings auto-generated on material deviations - [ ] Pipeline drift detection active: Airflow DAG changes, dbt model changes, Fivetran connector version changes, new source additions compared against the DR-Data-approved pipeline specification; deviations flagged - [ ] Lineage drift detection active: new upstream sources or downstream consumers appearing in the lineage graph since the last IR automatically open findings; any lineage edge not present in the approved flow design generates a finding - [ ] Classification drift detection active: Macie / BigID / Purview scan deltas vs. baseline run on a schedule; newly discovered PII or regulated data classes in an archetype not approved for that class open Critical or High findings - [ ] Cross-border flow drift detection: routing metadata compared against the approved cross-border transfer map; new jurisdictions or changed transfer mechanisms open IR findings within the detection SLA - [ ] Drift-detection pipeline health monitored: % Critical flows producing a fresh signal in the last 7 days; on-call alert configured for >48 hours of feed silence; alert test record on file

Outcome Metrics:

Metric	Baseline	Current	Target	Met?	Notes
% Critical-tier data flows under continuous drift detection (catalog, pipeline, lineage, classification-scan, vendor admin API)	___%	___%	≥90%	☐	Drift-detection telemetry
Median drift detection latency, Critical-tier	___ days	___ days	≤7 days	☐	IR telemetry
% Critical/High-tier flows with outbound LLM calls where no-train confirmed via vendor admin API recurrent probe	___%	___%	≥80%	☐	Vendor API probe log
Tier-cadence adherence (% of flows reviewed on their published cadence)	___%	___%	≥95%	☐	IR schedule × SM-Data inventory

Metric Collection Guidance: - Continuous drift detection coverage: count Critical-tier flows with all six signal sources active (catalog, pipeline, lineage, classification-scan, vendor admin API, cross-border) divided by total Critical-tier flows. Source: drift-detection pipeline configuration registry - Drift detection latency: median time from change-event timestamp to IR-finding-opened timestamp for Critical-tier drift findings. Source: IR telemetry × IM-Data - Vendor admin API recurrent probe coverage: count Critical/High-tier flows with outbound LLM calls where a vendor admin API probe result is dated within the current probe cycle divided by total Critical/High-tier flows with outbound LLM calls. Source: vendor API probe log - Tier-cadence adherence: count flows reviewed within their published cadence window divided by total flows. Source: IR schedule × SM-Data inventory last-IR-date

Evidence Location: _________

Validation Date: _________

Notes: _______

Question 5: Vendor Admin API Recurrent Probing for No-Train and Retention

Q2.2: Are no-train flags and retention settings verified via vendor admin APIs on a monthly (Critical) and quarterly (High) probing cadence, not from DPA text alone, covering ≥80% of Critical/High-tier flows with outbound LLM calls?

Evidence Required: - [ ] Vendor admin API probe records on file for Critical/High-tier flows with outbound LLM calls: OpenAI Org Settings API (data_controls.training_data_sharing = false); Anthropic Organization admin settings; Bedrock Service Control Policy and invocation logging config; Vertex AI / Gemini Organization Policy constraints - [ ] Probing cadence log showing Critical-tier probes monthly and High-tier probes quarterly, not one-time screenshots; all probe runs date-stamped with method (API or fallback UI) - [ ] Delta findings on file: any setting change detected (no-train toggle reset, retention period changed, logging config altered) generated an IR finding with severity matching the data-class impact of the change - [ ] Vendor API probing calendar maintained: missed probes tracked as process-metric failures with root cause and remediation on file - [ ] Classification-scan deltas compared to the DR-Data-approved baseline: Macie / BigID / Purview scan output compared against the approved classification scope; newly discovered classes generate findings - [ ] IR findings from probe deltas routed to IM-Data automatically with severity and owner pre-populated

Outcome Metrics:

Metric	Baseline	Current	Target	Met?	Notes
% Critical-tier data flows under continuous drift detection (catalog, pipeline, lineage, classification-scan, vendor admin API)	___%	___%	≥90%	☐	Drift-detection telemetry
Median drift detection latency, Critical-tier	___ days	___ days	≤7 days	☐	IR telemetry
% Critical/High-tier flows with outbound LLM calls where no-train confirmed via vendor admin API recurrent probe	___%	___%	≥80%	☐	Vendor API probe log
% Critical/High-tier flows contributing to DSAR surface with DSAR-query accuracy test on record (current IR cycle)	___%	___%	100%	☐	IR records

Metric Collection Guidance: - Vendor admin API recurrent probe coverage: count Critical/High-tier flows with a vendor admin API probe result dated within the current probe cycle divided by total flows with outbound LLM calls. Source: vendor API probe log - Delta finding rate: count probe cycles where a setting change was detected and a finding was generated vs. total probe cycles. Source: probe log × IM-Data - Probe cadence compliance: count monthly Critical-tier probes completed on schedule vs. planned. Source: probing calendar - Classification-scan delta coverage: % of Critical/High-tier flows with a classification-scan delta report compared to the DR-Data-approved baseline in the current IR cycle. Source: scan tooling output

Evidence Location: _________

Validation Date: _________

Notes: _______

Question 6: DSAR-Query Accuracy Verification and Tier-Calibrated Cadence

Q2.3: Are 100% of Critical/High-tier data flows contributing to the DSAR surface covered by DSAR-query accuracy tests in the current IR cycle, and is tier-cadence adherence ≥95% with Critical-tier findings aged per the SM-Data L2 tier-treatment matrix SLAs?

Evidence Required: - [ ] DSAR-query accuracy test records on file for Critical/High-tier flows contributing to the DSAR surface: canary subject record inserted, subject-access query executed via the DSAR fulfillment system, canary record returned with correct attributes, no co-mingling with other subjects, and canary record correctly excluded after a deletion request is processed - [ ] Cadence for DSAR tests documented and executed: Critical-tier quarterly, High-tier semi-annual; test records date-stamped with method and reviewer - [ ] Tier-cadence enforcement visible in SM-Data inventory: Critical-tier flows flagged when no IR in the last 180 days; escalation to program sponsor automated and test record on file - [ ] IR backlog tier-aware: Critical-tier findings reported separately; no Critical-tier finding waiting behind Low-tier queue items - [ ] DSAR-test failures documented as High findings (DSAR-surface exposure risk) with named owner and SLA-bound closure date - [ ] Tier-treatment matrix SLA compliance tracked: Critical-tier findings closed per SM-Data L2 SLA (High ≤7 days, Critical blocker before go-live); SLA breach rate reported to program sponsor

Outcome Metrics:

Metric	Baseline	Current	Target	Met?	Notes
% Critical-tier data flows under continuous drift detection (catalog, pipeline, lineage, classification-scan, vendor admin API)	___%	___%	≥90%	☐	Drift-detection telemetry
Median drift detection latency, Critical-tier	___ days	___ days	≤7 days	☐	IR telemetry
% Critical/High-tier flows contributing to DSAR surface with DSAR-query accuracy test on record (current IR cycle)	___%	___%	100%	☐	IR records
Tier-cadence adherence (% of flows reviewed on their published cadence)	___%	___%	≥95%	☐	IR schedule × SM-Data inventory

Metric Collection Guidance: - DSAR-query accuracy test coverage: count Critical/High-tier flows contributing to the DSAR surface with a DSAR-query accuracy test record in the current IR cycle divided by total Critical/High-tier flows contributing to the DSAR surface. Source: IR records - DSAR test pass rate: % of DSAR-query accuracy tests where canary record was included correctly and excluded correctly after deletion. Source: test records - Tier-cadence adherence: % of flows reviewed within their published cadence window. Source: SM-Data inventory last-IR-date × cadence policy - Tier-SLA breach rate: count Critical-tier findings exceeding the SM-Data L2 tier-treatment matrix SLA divided by total Critical-tier findings. Source: IM-Data timestamps

Evidence Location: _________

Validation Date: _________

Notes: _______

Maturity Level 3

Objective: Daily attestation per Critical-tier data flow, classification labels current, retention SLA met, encryption-key-rotation healthy, no-train probe results green, with drift auto-opening IM-Data tickets and configuration baseline schemas contributed to OpenSSF AI Data, DAMA, and OWASP SAMM AI

At this level, configuration for Critical-tier data flows is attested continuously across four dimensions. Every Critical data flow produces a daily attestation signal. Drift auto-opens IM-Data tickets.

Question 7: Daily Attestation Signal for Data Flows

Q3.1: Are ≥90% of Critical-tier AI/HAI data flows producing a daily attestation signal across all four dimensions (classification-label currency, retention SLA, encryption-key-rotation health, no-train probe results green), with deviations auto-opening IM-Data tickets within 1 hour?

Evidence Required: - [ ] Daily attestation pipeline operational covering four dimensions per Critical-tier flow: (1) classification labels current, automated catalog scan confirms labels match approved taxonomy and have not been silently modified in the last 24 hours; (2) retention SLA met, deletion-job execution log confirms job ran within expected window and no records past retention date remain; (3) encryption-key-rotation healthy, KMS confirms key is within declared rotation schedule and has not been exported outside vault; (4) no-train probe results green, vendor admin API probe result from the most recent probe cycle (≤30 days) confirms no-train setting active - [ ] Auto-open IM-Data ticket on drift: ticket created within 1 hour of detection; carries drift dimension, specific control that failed tolerance, and link to DR-Data decision record - [ ] Attestation-pipeline health monitored: % Critical flows with fresh signal in the last 24 hours; on-call paged for any Critical flow silent for >24 hours; pager history on file - [ ] Zero stale-evidence violations confirmed for Critical-tier REMs: no evidence citations outside their defined freshness window - [ ] Attestation artifacts regulator-consumable: machine-readable format confirmed suitable for GDPR Art. 35 DPIA evidence, EU AI Act Art. 10 data-governance records, and ISO/IEC 42001 AIMS operational records via test export - [ ] Configuration tolerance definitions documented per-control: tolerance rules versioned and linked from SM-Data inventory; tolerances tuned to avoid alert fatigue from minor pipeline dependency version bumps

Outcome Metrics:

Metric	Baseline	Current	Target	Met?	Notes
% Critical-tier data flows producing a daily attestation signal (all 4 dimensions)	___%	___%	≥90%	☐	Attestation telemetry
% attestation findings auto-opening IM-Data tickets within 1 hour of detection	___%	___%	≥95%	☐	IM-Data integration telemetry
Evidence freshness violations (stale evidence in active REMs)	___	___	0 for Critical; trending toward 0 for High	☐	Attestation telemetry
IR reviewer-hours per Critical data flow per year	___ hrs	___ hrs	trending down QoQ	☐	Reviewer time tracking

Metric Collection Guidance: - Daily attestation coverage: count Critical-tier flows with a signed attestation artifact dated within the last 24 hours covering all four dimensions divided by total Critical-tier flows. Source: attestation pipeline telemetry - Auto-open ticket latency: median time from drift-detection event to IM-Data ticket created. Source: attestation pipeline timestamp × IM-Data created-at timestamp - Stale-evidence violations: count Critical-tier REM rows with evidence citations outside their freshness window. Source: attestation pipeline evidence-freshness dimension output - IR reviewer-hours: total hours logged by reviewers for Critical-tier data IR activities per year. Source: reviewer time tracking

Evidence Location: _________

Validation Date: _________

Notes: _______

Question 8: External Data-Flow Schema Contribution

Q3.2: Has the program published per-archetype data-flow configuration baseline schemas to OpenSSF AI Data, DAMA International, OWASP SAMM AI, or CSA AI Safety Initiative, with documented external adoption and internal practice aligned to the published versions?

Evidence Required: - [ ] Per-archetype data-flow configuration baseline schemas published to at least one external body: OpenSSF AI Data working group, DAMA International AI Data Governance, OWASP SAMM AI extensions, or CSA AI Safety Initiative AI Controls Matrix - [ ] Internal practice aligned to published external versions: documented mapping between internal IR checklist and published schema; internal-only deviations proposed as upstream changes with a PR or issue link - [ ] External adoption tracked: citations, forks, direct acknowledgment from peer organizations, or inclusion in external tooling or assessment frameworks, evidence on file - [ ] Schema publication pipeline active: at least one schema in-draft, in-review, or published at any time; publication status tracker maintained - [ ] Adoption trend documented: at least one adoption data point on file since initial publication - [ ] Internal schema version pinned to published external version: divergence from the external version documented as a proposed upstream change

Outcome Metrics:

Metric	Baseline	Current	Target	Met?	Notes
% Critical-tier data flows producing a daily attestation signal (all 4 dimensions)	___%	___%	≥90%	☐	Attestation telemetry
% attestation findings auto-opening IM-Data tickets within 1 hour of detection	___%	___%	≥95%	☐	IM-Data integration telemetry
External adoption of published configuration baseline schemas	0	tracked	trending up	☐	External telemetry
IR reviewer-hours per Critical data flow per year	___ hrs	___ hrs	trending down QoQ	☐	Reviewer time tracking

Metric Collection Guidance: - External adoption: count citations, forks, and direct acknowledgments per quarter since publication. Source: GitHub stars/forks, external publication references, peer organization acknowledgments - Schema alignment rate: % of internal IR checklist items directly traceable to a published external schema item. Source: internal-to-external schema mapping document - Publication pipeline health: number of schemas in active publication stages. Source: publication tracker

Evidence Location: _________

Validation Date: _________

Notes: _______

Question 9: Automated Drift-to-IM Escalation and Post-Incident Feedback Loop

Q3.3: Is the post-incident IR feedback loop operational, IM-Data post-incident reviews include a mandatory IR-record re-examination step, and ≥1 attestation rule update is produced per material incident?

Evidence Required: - [ ] All IR findings (daily attestation and periodic reviews) flow into IM-Data automatically with severity and SLA pre-populated from the SM-Data L2 tier-treatment matrix; no manual entry required for attestation-generated findings - [ ] IM-Data SLA clock automated: overdue Critical findings escalate to the program sponsor at 50% and 100% of the SLA window automatically; escalation automation tested quarterly with synthetic finding, test record on file - [ ] Post-incident IR feedback loop documented in the IM-Data incident-review template: a mandatory IR-record re-examination step confirmed in the last material incident review record - [ ] Attestation rule updates produced from incident learning: at least one attestation rule update linked from a completed incident review in the last 12 months - [ ] Post-incident IR step completion tracked: % of material incidents in the last 12 months with a completed post-incident review including the IR-record re-examination step - [ ] Attestation-pipeline health log on file: ≥90% of Critical flows with fresh signal in the last 24 hours; pager history on file

Outcome Metrics:

Metric	Baseline	Current	Target	Met?	Notes
% Critical-tier data flows producing a daily attestation signal (all 4 dimensions)	___%	___%	≥90%	☐	Attestation telemetry
% attestation findings auto-opening IM-Data tickets within 1 hour of detection	___%	___%	≥95%	☐	IM-Data integration telemetry
Evidence freshness violations (stale evidence in active REMs)	___	___	0 for Critical; trending toward 0 for High	☐	Attestation telemetry
IR reviewer-hours per Critical data flow per year	___ hrs	___ hrs	trending down QoQ	☐	Reviewer time tracking

Metric Collection Guidance: - Auto-open ticket rate: count attestation-generated findings that created an IM-Data ticket within 1 hour divided by total attestation-generated findings. Source: attestation pipeline × IM-Data integration telemetry - Escalation automation test pass rate: % of quarterly escalation tests that confirmed correct automated escalation. Source: escalation test records - Post-incident IR step completion: % of material incident post-reviews with the IR-record re-examination step completed. Source: IM-Data incident review records - Attestation rule update rate: count attestation rule updates linked from post-incident reviews in the last 12 months. Source: attestation pipeline rule-change log × IM-Data incident reviews

Evidence Location: _________

Validation Date: _________

Notes: _______

Summary Scorecard

Level	Question	Score	Weight
L1	Q1: Per-Archetype Data IR Checklist	___	33%
L1	Q2: Review Triggers and No-Train Vendor Probing	___	33%
L1	Q3: Findings Tracking and SR-Data REM Loop	___	33%
L1 Total		___
L2	Q4: Continuous Data-Flow Drift Detection	___	33%
L2	Q5: Vendor Admin API Recurrent Probing	___	33%
L2	Q6: DSAR-Query Accuracy and Tier-Calibrated Cadence	___	33%
L2 Total		___
L3	Q7: Daily Attestation Signal for Data Flows	___	33%
L3	Q8: External Data-Flow Schema Contribution	___	33%
L3	Q9: Automated Escalation and Post-Incident Feedback	___	33%
L3 Total		___
Overall IR-Data Score		___

Current Maturity Level: ___

Key Findings:

Priority Improvements:

Document Version: HAIAMM v3.0 Practice: Implementation Review (IR) Domain: Data Questionnaire Date: 2026-05-15 Author: Verifhai

Instructions:

Answer based on current practices, not plans
“Yes” requires documented evidence
Complete all Level 1 questions before Level 2
Partial implementation = “No”

↓ Download as Markdown