Implementation Review (IR) - Processes Assessment

Assessment questionnaire for measuring maturity. Answer each question honestly based on current, implemented practices.

Implementation Review (IR) - Processes Domain

HAIAMM Assessment Questionnaire v3.0

Practice: Implementation Review (IR) Domain: Processes Purpose: Assess organizational maturity in verifying that the actually-running AI-embedded business workflow matches the design approved at DR, and stays there as the workflow and its underlying AI components evolve. Scoring Model: Evidence + Outcome Metrics (see Scoring Methodology below)


Instructions

  • Answer each question honestly based on current, implemented practices (not plans or aspirations)
  • Each question has two components: Evidence (what you did) and Outcome Metrics (how well it worked)
  • Scoring uses 4 tiers: Fully Mature (1.0), Implemented (0.67), Partial (0.33), Not Implemented (0.0)
  • Answer progressively, Complete all Level 1 questions before Level 2
  • Level progression, Achieve ALL questions at lower level before advancing
  • Baseline first, Record current metric values before setting targets

Scoring Methodology

Score Label Criteria
1.0 Fully Mature Evidence complete + ≥3 outcome metrics meet targets
0.67 Implemented Evidence complete + 2 outcome metrics meet targets
0.33 Partial Evidence partially complete + <2 metrics meet targets
0.0 Not Implemented No evidence of practice

Level Score = Average of the three question scores at that level Overall Score = Weighted average across all levels achieved


Maturity Level 1

Objective: Run per-archetype implementation reviews at go-live, annually, and on material change, verifying the running workflow matches the DR-approved design, HITL gates are substantive, Art. 50 disclosure is present, decision logging is complete, and affected-persons rights surface is responsive

At this level, the gap between the approved workflow design and the running workflow is systematically checked at the moments that matter most. Every review produces findings with severity tags, named owners, and SLA-bound resolution dates.


Question 1: Per-Archetype Workflow IR Checklist

Q1.1: Is there a published, per-archetype IR checklist, one per SM-Processes archetype (decision pipeline, customer-facing flow, human-AI collaboration chain, back-office augmentation, approval/review workflow, content-generation workflow, knowledge-management workflow), covering HITL gate substantiveness verification, Art. 50 disclosure presence check, decision-logging completeness, override-audit-trail queryability, affected-persons rights-response test, and fallback/kill-switch test?

Evidence Required: - [ ] Published checklist per SM-Processes archetype on file and linked from the SM-Processes inventory (decision pipeline, customer-facing flow, human-AI collaboration chain, back-office augmentation, approval/review workflow, content-generation workflow, knowledge-management workflow) - [ ] HITL gate substantiveness verified, not just a checkbox: test record on file confirming (a) gate fires for a declared trigger condition (synthetic trigger event, gate confirmed blocking progression), (b) reviewer SLA is met (HITL queue log sample with median review time within SLA), (c) override rationale is recorded for a stratified sample of reviewed items - [ ] Art. 50 disclosure presence verified via live UX sample-check (not staging environment): product-flow analytics confirm the disclosure element fires on every AI-touched interaction; screenshot audit of the deployed UI against the DR-approved disclosure design on file - [ ] Decision-logging completeness verified: a decision-log sample pulled and all required fields (decision identifier, timestamp, AI recommendation, human decision if HITL, reviewer identity, override flag, affected-person reference) confirmed present at expected rate - [ ] Override-audit-trail queryability tested: a sample query for a known override event executed and confirmed to return required fields within the declared SLA - [ ] Affected-persons rights-response test executed: a synthetic contestation request submitted; receipt, acknowledgment, and routing to review queue confirmed within the declared SLA; test record on file

Outcome Metrics:

Metric Baseline Current Target Met? Notes
% AI-embedded workflows with a go-live IR record ___% ___% 100% SM-Processes inventory × IR records
% active AI-embedded workflows with a current-year IR record ___% ___% ≥90% SM-Processes inventory × IR records
Critical / blocker findings open at go-live ___ ___ 0 Findings backlog
Median closure time for High findings ___ days ___ days ≤7 days Findings backlog

Metric Collection Guidance: - Go-live IR coverage: count workflows in SM-Processes inventory with a linked IR record dated at or before go-live divided by total workflows entering production. Formula: workflows_with_golive_IR / total_workflows_in_production × 100 - Current-year IR coverage: count workflows with an IR record dated within the last 12 months. Source: SM-Processes inventory last-IR-date field - Blocker findings at go-live: count Critical/blocker findings open on the go-live date. Source: IM-Processes findings history - High finding closure time: median calendar days from finding-opened to evidence-linked finding-closed for High-severity findings. Source: IM-Processes backlog timestamps

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No per-archetype checklist published)

Evidence Location: _________

Validation Date: _________

Notes: _______


Question 2: Review Triggers and Workflow Drift Sources

Q1.2: Do 100% of new AI-embedded workflows going live in the last 90 days carry a go-live IR record, with material-change triggers wired to SM-Processes inventory events, and are BPM-tool config drift sources reviewed at each IR?

Evidence Required: - [ ] Go-live IR records on file for all workflows entering production in the last 90 days; BPM tool config export and HITL queue log confirming the IR was completed before go-live - [ ] Material-change trigger wired to SM-Processes inventory: AI component swaps, HITL gate reconfigurations, new affected-person populations, Art. 50 disclosure changes, decision-logging scope reductions, contestation path changes, and class-shift monitor reconfigurations all generate a review-due alert within 5 business days - [ ] BPM-tool config drift sources reviewed since the last IR: Camunda BPMN file diff, Temporal workflow code diff, ServiceNow flow designer export, Argo workflow manifest diff reviewed and compared against the DR-approved design; deviations documented as findings - [ ] HITL queue logs reviewed: reviewer-throughput and override-rationale completion rate since the last IR reviewed and compared against declared targets - [ ] Product-flow analytics reviewed: where the AI step actually fires compared to the approved design, silent bypass confirmed absent; evidence: product-flow analytics export on file with IR record - [ ] Annual review calendar populated from the SM-Processes inventory with workflows nearing review-due date visible at least 30 days in advance

Outcome Metrics:

Metric Baseline Current Target Met? Notes
% AI-embedded workflows with a go-live IR record ___% ___% 100% SM-Processes inventory × IR records
% active AI-embedded workflows with a current-year IR record ___% ___% ≥90% SM-Processes inventory × IR records
% material changes to live workflows that trigger an IR before the change affects affected persons ___% ___% 100% SM-Processes change events × IR records
Median closure time for High findings ___ days ___ days ≤7 days Findings backlog

Metric Collection Guidance: - Material-change IR trigger rate: cross-reference SM-Processes inventory material-change events against IR records created within 5 business days of each event - BPM config drift detection rate: % of IRs where BPM tool config was actively diffed against the DR-approved design (not just the design text reviewed). Source: IR evidence records method field - HITL queue log review coverage: % of IRs where a HITL queue log sample was pulled and reviewer-throughput analyzed. Source: IR evidence records - Product-flow analytics review rate: % of IRs where product-flow analytics confirmed AI step firing location vs. approved design. Source: IR evidence records

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No material-change trigger or BPM drift source review)

Evidence Location: _________

Validation Date: _________

Notes: _______


Question 3: Findings Tracking and SR-Processes REM Loop

Q1.3: Are findings severity-tagged and tracked in IM-Processes with named owners and SLA-bound closure dates, and does every IR finding that reveals stale or inaccurate REM evidence trigger an SR-Processes REM row update before the finding is closed?

Evidence Required: - [ ] Findings backlog in IM-Processes showing all IR findings with severity tag (Critical / High / Medium / Low), named owner (not "the business team"), SLA-bound closure date, and linked after-fix evidence artifact (screenshot, config export, queue-log sample) - [ ] SR-Processes REM update loop active: IR findings that reveal stale or inaccurate REM evidence have a linked REM row update on file before the finding is closed - [ ] Severity calibration consistent: Critical findings include HITL gate absent or trivially bypassed on a High/Critical-tier workflow, Art. 50 disclosure absent on a customer-facing flow, and decision log not capturing required fields for an Annex III workflow, not downgraded to Medium - [ ] Override-audit logs reviewed since the last IR: schema (required fields present) and completeness (entries present at expected rate given workflow volume) verified and documented - [ ] Decision-distribution snapshot reviewed: a recent approval-rate or decision-rate distribution pulled and compared to the DR-approved monitoring baseline; drift documented as a finding if beyond declared threshold - [ ] Findings-aging dashboard reviewed at least monthly by the program sponsor: meeting record or dashboard screenshot on file

Outcome Metrics:

Metric Baseline Current Target Met? Notes
% AI-embedded workflows with a go-live IR record ___% ___% 100% SM-Processes inventory × IR records
% active AI-embedded workflows with a current-year IR record ___% ___% ≥90% SM-Processes inventory × IR records
Critical / blocker findings open at go-live ___ ___ 0 Findings backlog
Median closure time for High findings ___ days ___ days ≤7 days Findings backlog

Metric Collection Guidance: - REM update loop rate: count IR findings where the closure is linked to an SR-Processes REM row update vs. total IR findings that identified stale REM evidence. Source: IM-Processes × SR-Processes REM cross-reference - Named-owner coverage: count findings with a named individual as owner divided by total open findings. Source: IM-Processes - SLA adherence by severity: % of High findings closed within 7 days; % of Critical findings closed before go-live. Source: IM-Processes timestamps - Findings-aging review cadence: confirm monthly review via calendar record or dashboard export. Source: program sponsor review log

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No findings backlog or severity tagging in place)

Evidence Location: _________

Validation Date: _________

Notes: _______


Maturity Level 2

Objective: Detect workflow configuration drift continuously for Critical and High-tier workflows via BPM-tool change events, HITL-throughput monitoring, decision-distribution monitoring, and Art. 50 disclosure UI verification; probe affected-persons rights-response timing recurrently; calibrate IR cadence per SM-Processes tier

At this level, implementation review becomes a continuous signal. BPM-tool change webhooks, HITL-throughput monitoring, decision-distribution monitoring, and Art. 50 disclosure presence monitoring are all wired to automated detection.


Question 4: Continuous Workflow Drift Detection

Q2.1: Are ≥90% of Critical-tier AI-embedded workflows under continuous drift detection, via BPM-tool change events, HITL-throughput monitoring, decision-distribution monitoring, and Art. 50 disclosure presence monitoring, with median detection latency ≤7 days?

Evidence Required: - [ ] BPM-tool change event webhooks active for Critical and High-tier workflows: Camunda / Temporal / Argo / ServiceNow emit events on BPM model version changes; any change to HITL gate configuration, routing rules, or AI-step parameters triggers an automated diff against the DR-approved baseline; material deviations auto-open IR findings - [ ] HITL-throughput monitoring active: queue telemetry monitored for reviewer SLA miss (median review time exceeds declared SLA, alert within 24 hours), queue saturation (depth exceeds capacity model threshold, alert within 4 hours), and override-rationale completion drop (rate drops below declared minimum, alert within 24 hours) - [ ] Decision-distribution monitoring active: decision-rate and approval-rate distribution by class compared against the DR-approved baseline on a rolling 30-day window; drift beyond the declared threshold opens an IR finding automatically - [ ] Art. 50 disclosure UI A/B-test verification active: product-flow analytics monitored for the disclosure element's presence rate across all traffic slices; any traffic slice where disclosure is absent or fires at <100% rate opens an IR finding within 24 hours; A/B-test framework changes wired to the change-event trigger - [ ] Affected-persons rights-response monitoring active: contestation request response timing measured recurrently (monthly for Critical-tier); median response time compared against declared SLA; breaches open IR findings - [ ] Drift-detection pipeline health monitored: % Critical workflows producing a fresh signal in the last 7 days; on-call alert configured for >48 hours of feed silence; alert test record on file

Outcome Metrics:

Metric Baseline Current Target Met? Notes
% Critical-tier workflows under continuous drift detection (BPM-tool change events, HITL-throughput, decision-distribution, Art. 50 disclosure presence) ___% ___% ≥90% Drift-detection telemetry
Median drift detection latency, Critical-tier ___ days ___ days ≤7 days IR telemetry
% Critical/High-tier workflows with HITL substantiveness probe on record (current period) ___% ___% 100% IR records
Tier-cadence adherence (% of workflows reviewed on their published cadence) ___% ___% ≥95% IR schedule × SM-Processes inventory

Metric Collection Guidance: - Continuous drift detection coverage: count Critical-tier workflows with all four signal sources active (BPM-tool change events, HITL-throughput, decision-distribution, Art. 50 disclosure) divided by total Critical-tier workflows. Source: drift-detection pipeline configuration registry - Drift detection latency: median time from change-event timestamp to IR-finding-opened timestamp. Source: IR telemetry × IM-Processes - HITL substantiveness probe coverage: count Critical/High-tier workflows with a HITL substantiveness probe record in the current period divided by total Critical/High-tier workflows. Source: IR records - Tier-cadence adherence: count workflows reviewed within their published cadence window divided by total workflows. Source: IR schedule × SM-Processes inventory last-IR-date

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No automated drift detection in place)

Evidence Location: _________

Validation Date: _________

Notes: _______


Question 5: HITL Substantiveness Probing and Rights-Response Verification

Q2.2: Are HITL substantiveness probes completed monthly for Critical-tier and quarterly for High-tier, including decision-variance audits (escalation if ≥98% match rate), and are affected-persons rights-response probes completed on the same cadence?

Evidence Required: - [ ] HITL substantiveness probe records on file for Critical and High-tier workflows: stratified random sample audit of HITL queue decisions; probe checks reviewer decision time (unusually short decisions flagged), override-rationale quality (empty/boilerplate/below minimum character count flagged), and decision variance (human decision matches AI recommendation ≥98% of the time in sample, escalated to program sponsor and Business owner with documentation) - [ ] Escalation records on file where the ≥98% match rate was detected: escalation to program sponsor and Business owner documented with rationale and resolution - [ ] Probe cadence log showing Critical-tier probes monthly and High-tier probes quarterly, not one-time; all probe runs date-stamped with method and reviewer - [ ] Affected-persons rights-response probe records on file: synthetic contestation request submitted monthly (Critical) and quarterly (High); response receipt, acknowledgment, routing to review queue, and explanation generation all timed and compared against declared SLA; failures open IR findings with severity matching the rights-impact level of the workflow - [ ] Probe calendar maintained: missed probes tracked as process-metric failures with root cause and remediation on file - [ ] IR findings from HITL and rights-response probes routed to IM-Processes automatically with severity and owner pre-populated

Outcome Metrics:

Metric Baseline Current Target Met? Notes
% Critical-tier workflows under continuous drift detection (BPM-tool change events, HITL-throughput, decision-distribution, Art. 50 disclosure presence) ___% ___% ≥90% Drift-detection telemetry
Median drift detection latency, Critical-tier ___ days ___ days ≤7 days IR telemetry
% Critical/High-tier workflows with HITL substantiveness probe on record (current period) ___% ___% 100% IR records
% Critical/High-tier workflows with affected-persons rights-response probe on record (current period) ___% ___% 100% IR records

Metric Collection Guidance: - HITL substantiveness probe coverage: count Critical/High-tier workflows with a HITL substantiveness probe record in the current period divided by total Critical/High-tier workflows. Source: IR records - Decision variance escalation rate: count HITL audits where the ≥98% match rate was detected and escalated divided by total HITL audits. Source: probe records × escalation log - Rights-response probe coverage: count Critical/High-tier workflows with a rights-response probe record in the current period divided by total Critical/High-tier workflows. Source: IR records - Rights-response SLA breach rate: count probes where the response-time SLA was breached divided by total probes. Source: probe timing records

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No HITL substantiveness or rights-response probing)

Evidence Location: _________

Validation Date: _________

Notes: _______


Question 6: Tier-Calibrated Processes IR Cadence

Q2.3: Is tier-cadence adherence ≥95%, Critical on semi-annual + continuous drift detection, High on annual, Medium on annual, Low on go-live + material change, with Critical-tier findings aged per the SM-Processes L2 tier-treatment matrix SLAs?

Evidence Required: - [ ] Tier-cadence policy documented and published: Critical, go-live + semi-annual + material-change-triggered + continuous drift detection; High, go-live + annual + material-change-triggered; Medium, go-live + annual; Low, go-live + re-review on material change - [ ] IR schedule in the SM-Processes inventory confirmed against policy: every workflow carries a last-IR-date and next-IR-due field; Critical-tier workflows with no IR in the last 180 days flagged for escalation to the program sponsor - [ ] IR backlog tier-aware: Critical-tier findings reported separately; no Critical-tier finding waiting behind Low-tier queue items; tiered backlog dashboard or report on file - [ ] Drift findings from automated detection auto-open IM-Processes tickets, not dead-ending in an alert dashboard; confirmed via a sample of auto-opened findings from the last 90 days - [ ] HITL-throughput monitoring alerts treated as IR findings and opened in IM-Processes (not dismissed as operations noise): evidence of at least one HITL-throughput alert that generated an IM-Processes finding in the last 12 months - [ ] Tier-SLA breach rate tracked: count Critical-tier findings exceeding the SM-Processes L2 tier-treatment matrix SLA divided by total Critical-tier findings; rate reported to program sponsor monthly

Outcome Metrics:

Metric Baseline Current Target Met? Notes
% Critical-tier workflows under continuous drift detection (BPM-tool change events, HITL-throughput, decision-distribution, Art. 50 disclosure presence) ___% ___% ≥90% Drift-detection telemetry
Median drift detection latency, Critical-tier ___ days ___ days ≤7 days IR telemetry
% Critical/High-tier workflows with HITL substantiveness probe on record (current period) ___% ___% 100% IR records
Tier-cadence adherence (% of workflows reviewed on their published cadence) ___% ___% ≥95% IR schedule × SM-Processes inventory

Metric Collection Guidance: - Tier-cadence adherence: count workflows reviewed within their published cadence window divided by total workflows. Source: SM-Processes inventory last-IR-date × cadence policy - Critical-tier SLA breach rate: count Critical-tier findings exceeding the SM-Processes L2 tier-treatment matrix SLA divided by total Critical-tier findings. Source: IM-Processes timestamps - Auto-open ticket rate: count drift findings that auto-opened an IM-Processes ticket divided by total drift findings. Source: drift-detection telemetry × IM-Processes - HITL alert-to-finding conversion rate: count HITL-throughput alerts that generated an IM-Processes finding divided by total HITL-throughput alerts. Source: HITL monitoring × IM-Processes

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No tier-calibrated cadence in place)

Evidence Location: _________

Validation Date: _________

Notes: _______


Maturity Level 3

Objective: Daily attestation per Critical-tier workflow confirming HITL substantiveness, Art. 50 disclosure presence, decision-logging completeness, and override-audit freshness; drift opens an IM-Processes ticket automatically; contribute attestation schemas to ISO/IEC 42005 and sector AI governance bodies

At this level, every Critical-tier AI-embedded workflow produces a daily attestation signal across four operational dimensions. Drift auto-opens IM-Processes tickets. Per-archetype operational baseline schemas are contributed to ISO/IEC 42005 and sector AI governance bodies.


Question 7: Daily Attestation Signal for Workflows

Q3.1: Are ≥90% of Critical-tier AI-embedded workflows producing a daily attestation signal across all four dimensions (HITL gate health, decision-logging completeness, Art. 50 disclosure presence, override-audit freshness), with deviations auto-opening IM-Processes tickets within 1 hour?

Evidence Required: - [ ] Daily attestation pipeline operational covering four dimensions per Critical-tier workflow: (1) HITL gate health, HITL queue telemetry confirms reviewer SLA met, override-rationale completion at or above declared minimum, queue depth below saturation threshold; (2) decision-logging completeness, all required fields populated at expected volume; any field dropout opens a finding; (3) Art. 50 disclosure presence, product-flow analytics confirm disclosure element fires on ≥99.9% of AI-touched interactions across all traffic slices; any slice below threshold opens a finding within the hour; (4) override-audit freshness, override-audit log entries written at the rate implied by HITL queue throughput; schema drift or entry-rate drop opens a finding - [ ] Auto-open IM-Processes ticket on drift: ticket created within 1 hour of detection; carries drift dimension, specific signal that failed tolerance, and link to DR decision record - [ ] Attestation-pipeline health monitored: % Critical workflows with fresh signal in the last 24 hours; on-call paged for any Critical workflow silent for >24 hours; pager history on file - [ ] Zero Art. 50 disclosure-presence violations for Critical-tier workflows persisting beyond 1 hour: violation log on file; confirmed as zero in the last 30 days - [ ] Attestation artifacts regulator-consumable: machine-readable format confirmed suitable for EU AI Act Art. 9, Art. 26 deployer-duty documentation, and ISO/IEC 42001 AIMS operational records via test export - [ ] HITL-throughput thresholds calibrated to avoid alert fatigue: sensitivity tuning log on file; queue fluctuations (e.g., lunch-hour reviewer gaps) below a documented minimum meaningful threshold do not open findings

Outcome Metrics:

Metric Baseline Current Target Met? Notes
% Critical-tier workflows producing a daily attestation signal across all four dimensions ___% ___% ≥90% Attestation telemetry
% attestation findings auto-opening IM-Processes tickets within 1 hour of detection ___% ___% ≥95% IM-Processes integration telemetry
Art. 50 disclosure presence violations (disclosure absent from a traffic slice for >1 hour) ___ ___ 0 for Critical Attestation telemetry
IR reviewer-hours per Critical workflow per year ___ hrs ___ hrs trending down QoQ Reviewer time tracking

Metric Collection Guidance: - Daily attestation coverage: count Critical-tier workflows with a signed attestation artifact dated within the last 24 hours covering all four dimensions divided by total Critical-tier workflows. Source: attestation pipeline telemetry - Auto-open ticket latency: median time from drift-detection event to IM-Processes ticket created. Source: attestation pipeline timestamp × IM-Processes created-at timestamp - Art. 50 disclosure violations: count Critical-tier workflows with a disclosure-presence violation persisting beyond 1 hour in the last 30 days. Source: attestation pipeline Art. 50 dimension output - IR reviewer-hours: total hours logged for Critical-tier process IR activities per year. Source: reviewer time tracking

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No daily attestation signal in place)

Evidence Location: _________

Validation Date: _________

Notes: _______


Question 8: External Operational Baseline Schema Contribution

Q3.2: Has the program published per-archetype operational baseline schemas to ISO/IEC 42005 or sector AI governance bodies, with documented external adoption and internal practice aligned to the published versions?

Evidence Required: - [ ] Per-archetype operational baseline schemas published to at least one external body: ISO/IEC 42005 (AI impact assessment standard, operational monitoring criteria for AI-embedded workflows), sector AI governance bodies (financial-services AI supervisory guidance, healthcare AI governance, public-sector AI governance), or OWASP SAMM AI extensions - [ ] Internal practice aligned to published external versions: documented mapping between internal IR checklist and published schema; internal-only deviations proposed as upstream changes - [ ] External adoption tracked: citations, forks, direct acknowledgment from sector bodies or peer organizations, or inclusion in regulatory guidance, evidence on file - [ ] Schema publication pipeline active: at least one schema in-draft, in-review, or published at any time; publication status tracker maintained - [ ] Adoption trend documented: at least one adoption data point on file since initial publication - [ ] Internal schema version pinned to published external version: divergence documented as a proposed upstream change

Outcome Metrics:

Metric Baseline Current Target Met? Notes
% Critical-tier workflows producing a daily attestation signal across all four dimensions ___% ___% ≥90% Attestation telemetry
% attestation findings auto-opening IM-Processes tickets within 1 hour of detection ___% ___% ≥95% IM-Processes integration telemetry
External adoption of published operational baseline schemas 0 tracked trending up External telemetry
IR reviewer-hours per Critical workflow per year ___ hrs ___ hrs trending down QoQ Reviewer time tracking

Metric Collection Guidance: - External adoption: count citations, forks, and direct acknowledgments per quarter since publication. Source: external publication references, sector body acknowledgments, regulatory guidance citations - Schema alignment rate: % of internal IR checklist items directly traceable to a published external schema item. Source: internal-to-external schema mapping document - Publication pipeline health: number of schemas in active publication stages. Source: publication tracker

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No schemas published externally)

Evidence Location: _________

Validation Date: _________

Notes: _______


Question 9: Automated Drift-to-IM Escalation and Post-Incident Feedback Loop

Q3.3: Is the post-incident IR feedback loop operational, IM-Processes post-incident reviews include a mandatory IR-record re-examination step, and ≥1 attestation rule update is produced per material incident?

Evidence Required: - [ ] All IR findings (daily attestation and periodic reviews) flow into IM-Processes automatically with severity and SLA pre-populated from the SM-Processes L2 tier-treatment matrix; no manual entry required for attestation-generated findings - [ ] IM-Processes SLA clock automated: overdue Critical findings escalate to the program sponsor at 50% and 100% of the SLA window automatically; escalation automation tested quarterly with synthetic finding, test record on file - [ ] Post-incident IR feedback loop documented in the IM-Processes incident-review template: a mandatory IR-record re-examination step confirmed in the last material incident review record - [ ] Attestation rule updates produced from incident learning: at least one attestation rule update linked from a completed incident review in the last 12 months - [ ] Post-incident IR step completion tracked: % of material incidents in the last 12 months with a completed post-incident review including the IR-record re-examination step - [ ] Attestation-pipeline health log on file: ≥90% of Critical workflows with fresh signal in the last 24 hours; pager history on file

Outcome Metrics:

Metric Baseline Current Target Met? Notes
% Critical-tier workflows producing a daily attestation signal across all four dimensions ___% ___% ≥90% Attestation telemetry
% attestation findings auto-opening IM-Processes tickets within 1 hour of detection ___% ___% ≥95% IM-Processes integration telemetry
Art. 50 disclosure presence violations (disclosure absent from a traffic slice for >1 hour) ___ ___ 0 for Critical Attestation telemetry
IR reviewer-hours per Critical workflow per year ___ hrs ___ hrs trending down QoQ Reviewer time tracking

Metric Collection Guidance: - Auto-open ticket rate: count attestation-generated findings that created an IM-Processes ticket within 1 hour divided by total attestation-generated findings. Source: attestation pipeline × IM-Processes integration telemetry - Escalation automation test pass rate: % of quarterly escalation tests that confirmed correct automated escalation. Source: escalation test records - Post-incident IR step completion: % of material incident post-reviews with the IR-record re-examination step completed. Source: IM-Processes incident review records - Attestation rule update rate: count attestation rule updates linked from post-incident reviews in the last 12 months. Source: attestation pipeline rule-change log × IM-Processes incident reviews

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No automated escalation or post-incident feedback loop)

Evidence Location: _________

Validation Date: _________

Notes: _______


Summary Scorecard

Level Question Score Weight
L1 Q1: Per-Archetype Workflow IR Checklist ___ 33%
L1 Q2: Review Triggers and Workflow Drift Sources ___ 33%
L1 Q3: Findings Tracking and SR-Processes REM Loop ___ 33%
L1 Total ___
L2 Q4: Continuous Workflow Drift Detection ___ 33%
L2 Q5: HITL Substantiveness Probing and Rights-Response Verification ___ 33%
L2 Q6: Tier-Calibrated Processes IR Cadence ___ 33%
L2 Total ___
L3 Q7: Daily Attestation Signal for Workflows ___ 33%
L3 Q8: External Operational Baseline Schema Contribution ___ 33%
L3 Q9: Automated Escalation and Post-Incident Feedback ___ 33%
L3 Total ___
Overall IR-Processes Score ___

Current Maturity Level: ___

Key Findings:

Priority Improvements:


Document Version: HAIAMM v3.0 Practice: Implementation Review (IR) Domain: Processes Questionnaire Date: 2026-05-15 Author: Verifhai

Instructions:

  • Answer based on current practices, not plans
  • “Yes” requires documented evidence
  • Complete all Level 1 questions before Level 2
  • Partial implementation = “No”

↓ Download as Markdown