Assessment questionnaire for measuring maturity. Answer each question honestly based on current, implemented practices.
Practice: Implementation Review (IR) Domain: Processes Purpose: Assess organizational maturity in verifying that the actually-running AI-embedded business workflow matches the design approved at DR, and stays there as the workflow and its underlying AI components evolve. Scoring Model: Evidence + Outcome Metrics (see Scoring Methodology below)
| Score | Label | Criteria |
|---|---|---|
| 1.0 | Fully Mature | Evidence complete + ≥3 outcome metrics meet targets |
| 0.67 | Implemented | Evidence complete + 2 outcome metrics meet targets |
| 0.33 | Partial | Evidence partially complete + <2 metrics meet targets |
| 0.0 | Not Implemented | No evidence of practice |
Level Score = Average of the three question scores at that level Overall Score = Weighted average across all levels achieved
At this level, the gap between the approved workflow design and the running workflow is systematically checked at the moments that matter most. Every review produces findings with severity tags, named owners, and SLA-bound resolution dates.
Q1.1: Is there a published, per-archetype IR checklist, one per SM-Processes archetype (decision pipeline, customer-facing flow, human-AI collaboration chain, back-office augmentation, approval/review workflow, content-generation workflow, knowledge-management workflow), covering HITL gate substantiveness verification, Art. 50 disclosure presence check, decision-logging completeness, override-audit-trail queryability, affected-persons rights-response test, and fallback/kill-switch test?
Evidence Required: - [ ] Published checklist per SM-Processes archetype on file and linked from the SM-Processes inventory (decision pipeline, customer-facing flow, human-AI collaboration chain, back-office augmentation, approval/review workflow, content-generation workflow, knowledge-management workflow) - [ ] HITL gate substantiveness verified, not just a checkbox: test record on file confirming (a) gate fires for a declared trigger condition (synthetic trigger event, gate confirmed blocking progression), (b) reviewer SLA is met (HITL queue log sample with median review time within SLA), (c) override rationale is recorded for a stratified sample of reviewed items - [ ] Art. 50 disclosure presence verified via live UX sample-check (not staging environment): product-flow analytics confirm the disclosure element fires on every AI-touched interaction; screenshot audit of the deployed UI against the DR-approved disclosure design on file - [ ] Decision-logging completeness verified: a decision-log sample pulled and all required fields (decision identifier, timestamp, AI recommendation, human decision if HITL, reviewer identity, override flag, affected-person reference) confirmed present at expected rate - [ ] Override-audit-trail queryability tested: a sample query for a known override event executed and confirmed to return required fields within the declared SLA - [ ] Affected-persons rights-response test executed: a synthetic contestation request submitted; receipt, acknowledgment, and routing to review queue confirmed within the declared SLA; test record on file
Outcome Metrics:
| Metric | Baseline | Current | Target | Met? | Notes |
|---|---|---|---|---|---|
| % AI-embedded workflows with a go-live IR record | ___% | ___% | 100% | ☐ | SM-Processes inventory × IR records |
| % active AI-embedded workflows with a current-year IR record | ___% | ___% | ≥90% | ☐ | SM-Processes inventory × IR records |
| Critical / blocker findings open at go-live | ___ | ___ | 0 | ☐ | Findings backlog |
| Median closure time for High findings | ___ days | ___ days | ≤7 days | ☐ | Findings backlog |
Metric Collection Guidance:
- Go-live IR coverage: count workflows in SM-Processes inventory with a linked IR record dated at or before go-live divided by total workflows entering production. Formula: workflows_with_golive_IR / total_workflows_in_production × 100
- Current-year IR coverage: count workflows with an IR record dated within the last 12 months. Source: SM-Processes inventory last-IR-date field
- Blocker findings at go-live: count Critical/blocker findings open on the go-live date. Source: IM-Processes findings history
- High finding closure time: median calendar days from finding-opened to evidence-linked finding-closed for High-severity findings. Source: IM-Processes backlog timestamps
Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No per-archetype checklist published)
Evidence Location: _________
Validation Date: _________
Notes: _______
Q1.2: Do 100% of new AI-embedded workflows going live in the last 90 days carry a go-live IR record, with material-change triggers wired to SM-Processes inventory events, and are BPM-tool config drift sources reviewed at each IR?
Evidence Required: - [ ] Go-live IR records on file for all workflows entering production in the last 90 days; BPM tool config export and HITL queue log confirming the IR was completed before go-live - [ ] Material-change trigger wired to SM-Processes inventory: AI component swaps, HITL gate reconfigurations, new affected-person populations, Art. 50 disclosure changes, decision-logging scope reductions, contestation path changes, and class-shift monitor reconfigurations all generate a review-due alert within 5 business days - [ ] BPM-tool config drift sources reviewed since the last IR: Camunda BPMN file diff, Temporal workflow code diff, ServiceNow flow designer export, Argo workflow manifest diff reviewed and compared against the DR-approved design; deviations documented as findings - [ ] HITL queue logs reviewed: reviewer-throughput and override-rationale completion rate since the last IR reviewed and compared against declared targets - [ ] Product-flow analytics reviewed: where the AI step actually fires compared to the approved design, silent bypass confirmed absent; evidence: product-flow analytics export on file with IR record - [ ] Annual review calendar populated from the SM-Processes inventory with workflows nearing review-due date visible at least 30 days in advance
Outcome Metrics:
| Metric | Baseline | Current | Target | Met? | Notes |
|---|---|---|---|---|---|
| % AI-embedded workflows with a go-live IR record | ___% | ___% | 100% | ☐ | SM-Processes inventory × IR records |
| % active AI-embedded workflows with a current-year IR record | ___% | ___% | ≥90% | ☐ | SM-Processes inventory × IR records |
| % material changes to live workflows that trigger an IR before the change affects affected persons | ___% | ___% | 100% | ☐ | SM-Processes change events × IR records |
| Median closure time for High findings | ___ days | ___ days | ≤7 days | ☐ | Findings backlog |
Metric Collection Guidance: - Material-change IR trigger rate: cross-reference SM-Processes inventory material-change events against IR records created within 5 business days of each event - BPM config drift detection rate: % of IRs where BPM tool config was actively diffed against the DR-approved design (not just the design text reviewed). Source: IR evidence records method field - HITL queue log review coverage: % of IRs where a HITL queue log sample was pulled and reviewer-throughput analyzed. Source: IR evidence records - Product-flow analytics review rate: % of IRs where product-flow analytics confirmed AI step firing location vs. approved design. Source: IR evidence records
Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No material-change trigger or BPM drift source review)
Evidence Location: _________
Validation Date: _________
Notes: _______
Q1.3: Are findings severity-tagged and tracked in IM-Processes with named owners and SLA-bound closure dates, and does every IR finding that reveals stale or inaccurate REM evidence trigger an SR-Processes REM row update before the finding is closed?
Evidence Required: - [ ] Findings backlog in IM-Processes showing all IR findings with severity tag (Critical / High / Medium / Low), named owner (not "the business team"), SLA-bound closure date, and linked after-fix evidence artifact (screenshot, config export, queue-log sample) - [ ] SR-Processes REM update loop active: IR findings that reveal stale or inaccurate REM evidence have a linked REM row update on file before the finding is closed - [ ] Severity calibration consistent: Critical findings include HITL gate absent or trivially bypassed on a High/Critical-tier workflow, Art. 50 disclosure absent on a customer-facing flow, and decision log not capturing required fields for an Annex III workflow, not downgraded to Medium - [ ] Override-audit logs reviewed since the last IR: schema (required fields present) and completeness (entries present at expected rate given workflow volume) verified and documented - [ ] Decision-distribution snapshot reviewed: a recent approval-rate or decision-rate distribution pulled and compared to the DR-approved monitoring baseline; drift documented as a finding if beyond declared threshold - [ ] Findings-aging dashboard reviewed at least monthly by the program sponsor: meeting record or dashboard screenshot on file
Outcome Metrics:
| Metric | Baseline | Current | Target | Met? | Notes |
|---|---|---|---|---|---|
| % AI-embedded workflows with a go-live IR record | ___% | ___% | 100% | ☐ | SM-Processes inventory × IR records |
| % active AI-embedded workflows with a current-year IR record | ___% | ___% | ≥90% | ☐ | SM-Processes inventory × IR records |
| Critical / blocker findings open at go-live | ___ | ___ | 0 | ☐ | Findings backlog |
| Median closure time for High findings | ___ days | ___ days | ≤7 days | ☐ | Findings backlog |
Metric Collection Guidance: - REM update loop rate: count IR findings where the closure is linked to an SR-Processes REM row update vs. total IR findings that identified stale REM evidence. Source: IM-Processes × SR-Processes REM cross-reference - Named-owner coverage: count findings with a named individual as owner divided by total open findings. Source: IM-Processes - SLA adherence by severity: % of High findings closed within 7 days; % of Critical findings closed before go-live. Source: IM-Processes timestamps - Findings-aging review cadence: confirm monthly review via calendar record or dashboard export. Source: program sponsor review log
Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No findings backlog or severity tagging in place)
Evidence Location: _________
Validation Date: _________
Notes: _______
At this level, implementation review becomes a continuous signal. BPM-tool change webhooks, HITL-throughput monitoring, decision-distribution monitoring, and Art. 50 disclosure presence monitoring are all wired to automated detection.
Q2.1: Are ≥90% of Critical-tier AI-embedded workflows under continuous drift detection, via BPM-tool change events, HITL-throughput monitoring, decision-distribution monitoring, and Art. 50 disclosure presence monitoring, with median detection latency ≤7 days?
Evidence Required: - [ ] BPM-tool change event webhooks active for Critical and High-tier workflows: Camunda / Temporal / Argo / ServiceNow emit events on BPM model version changes; any change to HITL gate configuration, routing rules, or AI-step parameters triggers an automated diff against the DR-approved baseline; material deviations auto-open IR findings - [ ] HITL-throughput monitoring active: queue telemetry monitored for reviewer SLA miss (median review time exceeds declared SLA, alert within 24 hours), queue saturation (depth exceeds capacity model threshold, alert within 4 hours), and override-rationale completion drop (rate drops below declared minimum, alert within 24 hours) - [ ] Decision-distribution monitoring active: decision-rate and approval-rate distribution by class compared against the DR-approved baseline on a rolling 30-day window; drift beyond the declared threshold opens an IR finding automatically - [ ] Art. 50 disclosure UI A/B-test verification active: product-flow analytics monitored for the disclosure element's presence rate across all traffic slices; any traffic slice where disclosure is absent or fires at <100% rate opens an IR finding within 24 hours; A/B-test framework changes wired to the change-event trigger - [ ] Affected-persons rights-response monitoring active: contestation request response timing measured recurrently (monthly for Critical-tier); median response time compared against declared SLA; breaches open IR findings - [ ] Drift-detection pipeline health monitored: % Critical workflows producing a fresh signal in the last 7 days; on-call alert configured for >48 hours of feed silence; alert test record on file
Outcome Metrics:
| Metric | Baseline | Current | Target | Met? | Notes |
|---|---|---|---|---|---|
| % Critical-tier workflows under continuous drift detection (BPM-tool change events, HITL-throughput, decision-distribution, Art. 50 disclosure presence) | ___% | ___% | ≥90% | ☐ | Drift-detection telemetry |
| Median drift detection latency, Critical-tier | ___ days | ___ days | ≤7 days | ☐ | IR telemetry |
| % Critical/High-tier workflows with HITL substantiveness probe on record (current period) | ___% | ___% | 100% | ☐ | IR records |
| Tier-cadence adherence (% of workflows reviewed on their published cadence) | ___% | ___% | ≥95% | ☐ | IR schedule × SM-Processes inventory |
Metric Collection Guidance: - Continuous drift detection coverage: count Critical-tier workflows with all four signal sources active (BPM-tool change events, HITL-throughput, decision-distribution, Art. 50 disclosure) divided by total Critical-tier workflows. Source: drift-detection pipeline configuration registry - Drift detection latency: median time from change-event timestamp to IR-finding-opened timestamp. Source: IR telemetry × IM-Processes - HITL substantiveness probe coverage: count Critical/High-tier workflows with a HITL substantiveness probe record in the current period divided by total Critical/High-tier workflows. Source: IR records - Tier-cadence adherence: count workflows reviewed within their published cadence window divided by total workflows. Source: IR schedule × SM-Processes inventory last-IR-date
Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No automated drift detection in place)
Evidence Location: _________
Validation Date: _________
Notes: _______
Q2.2: Are HITL substantiveness probes completed monthly for Critical-tier and quarterly for High-tier, including decision-variance audits (escalation if ≥98% match rate), and are affected-persons rights-response probes completed on the same cadence?
Evidence Required: - [ ] HITL substantiveness probe records on file for Critical and High-tier workflows: stratified random sample audit of HITL queue decisions; probe checks reviewer decision time (unusually short decisions flagged), override-rationale quality (empty/boilerplate/below minimum character count flagged), and decision variance (human decision matches AI recommendation ≥98% of the time in sample, escalated to program sponsor and Business owner with documentation) - [ ] Escalation records on file where the ≥98% match rate was detected: escalation to program sponsor and Business owner documented with rationale and resolution - [ ] Probe cadence log showing Critical-tier probes monthly and High-tier probes quarterly, not one-time; all probe runs date-stamped with method and reviewer - [ ] Affected-persons rights-response probe records on file: synthetic contestation request submitted monthly (Critical) and quarterly (High); response receipt, acknowledgment, routing to review queue, and explanation generation all timed and compared against declared SLA; failures open IR findings with severity matching the rights-impact level of the workflow - [ ] Probe calendar maintained: missed probes tracked as process-metric failures with root cause and remediation on file - [ ] IR findings from HITL and rights-response probes routed to IM-Processes automatically with severity and owner pre-populated
Outcome Metrics:
| Metric | Baseline | Current | Target | Met? | Notes |
|---|---|---|---|---|---|
| % Critical-tier workflows under continuous drift detection (BPM-tool change events, HITL-throughput, decision-distribution, Art. 50 disclosure presence) | ___% | ___% | ≥90% | ☐ | Drift-detection telemetry |
| Median drift detection latency, Critical-tier | ___ days | ___ days | ≤7 days | ☐ | IR telemetry |
| % Critical/High-tier workflows with HITL substantiveness probe on record (current period) | ___% | ___% | 100% | ☐ | IR records |
| % Critical/High-tier workflows with affected-persons rights-response probe on record (current period) | ___% | ___% | 100% | ☐ | IR records |
Metric Collection Guidance: - HITL substantiveness probe coverage: count Critical/High-tier workflows with a HITL substantiveness probe record in the current period divided by total Critical/High-tier workflows. Source: IR records - Decision variance escalation rate: count HITL audits where the ≥98% match rate was detected and escalated divided by total HITL audits. Source: probe records × escalation log - Rights-response probe coverage: count Critical/High-tier workflows with a rights-response probe record in the current period divided by total Critical/High-tier workflows. Source: IR records - Rights-response SLA breach rate: count probes where the response-time SLA was breached divided by total probes. Source: probe timing records
Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No HITL substantiveness or rights-response probing)
Evidence Location: _________
Validation Date: _________
Notes: _______
Q2.3: Is tier-cadence adherence ≥95%, Critical on semi-annual + continuous drift detection, High on annual, Medium on annual, Low on go-live + material change, with Critical-tier findings aged per the SM-Processes L2 tier-treatment matrix SLAs?
Evidence Required: - [ ] Tier-cadence policy documented and published: Critical, go-live + semi-annual + material-change-triggered + continuous drift detection; High, go-live + annual + material-change-triggered; Medium, go-live + annual; Low, go-live + re-review on material change - [ ] IR schedule in the SM-Processes inventory confirmed against policy: every workflow carries a last-IR-date and next-IR-due field; Critical-tier workflows with no IR in the last 180 days flagged for escalation to the program sponsor - [ ] IR backlog tier-aware: Critical-tier findings reported separately; no Critical-tier finding waiting behind Low-tier queue items; tiered backlog dashboard or report on file - [ ] Drift findings from automated detection auto-open IM-Processes tickets, not dead-ending in an alert dashboard; confirmed via a sample of auto-opened findings from the last 90 days - [ ] HITL-throughput monitoring alerts treated as IR findings and opened in IM-Processes (not dismissed as operations noise): evidence of at least one HITL-throughput alert that generated an IM-Processes finding in the last 12 months - [ ] Tier-SLA breach rate tracked: count Critical-tier findings exceeding the SM-Processes L2 tier-treatment matrix SLA divided by total Critical-tier findings; rate reported to program sponsor monthly
Outcome Metrics:
| Metric | Baseline | Current | Target | Met? | Notes |
|---|---|---|---|---|---|
| % Critical-tier workflows under continuous drift detection (BPM-tool change events, HITL-throughput, decision-distribution, Art. 50 disclosure presence) | ___% | ___% | ≥90% | ☐ | Drift-detection telemetry |
| Median drift detection latency, Critical-tier | ___ days | ___ days | ≤7 days | ☐ | IR telemetry |
| % Critical/High-tier workflows with HITL substantiveness probe on record (current period) | ___% | ___% | 100% | ☐ | IR records |
| Tier-cadence adherence (% of workflows reviewed on their published cadence) | ___% | ___% | ≥95% | ☐ | IR schedule × SM-Processes inventory |
Metric Collection Guidance: - Tier-cadence adherence: count workflows reviewed within their published cadence window divided by total workflows. Source: SM-Processes inventory last-IR-date × cadence policy - Critical-tier SLA breach rate: count Critical-tier findings exceeding the SM-Processes L2 tier-treatment matrix SLA divided by total Critical-tier findings. Source: IM-Processes timestamps - Auto-open ticket rate: count drift findings that auto-opened an IM-Processes ticket divided by total drift findings. Source: drift-detection telemetry × IM-Processes - HITL alert-to-finding conversion rate: count HITL-throughput alerts that generated an IM-Processes finding divided by total HITL-throughput alerts. Source: HITL monitoring × IM-Processes
Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No tier-calibrated cadence in place)
Evidence Location: _________
Validation Date: _________
Notes: _______
At this level, every Critical-tier AI-embedded workflow produces a daily attestation signal across four operational dimensions. Drift auto-opens IM-Processes tickets. Per-archetype operational baseline schemas are contributed to ISO/IEC 42005 and sector AI governance bodies.
Q3.1: Are ≥90% of Critical-tier AI-embedded workflows producing a daily attestation signal across all four dimensions (HITL gate health, decision-logging completeness, Art. 50 disclosure presence, override-audit freshness), with deviations auto-opening IM-Processes tickets within 1 hour?
Evidence Required: - [ ] Daily attestation pipeline operational covering four dimensions per Critical-tier workflow: (1) HITL gate health, HITL queue telemetry confirms reviewer SLA met, override-rationale completion at or above declared minimum, queue depth below saturation threshold; (2) decision-logging completeness, all required fields populated at expected volume; any field dropout opens a finding; (3) Art. 50 disclosure presence, product-flow analytics confirm disclosure element fires on ≥99.9% of AI-touched interactions across all traffic slices; any slice below threshold opens a finding within the hour; (4) override-audit freshness, override-audit log entries written at the rate implied by HITL queue throughput; schema drift or entry-rate drop opens a finding - [ ] Auto-open IM-Processes ticket on drift: ticket created within 1 hour of detection; carries drift dimension, specific signal that failed tolerance, and link to DR decision record - [ ] Attestation-pipeline health monitored: % Critical workflows with fresh signal in the last 24 hours; on-call paged for any Critical workflow silent for >24 hours; pager history on file - [ ] Zero Art. 50 disclosure-presence violations for Critical-tier workflows persisting beyond 1 hour: violation log on file; confirmed as zero in the last 30 days - [ ] Attestation artifacts regulator-consumable: machine-readable format confirmed suitable for EU AI Act Art. 9, Art. 26 deployer-duty documentation, and ISO/IEC 42001 AIMS operational records via test export - [ ] HITL-throughput thresholds calibrated to avoid alert fatigue: sensitivity tuning log on file; queue fluctuations (e.g., lunch-hour reviewer gaps) below a documented minimum meaningful threshold do not open findings
Outcome Metrics:
| Metric | Baseline | Current | Target | Met? | Notes |
|---|---|---|---|---|---|
| % Critical-tier workflows producing a daily attestation signal across all four dimensions | ___% | ___% | ≥90% | ☐ | Attestation telemetry |
| % attestation findings auto-opening IM-Processes tickets within 1 hour of detection | ___% | ___% | ≥95% | ☐ | IM-Processes integration telemetry |
| Art. 50 disclosure presence violations (disclosure absent from a traffic slice for >1 hour) | ___ | ___ | 0 for Critical | ☐ | Attestation telemetry |
| IR reviewer-hours per Critical workflow per year | ___ hrs | ___ hrs | trending down QoQ | ☐ | Reviewer time tracking |
Metric Collection Guidance: - Daily attestation coverage: count Critical-tier workflows with a signed attestation artifact dated within the last 24 hours covering all four dimensions divided by total Critical-tier workflows. Source: attestation pipeline telemetry - Auto-open ticket latency: median time from drift-detection event to IM-Processes ticket created. Source: attestation pipeline timestamp × IM-Processes created-at timestamp - Art. 50 disclosure violations: count Critical-tier workflows with a disclosure-presence violation persisting beyond 1 hour in the last 30 days. Source: attestation pipeline Art. 50 dimension output - IR reviewer-hours: total hours logged for Critical-tier process IR activities per year. Source: reviewer time tracking
Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No daily attestation signal in place)
Evidence Location: _________
Validation Date: _________
Notes: _______
Q3.2: Has the program published per-archetype operational baseline schemas to ISO/IEC 42005 or sector AI governance bodies, with documented external adoption and internal practice aligned to the published versions?
Evidence Required: - [ ] Per-archetype operational baseline schemas published to at least one external body: ISO/IEC 42005 (AI impact assessment standard, operational monitoring criteria for AI-embedded workflows), sector AI governance bodies (financial-services AI supervisory guidance, healthcare AI governance, public-sector AI governance), or OWASP SAMM AI extensions - [ ] Internal practice aligned to published external versions: documented mapping between internal IR checklist and published schema; internal-only deviations proposed as upstream changes - [ ] External adoption tracked: citations, forks, direct acknowledgment from sector bodies or peer organizations, or inclusion in regulatory guidance, evidence on file - [ ] Schema publication pipeline active: at least one schema in-draft, in-review, or published at any time; publication status tracker maintained - [ ] Adoption trend documented: at least one adoption data point on file since initial publication - [ ] Internal schema version pinned to published external version: divergence documented as a proposed upstream change
Outcome Metrics:
| Metric | Baseline | Current | Target | Met? | Notes |
|---|---|---|---|---|---|
| % Critical-tier workflows producing a daily attestation signal across all four dimensions | ___% | ___% | ≥90% | ☐ | Attestation telemetry |
| % attestation findings auto-opening IM-Processes tickets within 1 hour of detection | ___% | ___% | ≥95% | ☐ | IM-Processes integration telemetry |
| External adoption of published operational baseline schemas | 0 | tracked | trending up | ☐ | External telemetry |
| IR reviewer-hours per Critical workflow per year | ___ hrs | ___ hrs | trending down QoQ | ☐ | Reviewer time tracking |
Metric Collection Guidance: - External adoption: count citations, forks, and direct acknowledgments per quarter since publication. Source: external publication references, sector body acknowledgments, regulatory guidance citations - Schema alignment rate: % of internal IR checklist items directly traceable to a published external schema item. Source: internal-to-external schema mapping document - Publication pipeline health: number of schemas in active publication stages. Source: publication tracker
Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No schemas published externally)
Evidence Location: _________
Validation Date: _________
Notes: _______
Q3.3: Is the post-incident IR feedback loop operational, IM-Processes post-incident reviews include a mandatory IR-record re-examination step, and ≥1 attestation rule update is produced per material incident?
Evidence Required: - [ ] All IR findings (daily attestation and periodic reviews) flow into IM-Processes automatically with severity and SLA pre-populated from the SM-Processes L2 tier-treatment matrix; no manual entry required for attestation-generated findings - [ ] IM-Processes SLA clock automated: overdue Critical findings escalate to the program sponsor at 50% and 100% of the SLA window automatically; escalation automation tested quarterly with synthetic finding, test record on file - [ ] Post-incident IR feedback loop documented in the IM-Processes incident-review template: a mandatory IR-record re-examination step confirmed in the last material incident review record - [ ] Attestation rule updates produced from incident learning: at least one attestation rule update linked from a completed incident review in the last 12 months - [ ] Post-incident IR step completion tracked: % of material incidents in the last 12 months with a completed post-incident review including the IR-record re-examination step - [ ] Attestation-pipeline health log on file: ≥90% of Critical workflows with fresh signal in the last 24 hours; pager history on file
Outcome Metrics:
| Metric | Baseline | Current | Target | Met? | Notes |
|---|---|---|---|---|---|
| % Critical-tier workflows producing a daily attestation signal across all four dimensions | ___% | ___% | ≥90% | ☐ | Attestation telemetry |
| % attestation findings auto-opening IM-Processes tickets within 1 hour of detection | ___% | ___% | ≥95% | ☐ | IM-Processes integration telemetry |
| Art. 50 disclosure presence violations (disclosure absent from a traffic slice for >1 hour) | ___ | ___ | 0 for Critical | ☐ | Attestation telemetry |
| IR reviewer-hours per Critical workflow per year | ___ hrs | ___ hrs | trending down QoQ | ☐ | Reviewer time tracking |
Metric Collection Guidance: - Auto-open ticket rate: count attestation-generated findings that created an IM-Processes ticket within 1 hour divided by total attestation-generated findings. Source: attestation pipeline × IM-Processes integration telemetry - Escalation automation test pass rate: % of quarterly escalation tests that confirmed correct automated escalation. Source: escalation test records - Post-incident IR step completion: % of material incident post-reviews with the IR-record re-examination step completed. Source: IM-Processes incident review records - Attestation rule update rate: count attestation rule updates linked from post-incident reviews in the last 12 months. Source: attestation pipeline rule-change log × IM-Processes incident reviews
Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No automated escalation or post-incident feedback loop)
Evidence Location: _________
Validation Date: _________
Notes: _______
| Level | Question | Score | Weight |
|---|---|---|---|
| L1 | Q1: Per-Archetype Workflow IR Checklist | ___ | 33% |
| L1 | Q2: Review Triggers and Workflow Drift Sources | ___ | 33% |
| L1 | Q3: Findings Tracking and SR-Processes REM Loop | ___ | 33% |
| L1 Total | ___ | ||
| L2 | Q4: Continuous Workflow Drift Detection | ___ | 33% |
| L2 | Q5: HITL Substantiveness Probing and Rights-Response Verification | ___ | 33% |
| L2 | Q6: Tier-Calibrated Processes IR Cadence | ___ | 33% |
| L2 Total | ___ | ||
| L3 | Q7: Daily Attestation Signal for Workflows | ___ | 33% |
| L3 | Q8: External Operational Baseline Schema Contribution | ___ | 33% |
| L3 | Q9: Automated Escalation and Post-Incident Feedback | ___ | 33% |
| L3 Total | ___ | ||
| Overall IR-Processes Score | ___ |
Current Maturity Level: ___
Key Findings:
Priority Improvements:
Document Version: HAIAMM v3.0 Practice: Implementation Review (IR) Domain: Processes Questionnaire Date: 2026-05-15 Author: Verifhai
Instructions: