Issue Management (IM) - Software Assessment

Assessment questionnaire for measuring maturity. Answer each question honestly based on current, implemented practices.

v3.0 rewrite: The canonical framing for the Software domain is Issue Management, unified backlog, tier-calibrated incident playbook, and regulatory SLA tracking for AI/HAI software issues. The fully v3.0 source-of-truth is ../practices/IM-Software-OnePager.md. Canonical subject and through-lines: ../HAIAMM-v3.0-Framing.md. Primary tactic: MITRE ATLAS TA0014 Impact.

Issue Management (IM) - Software Domain

HAIAMM Assessment Questionnaire v3.0

Practice: Issue Management (IM) Domain: Software Purpose: Assess organizational maturity in operating a unified AI/HAI software issue backlog, AI-specific incident playbook, and regulatory SLA tracker Scoring Model: Evidence + Outcome Metrics (see Scoring Methodology below)

Instructions

Answer each question honestly based on current, implemented practices (not plans or aspirations)
Each question has two components: Evidence (what you did) and Outcome Metrics (how well it worked)
Scoring uses 4 tiers: Fully Mature (1.0), Implemented (0.67), Partial (0.33), Not Implemented (0.0)
Answer progressively - Complete all Level 1 questions before Level 2
Level progression - Achieve ALL questions at lower level before advancing
Baseline first - Record current metric values before setting targets

Scoring Methodology

Each question is scored on a 4-tier scale:

Score	Label	Criteria
1.0	Fully Mature	Evidence complete + ≥3 outcome metrics meet targets
0.67	Implemented	Evidence complete + 2 outcome metrics meet targets
0.33	Partial	Evidence partially complete + <2 metrics meet targets
0.0	Not Implemented	No evidence of the practice

Level Score = Average of question scores within the level Overall IM-Software Score = Weighted average: L1 × 0.5 + L2 × 0.3 + L3 × 0.2

Maturity Level 1

Objective: Operate a single unified AI/HAI software issue backlog with a standard triage rubric, AI-specific incident playbook including containment plays for the primary HAI incident classes, and regulatory SLA tracking for GDPR Art. 33, EU AI Act Art. 73, HIPAA, NYDFS Part 500, and sector-specific obligations

Question 1: Unified AI/HAI Software Issue Backlog and Triage Rubric

Q1.1: Is there a single AI/HAI software issue backlog with standardized metadata (source, affected artifact linked to SM-Software inventory, severity rubric anchored to AI-specific axes, active exfiltration / agent damage / kill-switch failure / regulated-data breach for Critical; confirmed control failure with potential impact for High, etc., owner, SLA, regulatory flag, evidence link) capturing ≥95% of issues from all source practices (TA, SR, DR, IR, ST, ML, external)?

Evidence Required: - [ ] Single backlog record showing standardized metadata fields: Source (TA/SR/DR/IR/ST/ML/External), Affected artifact linked to SM-Software inventory with archetype and tier, Severity per AI-specific rubric, Owner, SLA, Regulatory flag, Evidence link - [ ] Triage rubric document with AI-specific severity anchors: Critical (active exfiltration, agent executing tool invocations with real-world damage, kill-switch failure in production, GDPR Art. 33 personal data breach), High (confirmed control failure with potential impact), Medium (non-production or compensated gap), Low (informational) - [ ] Backlog coverage audit showing ≥95% of AI/HAI software issues from all practices vs. reconciliation count from practice queues - [ ] Triage cadence records: daily Critical/High review, weekly Medium, monthly aging, meeting notes or ticket timestamps confirming cadence - [ ] Post-triage aging dashboard or report showing open issues by SLA bucket, refreshed at least monthly for program sponsor - [ ] SLA targets published: Critical acknowledge ≤4h / contain ≤48h / root-cause ≤30d; High ≤24h/7d/45d; Medium ≤48h/14d; Low ≤5bd/30d

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | % of AI/HAI software issues in single backlog vs. scattered queues | % | % | ≥95% | ☐ | | | % of issues with complete standardized metadata (all required fields populated) | % | % | ≥95% | ☐ | | | % of Critical/High issues acknowledged within SLA | % | % | 100% | ☐ | | | Monthly aging report delivered to program sponsor on schedule | ___ / 12 | ___ / 12 | 12 / 12 | ☐ | |

Metric Collection Guidance: - Backlog coverage: Monthly reconciliation, count issues in single backlog vs. sum of issues filed in TA, SR, DR, IR, ST, ML, and external advisory feeds. Formula: backlog_issues / total_issues_filed × 100 - Metadata completeness: Spot-audit 20 random tickets per month; count tickets where all 7 metadata fields are populated. Source: backlog export - SLA acknowledgement: issues_acknowledged_within_SLA / total_new_Critical_High_issues × 100. Source: backlog timestamp fields. Measured weekly - Aging report cadence: Count sponsor-facing aging reports delivered per year. Source: program reporting calendar

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence of unified backlog)

Evidence Location: _________

Metric Validation Date: _________

Notes: _______

Question 2: AI-Specific Incident Playbook, Software Containment Plays

Q1.2: Is the AI/HAI software incident playbook published with six named AI-specific incident classes (prompt-injection containment, agent rogue-action / kill-switch + tool-revoke, training-data-leakage / model rollback + eval-replay, silent model-family-swap rollback, RAG-poisoning containment, shadow-AI emergence), each with pre-assigned roles, containment plays, evidence-capture steps, and SLA targets, and has each class been exercised in at least one tabletop in the last 12 months?

Evidence Required: - [ ] Playbook document with six named AI-specific incident entries published and version-controlled: (1) prompt-injection containment, (2) agent rogue-action with kill-switch and tool-revoke steps, (3) training-data-leakage with model rollback and eval-replay, (4) silent model-family-swap rollback with model-version pinning, (5) RAG retrieval-source poisoning containment with source-disable steps, (6) shadow-AI emergence with egress-block steps - [ ] Each entry contains: trigger conditions, named roles (deployer-duty owner, AppSec on-call, Privacy/Legal, executive sponsor), step-by-step containment, artifacts to collect, evidence-capture instructions for deployer-duty record, closure criteria, SLA targets - [ ] Tabletop exercise records (agenda, attendees, scenario used, findings, follow-up actions) for each of the six playbook classes within the last 12 months, rotate one class per quarterly exercise minimum - [ ] Kill-switch function tested and documented: invocation path, scope, rollback confirmation procedure - [ ] Tool-revoke configuration records: agent tool allowlist management procedure with revoke path documented - [ ] Model rollback procedure documented with eval-harness replay steps and pass/fail criteria

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | % of AI/HAI software incidents handled on a published playbook entry | % | % | 100% | ☐ | | | Tabletop exercises completed per year across all six playbook classes | ___ | ___ | ≥4 (rotating) | ☐ | | | Kill-switch invocation test success rate (last 4 quarterly tests) | % | % | 100% | ☐ | | | Mean time from agent rogue-action detection to kill-switch invocation (drill) | ___ min | ___ min | ≤15 min | ☐ | |

Metric Collection Guidance: - Playbook coverage: incidents_handled_on_named_playbook_entry / total_incidents × 100. Source: incident records. Reviewed after each incident - Tabletop cadence: Count completed tabletop exercises per year; verify each covers one of the six named classes. Source: tabletop exercise log - Kill-switch test success: Count quarterly kill-switch function tests with pass result / total quarterly tests. Source: test event records - MTTA drill: Measure time from detection-simulation trigger to confirmed kill-switch invocation in tabletop scenarios. Source: tabletop event logs

Evidence Location: _________

Metric Validation Date: _________

Notes: _______

Question 3: Regulatory SLA Tracker and Post-Incident Review Loop

Q1.3: Is the regulatory SLA tracker live covering GDPR Art. 33 (72h), EU AI Act Art. 73, HIPAA (60d), NYDFS Part 500 (72h), and sector-specific obligations, with 100% adherence in the last 90 days, and does every Critical/blocker incident produce a post-incident review within 14 days with named update outputs flowing to SA, SR, EG, and ML?

Evidence Required: - [ ] Regulatory SLA tracker record showing: obligation name, regulatory citation, clock-start condition, notification deadline, named owner (Privacy/Legal for GDPR/EU AI Act/HIPAA; CISO/Privacy for NYDFS), current status for each active or recent incident - [ ] GDPR Art. 33 (72h) clock-start documented for each qualifying incident: first ML detection or IR finding constituting awareness is the clock start; daily status updates recorded until notification filed or clock expires - [ ] EU AI Act Art. 73 escalation path documented: Annex III artifact incident triggers immediate escalation to Privacy/Legal and executive sponsor; implementing-act timeline tracked - [ ] HIPAA 60-day breach notification window tracked for any AI/HAI software incident involving PHI; NYDFS Part 500 72-hour window tracked for material cybersecurity events - [ ] Post-incident review records for all Critical/blocker incidents within last 12 months: root cause, what caught it, what did not catch it, four update outputs (SA pattern update, SR requirements update, EG training update, ML detection update) all populated - [ ] Update outputs tracked as IM-Software improvement issues in the backlog, aging against same cadence as other issues; closed when downstream practice confirms resolution

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | Regulatory SLA adherence (0 missed notifications) in last 90 days | ___ | 0 missed | 0 missed | ☐ | | | Post-incident reviews completed within 14 days of Critical/blocker closure | % | % | 100% | ☐ | | | SA/SR/EG/ML update outputs from Critical reviews tracked and resolved | % | % | 100% produce ≥1 per target | ☐ | | | Median closure time for Critical AI/HAI software incidents (root-cause) | ___ days | ___ days | ≤30 days | ☐ | |

Metric Collection Guidance: - Regulatory SLA adherence: Count of qualifying incidents where regulatory notification was filed or confirmed not-required within the applicable window (GDPR 72h, HIPAA 60d, NYDFS 72h). Source: SLA tracker. Zero missed = 100% adherence - Post-incident review timeliness: reviews_completed_within_14d / total_Critical_blocker_closures × 100. Source: review records with timestamps - Update output completion: For each Critical review, verify SA/SR/EG/ML update rows exist and are tracked. Formula: reviews_with_all_4_outputs / total_Critical_reviews × 100 - MTTR: Median days from incident creation date to root-cause record filed. Source: backlog aging data. Calculated monthly

Evidence Location: _________

Metric Validation Date: _________

Notes: _______

Maturity Level 2

Objective: Calibrate incident response depth per SM-Software L2 risk tier; establish dedicated on-call rotation for Critical-tier artifacts; and automate cross-domain signal flow so that Software domain incidents affecting Vendors or Infrastructure generate coordinated response

Question 4: Tier-Calibrated Incident Playbook and 24/7 On-Call

Q2.1: Is a tier-calibrated incident playbook operational with Critical-tier MTTA ≤1 hour and MTTC ≤4 hours, 24/7 on-call coverage with a documented rotation including a current Critical-tier artifact briefing, and tier-movement in the SM-Software inventory automatically triggering IM configuration updates within 14 days (Critical re-tier)?

Evidence Required: - [ ] Tier-calibrated activation criteria documented per tier: Critical (CISO + Privacy/Legal + deployer-duty + executive sponsor, ≤1h MTTA, ≤4h MTTC, 24/7), High (AppSec + Privacy/Legal if regulated data, ≤4h MTTA, ≤24h MTTC, business hours + escalation path), Medium (standard queue, ≤1bd), Low (weekly aggregated handling) - [ ] 24/7 on-call rotation registry: named individuals per week, coverage handoff protocol, confirmed no-gap periods in last 90 days - [ ] On-call briefing document (current version): includes the current Critical-tier artifact list, their active detection set, kill-switch invocation path per artifact, playbook class mapping - [ ] Tier-movement trigger procedure: SM-Software inventory tier-change event triggers IM configuration update (on-call path, playbook variant, SLA targets) within 14 days for Critical re-tier; evidence of last 3 tier-movement updates - [ ] Pre-staged communication templates reviewed quarterly: internal escalation, customer-facing (if applicable), regulatory notification draft

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | Critical-tier MTTA (mean time to acknowledge) | ___ hr | ___ hr | ≤1 hour | ☐ | | | Critical-tier MTTC (mean time to contain) | ___ hr | ___ hr | ≤4 hours | ☐ | | | 24/7 on-call coverage: no-gap periods in last 90 days | ___ gaps | ___ gaps | 0 gaps | ☐ | | | Tier-movement → IM config update within 14 days (Critical re-tier) | % | % | 100% | ☐ | |

Metric Collection Guidance: - MTTA: Average time from first ML detection or backlog creation to acknowledged (named responder assigned) for Critical-tier incidents. Source: IM telemetry / backlog timestamps. Measured per incident - MTTC: Average time from acknowledgement to confirmed containment action complete for Critical-tier incidents. Source: IM telemetry. Measured per incident - On-call coverage gaps: Count weeks in last 90 days where on-call rotation had an unassigned period. Source: on-call rotation registry. Reviewed monthly - Tier-movement update latency: For each Critical re-tier event in SM-Software inventory, measure days until IM configuration updated. Target: 100% within 14 days. Source: SM-Software tier-change log × IM configuration update log

Evidence Location: _________

Metric Validation Date: _________

Notes: _______

Question 5: Post-Incident Review Auto-Flow Integration

Q2.2: Is a post-incident review auto-flow integration live routing Critical-tier review outputs to SA/SR/EG/ML practice backlogs, with ≥90% of downstream practice owners responding within 14 days and the sponsor reviewing output quality quarterly to distinguish substantive changes from nominal acknowledgements?

Evidence Required: - [ ] Integration configuration record: auto-ticket creation for each of four downstream practices on Critical-tier review closure, SA-Software (architecture-backlog ticket with incident reference), SR-Software (pack-backlog ticket with requirements-pack version and failing row), EG-Software (training-backlog ticket with affected population segment), ML-Software (detection-registry ticket with detection name, current query, proposed change) - [ ] Sample auto-created tickets from last 3 Critical-tier reviews showing correct metadata population and incident reference linking - [ ] Downstream practice backlog aging data: timestamps of auto-ticket creation vs. downstream owner response (accepted / rejected) for last 12 months; ≥90% responding within 14 days - [ ] Quarterly sponsor review records: review notes confirming sponsor examined post-incident update outputs for quality (substantive vs. nominal), at least one sample per quarter showing a flagged nominal output and remediation action - [ ] SLA enforcement record: accepted Critical-tier update outputs treated as High-severity in receiving practice backlog, evidence of High-severity designation in downstream tickets

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | Post-incident review outputs auto-flowing to SA/SR/EG/ML (% of Critical reviews) | % | % | 100% | ☐ | | | Downstream practice owner response to update outputs within 14 days | % | % | ≥90% | ☐ | | | Quarterly sponsor review of output quality completed on schedule | ___ / 4 | ___ / 4 | 4 / 4 | ☐ | | | Substantive (non-nominal) update output rate (sponsor-assessed) | % | % | ≥80% | ☐ | |

Metric Collection Guidance: - Auto-flow rate: Critical_reviews_with_all_4_auto_tickets_created / total_Critical_reviews × 100. Source: integration telemetry. Measured per review - Downstream response rate: update_tickets_responded_within_14d / total_update_tickets_created × 100. Source: downstream backlog aging. Measured monthly - Sponsor review cadence: Count quarterly reviews completed with documented output-quality assessment. Source: program reporting calendar - Substantive rate: Sponsor rates each reviewed update output as substantive (concrete change) or nominal (acknowledgement only). Formula: substantive_count / total_reviewed × 100

Evidence Location: _________

Metric Validation Date: _________

Notes: _______

Question 6: Cross-Domain Coordination Protocol

Q2.3: Is a cross-domain coordination protocol published and used for 100% of multi-domain AI/HAI software incidents, with named cross-domain contacts for Vendors, Infrastructure, and Processes domains verified quarterly, a single IC from the primary impacted domain, and joint post-incident reviews spanning all affected domains?

Evidence Required: - [ ] Cross-domain coordination protocol document: activation criteria for Software → Vendors (silent model-family swap at inference provider triggers Vendors-domain IM alongside Software rollback play), Software → Infrastructure (model-serving misconfiguration exposes training pipeline triggers Infrastructure EH and IM), Software → Processes (agent unauthorized writes to business workflow triggers Processes-domain coordinator) - [ ] Named cross-domain contacts registry: Vendors-domain IM contact, Infrastructure-domain IM contact, Processes-domain IM contact, verified current within last 90 days - [ ] Quarterly contact verification records: named contacts confirmed reachable, communication channels tested (last 4 quarters) - [ ] Multi-domain incident activation records (if applicable): shared status board used, single IC designated from primary impacted domain, joint post-incident review completed spanning all affected domains - [ ] IC designation procedure: pre-defined IC assignment rules (primary impacted domain takes IC role; tiebreaker criteria for ambiguous incidents)

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | Cross-domain coordination protocol used for 100% of multi-domain incidents | % | % | 100% | ☐ | | | Cross-domain contact registry verified current within last 90 days | Yes/No | Yes/No | Yes | ☐ | | | Joint post-incident reviews completed for multi-domain incidents | % | % | 100% | ☐ | | | Quarterly contact verification completed on schedule | ___ / 4 | ___ / 4 | 4 / 4 | ☐ | |

Metric Collection Guidance: - Protocol usage: multi_domain_incidents_using_protocol / total_multi_domain_incidents × 100. Source: incident coordination records. Reviewed per incident - Contact currency: Date of last contact verification across all three domains. Target: within 90 days. Source: contact registry - Joint review completion: multi_domain_incidents_with_joint_PIR / total_multi_domain_incidents × 100. Source: post-incident review records - Verification cadence: Count quarterly contact verification events completed. Source: program calendar

Evidence Location: _________

Metric Validation Date: _________

Notes: _______

Maturity Level 3

Objective: Contribute incident patterns and playbook templates to ISACs, MITRE ATLAS TA0014 Impact, and AVID; automate runbook decisioning for low-severity, high-confidence detections; and benchmark MTTR against industry peers

Q3.1: Does the program contribute ≥4 anonymized AI incident-classification entries per year to sector ISACs, ≥2 entries per year to AVID, and ≥1 contribution per year to MITRE ATLAS TA0014 Impact tactic documentation, with all contributions maintained current, legally vetted, and tracked for external adoption?

Evidence Required: - [ ] ISAC participation records: sector ISAC membership confirmation (FS-ISAC AI WG, H-ISAC, IT-ISAC, or sector-specific); ISAC incident feed integration into IM-Software external-advisory source; ≥4 anonymized contribution records per year (incident type, ATLAS tactic tag, HAI-TTP tag, containment play used, MTTR achieved) - [ ] AVID submission records: ≥2 AI/HAI software vulnerability entries submitted per year; entry maintenance log showing entries updated when the vulnerability class evolves; legal sign-off for each submission - [ ] MITRE ATLAS TA0014 Impact contribution record: ≥1 technique observation or mitigation entry submitted per year for Impact-tactic techniques (AML.T0048–AML.T0053 range and successors); contribution tracking log showing submission status (draft/submitted/published) - [ ] Legal vetting records: legal or privacy review sign-off for each external contribution confirming anonymization and no regulated data leakage - [ ] External adoption tracking: log of any citations, adoptions, or references to the org's contributed artifacts by ISACs, AVID, ATLAS, or standards bodies

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | ISAC AI incident contributions per year | 0 | ___ | ≥4 | ☐ | | | AVID entries submitted per year | 0 | ___ | ≥2 | ☐ | | | ATLAS TA0014 Impact contributions per year | 0 | ___ | ≥1 | ☐ | | | External adoption events (citations/adoptions tracked) | 0 | ___ | ≥1 | ☐ | |

Metric Collection Guidance: - ISAC contributions: Count anonymized incident-classification submissions to sector ISACs per calendar year. Source: ISAC contribution log. Updated per submission - AVID entries: Count AVID database entries created or materially updated per calendar year. Source: AVID contribution log. Updated per submission - ATLAS contributions: Count ATLAS technique observation or mitigation entries submitted per calendar year. Source: ATLAS contribution log. Updated per submission - Adoption events: Count external citations or documented adoptions of the org's contributed artifacts. Source: contribution tracking log. Reviewed semi-annually

Evidence Location: _________

Metric Validation Date: _________

Notes: _______

Question 8: Pre-Authorized Automated Runbook Decisioning

Q3.2: Are ≥3 pre-authorized automated containment actions live (kill-switch, egress-block, retrieval-source disable, or tool-revoke classes), vetted by Privacy/Legal and the executive sponsor, producing 100% audit records plus human-review tickets on execution, with the pre-authorization policy reviewed quarterly and any unexpected outcome triggering an out-of-cycle review?

Evidence Required: - [ ] Pre-authorization policy document: published list of ≥3 pre-authorized automated actions with confidence thresholds (e.g., kill-switch for Low/Medium-tier agent on AGH detection ≥95%; egress-block for shadow-AI on non-Critical-tier service on first detection; retrieval-source disable for flagged source document on injection-defense detection; tool-revoke for auto-revocable tool category on tool-scope-violation detection) - [ ] Legal/Privacy and executive sponsor sign-off records for the pre-authorization policy, dated, versioned - [ ] Critical-tier handling documented: pre-authorized actions for Critical-tier require human confirmation within 15 minutes; timer-based fallback fires with executive notification if no confirmation received - [ ] Automation execution log samples: each automated action produces a full audit log entry in IM-Software backlog, a human-review ticket auto-created at execution time, and notification to deployer-duty owner, 5 sample execution records showing all three outputs - [ ] Quarterly pre-authorization policy review records (last 4 quarters): review notes, any threshold adjustments, any unexpected-outcome triggered out-of-cycle reviews

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | Pre-authorized automated containment actions operational | 0 | ___ | ≥3 defined, vetted, live | ☐ | | | % pre-authorized actions producing full audit record + human-review ticket | % | % | 100% | ☐ | | | Quarterly pre-authorization policy review completed on schedule | ___ / 4 | ___ / 4 | 4 / 4 | ☐ | | | Unexpected-outcome triggered out-of-cycle reviews completed within 5 business days | % | % | 100% | ☐ | |

Metric Collection Guidance: - Pre-authorized actions live: Count unique automated containment action types defined in the policy, vetted, and confirmed live in production (not just planned). Source: pre-authorization policy + automation deployment record - Audit completeness: automated_actions_with_audit_record_AND_review_ticket / total_automated_actions × 100. Source: automation telemetry. Measured per execution - Policy review cadence: Count quarterly reviews completed with documented outcomes. Source: policy review calendar - Out-of-cycle review timeliness: unexpected_outcome_reviews_within_5bd / total_unexpected_outcomes × 100. Source: out-of-cycle review log

Evidence Location: _________

Metric Validation Date: _________

Notes: _______

Question 9: MTTR Benchmarking Against Industry Peers

Q3.3: Is a quarterly MTTR benchmark brief published to the sponsor comparing the program's MTTR per incident class and per tier against ISAC-sourced and peer-sourced benchmarks, with Critical-tier MTTR at or below benchmark for ≥3 of 6 incident classes and deltas above benchmark linked to specific practice gaps and investment proposals?

Evidence Required: - [ ] MTTR benchmark data sources documented: sector ISAC AI incident data exchange records, BSIMM-style observational data, MITRE ATLAS practitioner community data, peer roundtable inputs, at least two external sources active and refreshed at least semi-annually - [ ] Quarterly MTTR benchmark brief (last 4 quarters): MTTR per incident class (prompt-injection, agent rogue-action, training-data-leakage, shadow-AI emergence, RAG poisoning, model-swap rollback) vs. benchmark; MTTR per tier (Critical/High/Medium) vs. benchmark; delta trend (improving/stable/degrading) - [ ] Investment-driver section in each brief: incidents classes where MTTR is above benchmark have root-cause mapped to a specific practice gap (missing detection, unclear playbook, on-call latency) with a budget-linked improvement proposal - [ ] Benchmark brief delivery records: distributed to program sponsor within schedule commitment (last 4 quarters on-time delivery) - [ ] Benchmark data source refresh log: external benchmark inputs updated at least semi-annually; stale benchmarks flagged in brief

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | MTTR benchmark brief published quarterly to sponsor | ___ / 4 | ___ / 4 | 4 / 4 | ☐ | | | MTTR per incident class vs. benchmark (Critical-tier, at or below for ≥3 of 6 classes) | ___ / 6 | ___ / 6 | ≥3 of 6 | ☐ | | | Above-benchmark MTTR classes with investment proposals linked | % | % | 100% | ☐ | | | Benchmark data source refresh within last 6 months | Yes/No | Yes/No | Yes | ☐ | |

Metric Collection Guidance: - Brief cadence: Count MTTR benchmark briefs delivered on schedule per year. Source: program reporting calendar. Target: 4 / 4 quarterly - Benchmark performance: For each of 6 incident classes, compare Critical-tier MTTR to the benchmark value. Count classes at or below benchmark. Source: MTTR benchmark brief. Measured quarterly - Investment proposals: above_benchmark_classes_with_investment_proposal / total_above_benchmark_classes × 100. Source: benchmark brief investment-driver section. Reviewed quarterly - Benchmark freshness: Date of most recent external data source update vs. today. Target: within 180 days. Source: benchmark data source refresh log

Evidence Location: _________

Metric Validation Date: _________

Notes: _______

Summary Scorecard

Question	Level	Activity	Score
Q1: Unified backlog and triage rubric	L1	A	☐ 1.0 ☐ 0.67 ☐ 0.33 ☐ 0.0
Q2: AI-specific incident playbook (6 software plays)	L1	B	☐ 1.0 ☐ 0.67 ☐ 0.33 ☐ 0.0
Q3: Regulatory SLA tracker and post-incident review loop	L1	C	☐ 1.0 ☐ 0.67 ☐ 0.33 ☐ 0.0
Q4: Tier-calibrated playbook and 24/7 on-call	L2	A	☐ 1.0 ☐ 0.67 ☐ 0.33 ☐ 0.0
Q5: Post-incident review auto-flow integration	L2	B	☐ 1.0 ☐ 0.67 ☐ 0.33 ☐ 0.0
Q6: Cross-domain coordination protocol	L2	C	☐ 1.0 ☐ 0.67 ☐ 0.33 ☐ 0.0
Q7: ISAC / AVID / ATLAS contributions	L3	A	☐ 1.0 ☐ 0.67 ☐ 0.33 ☐ 0.0
Q8: Pre-authorized automated runbook	L3	B	☐ 1.0 ☐ 0.67 ☐ 0.33 ☐ 0.0
Q9: MTTR benchmarking	L3	C	☐ 1.0 ☐ 0.67 ☐ 0.33 ☐ 0.0

Level 1 Score (avg Q1–Q3): _____ / 1.0

Level 2 Score (avg Q4–Q6): _____ / 1.0

Level 3 Score (avg Q7–Q9): _____ / 1.0

Overall IM-Software Score (L1×0.5 + L2×0.3 + L3×0.2): _____ / 1.0

Assessment Date: _________

Assessor: _________

Next Assessment Due: _________

Document Version: HAIAMM v3.0, 2026-05-15, Verifhai Practice: Issue Management (IM) Domain: Software Source of Truth: docs/practices/IM-Software-OnePager.md

Instructions:

Answer based on current practices, not plans
“Yes” requires documented evidence
Complete all Level 1 questions before Level 2
Partial implementation = “No”

↓ Download as Markdown

Issue Management (IM) - Software Assessment

Issue Management (IM) - Software Domain

HAIAMM Assessment Questionnaire v3.0

Instructions

Scoring Methodology

Maturity Level 1

Question 1: Unified AI/HAI Software Issue Backlog and Triage Rubric

Question 2: AI-Specific Incident Playbook, Software Containment Plays

Question 3: Regulatory SLA Tracker and Post-Incident Review Loop

Maturity Level 2

Question 4: Tier-Calibrated Incident Playbook and 24/7 On-Call

Question 5: Post-Incident Review Auto-Flow Integration

Question 6: Cross-Domain Coordination Protocol

Maturity Level 3

Question 7: Industry-Coordinated Incident Sharing and Contribution

Question 8: Pre-Authorized Automated Runbook Decisioning

Question 9: MTTR Benchmarking Against Industry Peers

Summary Scorecard