Implementation Review (IR) - Software Assessment

Assessment questionnaire for measuring maturity. Answer each question honestly based on current, implemented practices.

Implementation Review (IR) - Software Domain

HAIAMM Assessment Questionnaire v3.0

Practice: Implementation Review (IR) Domain: Software Purpose: Assess organizational maturity in verifying that the actual code and configuration of AI/HAI software artifacts match the design approved at DR, and stay there as artifacts evolve. Scoring Model: Evidence + Outcome Metrics (see Scoring Methodology below)

Instructions

Answer each question honestly based on current, implemented practices (not plans or aspirations)
Each question has two components: Evidence (what you did) and Outcome Metrics (how well it worked)
Scoring uses 4 tiers: Fully Mature (1.0), Implemented (0.67), Partial (0.33), Not Implemented (0.0)
Answer progressively, Complete all Level 1 questions before Level 2
Level progression, Achieve ALL questions at lower level before advancing
Baseline first, Record current metric values before setting targets

Scoring Methodology

Score	Label	Criteria
1.0	Fully Mature	Evidence complete + ≥3 outcome metrics meet targets
0.67	Implemented	Evidence complete + 2 outcome metrics meet targets
0.33	Partial	Evidence partially complete + <2 metrics meet targets
0.0	Not Implemented	No evidence of practice

Level Score = Average of the three question scores at that level Overall Score = Weighted average across all levels achieved

Maturity Level 1

Objective: Run per-archetype implementation reviews at go-live, annually, and on material change, verifying code matches the SA pattern, config matches the DR decision, and the SR REM evidence is current

At this level, the gap between approved design and running system is systematically checked at the moments it matters most. Every review produces findings with severity tags, named owners, and SLA-bound resolution dates.

Question 1: Per-Archetype IR Checklist

Q1.1: Is there a published, per-archetype IR checklist, one per SM-Software archetype (LLM-integrated app, agent, RAG, fine-tune/training, eval harness, model-serving service, classical ML), covering code-matches-pattern verification, config-matches-DR verification, SR REM evidence currency check, logging-event production verification, and kill-switch test execution?

Evidence Required: - [ ] Published checklist per SM-Software archetype on file and linked from the SM inventory (LLM-integrated app, agent, RAG, fine-tune/training, eval harness, model-serving, classical ML) - [ ] Agent checklist explicitly verifies tool allowlist enforcement in deployed code (not only in DR checklist), per-tool scope, HITL gate function, and tool-call logging completeness - [ ] Code-repo drift sources reviewed since last IR: commits touching SA-pattern-control files (output filter, tool allowlist, session bounds, vault bindings, HITL gate, kill-switch, egress config, prompt template) reviewed and delta documented with findings - [ ] Kill-switch test executed and recorded with test input, expected behavior, actual behavior, test date, and reviewer on file, not a checkbox without a test record - [ ] SR REM evidence currency check present: a stratified sample of REM rows verified against current observable reality (no-train vendor admin console state, secrets-scan result currency, kill-switch test recency) - [ ] Logging-event production verified: a sample of prompt/completion/tool-call/admin-audit events pulled from the pipeline and confirmed present with correct format and retention policy

Outcome Metrics:

Metric	Baseline	Current	Target	Met?	Notes
% AI/HAI software artifacts with a go-live IR record	___%	___%	100%	☐	SM inventory × IR records
% active AI/HAI software artifacts with a current-year IR record	___%	___%	≥90%	☐	SM inventory × IR records
Critical / blocker findings open at go-live	___	___	0	☐	Findings backlog
Median closure time for High findings	___ days	___ days	≤7 days	☐	Findings backlog

Metric Collection Guidance: - Go-live IR coverage: count artifacts in SM inventory with a linked IR record dated at or before production cutover divided by total artifacts entering production. Formula: artifacts_with_golive_IR / total_artifacts_in_production × 100 - Current-year IR coverage: count artifacts with an IR record dated within the last 12 months divided by total active artifacts. Source: SM inventory last-IR-date field - Blocker findings at go-live: count Critical/blocker findings that were open on the go-live date. Source: IM-Software findings history - High finding closure time: median calendar days from finding-opened to evidence-linked finding-closed for High-severity findings. Source: IM-Software backlog timestamps

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No per-archetype checklist published)

Evidence Location: _________

Validation Date: _________

Notes: _______

Question 2: Review Triggers and Timing

Q1.2: Do 100% of new AI/HAI software artifacts going to production in the last 90 days carry a go-live IR record, and do ≥90% of all active artifacts carry a current-year IR record, with material-change triggers wired to SM inventory events?

Evidence Required: - [ ] Go-live IR records on file for all artifacts entering production in the last 90 days; deploy-event logs and model-registry entries confirm IR was completed before production cutover - [ ] Material-change trigger wired to SM inventory: model swaps, new agent tools, new data classes, customer-exposure changes, and new retrieval sources all generate a review-due alert within 5 business days - [ ] IaC plan-vs-apply state reviewed at each IR: Terraform / Pulumi planned vs. applied state compared against the DR-approved baseline; deviations documented as findings with severity tags - [ ] CI/CD job parameter review present: build-job parameters (model version pins, provider endpoint references, security-control toggles) compared against the DR-approved specification for artifacts reviewed in the last 90 days - [ ] Deploy-event configuration changes logged and reviewed: environment variable changes, feature flag changes, and secrets-rotation events captured at deploy time and reviewed against SA-pattern-control variables - [ ] Annual review calendar populated from the SM inventory with artifacts nearing review-due date visible at least 30 days in advance

Outcome Metrics:

Metric	Baseline	Current	Target	Met?	Notes
% AI/HAI software artifacts with a go-live IR record	___%	___%	100%	☐	SM inventory × IR records
% active AI/HAI software artifacts with a current-year IR record	___%	___%	≥90%	☐	SM inventory × IR records
% material changes to production artifacts that trigger an IR before the change ships	___%	___%	100%	☐	SM change events × IR records
Median closure time for High findings	___ days	___ days	≤7 days	☐	Findings backlog

Metric Collection Guidance: - Material-change IR trigger rate: cross-reference SM inventory material-change events (model swaps, new tool additions, new data classes) against IR records created within 5 business days of each event - Current-year IR coverage: count artifacts with an IR record dated within the last 365 days divided by total active artifacts in SM inventory - Blocker at go-live: query IM-Software findings for Critical/blocker findings in open status on the go-live date for each artifact - High closure time: median days from finding-opened to evidence-linked closure for High findings. Source: IM-Software timestamps

Evidence Location: _________

Validation Date: _________

Notes: _______

Question 3: Findings Tracking and SR REM Loop

Q1.3: Are findings severity-tagged and tracked in IM-Software with named owners and SLA-bound closure dates, and does every IR finding that reveals stale or inaccurate REM evidence trigger an SR REM row update before the finding is closed?

Evidence Required: - [ ] Findings backlog in IM-Software showing all IR findings with severity tag (Critical / High / Medium / Low), named owner (not "the engineering team"), SLA-bound closure date, and linked after-fix evidence artifact - [ ] SR REM update loop active: IR findings that reveal stale or inaccurate REM evidence have a linked REM row update on file before the finding is closed - [ ] Model-registry drift source verified: model version and fine-tune lineage changes since last IR reviewed and compared against SM inventory record and DR-approved version pin - [ ] Severity calibration consistent: Critical findings include at minimum agent acting with broader tool scope than DR-approved, kill-switch non-functional, and no-train setting unverified in vendor admin console, not downgraded to Medium - [ ] Findings-aging dashboard reviewed at least monthly by the program sponsor: meeting record, dashboard screenshot, or audit log confirming monthly review on file - [ ] No-train assertions and retention settings verified via vendor admin console state for fine-tune workloads (OpenAI / Anthropic / Bedrock / Vertex admin APIs), not from DPA text alone, probe result on file with IR record

Outcome Metrics:

Metric	Baseline	Current	Target	Met?	Notes
% AI/HAI software artifacts with a go-live IR record	___%	___%	100%	☐	SM inventory × IR records
% active AI/HAI software artifacts with a current-year IR record	___%	___%	≥90%	☐	SM inventory × IR records
Critical / blocker findings open at go-live	___	___	0	☐	Findings backlog
Median closure time for High findings	___ days	___ days	≤7 days	☐	Findings backlog

Metric Collection Guidance: - REM update loop rate: count IR findings where the closure is linked to an SR REM row update vs. total IR findings that identified stale or inaccurate REM evidence. Source: IM-Software × SR REM cross-reference - Named-owner coverage: count findings with a named individual (not a team) as owner divided by total open findings. Source: IM-Software - SLA adherence by severity: % of High findings closed within 7 days; % of Critical findings closed before go-live or rollback initiated. Source: IM-Software timestamps - Findings-aging review cadence: confirm monthly review via calendar record, dashboard export, or meeting minutes. Source: program sponsor review log

Evidence Location: _________

Validation Date: _________

Notes: _______

Maturity Level 2

Objective: Detect configuration drift continuously for Critical and High-tier artifacts via code-repo webhooks, model-registry events, IaC scan tooling, and vendor admin APIs; probe no-train and retention settings recurrently; calibrate IR cadence per SM tier

At this level, implementation review becomes a continuous signal for Critical and High-tier artifacts. No-train and retention settings are validated recurrently via vendor admin APIs. Tool scope for agent artifacts is tested at each IR cycle.

Question 4: Continuous Drift Detection

Q2.1: Are ≥90% of Critical-tier AI/HAI software artifacts under continuous drift detection, via code-repo webhooks, model-registry events, IaC scan tooling, deploy-event telemetry, and CI/CD parameter monitoring, with median detection latency ≤7 days and automated finding creation on material deviations?

Evidence Required: - [ ] Code-repo webhook configuration on file: webhooks trigger on commits to SA-pattern-control files (output filter, tool allowlist, session bounds, vault bindings, HITL gate, kill-switch, egress config, prompt template) and produce automated diff against DR-approved baseline for Critical and High-tier artifacts - [ ] Model-registry event pipeline active: model version promotions, fine-tune lineage changes, and deprecation events trigger an IR re-review gate; a model swap without a corresponding DR material-change review generates a Critical finding automatically - [ ] IaC plan-vs-apply drift scans running on each deploy for Critical and High artifacts (Terraform Cloud / Atlantis / Pulumi); scan results on file with findings auto-generated on deviations from the approved IaC module - [ ] Deploy-event telemetry capturing environment variable changes, feature flag changes, and secrets-rotation events; changes to SA-pattern-control variables since the last IR auto-open findings - [ ] Drift-detection pipeline health monitored: % Critical artifacts producing a fresh signal in the last 7 days; on-call alert configured for >48 hours of feed silence; alert test record on file - [ ] Detection latency evidence: telemetry log showing Critical-tier drift findings opened within 7 days of change event for the last 90-day period

Outcome Metrics:

Metric	Baseline	Current	Target	Met?	Notes
% Critical-tier artifacts under continuous drift detection (code-repo, model-registry, IaC, deploy telemetry)	___%	___%	≥90%	☐	Drift-detection telemetry
Median drift detection latency, Critical-tier	___ days	___ days	≤7 days	☐	IR telemetry
% Critical/High-tier artifacts with no-train and retention settings verified via vendor admin API (not DPA text alone)	___%	___%	≥80%	☐	Vendor API probing log
Tier-cadence adherence (% of artifacts reviewed on their published cadence)	___%	___%	≥95%	☐	IR schedule × SM inventory

Metric Collection Guidance: - Continuous drift detection coverage: count Critical-tier artifacts with all four signal sources active divided by total Critical-tier artifacts. Source: drift-detection pipeline configuration registry - Drift detection latency: median time from change-event timestamp to IR-finding-opened timestamp for Critical-tier drift findings. Source: IR telemetry × IM-Software - Vendor admin API coverage: count Critical/High-tier artifacts with a vendor admin API probe result within the current probe cycle (≤30 days Critical, ≤90 days High) divided by total Critical/High-tier artifacts with outbound LLM calls. Source: vendor API probing log - Tier-cadence adherence: count artifacts reviewed within their published cadence window (Critical: semi-annual; High: annual) divided by total artifacts. Source: IR schedule × SM inventory last-IR-date

Evidence Location: _________

Validation Date: _________

Notes: _______

Question 5: Vendor Admin API Probing for No-Train and Retention Settings

Q2.2: Are no-train and retention settings verified via vendor admin APIs on a monthly (Critical) and quarterly (High) probing cadence, not from DPA text alone, covering ≥80% of Critical/High-tier artifacts with outbound LLM calls?

Evidence Required: - [ ] Vendor admin API probe records on file for all Critical/High-tier artifacts using LLM providers: OpenAI Org Settings API (data_controls.training_data_sharing = false); Anthropic Organization admin settings confirming no-train terms; Bedrock Service Control Policy and invocation logging config; Vertex AI / Gemini Organization Policy constraints - [ ] Probing cadence log showing Critical-tier probes monthly and High-tier quarterly, not one-time screenshots; delta from the previous probe documented with IR findings when settings changed - [ ] Delta findings on file: any setting change detected by a probe (no-train toggle reset, retention period changed, logging config altered) generated an IR finding with severity matching the data-class impact of the change - [ ] Vendor API probing calendar maintained: missed probes tracked as process-metric failures with root cause and remediation on file - [ ] IR findings generated from probe deltas routed to IM-Software automatically with severity and owner pre-populated - [ ] UI-based verification with screenshot evidence documented as fallback where vendor APIs are not available, with a plan to migrate to API-based verification

Outcome Metrics:

Metric	Baseline	Current	Target	Met?	Notes
% Critical-tier artifacts under continuous drift detection (code-repo, model-registry, IaC, deploy telemetry)	___%	___%	≥90%	☐	Drift-detection telemetry
Median drift detection latency, Critical-tier	___ days	___ days	≤7 days	☐	IR telemetry
% Critical/High-tier artifacts with no-train and retention settings verified via vendor admin API (not DPA text alone)	___%	___%	≥80%	☐	Vendor API probing log
% Critical/High-tier agent artifacts with tool-scope boundary tests on record (current IR cycle)	___%	___%	100%	☐	IR records

Metric Collection Guidance: - Vendor admin API coverage: count Critical/High-tier artifacts with a vendor admin API probe result dated within the current probe cycle divided by total Critical/High-tier artifacts with outbound LLM calls. Source: vendor API probing log - Delta finding rate: count probe cycles where a setting change was detected and an IR finding was generated vs. total probe cycles. Source: probing log × IM-Software - Probe cadence compliance: count monthly Critical-tier probes completed on schedule vs. planned. Source: probing calendar × probing log - API vs. UI ratio: % of probes conducted via vendor admin API vs. UI-based screenshot fallback. Source: probing log method field

Evidence Location: _________

Validation Date: _________

Notes: _______

Question 6: Tool-Scope Boundary Testing and Tier-Calibrated Cadence

Q2.3: Are 100% of Critical/High-tier agent artifacts covered by tool-scope boundary tests in the current IR cycle, and is tier-cadence adherence ≥95% with Critical-tier findings aged per the SM L2 tier-treatment matrix SLAs?

Evidence Required: - [ ] Tool-scope boundary test records on file for every Critical/High-tier agent artifact: out-of-scope argument rejection confirmed (file path outside declared scope, record ID outside customer scope), non-allowlisted tool-call blocking confirmed at the allowlist-enforcement layer, HITL gate trigger verified synchronously, and kill-switch SLA confirmed - [ ] Kill-switch test record on file: agent process and all active tool invocations confirmed to stop within the specified SLA; test date within the current IR cycle - [ ] Tier-cadence enforcement visible in SM inventory: Critical-tier artifacts flagged when no IR in the last 180 days; escalation to program sponsor automated and test record on file - [ ] IR backlog tier-aware: Critical-tier findings reported separately; no Critical-tier finding waiting behind Low-tier queue items - [ ] Tool-scope test failures documented as Critical findings for Critical-tier agents and High findings for High-tier agents; no downgraded severity - [ ] Tier-treatment matrix SLA compliance tracked: Critical-tier findings closed per SM L2 SLA; SLA breach rate reported to program sponsor

Outcome Metrics:

Metric	Baseline	Current	Target	Met?	Notes
% Critical-tier artifacts under continuous drift detection (code-repo, model-registry, IaC, deploy telemetry)	___%	___%	≥90%	☐	Drift-detection telemetry
Median drift detection latency, Critical-tier	___ days	___ days	≤7 days	☐	IR telemetry
% Critical/High-tier agent artifacts with tool-scope boundary tests on record (current IR cycle)	___%	___%	100%	☐	IR records
Tier-cadence adherence (% of artifacts reviewed on their published cadence)	___%	___%	≥95%	☐	IR schedule × SM inventory

Metric Collection Guidance: - Tool-scope boundary test coverage: count Critical/High-tier agent artifacts with a complete boundary test record in the current IR cycle divided by total Critical/High-tier agent artifacts. Source: IR records - Kill-switch SLA confirmation rate: % of kill-switch tests where the SLA was confirmed met (agent stopped within specified window). Source: kill-switch test records - Tier-cadence adherence: % of artifacts reviewed within their published cadence window. Source: SM inventory last-IR-date × cadence policy - Tier-SLA breach rate: count Critical-tier findings exceeding the SM L2 tier-treatment matrix SLA divided by total Critical-tier findings. Source: IM-Software timestamps

Evidence Location: _________

Validation Date: _________

Notes: _______

Maturity Level 3

Objective: Continuous configuration attestation for Critical-tier artifacts, daily attestation signal confirming pattern compliance and evidence freshness, automatic IM ticket on drift, and contribution to OpenSSF AI reference attestation schemas and OWASP SAMM AI

At this level, Critical-tier AI/HAI software artifacts are attested continuously. Every Critical artifact produces a daily attestation signal across pattern compliance, evidence freshness, and configuration tolerance. Drift auto-opens IM-Software tickets.

Question 7: Daily Attestation Signal

Q3.1: Are ≥90% of Critical-tier AI/HAI software artifacts producing a daily attestation signal across all three dimensions (pattern compliance, evidence freshness, configuration tolerance), with deviations auto-opening IM-Software tickets within 1 hour?

Evidence Required: - [ ] Daily attestation pipeline operational: machine-readable, signed attestation artifacts produced daily per Critical-tier artifact covering (1) pattern compliance, output filter status, tool allowlist enforcement, vault binding, HITL gate logic, egress allowlist, session memory bounds; (2) evidence freshness, secrets scan ≤7 days, kill-switch test ≤90 days, no-train API probe ≤30 days, tool-scope boundary test ≤180 days; (3) configuration tolerance, model version, env variables, IaC outputs within declared tolerances - [ ] Auto-open IM-Software ticket on drift: ticket created within 1 hour of detection; carries drift dimension, specific control that failed tolerance, and link to DR decision record - [ ] Attestation-pipeline health monitored: % Critical artifacts with fresh signal in the last 24 hours; on-call paged for any Critical artifact silent for >24 hours; pager history on file - [ ] Zero stale-evidence violations confirmed for Critical-tier REMs: no evidence citations in active Critical-tier REMs outside their defined freshness window - [ ] Attestation artifacts regulator-consumable: machine-readable format confirmed suitable for EU AI Act Art. 9 risk-management evidence and ISO/IEC 42001 AIMS operational records via test export - [ ] Configuration tolerance definitions documented per-control: model patch versions within same major version tolerated; model family changes not tolerated without DR re-review; tolerance rules versioned and linked from SM inventory

Outcome Metrics:

Metric	Baseline	Current	Target	Met?	Notes
% Critical-tier artifacts producing a daily attestation signal	___%	___%	≥90%	☐	Attestation telemetry
% attestation findings auto-opening IM tickets within 1 hour of detection	___%	___%	≥95%	☐	IM-Software integration telemetry
Evidence freshness violations (stale evidence in active REMs)	___	___	0 for Critical; trending toward 0 for High	☐	Attestation telemetry
IR reviewer-hours per Critical artifact per year	___ hrs	___ hrs	trending down QoQ	☐	Reviewer time tracking

Metric Collection Guidance: - Daily attestation coverage: count Critical-tier artifacts with a signed attestation artifact dated within the last 24 hours divided by total Critical-tier artifacts. Source: attestation pipeline telemetry - Auto-open ticket latency: median time from drift-detection event to IM-Software ticket created. Source: attestation pipeline timestamp × IM-Software created-at timestamp - Stale-evidence violations: count Critical-tier REM rows with evidence citations outside their freshness window. Source: attestation pipeline evidence-freshness dimension output - IR reviewer-hours: total hours logged by reviewers for Critical-tier IR activities per year. Source: reviewer time tracking or sprint-board estimates

Evidence Location: _________

Validation Date: _________

Notes: _______

Question 8: External Schema Contribution

Q3.2: Has the program published per-archetype configuration baseline schemas to OpenSSF AI, OWASP SAMM AI, or CSA AI Safety Initiative, with documented external adoption and internal practice aligned to the published versions?

Evidence Required: - [ ] Per-archetype configuration baseline schemas published to at least one external body: OpenSSF AI working group (reference attestation schema compatible with SLSA / in-toto), OWASP SAMM AI extensions (Verification function, Implementation Review stream), or CSA AI Safety Initiative AI Controls Matrix - [ ] Internal practice aligned to published external versions: documented mapping between internal IR checklist and published schema; internal-only deviations proposed as upstream changes with a PR or issue link - [ ] External adoption tracked: citations, forks, direct acknowledgment from peer organizations, or inclusion in external tooling or assessment frameworks, evidence on file - [ ] Schema publication pipeline active: at least one schema in-draft, in-review, or published at any time; publication status tracker maintained - [ ] Adoption trend documented: at least one adoption data point (citation, fork, or direct acknowledgment) on file since initial publication - [ ] Internal schema version pinned to published external version: divergence from the external version documented as a proposed upstream change

Outcome Metrics:

Metric	Baseline	Current	Target	Met?	Notes
% Critical-tier artifacts producing a daily attestation signal	___%	___%	≥90%	☐	Attestation telemetry
% attestation findings auto-opening IM tickets within 1 hour of detection	___%	___%	≥95%	☐	IM-Software integration telemetry
External adoption of published configuration baseline schemas	0	tracked	trending up	☐	External telemetry
IR reviewer-hours per Critical artifact per year	___ hrs	___ hrs	trending down QoQ	☐	Reviewer time tracking

Metric Collection Guidance: - External adoption: count citations, forks, and direct acknowledgments per quarter since publication. Source: GitHub stars/forks, external publication references, peer organization acknowledgments - Schema alignment rate: % of internal IR checklist items directly traceable to a published external schema item. Source: internal-to-external schema mapping document - Publication pipeline health: number of schemas in active publication stages (draft / review / published). Source: publication tracker

Evidence Location: _________

Validation Date: _________

Notes: _______

Question 9: Automated Drift-to-IM Escalation and Post-Incident Feedback Loop

Q3.3: Is the post-incident IR feedback loop operational, IM-Software post-incident reviews include a mandatory IR-record re-examination step, and ≥1 attestation rule update is produced per material incident?

Evidence Required: - [ ] All IR findings (daily attestation and periodic reviews) flow into IM-Software automatically with severity and SLA pre-populated from the SM L2 tier-treatment matrix; no manual entry required for attestation-generated findings - [ ] IM-Software SLA clock automated: overdue Critical findings escalate to the program sponsor at 50% and 100% of the SLA window automatically; escalation automation tested quarterly with synthetic finding, test record on file - [ ] Post-incident IR feedback loop documented in the IM-Software incident-review template: a mandatory IR-record re-examination step confirmed in the last material incident review record - [ ] Attestation rule updates produced from incident learning: at least one attestation rule update linked from a completed incident review in the last 12 months - [ ] Post-incident IR step completion tracked: % of material incidents in the last 12 months with a completed post-incident review including the IR-record re-examination step - [ ] Attestation-pipeline health log on file: ≥90% of Critical artifacts with fresh signal in the last 24 hours; pager history showing no more than one silent-period alert per month

Outcome Metrics:

Metric	Baseline	Current	Target	Met?	Notes
% Critical-tier artifacts producing a daily attestation signal	___%	___%	≥90%	☐	Attestation telemetry
% attestation findings auto-opening IM tickets within 1 hour of detection	___%	___%	≥95%	☐	IM-Software integration telemetry
Evidence freshness violations (stale evidence in active REMs)	___	___	0 for Critical; trending toward 0 for High	☐	Attestation telemetry
IR reviewer-hours per Critical artifact per year	___ hrs	___ hrs	trending down QoQ	☐	Reviewer time tracking

Metric Collection Guidance: - Auto-open ticket rate: count attestation-generated findings that created an IM-Software ticket within 1 hour divided by total attestation-generated findings. Source: attestation pipeline × IM-Software integration telemetry - Escalation automation test pass rate: % of quarterly escalation tests that confirmed correct automated escalation. Source: escalation test records - Post-incident IR step completion: % of material incident post-reviews with the IR-record re-examination step completed. Source: IM-Software incident review records - Attestation rule update rate: count attestation rule updates linked from post-incident reviews in the last 12 months. Source: attestation pipeline rule-change log × IM-Software incident reviews

Evidence Location: _________

Validation Date: _________

Notes: _______

Summary Scorecard

Level	Question	Score	Weight
L1	Q1: Per-Archetype IR Checklist	___	33%
L1	Q2: Review Triggers and Timing	___	33%
L1	Q3: Findings Tracking and SR REM Loop	___	33%
L1 Total		___
L2	Q4: Continuous Drift Detection	___	33%
L2	Q5: Vendor Admin API Probing, No-Train and Retention	___	33%
L2	Q6: Tool-Scope Boundary Testing and Tier-Calibrated Cadence	___	33%
L2 Total		___
L3	Q7: Daily Attestation Signal	___	33%
L3	Q8: External Schema Contribution	___	33%
L3	Q9: Automated Escalation and Post-Incident Feedback	___	33%
L3 Total		___
Overall IR-Software Score		___

Current Maturity Level: ___

Key Findings:

Priority Improvements:

Document Version: HAIAMM v3.0 Practice: Implementation Review (IR) Domain: Software Questionnaire Date: 2026-05-15 Author: Verifhai

Instructions:

Answer based on current practices, not plans
“Yes” requires documented evidence
Complete all Level 1 questions before Level 2
Partial implementation = “No”

↓ Download as Markdown