Strategy & Metrics (SM) - Infrastructure Assessment

Assessment questionnaire for measuring maturity. Answer each question honestly based on current, implemented practices.

Strategy & Metrics (SM) - Infrastructure Domain

HAIAMM Assessment Questionnaire v3.0

Canonical source-of-truth: ../practices/SM-Infrastructure-OnePager.md. This questionnaire's questions, evidence requirements, and outcome metrics are derived from that one-pager. The canonical v3.0 model: ../HAIAMM-v3.0-Framing.md.


Practice: Strategy & Metrics (SM) Domain: Infrastructure Purpose: Stand up an AI/HAI Infrastructure Assurance program that discovers, inventories, and strategically governs all infrastructure that hosts and serves AI/HAI systems, with shadow AI infrastructure prevention as the primary L1 outcome and a defensible risk-tier rubric as the primary L2 deliverable. Scoring Model: Evidence + Outcome Metrics (see Scoring Methodology below)


Instructions

  • Answer each question honestly based on current, implemented practices (not plans or aspirations)
  • Each question has two components: Evidence (what you did) and Outcome Metrics (how well it worked)
  • Scoring uses 4 tiers: Fully Mature (1.0), Implemented (0.67), Partial (0.33), Not Implemented (0.0)
  • Answer progressively, Complete all Level 1 questions before Level 2
  • Level progression, Achieve ALL questions at lower level before advancing
  • Baseline first, Record current metric values before setting targets

Scoring Methodology

Tier Score Criteria
Fully Mature 1.0 Evidence complete + ≥3 outcome metrics meet targets
Implemented 0.67 Evidence complete + 2 outcome metrics meet targets
Partial 0.33 Evidence partially complete + <2 outcome metrics meet targets
Not Implemented 0.0 No substantive evidence of practice

Practice maturity level achieved = the highest level where all 3 questions score ≥ 0.67.


Maturity Level 1

Objective: Stand up the AI/HAI Infrastructure Assurance program, build an inventory of the seven infrastructure archetypes, and establish baseline metrics that prove shadow AI infrastructure is decreasing

Question 1: Charter the AI/HAI Infrastructure Assurance program

Q1.1: Do you have a published AI/HAI Infrastructure Assurance program charter that names the problem (shadow AI infrastructure, ungoverned GPU fleets, untracked inference endpoints, unregistered model deployments, orchestrator control planes without security review), defines the seven in-scope infrastructure archetypes, names an executive sponsor (CISO + VP Infrastructure / Head of Platform Engineering), establishes a cross-functional working group, and defines decision rights for provisioning approval, block, exception, and go-live?

Evidence Required: - [ ] Published program charter with named executive sponsor (CISO + VP Infrastructure / Head of Platform Engineering) and CTO co-signature where AI infrastructure is a product differentiator - [ ] Problem statement covering inference-endpoint attack surfaces, GPU-fleet credential risks, model-registry supply-chain control, orchestrator EA/TM/RA risk, and deployer duties under EU AI Act Art. 15 and GDPR Art. 32 - [ ] Seven in-scope infrastructure archetypes listed: inference endpoint/model-serving cluster, model registry, GPU/accelerator fleet, orchestrator/control plane, vector-store infrastructure, AI-specific CI/CD, feature store/online serving cache - [ ] Working group roster: Security, Platform/SRE, Cloud Architecture, ML Platform, AI/ML Engineering, Privacy/Legal, FinOps - [ ] Decision rights defined: who approves a new AI/HAI infrastructure component for production hosting, who blocks, who handles exceptions, who owns the go-live gate - [ ] Year-one success definition with numerical targets for L1 outcome metrics

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |---|---|---|---|---|---| | AI/HAI infrastructure inventory coverage (% of discovered infra instances in inventory) | ___ | ___ | ≥90% within 12 months | ☐ | | | Shadow-AI-infra ratio (unsanctioned AI/HAI infra instances in production ÷ total AI/HAI infra instances) | ___ | ___ | ≤15% and trending down | ☐ | | | % platform/SRE headcount with acknowledged AI Infrastructure Standards and GPU AUP | ___ | ___ | ≥95% of platform and SRE | ☐ | | | % AI/HAI infra instances in production with a named owning team | ___ | ___ | 100% | ☐ | | | Known data-exposure events from AI/HAI infrastructure (per quarter) | ___ | ___ | trending down QoQ | ☐ | |

Metric Collection Guidance: - Inventory coverage: Reconcile inventory count against cloud-provider API signals, Kubernetes workload discovery, IaC repo scanning, model-registry APIs, GPU-spend signals, egress logs, and vector-store listings. Formula: inventory_count / discovered_count × 100 - Shadow-AI-infra ratio: From inventory status field, count in-production instances not in "Sanctioned" status; divide by total in-production instances; trend quarterly - Policy attestation: HR/LMS acknowledgment records for AI Infrastructure Standards and GPU AUP filtered to platform and SRE headcount; denominator is that headcount - Named owning team: Count inventory records with non-null owning_team field vs. total; automated check at record creation - Data-exposure events from infrastructure: Aggregate DLP alerts, incident-tracker entries, and egress-log anomaly findings per quarter; trend over time

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)

Evidence Location: _____ Metric Validation Date: ____ Notes: __________


Question 2: Build the AI/HAI infrastructure inventory and discover shadow AI infrastructure

Q1.2: Do you maintain a single AI/HAI infrastructure inventory seeded from cloud-provider APIs (AWS/GCP/Azure AI service listings, GPU instance types), Kubernetes workload signals (GPU resource requests, ML base image containers), IaC repos (Terraform/Pulumi modules with AI-infra tags), model-registry APIs, cloud-spend/GPU-spend signals, egress logs to AI provider domains, and vector-store listings, covering all seven archetypes with a minimum field set including archetype, data classification of data passing through, isolation posture, and approval status?

Evidence Required: - [ ] Single authoritative infrastructure inventory with minimum fields: instance name, owning team, archetype, production status, customer-facing flag, AI/HAI software artifacts hosted, data classification of data passing through, geographic scope, isolation posture, compute scale, decision-affecting use hosted, approval status, linked artifacts - [ ] Cloud-provider API discovery active: AWS SageMaker/Bedrock/EKS GPU listings, GCP Vertex AI/GKE GPU node pools, Azure OpenAI/AKS GPU node pools - [ ] Kubernetes/container registry scanning: pods with GPU resource requests, containers from ML base registries (nvidia/cuda, huggingface, pytorch, vllm), Helm releases for ML-platform charts - [ ] IaC repo scanning: Terraform/Pulumi modules tagged "ai-infra," "ml-platform," "llm," "gpu-fleet," "vector-store," "model-registry," "inference-endpoint"; state files listing GPU instance groups or SageMaker resources - [ ] Cloud-spend/FinOps signals used: GPU spend by tag or cost-allocation group; unexplained GPU spend spikes treated as shadow-infra candidates - [ ] Egress logs monitored: outbound calls from internal hosts to known AI provider domains not attributable to a known inventory entry are shadow-infra candidates

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |---|---|---|---|---|---| | AI/HAI infrastructure inventory coverage (% of discovered infra instances in inventory) | ___ | ___ | ≥90% within 12 months | ☐ | | | Shadow-AI-infra ratio (unsanctioned AI/HAI infra instances in production ÷ total AI/HAI infra instances) | ___ | ___ | ≤15% and trending down | ☐ | | | % platform/SRE headcount with acknowledged AI Infrastructure Standards and GPU AUP | ___ | ___ | ≥95% of platform and SRE | ☐ | | | % AI/HAI infra instances in production with a named owning team | ___ | ___ | 100% | ☐ | | | Known data-exposure events from AI/HAI infrastructure (per quarter) | ___ | ___ | trending down QoQ | ☐ | |

Metric Collection Guidance: - Inventory coverage: Monthly reconciliation comparing inventory records to cloud-provider API results + Kubernetes GPU workload list + IaC state files + model-registry listings + GPU-spend tags + egress-log anomalies + vector-store listings; unmatched signals are shadow-infra candidates - Shadow-AI-infra ratio: Filter inventory for in-production instances with approval status != "Sanctioned"; divide by total in-production; trend quarterly - Named owning team: Automated check, instances with null owning_team flagged and assigned to triage owner within 5 BD - Data-exposure events from infrastructure: DLP alerts, incident-tracker entries, and egress-log review findings combined per quarter; trend over time - Policy attestation: HR/LMS query for platform and SRE headcount who completed AI Infrastructure Standards and GPU AUP acknowledgment

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)

Evidence Location: _____ Metric Validation Date: ____ Notes: __________


Question 3: Establish foundational metrics and the shadow AI infra scoreboard

Q1.3: Do you baseline and report quarterly to the executive sponsor a shadow AI infra scoreboard covering inventory state by archetype, new infrastructure instances discovered and their intake status, shadow-AI-infra ratio trend over the last four quarters, AI Infrastructure Standards and GPU AUP attestation coverage, and the top five unmitigated infrastructure risks with named owners and remediation status?

Evidence Required: - [ ] Quarterly shadow AI infra scoreboard published and delivered to executive sponsor, at least two consecutive quarters on record - [ ] Scoreboard includes archetype-level breakdown (inference endpoint/model-serving cluster, model registry, GPU/accelerator fleet, orchestrator/control plane, vector-store infrastructure, AI-specific CI/CD, feature store/online serving cache) - [ ] Shadow-AI-infra ratio trended over last 4 quarters with commentary on direction - [ ] AI Infrastructure Standards and GPU AUP attestation percentage reported with platform/SRE headcount denominator - [ ] Top 5 unmitigated infrastructure risks listed with named owner and remediation status (TA-flagged, misconfiguration-scanner-flagged, or external-advisory-flagged) - [ ] Intake SLA tracked: new AI/HAI infrastructure intake triaged within 5 BD; provisional approval within 10 BD for Low-tier archetypes (internal-only, no regulated data)

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |---|---|---|---|---|---| | AI/HAI infrastructure inventory coverage (% of discovered infra instances in inventory) | ___ | ___ | ≥90% within 12 months | ☐ | | | Shadow-AI-infra ratio (unsanctioned AI/HAI infra instances in production ÷ total AI/HAI infra instances) | ___ | ___ | ≤15% and trending down | ☐ | | | % platform/SRE headcount with acknowledged AI Infrastructure Standards and GPU AUP | ___ | ___ | ≥95% of platform and SRE | ☐ | | | % AI/HAI infra instances in production with a named owning team | ___ | ___ | 100% | ☐ | | | Known data-exposure events from AI/HAI infrastructure (per quarter) | ___ | ___ | trending down QoQ | ☐ | |

Metric Collection Guidance: - Scoreboard delivery cadence: Confirm last two quarters have a dated scoreboard delivered to exec sponsor with acknowledgment on record - Archetype breakdown: Scoreboard section shows counts per archetype (sanctioned / provisional / prohibited / awaiting intake); detects archetype classes growing unchecked - Shadow-AI-infra ratio trend: Four-quarter chart or table; downward trend is the L1 success signal; source is inventory status field reconciled monthly against all discovery signals - Policy attestation: Percentage with platform/SRE denominator explicitly stated; HR/LMS is the authoritative source; updated each quarter - Top-5 risks: Each entry lists risk description, source, named owner, and remediation status (open / in-progress / mitigated)

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)

Evidence Location: _____ Metric Validation Date: ____ Notes: __________


Maturity Level 2

Objective: Risk-tier the AI/HAI infrastructure inventory using the canonical rubric, calibrate the program's intensity per tier, and measure practice maturity and shadow-infra reduction per tier, establishing the tier rubric every other Infrastructure-domain L2 practice depends on

Question 1: Define the AI/HAI infrastructure risk-tier rubric

Q2.1: Do you have a published risk-tier rubric (Critical / High / Medium / Low) assigning a tier to every AI/HAI infrastructure instance based on seven auditable dimensions, tier of AI/HAI software hosted (inherited from SM-Software L2), multi-tenancy isolation, customer exposure, compute scale and concentration, data classification of data passing through, decision-affecting use hosted, and geographic scope, with tier derivation deterministic, human overrides recorded, and 100% of inventory records carrying a current tier?

Evidence Required: - [ ] Published tier-rubric document listing all seven auditable dimensions with deterministic assignment logic - [ ] 100% of inventory records carry a current tier assignment derived from the rubric - [ ] Hosted-software-tier dimension: infrastructure hosting a Critical-tier Software artifact (per SM-Software L2) → Critical Infrastructure; tier is the maximum of hosted-software tier and all infrastructure-specific dimensions - [ ] Multi-tenancy dimension: multi-tenant serving with no per-tenant isolation → elevate tier; isolation present → no elevation from this dimension - [ ] Data classification dimension: regulated data (PII/PHI/PCI/source code/customer confidential) transiting or stored at inference/training/retrieval → Critical or High; cross-border personal-data flow triggers GDPR Art. 44–49 assessment → elevate - [ ] Decision-affecting use dimension: EU AI Act Annex III or GDPR Art. 22 system hosted → Critical - [ ] Human override log maintained: overrides recorded with rationale and reviewed by working group

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |---|---|---|---|---|---| | % of inventory with a current tier assignment | ___ | ___ | 100% | ☐ | | | Tier-treatment matrix adherence, % Critical infra instances with full-scope treatment in last 12 months | ___ | ___ | ≥95% | ☐ | | | Tier-weighted shadow-AI-infra ratio (Critical-weighted) | ___ | ___ | Critical = 0 unsanctioned in production; overall trending down | ☐ | | | Per-tier SLA adherence across practices (intake, IR, ST, ML, IM) | ___ | ___ | ≥90% per tier | ☐ | | | Critical infra instances with per-tenant isolation and HSM/KMS-rooted encryption | ___ | ___ | 100% | ☐ | | | Tier drift rate (tier changes per year) | ___ | ___ | tracked; unexplained changes = 0 | ☐ | |

Metric Collection Guidance: - % inventory with tier assignment: Automated check, records with null risk_tier or not re-confirmed after a material change are flagged for remediation within 5 BD - Tier-treatment matrix adherence: Cross-reference Critical-tier instances against full-scope treatment evidence: per-tenant isolation confirmed, HSM/KMS key confirmed, egress allowlist set, least-privilege IAM, semi-annual IR completed, full ST battery completed - Tier-weighted shadow-AI-infra ratio: Critical-tier in-production instances not in "Sanctioned" status must be 0; overall ratio should trend down - Per-tier SLA adherence: From intake, IR, ST, and IM trackers; % on-time per tier; report monthly - Critical instances with isolation and encryption: Infrastructure attestation report listing each Critical instance with isolation posture and key-management configuration; 100% must show per-tenant isolation AND HSM/KMS-rooted encryption

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)

Evidence Location: _____ Metric Validation Date: ____ Notes: __________


Question 2: Calibrate program intensity per tier

Q2.2: Do you have a published tier-treatment matrix defining differential controls across all downstream Infrastructure-domain practices (PC, TA, SR, SA, DR, IR, ST, EH, ML, IM) for each tier, and is this matrix enforced, with Critical-tier infrastructure instances receiving per-tenant isolation, HSM/KMS-rooted encryption, egress allowlisting, least-privilege per-instance service accounts, semi-annual IR, the full ST battery, and mandatory re-review within 14 days of any material change?

Evidence Required: - [ ] Tier-treatment matrix published covering all downstream practices with explicit controls per tier - [ ] Critical-tier treatment documented: full SR pack + REM + executive + security sign-off at intake; per-tenant/dedicated namespace isolation; HSM-rooted or KMS key-per-instance + TLS 1.2+ in transit; egress allowlist to named model providers; least-privilege per-instance service accounts with no standing human IAM and secrets in vault; per-instance deep TA with model-extraction/inference-endpoint/AGH/EA/TM/RA vectors; semi-annual IR + on material change; full ST battery; IM SLA ack ≤4h/mitigate ≤48h; mandatory re-review within 14 days of material change - [ ] FedRAMP/regional compliance gating enforced for Critical-tier instances in applicable contexts before go-live - [ ] Low-tier fast-track documented: provisionally approved within 10 BD; shared cluster; managed encryption; egress logged; service accounts; spot-check ST; baseline logging; IR at go-live only - [ ] Downstream practices acknowledged calibration via working-group decision record

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |---|---|---|---|---|---| | % of inventory with a current tier assignment | ___ | ___ | 100% | ☐ | | | Tier-treatment matrix adherence, % Critical infra instances with full-scope treatment in last 12 months | ___ | ___ | ≥95% | ☐ | | | Tier-weighted shadow-AI-infra ratio (Critical-weighted) | ___ | ___ | Critical = 0 unsanctioned in production; overall trending down | ☐ | | | Per-tier SLA adherence across practices (intake, IR, ST, ML, IM) | ___ | ___ | ≥90% per tier | ☐ | | | Critical infra instances with per-tenant isolation and HSM/KMS-rooted encryption | ___ | ___ | 100% | ☐ | | | Tier drift rate (tier changes per year) | ___ | ___ | tracked; unexplained changes = 0 | ☐ | |

Metric Collection Guidance: - Tier-treatment matrix adherence: For each Critical-tier instance, verify: isolation posture confirmed, encryption confirmed, egress allowlist set, least-privilege IAM confirmed, semi-annual IR on schedule, full ST battery completed; ≥95% must show all treatments in last 12 months - FedRAMP/regional compliance gating: For Critical-tier instances in US federal or public-sector context, verify compliance evidence on file before production status; no instance in production without the evidence - Critical instances with isolation and encryption: Infrastructure attestation; 100% of Critical-tier instances must show per-tenant isolation AND HSM/KMS-rooted encryption confirmed - Per-tier SLA adherence: Aggregated from intake, IR, ST, and IM trackers; % on-time per tier; reported monthly - Tier drift rate: Governance log reviewed at each working-group meeting; changes without dimension-change rationale are unexplained; target = 0

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)

Evidence Location: _____ Metric Validation Date: ____ Notes: __________


Question 3: Per-tier scoreboard and governance

Q2.3: Does the quarterly shadow AI infra scoreboard report inventory state per tier and per archetype with Critical-tier unsanctioned infrastructure in production explicitly tracked at zero, include a tier-movement log with rationale, report per-tier SLA adherence, include FedRAMP/regional compliance gating status for Critical-tier instances, and is it reviewed by the executive sponsor who discusses tier-balance?

Evidence Required: - [ ] Quarterly scoreboard includes a tier × archetype breakdown table (Critical/High/Medium/Low rows by archetype columns) - [ ] Critical-tier unsanctioned infrastructure in production is a named metric; target is 0; any non-zero value is a headline finding requiring sponsor action - [ ] Tier-movement log included: instances that moved up or down in the quarter, with dimension(s) that changed and rationale for each move - [ ] SLA adherence per tier reported for intake, IR, ST, ML, and IM - [ ] FedRAMP/regional compliance gating status included: Critical-tier instances in applicable contexts showing compliance evidence on file before production status - [ ] Quarterly executive review documented (agenda + minutes) showing tier-balance discussion, GPU-spend-by-tier review, and sponsor sign-off

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |---|---|---|---|---|---| | % of inventory with a current tier assignment | ___ | ___ | 100% | ☐ | | | Tier-treatment matrix adherence, % Critical infra instances with full-scope treatment in last 12 months | ___ | ___ | ≥95% | ☐ | | | Tier-weighted shadow-AI-infra ratio (Critical-weighted) | ___ | ___ | Critical = 0 unsanctioned in production; overall trending down | ☐ | | | Per-tier SLA adherence across practices (intake, IR, ST, ML, IM) | ___ | ___ | ≥90% per tier | ☐ | | | Critical infra instances with per-tenant isolation and HSM/KMS-rooted encryption | ___ | ___ | 100% | ☐ | | | Tier drift rate (tier changes per year) | ___ | ___ | tracked; unexplained changes = 0 | ☐ | |

Metric Collection Guidance: - Per-tier scoreboard delivery: Last two consecutive quarterly scoreboards must include tier × archetype table; each shows delta from prior quarter; GPU spend by tier included as a FinOps dimension - Critical-tier unsanctioned count: Named metric; source is inventory filtered on tier=Critical AND status != Sanctioned AND production_status = in-production; target = 0 - Tier-movement log completeness: Each entry must have instance name, prior tier, new tier, dimension(s) that changed, reviewer name, and date; unexplained changes target = 0 - SLA adherence per tier: Pulled from intake, IR, ST, and IM systems; aggregated per tier; reported as % on-time per tier per quarter - Executive review: Filed governance document confirming exec sponsor reviewed tier-balance and GPU-spend-by-tier sections and issued follow-up actions or signed off

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)

Evidence Location: _____ Metric Validation Date: ____ Notes: __________


Maturity Level 3

Objective: Automate inventory and tier maintenance from cloud-provider APIs, IaC telemetry, and runtime signals; benchmark the program against external infrastructure security peers; and contribute to CNCF, OpenSSF AI, and ML-platform community infrastructure security standards

Question 1: Continuous inventory and tier automation from cloud, IaC, and runtime signals

Q3.1: Does the AI/HAI infrastructure inventory auto-update from live cloud-provider asset API events, IaC state events, Kubernetes admission webhook telemetry, model-registry events, GPU-spend deltas, and egress-log anomalies, with tier assignments rule-based and replayable, tier changes auto-triggering downstream obligations within 24 hours, and a published data-quality SLO of ≥99% correctly tiered within 48 hours of a material change?

Evidence Required: - [ ] Published data-quality SLO: ≥99% of active AI/HAI infrastructure instances correctly tiered within 48 hours of a material change; ≥95% inventory completeness against discovery-source reconciliation - [ ] Automated feeds operational: cloud-provider asset API events (new SageMaker endpoint, new Bedrock provisioned throughput, new GKE GPU node pool), IaC state events (Terraform AI-infra module applied/destroyed), Kubernetes admission webhook events (new pod with GPU request or AI/ML image), model-registry events (new model registered or version promoted), GPU-spend deltas (unexplained new tag is shadow-infra signal), egress-log anomalies (new outbound call to AI provider domain not in inventory), vector-store collection creation events, self-attestation and intake - [ ] Tier rules documented as versioned, replayable logic; rule changes change-logged and replayable against historical inventory state - [ ] Tier-change events auto-trigger downstream obligations (e.g., Medium→Critical triggers isolation gate, encryption upgrade, IR reconfiguration, FedRAMP/regional-compliance check) within 24h; monitored via workflow telemetry - [ ] Human curation queue defined for: new archetype patterns, ambiguous multi-workload instances, dimensional-input conflicts between Software and Infrastructure tier derivations - [ ] Automation health dashboard: on-call paged when any feed exceeds staleness threshold

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |---|---|---|---|---|---| | Inventory auto-update latency | ___ | ___ | ≤48h for material changes | ☐ | | | % inventory entries auto-curated vs. human-curated | ___ | ___ | ≥80% auto | ☐ | | | Inventory completeness against discovery-source reconciliation | ___ | ___ | ≥99% | ☐ | | | Tier-rule auto-trigger of downstream obligations on tier change | ___ | ___ | 100% within 24h | ☐ | | | External benchmarks tracked | ___ | ___ | ≥5 peer-comparable metrics (CNCF, OpenSSF, FinOps, ISAC, ML-platform community) | ☐ | | | Industry contributions per year | ___ | ___ | ≥4 substantive | ☐ | | | Executive ROI narrative refreshed with external benchmarks | ___ | ___ | semi-annual | ☐ | |

Metric Collection Guidance: - Auto-update latency: Measure time from a known material change event (new GPU node pool added, new SageMaker endpoint deployed) to the corresponding inventory record update; P95 across 20 sampled events per quarter - % auto-curated: From the curation log, count records updated by automated feeds vs. human-initiated edits; report as a ratio per quarter - Inventory completeness: Full discovery-source reconciliation across all seven archetype signal sources; report completeness % and list archetypes below 99% - Downstream obligation auto-trigger: Workflow telemetry showing each tier-change event produced an isolation gate, encryption-upgrade task, IR reconfiguration, or FedRAMP check event within 24h; report % within SLO - External benchmarks tracked: Count distinct benchmark data points in semi-annual brief; each traceable to CNCF, OpenSSF, FinOps Foundation, sector ISAC, or ML-platform community source

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)

Evidence Location: _____ Metric Validation Date: ____ Notes: __________


Question 2: External benchmarking

Q3.2: Do you publish a semi-annual external-benchmarking brief comparing the program against at least five peer-comparable metrics via CNCF AI/ML working groups, OpenSSF AI supply-chain security working groups, FinOps Foundation AI infrastructure working groups, MLOps/ML-platform community, and sector ISACs with AI infrastructure working groups, and do benchmark deltas explicitly inform program investment decisions?

Evidence Required: - [ ] Semi-annual benchmarking brief published, two most recent on file with dates, each containing ≥5 peer-comparable metrics from named external sources - [ ] Benchmarking sources include at least two of: CNCF AI/ML / CNCF TAG Security AI infrastructure / OpenSSF AI supply-chain / FinOps Foundation AI infrastructure / MLOps/ML-platform community (KubeCon ML, Apply(ML)) / sector ISACs / formal CISO peer roundtables - [ ] Metrics benchmarked cover: inventory coverage, shadow-AI-infra ratio, per-tier SLA adherence, isolation posture of Critical-tier instances, automation level, IR drift detection rate, ST coverage rate, time from "provisioning request" to "provisional approval" - [ ] Benchmark deltas explicitly referenced in a program investment or prioritization decision; documentation filed within 90 days of each brief - [ ] Peer selection rationale documented, peers chosen to stretch the program, not flatter it

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |---|---|---|---|---|---| | Inventory auto-update latency | ___ | ___ | ≤48h for material changes | ☐ | | | % inventory entries auto-curated vs. human-curated | ___ | ___ | ≥80% auto | ☐ | | | Inventory completeness against discovery-source reconciliation | ___ | ___ | ≥99% | ☐ | | | Tier-rule auto-trigger of downstream obligations on tier change | ___ | ___ | 100% within 24h | ☐ | | | External benchmarks tracked | ___ | ___ | ≥5 peer-comparable metrics (CNCF, OpenSSF, FinOps, ISAC, ML-platform community) | ☐ | | | Industry contributions per year | ___ | ___ | ≥4 substantive | ☐ | | | Executive ROI narrative refreshed with external benchmarks | ___ | ___ | semi-annual | ☐ | |

Metric Collection Guidance: - External benchmarks tracked: Each brief lists ≥5 named benchmark data points; each traceable to a CNCF/OpenSSF/FinOps/ISAC/ML-platform source - Benchmark-driven investment: Program planning or budget document explicitly citing a benchmark delta as rationale; filed within 90 days of each brief - Semi-annual cadence: Two briefs within a 12-month window; no gap > 7 months between consecutive briefs - Executive ROI narrative: Annual exec/board deck includes benchmark comparisons, GPU fleet spend by tier, and avoided-loss examples (Critical-tier multi-tenant cluster caught at DR before a model-extraction event)

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)

Evidence Location: _____ Metric Validation Date: ____ Notes: __________


Question 3: Contribute to CNCF, OpenSSF AI, and ML-platform community infrastructure security standards

Q3.3: Does the program contribute at least four substantive, anonymized artifacts per year to the AI/HAI infrastructure security ecosystem through CNCF AI/ML Working Group, CNCF TAG Security, OpenSSF AI, FinOps Foundation AI Infrastructure SIG, MITRE ATLAS, NIST AI RMF Playbook, CSA AI Safety Initiative, or sector ISACs, with each contribution anonymized, legally vetted, and traceable to a published working-group output?

Evidence Required: - [ ] Contribution log maintained listing all submissions: target body, submission type (inference-endpoint hardening guidance, model-registry security pattern, Kubernetes GPU workload isolation pattern, TTP, etc.), date submitted, anonymization review completed, status - [ ] At least 4 substantive contributions per year in the most recent 12-month window; each is a technical artifact accepted or in active review by the named body - [ ] Each contribution has a legal/privacy review sign-off confirming anonymization before submission - [ ] Contributions traceable to published outputs: CNCF working-group deliverables, OpenSSF advisories, FinOps Foundation SIG outputs, MITRE ATLAS technique entries, NIST AI RMF Playbook references, CSA controls matrix updates - [ ] Contribution pipeline shows ≥2 items in-flight (draft, in-review, or being prepared) at any working-group review

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |---|---|---|---|---|---| | Inventory auto-update latency | ___ | ___ | ≤48h for material changes | ☐ | | | % inventory entries auto-curated vs. human-curated | ___ | ___ | ≥80% auto | ☐ | | | Inventory completeness against discovery-source reconciliation | ___ | ___ | ≥99% | ☐ | | | Tier-rule auto-trigger of downstream obligations on tier change | ___ | ___ | 100% within 24h | ☐ | | | External benchmarks tracked | ___ | ___ | ≥5 peer-comparable metrics (CNCF, OpenSSF, FinOps, ISAC, ML-platform community) | ☐ | | | Industry contributions per year | ___ | ___ | ≥4 substantive | ☐ | | | Executive ROI narrative refreshed with external benchmarks | ___ | ___ | semi-annual | ☐ | |

Metric Collection Guidance: - Industry contributions per year: Count entries in the contribution log for trailing 12 months where status = submitted or accepted to a named body; conference talks and press releases do not count - Contribution pipeline health: At any working-group meeting, pipeline log shows ≥2 items not yet in submitted status; noted in working-group minutes - Legal/privacy review: Each contribution log entry must have reviewer name and date; no contribution submitted without this sign-off - Executive ROI narrative: Filed annually to exec/board; references external benchmarks, GPU fleet spend by tier, and avoided-loss examples; faster sanctioned provisioning time is a key metric

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)

Evidence Location: _____ Metric Validation Date: ____ Notes: __________


Summary Scorecard

Level Q1 Q2 Q3 Avg Achieved?
L1 __ __ __ __
L2 __ __ __ __
L3 __ __ __ __

Practice maturity level achieved: ___ (highest level where all 3 questions score ≥ 0.67)


Document Version: HAIAMM v3.0 Practice: Strategy & Metrics (SM) Domain: Infrastructure Last Updated: 2026-05-15 Author: Verifhai

Instructions:

  • Answer based on current practices, not plans
  • “Yes” requires documented evidence
  • Complete all Level 1 questions before Level 2
  • Partial implementation = “No”

↓ Download as Markdown