Strategy & Metrics (SM)

Infrastructure Domain - HAIAMM v3.0

Practice Overview

Objective: Stand up an AI/HAI Infrastructure Assurance program that discovers, inventories, and strategically governs all infrastructure that hosts and serves AI/HAI systems, with shadow AI infrastructure prevention as the primary L1 outcome and a defensible risk-tier rubric as the primary L2 deliverable.

Description: The Infrastructure domain governs the compute and platform layer that AI/HAI systems run on, inference endpoints and model-serving clusters, model registries, GPU and accelerator fleets, orchestrator control planes, vector-store infrastructure, AI-specific CI/CD pipelines, and feature stores or online serving caches. SM-Infrastructure establishes the program charter, an authoritative inventory of these infrastructure archetypes, and the practice-maturity metrics that prove the program is working. SM-Infrastructure L2 produces the risk-tier rubric every other Infrastructure-domain L2 practice depends on (per §9.3 of the v3.0 framing).

Context: Infrastructure teams adopt AI workloads faster than the security and platform governance programs that are meant to track them. A platform engineer spins up a vLLM serving cluster from an IaC module; an ML team registers a new fine-tuned model into a SageMaker Model Registry silo security has never reviewed; a researcher provisions GPU spot instances for an overnight training run and leaves persistent credentials behind. None of this is malicious, it is the normal pace of AI-enabled infrastructure provisioning. But it bypasses threat modeling (TA), configuration requirements (SR), reference architecture (SA), and the deployer duties that EU AI Act Art. 15 (accuracy/robustness/cybersecurity of high-risk systems) and GDPR Art. 32 (security of processing) place on whoever operates the system. The AI/HAI Infrastructure Assurance program makes this surface visible, attaches accountable ownership per archetype, and puts a light-touch intake gate on the path from prototype provisioning to production hosting, so sanctioned infrastructure ships faster and shadow AI infrastructure cannot quietly accumulate.

Maturity Level 1

Objective: Stand up the AI/HAI Infrastructure Assurance program, build an inventory of the seven infrastructure archetypes, and establish baseline metrics that prove shadow AI infrastructure is decreasing

At this level, the organization makes its AI/HAI hosting and serving infrastructure visible, assigns accountability per archetype instance, and begins measuring the reduction of shadow AI infrastructure, unsanctioned inference endpoints, untracked GPU workloads, unregistered model-serving deployments, and ungoverned orchestrator control planes.

Dependencies

• None, entry-point practice for the Infrastructure domain. SM-Infrastructure L1 precedes all other Infrastructure-domain L1s.
• Alignment (not a hard dependency): enterprise-wide SM strategy, SM-Software L1 (the software artifacts hosted here are already in the Software inventory), SM-Data L1 (the data passing through this infrastructure is in the Data inventory), existing CMDB or cloud-asset-management program, so the AI/HAI infrastructure program connects to existing asset catalogs rather than forming a parallel stack.
• Supports / unblocks: PC-Infrastructure L1 (policies need the inventory and archetype taxonomy), TA-Infrastructure L1 (threat modeling needs the asset list), SR-Infrastructure L1 (requirements packs key on archetype), SA-Infrastructure L1 (reference patterns need the archetype list), IM-Infrastructure L1 (incident routing needs the owner and sponsor structure), ML-Infrastructure L1 (logging baseline needs the inventory).

Desired Outcomes

• Shadow AI infrastructure is visible, attributed to a named owning team, and trending down quarter-over-quarter.
• A single AI/HAI infrastructure inventory is the authoritative source of truth across Security, Platform/SRE, Cloud, ML Platform, and Engineering.
• An accountable executive owns AI/HAI infrastructure risk; decision rights for provisioning approval, block, exception, and go-live are unambiguous.
• Practice maturity is measurable from a small, automatable metric set rather than from activity counts (scans run, tickets closed).
• The program is positioned as an enabler, fast-track intake for Low-tier internal-only vector stores over public data, full review for Critical-tier customer-facing inference endpoints processing regulated data, so platform teams work through it rather than around it.

Activities

A) Charter the AI/HAI Infrastructure Assurance program

Publish a short program charter that names the problem (shadow AI infrastructure, ungoverned GPU fleets, untracked inference endpoints, unregistered model deployments, orchestrator control planes with no security review), defines scope, and assigns accountable ownership. The program does not require a new team, it requires a named owner, a cross-functional working group, and a clear intake gate for new AI/HAI infrastructure before it reaches production hosting.

Charter elements: - Problem statement, why AI/HAI infrastructure is a distinct first-party risk category: inference endpoints are high-value attack surfaces (model extraction, prompt injection at the serving layer, data exfiltration via API); GPU fleets concentrate sensitive training workloads and their credentials; model registries are supply-chain control points for every artifact downstream; orchestrator control planes carry EA/TM/RA risk proportional to the agent workloads they run; vector stores hold invertible embeddings and retrieval context that feeds AGH; AI-specific CI/CD pipelines are software-supply-chain attack surfaces for model promotion; deployer duties under EU AI Act Art. 15 and GDPR Art. 32 require the org operating this infrastructure to implement appropriate cybersecurity measures. - In-scope AI/HAI infrastructure archetypes, the seven canonical Infrastructure-domain asset types: 1. Inference endpoint / model-serving cluster, own-hosted serving infrastructure: vLLM, TGI, Ray Serve, SageMaker / Vertex / Bedrock-hosted endpoints the org operates, Azure OpenAI deployments the org manages. 2. Model registry, MLflow, Weights & Biases, SageMaker Model Registry, Vertex AI Model Registry instances hosting model artifacts, lineage metadata, and promotion gates. 3. GPU / accelerator fleet, training compute, inference compute, on-prem GPU nodes, cloud accelerator reservations (A100, H100, TPU pods), spot-instance pools used for ML workloads. 4. Orchestrator / control plane, agent platforms, workflow orchestrators running AI/HAI workloads: LangGraph deployments, Temporal clusters, Airflow instances running ML pipelines, Ray clusters, Kubeflow Pipelines. 5. Vector-store infrastructure, own-hosted Weaviate, Qdrant, Milvus, Chroma clusters; pgvector-enabled databases; Pinecone tenant instances the org administers. 6. AI-specific CI/CD, training pipelines, model-promotion pipelines, eval-gate jobs, automated fine-tuning triggers, model-deployment automation; distinct from general CI/CD by virtue of model artifacts and training-data access. 7. Feature store / online serving cache, Feast, Tecton, Vertex Feature Store, Redis-backed feature-serving layers that supply features to AI/HAI inference. - Executive sponsor, typically the CISO co-sponsored by the VP of Infrastructure / Head of Platform Engineering / VP of Engineering; co-signed by the CTO where AI infrastructure is a product differentiator. - Working group, Security, Platform/SRE, Cloud Architecture, ML Platform, AI/ML Engineering, Privacy/Legal (for data-classification of data passing through infrastructure), FinOps (GPU spend visibility). - Decision rights, who can approve a new infrastructure component for hosting AI/HAI workloads; who can block one; who handles exceptions; who owns the go-live gate. - Success definition for year one, a numerical target for the L1 outcome metrics below (e.g., "≥90% of AI/HAI infrastructure instances in production inventoried within 12 months").

B) Build the AI/HAI infrastructure inventory and discover shadow AI infrastructure

Establish a single AI/HAI infrastructure inventory as the program's source of truth. Seed it from authoritative infrastructure signals, then actively discover shadow AI infrastructure using signals already available to platform, security, and FinOps teams.

Inventory fields (minimum): - Instance name, owning team, archetype (one of the seven above). - Production status: prototype / staging / production / deprecated. - Customer-facing? Internal-only? Multi-tenant (serves multiple teams / products)? - AI/HAI software artifacts hosted: links to SM-Software inventory records for the models, agents, RAG pipelines, or training workloads running on this infrastructure. - Data classification of data passing through: public, internal, confidential, regulated (PII / PHI / PCI / source code), pulled from SM-Data inventory where available. - Geographic scope: single-region, multi-region, cross-border (triggers GDPR Art. 44–49 assessment). - Isolation posture: dedicated (single workload), shared (multiple workloads), multi-tenant (multiple teams or customers). - Compute scale: approximate GPU-hours or vCPU scale; concentration flag if a single fleet serves multiple Critical-tier AI software artifacts. - Decision-affecting use: hosts an EU AI Act Annex III / Art. 22 system (yes/no), inherits from SM-Software inventory. - Approval status: Sanctioned / Provisional / Under review / Prohibited / Awaiting Intake. - Risk tier assignment (populated at L2, see SM-Infrastructure L2 Activity A). - Linked artifacts: TA threat snapshot, SR requirements-evidence map (REM), SA reference pattern, latest IR finding, ML logging-baseline status.

Discovery sources (at L1, use what platform and cloud teams already have): - Cloud-provider APIs, AWS: SageMaker endpoint listings, Bedrock provisioned-throughput inventory, EKS workload labels matching "ml," "llm," or "ai," EC2 GPU instance types (p4, p3, g4, g5, inf1/2), ECR image repositories with AI/ML tags; GCP: Vertex AI endpoint listings, GKE node-pool GPU labels, Cloud Run AI deployments; Azure: Azure OpenAI deployment listings, AKS GPU node-pool inventory, Azure ML endpoint catalog. - Kubernetes / container registry scans, pods with GPU resource requests, container images from known ML base registries (nvidia/cuda, huggingface, pytorch, tensorflow, vllm), Helm releases with ML-platform chart names (kubeflow, seldon, ray, temporal, airflow), namespaces matching "ml-," "ai-," "llm-," "model-." - Model-registry APIs, MLflow experiment and model-registry endpoints, W&B project/artifact listings, SageMaker model-package groups, Vertex AI model resource names, HuggingFace Hub model repos linked to the org. - IaC repos, Terraform / Pulumi module catalogs: scan for modules tagged "ai-infra," "ml-platform," "llm," "gpu-fleet," "vector-store," "model-registry," "inference-endpoint"; state files listing GPU instance groups or SageMaker resources. - Cloud spend / FinOps signals, GPU spend by tag or cost-allocation group: AWS Cost Explorer filtered on EC2 GPU instance types, Bedrock API call costs, SageMaker training-job costs; GCP billing filtered on Vertex AI and TPU; Azure Cost Management filtered on Azure OpenAI and AML compute. Unexpected GPU spend spikes are shadow-infra signals. - Egress logs to AI provider domains, service mesh, API gateway, VPC flow logs: outbound calls from internal hosts to known AI provider domains (api.openai.com, api.anthropic.com, generativelanguage.googleapis.com, bedrock-runtime..amazonaws.com, cognitiveservices.azure.com, pinecone.io, weaviate.io, qdrant.io) that cannot be attributed to a known inventory entry are shadow-infra candidates. - Vector-store listings, Weaviate REST /v1/schema, Qdrant /collections, Pinecone list-indexes, Milvus collection listings, pgvector schemas: any collection or index not in the inventory is a candidate. - Self-attestation*, a 60-second intake form publicized to platform, SRE, and ML engineering through team channels; amnesty window for disclosing unsanctioned AI/HAI infrastructure already in production.

C) Establish foundational metrics that measure practice maturity and shadow AI infrastructure reduction

Baseline and track a small set of outcome, process, and effectiveness metrics. Keep L1 metrics simple, automatable, and tied to the L1 outcome (shadow infrastructure reduction and inventory coverage of what the org hosts).

Shadow AI infra scoreboard (published quarterly to the executive sponsor): 1. AI/HAI infrastructure instances in inventory (total / sanctioned / provisional / prohibited / awaiting intake), broken out by archetype. 2. New infrastructure instances discovered this quarter and their intake status. 3. Shadow-AI-infra ratio trend (last 4 quarters): instances hosting AI/HAI workloads without a known owner or inventory record. 4. AI Infrastructure Standards Policy and GPU / Accelerator AUP attestation coverage across platform and SRE headcount. 5. Top 5 unmitigated infrastructure risks (TA-flagged, misconfiguration-scanner-flagged, or external-advisory-flagged) with owners and remediation status.

Outcome Metrics (L1)

Process Metrics (leading)

• Discovery cadence, shadow-AI-infra discovery sweeps (cloud-provider APIs + Kubernetes + IaC repos + model-registry APIs + cloud-spend / GPU-spend + egress logs + vector-store listings) run at least monthly.
• Intake SLA, new AI/HAI infrastructure intake triaged within 5 business days; provisional approval within 10 BD for Low-tier archetypes (internal-only, no regulated data).
• Inventory freshness, ≥80% of inventory records reviewed or updated in the last 90 days, tied to last-deployment or last-API-discovery timestamp.
• Working-group cadence, at least monthly; minutes published.

Effectiveness Metrics (business value)

• Platform cycle-time impact, time from "platform team requests intake for a new AI infra component" to "provisional approval issued" should decrease as the program matures; the program is not a bottleneck.
• Reuse rate, % of AI/HAI infrastructure instances using a sanctioned IaC module or reference architecture vs. a bespoke provisioning approach; rising reuse indicates the program scales.
• Avoided-incident stories, documented cases where early discovery caught an AI infrastructure risk before production landing (unencrypted model registry, GPU fleet with overly permissive IAM, vector store with public endpoint, orchestrator with unrestricted tool-call scope).

Success Criteria

• Program charter published and sponsored by an accountable executive (CISO + VP Infrastructure / Head of Platform Engineering).
• AI/HAI infrastructure inventory exists as a single source of truth with ≥90% coverage of discovered infrastructure instances within 12 months, broken out by the seven archetypes.
• Shadow-AI-infra ratio baselined and trending down for two consecutive quarters.
• ≥95% of platform and SRE headcount has acknowledged the AI Infrastructure Standards and GPU AUP policies.
• Quarterly shadow AI infra scoreboard delivered to the executive sponsor with archetype-level breakdown.

Maturity Level 2

Objective: Risk-tier the AI/HAI infrastructure inventory using the canonical rubric, calibrate the program's intensity per tier, and measure practice maturity and shadow-infra reduction per tier, establishing the tier rubric every other Infrastructure-domain L2 practice depends on

At this level, the program stops treating every infrastructure instance the same. Risk tiers drive how deep intake goes, how often reviews happen, what isolation controls are required, and what the sponsor sees on the scoreboard. One unsanctioned Critical-tier customer-facing inference endpoint serving a regulated workload is not equivalent to five Low-tier internal-only vector-store prototypes over public data. Per §9.3 of the v3.0 framing, the rubric established here is the prerequisite for L2 at PC, TA, SR, SA, DR, IR, ST, EH, ML, and IM in the Infrastructure domain.

Dependencies

• SM-Infrastructure L1 (required): inventory, charter, working group, and baseline metrics are the substrate L2 tiers and calibrates.
• PC-Infrastructure L1 (required): the priority compliance map provides several tier dimensions (EU AI Act Art. 15 / Annex III scope, GDPR Art. 32/44–49, sector-specific infrastructure requirements).
• TA-Infrastructure L1 (required): the threat library provides infrastructure threat dimensions (model-extraction surface, inference-endpoint attack surface, GPU-fleet IAM posture, vector-store inversion risk, orchestrator EA/TM/RA exposure).
• SM-Software L2 (alignment): the tier of AI/HAI software hosted is the primary driver of infrastructure tier, inheriting tier from the Software inventory prevents double-tier-derivation; the two inventories must reconcile.
• Supports / unblocks: PC-Infrastructure L2 (tier-driven policy depth), TA-Infrastructure L2 (per-instance deep threat models), SR-Infrastructure L2 (per-tier requirements packs), SA-Infrastructure L2 (tier-conditional reference architectures), DR/IR/ST/EH/ML/IM-Infrastructure L2 (all per-tier calibrated).

Desired Outcomes

• Every AI/HAI infrastructure instance in the inventory carries a risk-tier assignment tied to explicit, auditable dimensions, not reviewer judgment.
• Program intensity is visibly differentiated: Critical-tier customer-facing inference endpoints processing regulated data get per-tenant isolation, encrypted model storage, egress allowlisting, and semi-annual IR; Low-tier internal vector stores over public data get baseline documentation and annual review.
• The quarterly scoreboard splits by tier and archetype; the sponsor can see which tiers and archetype classes are healthy and which are drifting.
• Tier movements (an instance upgraded when it gains customer exposure, regulated data flow, multi-tenant use, or a Critical-tier software artifact is hosted; downgraded when scope shrinks) are tracked, rationale-logged, and sponsor-visible.
• Practice maturity is defensible per tier: "we are mature at Critical-tier inference endpoints and still building at Medium-tier vector stores" is a real, evidenced statement.

Activities

A) Define the AI/HAI infrastructure risk-tier rubric

Four tiers, Critical / High / Medium / Low, assigned from a small set of auditable dimensions specific to AI/HAI infrastructure. This rubric is the canonical source of truth; all downstream Infrastructure-domain L2 practices inherit it.

Tier of AI/HAI software hosted: - Hosts a Critical-tier Software artifact (per SM-Software L2 rubric) → Critical Infrastructure. The infrastructure tier is the maximum of the hosted-software tier and all infrastructure-specific dimensions below.

Multi-tenancy isolation: - Multi-tenant serving infrastructure (multiple teams, products, or customers share inference compute or vector-store namespace) with no per-tenant isolation → elevate tier. Isolation present → no elevation.

Customer exposure: - Customer-facing inference endpoint (public or B2B) → Critical or High. - Internal-employee-facing → neutral. Developer-only or eval-only infrastructure → lower.

Compute scale and concentration: - Single GPU fleet or inference cluster serving multiple Critical-tier AI workloads → elevate (concentration risk: a compromise affects multiple critical workloads simultaneously).

Data classification of data passing through: - Regulated data (PII / PHI / PCI / source code / customer confidential) transits or is stored on this infrastructure at inference, training, or retrieval → Critical or High. - Cross-border data flow (personal data processed in a third country) → elevate; triggers GDPR Art. 44–49 assessment.

Decision-affecting use hosted: - An EU AI Act Annex III high-risk system or a GDPR Art. 22 automated-decisioning system runs on this infrastructure → Critical.

Geographic scope: - Multi-region, cross-border deployment where personal data transits between regions → elevate. Single-region, no cross-border flows → neutral.

Tier derivation is deterministic from the rubric inputs; human overrides are allowed but recorded with rationale and reviewed by the working group.

B) Calibrate program intensity per tier

Publish a tier-treatment matrix, what each tier gets from the Infrastructure-domain program. Every downstream practice inherits this calibration.

Each downstream Infrastructure-domain L2 practice inherits this calibration.

C) Per-tier scoreboard and governance

The L1 shadow AI infra scoreboard becomes tier-aware at L2: - Inventory state reported by tier and by archetype (a Critical-tier customer-facing inference cluster is its own row; the count of Low-tier internal vector-store prototypes is one line). - Shadow-AI-infra ratio reported per tier, a Critical-tier unsanctioned inference endpoint is a headline; a Low-tier one is a line item. - SLA adherence per tier (intake, IR, ST, ML, IM) reported monthly. - Tier-movement log, instances that moved up a tier this quarter (tighter treatment now applies) and those that moved down (with rationale). - Quarterly executive review explicitly discusses tier-balance, is the program's effort matching the risk profile?

Outcome Metrics (L2)

Process Metrics (leading)

• Tier-rubric review cadence, reviewed every 2 quarters; changes change-logged.
• Tier calibration exercise, at least quarterly, a sample of 20 instances re-tiered by a second reviewer; drift tracked.
• Per-tier queue depth monitored; no tier's backlog exceeds a published threshold.
• FedRAMP / regional compliance gating enforced for Critical-tier instances in applicable contexts before go-live.

Effectiveness Metrics (business value)

• Effort allocation match, % of reviewer hours spent on Critical+High tiers vs. Medium+Low; should rise relative to L1.
• Platform throughput at Low/Medium tiers, sanctioned-IaC-module reuse accelerates provisioning time vs. bespoke design.
• Avoided-incident stories where tier-differentiation caught risk earlier (Critical-tier multi-tenant inference cluster caught at DR with missing isolation controls vs. discovered after a model-extraction event).
• Scoreboard drives FinOps, GPU fleet spend by tier informs budget and concentration-risk conversations.

Success Criteria

• Risk-tier rubric published and applied; tier assigned to 100% of inventory.
• Tier-treatment matrix published; downstream practices (PC, TA, SR, SA, DR, IR, ST, EH, ML, IM) calibrated to it.
• Per-tier shadow-AI-infra ratio reported quarterly; Critical-tier unsanctioned infrastructure in production = 0.
• Per-tier SLA adherence ≥90% across practices.
• Tier-movement governance active, changes logged with rationale and reviewed by the sponsor.

Maturity Level 3

Objective: Automate inventory and tier maintenance from cloud-provider APIs, IaC telemetry, and runtime signals; benchmark the program against external infrastructure security peers; and contribute to CNCF, OpenSSF AI, and ML-platform community infrastructure security standards

At this level, the program is predominantly signal-driven rather than ticket-driven. Inventory and tiering update from cloud-provider asset APIs, IaC state events, Kubernetes admission webhook telemetry, model-registry events, and GPU-fleet spend feeds; human review is exception-based. The program benchmarks against external peers on specific infrastructure security metrics, and contributes to the AI/HAI infrastructure security ecosystem through CNCF AI/ML working groups, OpenSSF AI supply-chain security, FinOps Foundation AI infrastructure working groups, and ML platform community standards efforts.

Dependencies

• SM-Infrastructure L2 (required): tiering and calibration must be settled before automation is trustworthy.
• ML-Infrastructure L2+ (required): signals (cloud-provider API events, IaC state events, Kubernetes events, model-registry events, GPU-spend deltas, egress-log telemetry) need the monitoring pipeline behind them.
• EG-Infrastructure L2+ (required): the platform and SRE literacy that lets teams self-attest inventory accurately at the instance level.
• Supports / unblocks: the other 11 Infrastructure-domain practices can move to L3 automation patterns because SM now supplies automated inventory and tier data.

Desired Outcomes

• Inventory accuracy is measured in hours of latency, not months.
• Tier assignments adjust automatically when dimensional inputs change (a new AI workload hosted on an instance, a new customer tenant added, regulated data begins transiting a vector store, a multi-region expansion triggers GDPR Art. 44–49 scope); humans intervene only on exceptions.
• External benchmarking is routine, the program sponsor can answer "how do we compare on AI infrastructure security maturity?" with specific deltas, not hand-waving.
• The organization is a net contributor to the AI/HAI infrastructure security ecosystem, CNCF, OpenSSF AI, FinOps Foundation AI, and ML platform communities reference program outputs.

Activities

A) Continuous inventory and tier automation from cloud, IaC, and runtime signals

• Inventory auto-updates from: cloud-provider asset API events (new SageMaker endpoint created, new Bedrock provisioned throughput, new GKE GPU node pool, new Azure ML compute cluster), IaC state events (new Terraform module with AI-infra tags applied or destroyed), Kubernetes admission webhook events (new pod with GPU resource request or AI/ML image deployed), model-registry events (new model registered or version promoted triggers serving-infra linkage check), GPU-spend deltas (unexplained new spend tag is a shadow-infra signal), egress-log anomalies (new outbound call to AI provider domain not attributable to a known inventory instance), vector-store collection creation events, self-attestation, and intake.
• Tier assignments are rule-based on the L2 rubric inputs; rule changes are versioned and replayable; tier changes auto-trigger downstream practice obligations (e.g., a Medium→Critical upgrade triggers isolation gate, encryption upgrade, IR reconfiguration, FedRAMP/regional-compliance check).
• Human curation handles: new archetype patterns not matching existing rules, ambiguous multi-workload instances, dimensional-input conflicts between Software and Infrastructure tier derivations.
• A data-quality SLO is published: ≥99% of active AI/HAI infrastructure instances correctly tiered within 48 hours of a material change; ≥95% inventory completeness against discovery-source reconciliation.

B) External benchmarking

• Program metrics compared against peer benchmarks via:
• CNCF AI/ML working groups and CNCF Security Technical Advisory Group (TAG Security) AI infrastructure guidance.
• OpenSSF AI supply-chain security working groups (model-registry integrity, CI/CD pipeline security, IaC security for AI infra).
• FinOps Foundation AI infrastructure working groups (GPU fleet cost, concentration risk, efficiency benchmarks).
• MLOps / ML platform community (MLOps World, Apply(ML), Ray Summit, KubeCon ML track).
• Sector ISACs with AI infrastructure security working groups (FS-ISAC, H-ISAC, IT-ISAC).
• Formal peer roundtables (CISO communities with AI infrastructure scope, cloud-provider security practitioner circles).
• A published "how we compare" brief refreshed semi-annually covers: inventory coverage, shadow-AI-infra ratio, per-tier SLA adherence, isolation posture of Critical-tier instances, automation level, IR drift detection rate, ST coverage rate, time from "provisioning request" to "provisional approval."
• Benchmark deltas inform program investment, board-level narrative, and the next year's L2/L3 work priorities.

C) Contribute to CNCF, OpenSSF AI, and ML platform community infrastructure security standards

• Contribute to:
• CNCF AI/ML Working Group and CNCF TAG Security AI infrastructure guidance (inference-endpoint hardening, model-registry security, Kubernetes GPU workload isolation patterns).
• OpenSSF AI (model-supply-chain integrity, AI-specific CI/CD security, model-registry signing and verification).
• FinOps Foundation AI Infrastructure Special Interest Group (GPU fleet governance, concentration-risk frameworks).
• MITRE ATLAS (new TTPs observed in own-operated AI/HAI infrastructure, inference-endpoint extraction, model-registry tampering, orchestrator goal-hijack via control-plane compromise).
• NIST AI RMF Playbook (infrastructure-domain operational guidance).
• CSA AI Safety Initiative (AI infrastructure controls matrix).
• ISO/IEC 42001 AIMS community (infrastructure-domain operational controls).
• Target: minimum 4 substantive contributions per year; quality over volume; every contribution anonymized and legally vetted.

Outcome Metrics (L3)

Process Metrics (leading)

• Automation health, signal-feed freshness and error rate monitored; on-call paged when a feed staleness threshold is exceeded.
• Benchmarking cadence honored (semi-annual brief published on schedule).
• Contribution pipeline always has ≥2 items in-flight (draft, in-review, or being prepared).
• Tier-rule change-log healthy, rule changes versioned, replayable, reviewed quarterly by the working group.

Effectiveness Metrics (business value)

• Sponsor decisions (budget, headcount, GPU fleet sizing) citing benchmark data and tier-level metrics.
• Industry recognition, invitations to CNCF / OpenSSF / FinOps working groups, citations of program contributions, peer adoption of published IaC reference modules.
• Talent, the program attracts experienced platform security and ML platform engineers because of its external profile.
• Faster sanctioned provisioning, time from "platform team proposes new AI infra component" to "provisional approval issued" is industry-leading.

Success Criteria

• Inventory auto-update SLO published and met.
• Tier-assignment automation operational with published rules, replayable change-log, and exception-based human review.
• Semi-annual external-benchmarking brief published to the sponsor with ≥5 peer-comparable metrics.
• ≥4 substantive industry contributions per year, anonymized and cited.
• ROI narrative including external benchmarks delivered to exec/board at least annually.

Key Success Indicators

Level 1: - AI/HAI Infrastructure Assurance program charter published and sponsored by an accountable executive (CISO + VP Infrastructure / Head of Platform Engineering), with a cross-functional working group (Security, Platform/SRE, Cloud Architecture, ML Platform, AI/ML Engineering, Privacy/Legal, FinOps). - AI/HAI infrastructure inventory exists as a single source of truth, covering all seven archetypes (inference endpoint / model-serving cluster, model registry, GPU / accelerator fleet, orchestrator / control plane, vector-store infrastructure, AI-specific CI/CD, feature store / online serving cache). - Shadow AI infrastructure actively discovered each month from cloud-provider APIs, Kubernetes workload signals, IaC repos, model-registry APIs, cloud-spend / GPU-spend signals, egress logs to AI provider domains, and vector-store listings, reconciled against the inventory. - AI Infrastructure Standards Policy and GPU / Accelerator AUP acknowledged by ≥95% of platform and SRE headcount. - Foundational metrics baselined: inventory coverage, shadow-AI-infra ratio, policy attestation, intake SLA; quarterly shadow AI infra scoreboard delivered to the exec sponsor with archetype-level breakdown.

Level 2: - Risk-tier rubric published and applied, 100% of inventory carries a current tier from the seven auditable dimensions (tier of AI/HAI software hosted, multi-tenancy isolation present, customer exposure, compute scale and concentration, data classification of data passing through, decision-affecting use hosted, geographic scope). - Tier-treatment matrix published; downstream practices (PC, TA, SR, SA, DR, IR, ST, EH, ML, IM) calibrated to it. - Quarterly shadow AI infra scoreboard reports per tier and per archetype; Critical-tier unsanctioned infrastructure in production = 0. - Per-tier SLA adherence ≥90% across program activities. - Tier-movement governance operating with logged rationale and sponsor review.

Level 3: - Inventory auto-update latency ≤48 hours for material changes; ≥80% of curation automated; ≥99% inventory completeness against discovery-source reconciliation. - Tier-assignment automation operates on a published, versioned rule set with exception-based human review; tier changes auto-trigger downstream practice obligations within 24 hours. - Semi-annual external-benchmarking brief published to the sponsor, citing ≥5 peer-comparable metrics from CNCF / OpenSSF AI / FinOps Foundation / sector ISACs / ML-platform community. - ≥4 substantive anonymized industry contributions per year (CNCF, OpenSSF AI, FinOps Foundation, MITRE ATLAS, NIST AI RMF, CSA AI Safety Initiative, sector ISACs). - Executive / board ROI narrative refreshed at least annually with external benchmarks and documented avoided-loss examples.

Common Pitfalls

Level 1: - ❌ Inventory is seeded only from "AI infrastructure the ML team told us about", misses GPU spot instances provisioned by researchers, model-serving deployments added to Kubernetes by engineering teams, vector stores stood up as database extensions, and orchestrators repurposed for AI workloads. - ❌ Treating AI/HAI infrastructure as a general CMDB concern without the seven-archetype taxonomy, Critical inference endpoints and Low internal vector stores are conflated; the program cannot tier without re-inventorying. - ❌ Program positioned as a blocker, intake SLA unpublished, platform team provisioning cycles balloon, teams route around the program by deploying from personal cloud accounts or untagged resources. - ❌ Executive sponsor is security-only; VP Infrastructure or Head of Platform Engineering is not a co-owner, so the program lacks infrastructure authority to enforce intake gates. - ❌ Metrics count activity (scans run, tickets closed) instead of outcomes (shadow-AI-infra ratio down, Critical infra with isolation controls trending up). - ❌ No amnesty window, platform engineers hide untracked GPU workloads and inference deployments rather than surface them. - ❌ Discovery relies solely on self-attestation, cloud-spend signals, egress-log anomalies, and Kubernetes GPU workload discovery are never queried; inventory coverage stays low.

Level 2: - ❌ Tier rubric ignores the hosted-software-tier dimension, an inference cluster serving a Critical-tier customer-facing agent is tiered as Low because the infra team did not check the SM-Software inventory. - ❌ Tier-treatment matrix published but not enforced, Critical-tier multi-tenant inference endpoints share namespaces with Medium-tier workloads because isolation was never made a go-live condition. - ❌ Scoreboard reported in aggregate, hiding that Critical-tier shadow infrastructure is present because overall averages look acceptable. - ❌ Tier upgrades get resistance from platform teams because they trigger isolation and encryption requirements, no governance on tier-movement leaves instances under-tiered as workload scope expands. - ❌ FedRAMP / regional compliance gating acknowledged but not operationalized, Critical-tier infrastructure in applicable contexts ships without evidence, creating regulatory exposure. - ❌ Rubric over-engineered, too many sub-dimensions, tier derivation becomes a long committee discussion rather than a deterministic computation.

Level 3: - ❌ Automation runs without a data-quality SLO, signal-driven inventory silently drifts as cloud-provider API schemas change or Kubernetes label conventions shift; platform teams stop trusting it. - ❌ Benchmarking chooses peers that flatter the program (comparing startup GPU-fleet governance to startup benchmarks when operating at regulated-enterprise scale). - ❌ Industry "contributions" are conference talks and whitepapers, not technical artifacts that land in CNCF / OpenSSF / FinOps working-group outputs. - ❌ Automated tiering rules encode historical bias, the inference-endpoint archetype is over-weighted because it was the first discovered; orchestrators and AI-specific CI/CD are systematically under-tiered. - ❌ ROI narrative decouples from reality, external benchmarks cited but the program's own inventory and tier-treatment metrics are stale; sponsor stops trusting the deck. - ❌ Tier-change downstream-trigger automation fires on every node restart or autoscale event, platform teams disable the signal-source rather than fix rule sensitivity.

Practice Maturity Questions

Level 1: 1. Is there a published AI/HAI Infrastructure Assurance program charter with a named executive sponsor (CISO + VP Infrastructure / Head of Platform Engineering), a cross-functional working group, and clear decision rights for approval, block, exception, and go-live across all seven infrastructure archetypes (inference endpoint / model-serving cluster, model registry, GPU / accelerator fleet, orchestrator / control plane, vector-store infrastructure, AI-specific CI/CD, feature store / online serving cache)? 2. Does a single AI/HAI infrastructure inventory exist, seeded from cloud-provider APIs, Kubernetes workload signals, IaC repos, model-registry APIs, cloud-spend / GPU-spend signals, egress logs to AI provider domains, and vector-store listings, covering all seven archetypes with ≥90% coverage of discovered instances within 12 months? 3. Are the L1 outcome metrics baselined and reported quarterly to the sponsor, inventory coverage, shadow-AI-infra ratio (≤15% and trending down), policy attestation (≥95% of platform and SRE), AI/HAI infra instances with named owning team (100%), and known data-exposure events from AI/HAI infrastructure?

Level 2: 1. Is every AI/HAI infrastructure instance in the inventory assigned a risk tier based on the seven auditable dimensions, tier of AI/HAI software hosted, multi-tenancy isolation present, customer exposure, compute scale and concentration, data classification of data passing through, decision-affecting use hosted, and geographic scope, with a published tier-treatment matrix driving differential intensity? 2. Is there a published tier-treatment matrix driving differential controls across PC, TA, SR, SA, DR, IR, ST, EH, ML, IM, with ≥95% of Critical-tier infrastructure instances receiving full-scope treatment (per-tenant isolation, HSM/KMS-rooted encryption, egress allowlisting, least-privilege IAM, semi-annual IR, full ST battery) in the last 12 months? 3. Does the quarterly shadow AI infra scoreboard report per tier and per archetype (with Critical-tier unsanctioned infrastructure in production explicitly tracked at zero), and does tier-movement get logged and reviewed by the program sponsor, including FedRAMP / regional compliance gating for Critical-tier instances in applicable contexts?

Level 3: 1. Does inventory and tier assignment auto-update from live cloud-provider API events, IaC state events, Kubernetes admission webhook telemetry, model-registry events, GPU-spend deltas, and egress-log anomalies, with a published data-quality SLO and ≥80% of curation handled automatically with exception-based human review? 2. Do you publish a semi-annual external-benchmarking brief comparing the program against ≥5 peer-comparable metrics via CNCF / OpenSSF AI / FinOps Foundation / sector ISACs / ML-platform community, and does it drive program investment decisions? 3. Does the program contribute ≥4 substantive, anonymized artifacts per year to the AI/HAI infrastructure security ecosystem (CNCF, OpenSSF AI, FinOps Foundation, MITRE ATLAS, NIST AI RMF, CSA AI Safety Initiative, sector ISACs), and does the exec/board ROI narrative cite external benchmarks?

Document Version: HAIAMM v3.0 Practice: Strategy & Metrics (SM) Domain: Infrastructure Last Updated: 2026-05-14 Author: Verifhai