Monitoring & Logging (ML)

Infrastructure Domain - HAIAMM v3.0

Practice Overview

Objective: Establish the per-archetype logging baseline for all seven AI infrastructure archetypes, operate a small high-signal detection set targeting the top threats from TA-Infrastructure, and produce the evidence trail that satisfies EU AI Act Art. 12 deployer-duty logs, GDPR processor obligations, and applicable cloud-security regulatory requirements, on demand, inside a published SLA.

Description: ML-Infrastructure captures the signals produced by the infrastructure that hosts and serves AI systems, inference endpoints / model-serving clusters, model registries, GPU/accelerator fleets, orchestrator / control planes, vector-store infrastructure, AI-specific CI/CD, and feature stores / online serving caches. For each archetype it specifies the exact events to capture (request events, admin events, rate-limit and abuse events, residual-state events, identity events), the retention window required to satisfy the longest applicable regulation, and the export path that supports auditor review. On top of the logging baseline it operates a bounded, purposeful detection set, each detection tied to a TA-Infrastructure threat with a named owner, a defined query, and an active tuning record. The resulting log corpus serves as the primary evidence artifact for regulatory compliance obligations that apply to AI-hosting infrastructure, including EU AI Act Art. 12 (high-risk system logging), sector cloud regulations (FedRAMP, ISO/IEC 27001 A.12), and GDPR processor obligations.

Context: Logging AI infrastructure is not the same as logging classic compute infrastructure. An inference-endpoint request event requires principal identity, model version, data classification label, token counts, latency, error code, and region alongside the HTTP status code. A GPU job event must carry the workload classification, the residual-state-clearing confirmation, and anomalous-utilization flags, not just CPU/memory metrics. A model registry upload event must capture the signing verification result, the promotion approval chain, and the SLSA provenance attestation reference. A vector-store query event must carry the tenant ID, the query hash (not the raw query text where regulated data may be present), retrieved document IDs, and classification labels. None of this exists by default in cloud-native logging unless someone has explicitly instrumented the archetype's event schema. ML-Infrastructure makes that schema explicit, per archetype, from day one.

Maturity Level 1

Objective: Establish the per-archetype logging baseline for all seven AI infrastructure archetypes, operate a small high-signal detection set targeting the top TA-Infrastructure threats, and produce an on-demand evidence trail satisfying EU AI Act Art. 12, applicable sector cloud regulations, and GDPR processor obligations within a published SLA

At this level, the program makes every production AI infrastructure archetype observable with a defined minimum event schema, closes the most dangerous detection gaps (cross-tenant access, silent model swap, GPU residual-state clearing failure, unsigned model artifact promotion, vector-store mass-extraction, CI/CD pipeline integrity failure, orchestrator workflow injection, shadow inference endpoint), and demonstrates that the resulting log corpus can produce deployer-duty evidence on demand inside the published SLA.

Dependencies

• SM-Infrastructure L1 (required): the inventory and archetype taxonomy define what must be instrumented; ML-Infrastructure L1 cannot baseline what SM has not yet catalogued.
• SA-Infrastructure L1 (required): reference patterns specify where logs originate (inference endpoint API gateway, model registry webhook, GPU scheduler, orchestrator event bus, vector-store query layer, CI/CD pipeline runner, feature-store serving layer); patterns specify what to log.
• EH-Infrastructure L1 (required): hardened identity controls (workload identity, SSO, JIT access, signed artifacts) produce the identity and supply-chain events that ML-Infrastructure captures; without EH instrumentation there are no usable audit-log sources.
• TA-Infrastructure L1 (required): archetype threat library drives detection priority; the high-signal detection set is selected from TA's top threat list.
• Supports / unblocks: IM-Infrastructure L1 (detections become the primary input to the issue backlog); PC-Infrastructure L1 (ML log retention and export path constitutes the compliance evidence trail the PC compliance map requires).

Desired Outcomes

• Every production AI infrastructure archetype emits the per-archetype minimum event schema; no archetype produces logs that are architecturally insufficient for regulatory evidence.
• Retention meets or exceeds the longest applicable compliance window across the active regulatory set; export path demonstrated for each archetype at least annually.
• A small, bounded detection set is live, each detection tied to a named TA-Infrastructure archetype threat, each with an owner, a query, a SLA, and a tuning record; false-positive rate tracked per detection.
• EU AI Act Art. 12 high-risk-system log obligations, applicable sector cloud regulations (FedRAMP IR, ISO/IEC 27001 A.12), and GDPR processor obligations can be satisfied from ML-Infrastructure log stores within the published SLA (on-demand pull ≤24 hours).
• Shadow inference endpoint emergence is detectable from ML signals alone (new inference endpoint appearing in cloud API discovery that does not appear in the SM-Infrastructure inventory surfaces immediately in the detection set).

Activities

A) Establish the per-archetype logging baseline

Define and instrument the minimum event schema for each archetype in the SM-Infrastructure inventory. Each event record includes: event-id / correlation-id, principal (workload identity or human identity), timestamp, archetype tag, region, classification tier, and the archetype-specific fields below. PII scrubbing applied at the logging layer before writing to long-term log stores where applicable.

Inference endpoint / model-serving cluster: - Request event: request-id, principal (service account or end-user token), model name + version, data classification label (from request metadata), input token count, output token count, latency ms, error code, region, tenant-id (for multi-tenant endpoints). - Admin event: configuration change (scaling policy, model swap, rate-limit change), scaling event (scale-up, scale-down, trigger source), model-swap event (from-version, to-version, approver, timestamp). - Rate-limit / abuse event: principal, rate-rule triggered, action taken (throttle / block / alert), request volume in the triggering window.

Model registry: - Upload event: artifact-id, uploader principal, artifact hash, SLSA provenance attestation reference, classification label, timestamp. - Promotion event: artifact-id, from-environment, to-environment, approver principal, eval-gate-result reference, signature verification result (pass / fail), timestamp. - Deletion event: artifact-id, deleting principal, reason, timestamp. - Access event: principal, artifact-id, action (download / read-metadata / list), timestamp, captures who downloaded what model when. - Signature-verification event: artifact-id, verification result, verifying component identity, timestamp.

GPU / accelerator fleet: - Job-schedule event: job-id, workload principal, data classification label of the workload, node-pool assigned, node-id, timestamp. - Residual-state-clearing event: node-id, job-id-prior, clearing method, clearing result (success / failure), timestamp, required before next job starts on the same node. - Anomalous-utilization event: node-id, job-id, utilization metric (GPU memory usage, utilization percentage), threshold exceeded, timestamp.

Orchestrator / control plane: - Workflow-execution event: workflow-id, initiating principal, workflow definition reference, start timestamp, completion status. - Step-principal event: workflow-id, step-id, step type, executing principal (service account for each step), tool or action invoked, timestamp. - Control-plane API audit: API operation, requesting principal, resource affected, result, timestamp, captures all control-plane mutations. - Agent-state event: agent-id, state transition (created / running / paused / terminated / killed), invoking principal, timestamp.

Vector-store infrastructure: - Query event: query-id, principal, tenant-id, query hash (not raw query text where regulated data may be present), retrieved document IDs, classification labels of retrieved documents, retrieval-policy-decision result (allowed / denied / filtered), latency ms, timestamp. - Ingest event: document-id, ingesting principal, classification label, source, timestamp. - Retrieval-policy-decision event: policy name, principal, tenant-id, decision, reason, timestamp.

AI-specific CI/CD: - Pipeline-run event: pipeline-id, triggering principal, repository reference, run result (success / failure), timestamp. - Eval-gate-decision event: pipeline-id, eval suite reference, decision (pass / fail), approver principal, timestamp. - Promotion event: artifact-id, pipeline-id, target environment, approver, timestamp. - Signature-verification event: artifact-id, verification result, timestamp.

Feature store / online serving cache: - Read event: feature-set-id, reading principal, tenant-id, classification label, timestamp. - Write event: feature-set-id, writing principal, source pipeline-id, classification label, timestamp. - Skew-detection event: feature-set-id, skew type (training-serving), magnitude vs. baseline, threshold crossed (yes/no), alert routed.

Admin-audit events (all archetypes): - IAM changes: principal modified, role or policy changed, requestor, timestamp. - Network-policy changes: policy name, change type, requestor, timestamp. - Encryption-key rotation: key-id, rotation event type, initiating principal, timestamp. - Region-config changes: resource affected, from-region, to-region, approver, timestamp.

Identity events (cross-archetype): - SSO sign-ins to model registry, vector-store, orchestrator, and cloud ML consoles: principal, console, result, timestamp. - Workload-identity token issuance: workload identity, audience, issuing authority, timestamp. - JIT access grants: principal, resource, scope, grant duration, approver, timestamp.

Retention and exportability: - Retention window: meets or exceeds the longest applicable requirement (EU AI Act Art. 12 high-risk system logs ≥6 months; GDPR Art. 30 per data class and processing purpose; sector-specific, FedRAMP IR ≥90 days, ISO/IEC 27001 A.12.4 per org's ISMS statement; HIPAA where applicable ≥6 years). Where multiple windows apply, the longest governs. - Export path: JSON or structured export from the log store tested at least annually; on-demand pull SLA ≤24 hours for evidence requests from auditors, regulators, or legal hold. - Log integrity: write-once or append-only storage for admin-audit and deployer-duty evidence tiers; access-control separation between infrastructure operations teams and log store administrators.

B) Operate a small high-signal detection set

L1 target: ≤12 detections, each tied to a TA-Infrastructure archetype threat. Each detection has: owner, detection query, SLA (time-to-IM-ticket), and last-tuned date. False-positive rate tracked per detection; monthly tuning review.

Core detection set:

• Cross-tenant access attempt, a request event on a multi-tenant inference endpoint or vector-store carries a principal matching tenant A but retrieval-policy-decision logs show access to documents or model versions belonging to tenant B; fires on any mismatch between principal tenant-id and retrieved-resource tenant-id.
• Model swap without approval, a model-swap event on an inference endpoint or a promotion event in the model registry where no approver principal is recorded or the promotion event does not carry a matching eval-gate-decision event; fires on any model version flip without an auditable approval chain.
• GPU residual-state clearing failure, a residual-state-clearing event with result = failure; fires immediately; node should be drained; routes to IM-Infrastructure as Critical.
• Unsigned model artifact promoted to production, a promotion event in the model registry where the signature-verification event result is failure or absent; fires on any production-environment promotion without a verified signature.
• Vector-store extraction pattern, anomalous retrieval volume: a principal's query event count or retrieved document count exceeds the archetype's declared operational profile by a configurable multiple (default 10×) within a rolling time window; fires as a mass-extraction alert.
• CI/CD pipeline integrity failure, a signature-verification event on a pipeline artifact result = failure, or a pipeline-run event where the triggering principal is not in the declared CI/CD service-account allowlist; fires on any unsigned or unauthorized pipeline execution touching model promotion.
• Orchestrator workflow injection, a step-principal event where the executing principal for a step does not match the declared service account for that step type in the workflow definition reference; fires on principal-mismatch between declared and observed step executor.
• Shadow inference endpoint detected, a cloud API discovery event (AWS EC2/EKS, GCP GKE, Azure AKS scan) surfaces a new inference service endpoint that does not appear in the SM-Infrastructure inventory; fires on first-seen detection per endpoint-identity pair.

Each detection routes to the IM-Infrastructure backlog on fire; median detection-to-ticket time target ≤1 hour for Critical-tier archetype components.

C) Produce and drill the deployer-duty evidence trail

Wire the log store to the compliance requirements:

• EU AI Act Art. 12 (high-risk system logging for deployer duties): for every infrastructure archetype component assessed as hosting an Annex III high-risk AI system or a customer-facing decision-affecting AI output, confirm that request events, admin events, and identity events are captured and retained at the required window; produce a deployer-duty evidence view for each such component.
• Applicable sector cloud regulations (FedRAMP IR, ISO/IEC 27001 A.12.4, NIST AI RMF MANAGE): infrastructure log completeness and retention are mapped to the active regulatory set in the PC-Infrastructure priority compliance map; gaps between the required event schema and the deployed logging baseline are open IM-Infrastructure findings.
• GDPR processor obligations (Art. 30 records of processing): for any AI infrastructure component processing personal data (inference endpoints serving user inputs, vector stores holding PII-containing embeddings, CI/CD pipelines processing training datasets with personal data), the request and ingest events with classification labels and tenant identifiers constitute operational records of processing; link the log-store retention policy to the Art. 30 record for each component.

Quarterly deployer-duty drill: pull the deployer-duty evidence package for one randomly selected production archetype component per archetype within the published SLA (≤24 hours from request to assembled package). Record drill results; gaps route to IM-Infrastructure.

Outcome Metrics (L1)

Process Metrics (leading)

• Archetype-baseline gap list maintained, every production component scored against its archetype baseline; gaps on IM-Infrastructure backlog with named owner.
• Detection tuning cadence, monthly review per detection; stale or superseded detections retired; new detection candidates from IM post-incident reviews queued.
• Retention / export test cadence, at least annually per archetype, with documented evidence.
• Deployer-duty drill cadence honored, quarterly, covering all archetypes in rotation.

Effectiveness Metrics (business value)

• Incidents detected internally before external notification, trend over quarters.
• Regulator / auditor evidence requests turned around inside the published SLA (≤24 hours).
• Detection-to-prevention conversion, detections that led to a preemptive control change (EH update, SA pattern update) rather than only an incident finding.

Success Criteria

• Per-archetype logging baseline published and instrumented for ≥90% of production AI infrastructure components.
• ≤12-detection high-signal set live (core 8 detections from Activity B), each with owner, detection query, SLA, and monthly tuning record; false-positive rate tracked per detection.
• Retention meets the longest applicable regulatory window for every production component; export path tested at least annually.
• EU AI Act Art. 12, applicable sector cloud regulations, and GDPR processor-obligation evidence wiring documented; quarterly deployer-duty drill executed inside the ≤24-hour SLA.

Maturity Level 2

Objective: Calibrate logging depth and detection set to the SM-Infrastructure L2 tier rubric (Critical / High / Medium / Low); integrate with SIEM for cross-archetype correlation; and feed incident-driven and ST-driven detection updates into a continuous tuning loop

At this level, monitoring intensity matches risk tier. Critical-tier archetype components receive full event corpora retained for the longest regulatory window; Low-tier components receive the baseline only. The SIEM ingests ML-Infrastructure log feeds and executes cross-archetype correlation rules. The detection set evolves continuously from IM-Infrastructure post-incident reviews and ST-Infrastructure findings rather than remaining static. Anomaly detection on request, utilization, and access corpora supplements the rule-based set for Critical and High-tier components.

Dependencies

• ML-Infrastructure L1 (required): per-archetype logging baseline, detection set, and deployer-duty evidence trail must be operational before tier calibration is meaningful.
• SM-Infrastructure L2 (required): tier rubric (Critical / High / Medium / Low) and tier-treatment matrix drive per-tier logging depth and detection priority.
• EH-Infrastructure L2 (required): tier-calibrated hardening controls produce richer identity, supply-chain, and isolation signals; ML-Infrastructure L2 consumes them for anomaly baselining.
• IM-Infrastructure L1+ (required): post-incident reviews from IM feed detection-tuning and new-detection requests into ML's review cycle.
• ST-Infrastructure L1+ (required): security test findings identify gaps in the detection set and drive new detection candidates.
• Supports / unblocks: PC-Infrastructure L2 (tier-calibrated compliance evidence bundles require ML L2 log completeness signals); IM-Infrastructure L2 (richer detections feed the tiered incident playbook with higher-fidelity severity signals).

Desired Outcomes

• Logging intensity visibly differentiates by tier: Critical-tier components retain full event corpora for the longest regulatory window; Low-tier components produce only the baseline schema.
• SIEM integration enables cross-archetype correlation, an attacker session interacting with the model registry and then pivoting to an inference endpoint surfaces as a correlated detection rather than isolated per-archetype alerts.
• The detection set evolves quarterly from a defined, governed feedback loop (IM post-incident reviews + ST findings + external advisory updates from ATLAS, CNCF security advisories, cloud-provider bulletins).
• Anomaly detection establishes behavioral baselines for Critical and High-tier components; rule-based detections that fire frequently without incident are refined or retired.

Activities

A) Tier-calibrated logging depth

Apply the SM-Infrastructure L2 tier-treatment matrix to logging configuration:

• Critical tier: full request event corpus (including request content or content hash) retained for the longest regulatory window; full admin, identity, and supply-chain event corpora retained; all detections tuned to the component; per-tenant isolation enforced at the log store (Critical-tier component logs partitioned from other tier logs).
• High tier: full request and admin event corpus retained; identity events at full fidelity; core detections active.
• Medium tier: request-event summaries (no content, tokens/latency/error retained) for regulatory window; standard admin events; shadow-endpoint and clearing-failure detections active.
• Low tier: baseline logging schema only; shadow-endpoint detection only.

For every Critical-tier component, the ML-Infrastructure log store is the primary source for PC-Infrastructure compliance evidence bundles (per PC-Infrastructure L2 staleness thresholds: ML logging-baseline validation ≤30 days for Critical-tier).

B) SIEM integration and cross-archetype correlation

• Ingest all tier-appropriate ML-Infrastructure log feeds into the SIEM.
• Author and maintain at least three cross-archetype correlation rules at L2:
• Registry-to-endpoint pivot: a promotion event in the model registry for artifact-id X correlates with a model-swap event on an inference endpoint within the same time window where the swap was not pre-approved in the change calendar, fires a unified incident.
• Identity pivot on supply-chain compromise: a CI/CD pipeline integrity failure (unsigned pipeline execution) correlates with a subsequent model promotion event from the same pipeline-id within the same session window, escalates to Critical regardless of tier.
• Vector-store mass-extraction + JIT access anomaly: a vector-store extraction pattern detection correlates with a JIT access grant to the vector-store admin interface from the same principal or a related principal in the same time window.
• Cross-archetype correlation alerts route to IM-Infrastructure at the tier of the highest-tier component involved.

C) Detection tuning loop: IM post-incident and ST feedback

Operate a quarterly detection review cycle:

• IM-Infrastructure post-incident reviews that touch a logging or detection gap generate a detection-update request (new detection, tuned query, or retired false-positive rule).
• ST-Infrastructure test findings that are not caught by the current detection set generate a detection-gap finding routed to ML-Infrastructure.
• External advisory updates (MITRE ATLAS new techniques, CNCF security advisories, cloud-provider security bulletins, ATLAS updates for AI infrastructure TTPs) are assessed quarterly; each applicable update either adds a new detection candidate or updates an existing detection's query.
• Monthly anomaly-baseline refresh for Critical and High-tier components: normal behavior baseline (request volume, retrieval volume, GPU utilization distribution, model-swap frequency) refreshed from the previous 30-day window; anomaly threshold auto-tunes to maintain target false-positive rate.
• Each detection has a last-tuned date and a false-positive rate; detections that have not fired a true positive in 90 days or that exceed 20% false-positive rate are reviewed for retirement at the quarterly cycle.

Outcome Metrics (L2)

Process Metrics (leading)

• Anomaly baseline refresh cadence honored (monthly for Critical; quarterly for High).
• SIEM correlation rule health monitored (rules producing alerts within expected frequency; no rule silent for >90 days without investigation).
• Detection-gap review calendar on schedule; IM and ST feedback queues reviewed at each quarterly cycle.
• Retention-tier calibration reconciled with SM-Infrastructure inventory tier changes, when a component is re-tiered, logging depth updated within 14 days (Critical re-tier) or 30 days (other tiers).

Effectiveness Metrics (business value)

• True-positive rate improvement quarter-over-quarter as detection tuning loop matures.
• Cross-archetype correlation incidents that would have appeared as isolated per-archetype findings at L1 now unified, reduces IM mean-time-to-understand for complex infrastructure incidents.
• Compliance evidence bundle ML validation element completing inside PC-Infrastructure staleness threshold with no manual intervention.

Success Criteria

• Tier-calibrated logging depth applied to 100% of SM-Infrastructure inventory with current tier assignments; Critical-tier full corpus retention confirmed.
• SIEM integration live; ≥3 cross-archetype correlation rules active.
• Quarterly detection tuning loop operating with IM-Infrastructure and ST-Infrastructure feedback; ≥1 net change per cycle.
• ≥90% of Critical/High-tier components with anomaly-detection baselines; FP rate tracked and trending down.
• ML logging-baseline validation element fresh (≤30 days) for all Critical-tier components in PC-Infrastructure compliance evidence bundles.

Maturity Level 3

Objective: Express detections as code with automated deployment; apply anomaly detection to request, access, and utilization corpora; contribute anonymized detection signatures and telemetry schemas to CNCF observability, OpenSSF AI, MITRE ATLAS, and sector ISACs

At this level, detections are version-controlled software artifacts deployed through CI/CD. Anomaly detection on the full request, access, and utilization corpora for Critical and High-tier components surfaces novel attack patterns that rule-based detections miss. The detection library and telemetry schemas are contributed back to the AI-assurance ecosystem, CNCF observability working groups, OpenSSF AI Infrastructure, MITRE ATLAS, and sector ISACs.

Dependencies

• ML-Infrastructure L2 (required): tier-calibrated logging, SIEM integration, and detection tuning loop must be mature before automation is trustworthy.
• PC-Infrastructure L3 (required): continuous compliance attestation pipeline consumes ML-Infrastructure log signals; attestation SLO depends on ML L3 log freshness guarantees.
• SM-Infrastructure L3 (required): automated inventory and tier-maintenance events trigger automated detection-set updates (new Critical-tier component auto-provisions the full detection set for that archetype).

Desired Outcomes

• Detection-as-code: every detection is a version-controlled artifact deployed via the same CI/CD pipeline as the AI infrastructure it monitors; a merge request to a detection changes it in production.
• Anomaly detection on request, access, and utilization corpora identifies attacker-session patterns, unusual retrieval-distribution shifts, and novel model-swap sequences that elude rule-based detections.
• The organization is a net contributor to AI-infrastructure observability standards: CNCF observability working group semantic conventions, OpenSSF AI Infrastructure telemetry schemas, MITRE ATLAS detection mitigations.
• Industry peers can adopt contributed detection schemas and telemetry standards without significant adaptation.

Activities

A) Detection-as-code

• Every detection in the set expressed as a versioned, tested artifact in source control (detection query + metadata: owner, SLA, ATLAS-tactic tag, false-positive threshold, last-test-result).
• Detection CI/CD pipeline: changes to detection code trigger a test suite (unit tests over synthetic log data, integration tests against a log replay environment) before production deployment.
• Detection deployment via the same change-management pipeline as AI infrastructure IaC; detection changes are reviewed, not applied ad hoc in the SIEM console.
• Detection coverage automatically checked on SM-Infrastructure inventory change events: when a new archetype component is registered or a component is re-tiered to Critical, the automation verifies the required detection set is active for that component and opens a gap finding if not.

B) Anomaly detection on AI infrastructure corpora

Apply anomaly models to the request, access, and utilization corpora for Critical and High-tier archetype components:

• Request-volume anomaly on inference endpoints: sessions or service accounts whose request-volume or token-count distribution is a statistical outlier from normal operational profiles (abuse patterns, prompt-bomb campaigns, scraping sessions).
• Retrieval-distribution anomaly on vector stores: query sessions whose retrieved-document distribution shifts from baseline on a rolling window (potential systematic extraction of a specific document corpus or tenant's data).
• Model-swap frequency anomaly on model registry: model promotion frequency that deviates from the established baseline (e.g., multiple unscheduled model swaps in a short window suggesting supply-chain tampering).
• GPU utilization anomaly: GPU node utilization patterns that deviate from the baseline for the registered workload (potential cryptomining, unauthorized training job, or resource hijacking).

Anomaly model outputs feed the same detection-to-IM-ticket pipeline as rule-based detections; anomaly severity is tagged to the component's tier. Anomaly models retrained monthly.

C) Contribute detection signatures and telemetry schemas

• CNCF observability working group, contribute OpenTelemetry semantic conventions for AI infrastructure event types: inference-endpoint request spans, model-registry promotion traces, GPU-job traces, vector-store query spans, orchestrator workflow traces; schema-spec format compatible with the OTel specification process.
• OpenSSF AI Infrastructure Working Group, contribute detection-pattern examples for supply-chain events (unsigned model promotion, CI/CD pipeline integrity failure) and inference-infrastructure events (shadow endpoint, cross-tenant access) as OpenSSF reference detection modules.
• MITRE ATLAS, for each detection in the set that corresponds to an ATLAS tactic/technique, propose or validate an AML.M00xx mitigation entry (detection-based mitigation type); priority: cross-tenant access (TA0009 Collection), model swap without approval (TA0010 ML Supply Chain Compromise), orchestrator workflow injection (TA0008 Defense Evasion), GPU residual-state leakage (TA0013 Exfiltration).
• Sector ISACs (FS-ISAC, H-ISAC, IT-ISAC AI working groups), share anonymized, generalized detection signatures for AI infrastructure threats; target ≥12 signatures per year.
• Target: ≥2 telemetry-standard contributions per year and ≥12 ISAC detection signatures per year; all contributions anonymized, legally vetted, and maintained.

Outcome Metrics (L3)

Process Metrics (leading)

• Detection CI/CD pipeline health monitored; failed detection-deployment builds are on-call paged the same as production infrastructure failures.
• Anomaly model drift monitored (detection model performance against labeled incident records); retraining SLO met monthly.
• Contribution pipeline active, ≥2 telemetry-standard items in-flight (draft, in-review, or submitted) at any time.
• ISAC submission cadence, at least one anonymized detection signature submitted per month.

Effectiveness Metrics (business value)

• True-positive incidents surfaced first by anomaly detection (vs. rule-based or external notification), trending up quarter-over-quarter.
• Detection deployment lead time (merge to production) measured in hours, not weeks.
• Industry adoption of contributed detection schemas and telemetry conventions tracked; citations, integrations, or standards-body acknowledgments documented.
• ISAC partner organizations citing contributed signatures in their own detection catalogs.

Success Criteria

• ≥90% of the detection set expressed as version-controlled, CI/CD-deployed artifacts; detection changes reviewed and deployed through the same change pipeline as AI infrastructure IaC.
• Detection coverage auto-verified for 100% of new or re-tiered SM-Infrastructure inventory entries within 24 hours.
• ≥90% of Critical/High-tier components with anomaly detection active; anomaly models retrained monthly on schedule.
• ≥2 telemetry-standard contributions per year to CNCF observability working group or OpenSSF AI Infrastructure; ≥12 anonymized detection signatures per year to sector ISACs; ≥2 ATLAS AML.M00xx mitigation entries proposed or validated.

Key Success Indicators

Level 1: - Per-archetype logging baseline published and instrumented for ≥90% of production AI infrastructure components, covering request events, admin events, supply-chain events, residual-state events, and identity events per archetype with the fields specified in Activity A. - ≤12-detection high-signal set live (core 8: cross-tenant access, model swap without approval, GPU clearing failure, unsigned model promotion, vector-store extraction, CI/CD integrity failure, orchestrator workflow injection, shadow inference endpoint), each with owner, detection query, SLA, and monthly tuning record. - Retention meets the longest applicable regulatory window; export path tested annually; EU AI Act Art. 12, applicable sector cloud regulations, and GDPR processor-obligation evidence wiring documented. - Quarterly deployer-duty drill executed inside the ≤24-hour SLA; gaps routed to IM-Infrastructure.

Level 2: - Tier-calibrated logging depth applied per SM-Infrastructure L2 tier-treatment matrix; Critical-tier components retain full event corpora at the longest regulatory window. - SIEM integration live with ≥3 cross-archetype correlation rules; ≥90% of Critical/High-tier components with anomaly-detection behavioral baselines and FP rate trending down. - Quarterly detection tuning loop operating from IM-Infrastructure post-incident and ST-Infrastructure finding inputs; ≥1 net detection change per cycle. - ML logging-baseline validation element fresh (≤30 days) for all Critical-tier components in PC-Infrastructure compliance evidence bundles.

Level 3: - ≥90% of the detection set expressed as version-controlled, CI/CD-deployed artifacts; detection coverage auto-verified on SM-Infrastructure inventory changes within 24 hours. - ≥90% of Critical/High-tier components with anomaly detection active; models retrained monthly. - ≥2 telemetry-standard contributions per year; ≥12 anonymized ISAC detection signatures per year; ≥2 MITRE ATLAS AML.M00xx detection-mitigation entries proposed or validated.

Common Pitfalls

Level 1: - ❌ Logging baseline defined for inference endpoints and model registry but GPU fleet, orchestrator, and vector-store archetypes are left on generic cloud monitoring, residual-state clearing failures, workflow injection events, and retrieval-policy decisions are invisible until an incident forces forensic reconstruction from incomplete telemetry. - ❌ Model registry upload and promotion events are logged at the HTTP level (status code + endpoint) but not at the artifact level, the "who downloaded what model when" audit trail required for supply-chain incident investigation does not exist. - ❌ Vector-store query events log the raw query text instead of a query hash when regulated data may be present in the queries, the log store becomes a secondary repository of regulated data with weaker access controls than the primary data store. - ❌ Detection set includes GPU clearing failure and shadow endpoint but not model swap without approval, a silent model version flip on a Critical-tier inference endpoint goes undetected until output-integrity regression surfaces downstream. - ❌ Deployer-duty evidence view exists as a template document but is never populated for specific archetype components, the quarterly drill is skipped because the infrastructure team assumes the cloud-native logs are sufficient without testing the export path. - ❌ Retention meets ISO/IEC 27001 A.12.4 but not EU AI Act Art. 12 high-risk-system windows, evidence requests for Annex III-adjacent inference endpoints cannot be satisfied because the relevant log tier was retained for only 30 days.

Level 2: - ❌ Tier-calibrated logging configured at deployment time but not maintained, when an inference cluster is re-tiered from Medium to Critical, logging depth is not updated; full corpora are absent for the cluster when the first Critical-tier incident fires. - ❌ SIEM correlation rules are built once and never validated, a cross-archetype correlation rule that has not fired in 90 days may be broken (log format changed after a Kubernetes upgrade) rather than evidence that no correlatable events occurred. - ❌ Anomaly baselines established at cluster launch and never refreshed, normal usage evolves as models are swapped and traffic patterns change; stale baselines produce FP spikes that overwhelm the IM team. - ❌ Detection tuning loop exists on paper but IM and ST feedback never feeds into the review cycle, the same false-positive GPU-utilization alert remains in the set for quarters because the quarterly process has no dedicated infrastructure-security owner. - ❌ Cross-archetype correlation produces high-severity unified incidents that land in IM-Infrastructure without the component-archetype context needed for triage, correlation helps detection but hurts triage unless the correlated event package includes links to the individual archetype findings.

Level 3: - ❌ Detection-as-code pipeline deployed but detection tests use synthetic log data that does not resemble production infrastructure log formats, tests pass in CI and detections fail silently in production after a log-schema change in the cloud provider's managed service. - ❌ Anomaly models retrained on the full log corpus including attacker-session logs from past incidents, poisoned baseline; the model learns to treat GPU residual-state clearing failures from past incidents as normal operational events. - ❌ Contributed telemetry schemas published as point-in-time artifacts and then diverge from internal practice, external adopters build against v1.0 OpenTelemetry semantic conventions while the org operates v1.3 internally; trust erodes. - ❌ ISAC detection signatures generalized to the point of uselessness, partner organizations cannot implement cross-tenant access or vector-store extraction detections without reconstructing the tenant-id and retrieval-volume context removed for anonymization. - ❌ ATLAS AML.M00xx contribution targets treated as compliance checkboxes, entries proposed but never followed through to publication because internal legal review for the orchestrator-workflow-injection technique creates indefinite delay.

Practice Maturity Questions

Level 1: 1. Has a per-archetype logging baseline been published specifying the minimum event schema, fields, retention window, and export path for each AI infrastructure archetype in the SM-Infrastructure inventory (inference endpoint, model registry, GPU fleet, orchestrator, vector store, AI CI/CD, feature store), and has compliance of each production component been measured against it within the last quarter? 2. Is a high-signal detection set of ≤12 detections active, each with a named owner, detection query, SLA, and last-tuned date, including cross-tenant access, model swap without approval, GPU residual-state clearing failure, unsigned model artifact promotion, vector-store extraction pattern, CI/CD integrity failure, orchestrator workflow injection, and shadow inference endpoint, with false-positive rates tracked per detection and monthly tuning reviews occurring? 3. Has the evidence trail for EU AI Act Art. 12, applicable sector cloud regulations, and GDPR processor obligations been wired to the ML-Infrastructure log store, and has a quarterly deployer-duty drill confirmed that the evidence package for a randomly selected production component can be assembled within the ≤24-hour SLA?

Level 2: 1. Is tier-calibrated logging depth applied per the SM-Infrastructure L2 tier-treatment matrix, Critical-tier components retaining full event corpora at the longest regulatory window, Low-tier components receiving baseline only, and is this calibration automatically updated when a component is re-tiered? 2. Is the SIEM ingesting ML-Infrastructure log feeds with ≥3 cross-archetype correlation rules active (covering at minimum registry-to-endpoint pivot, identity pivot on supply-chain compromise, and vector-store mass-extraction plus JIT access anomaly), and is a quarterly detection tuning cycle operating from IM-Infrastructure post-incident and ST-Infrastructure finding inputs? 3. Are ≥90% of Critical/High-tier components running anomaly-detection baselines with behavioral profiles refreshed monthly and FP rates tracked and trending down, and is the ML logging-baseline validation element completing inside the ≤30-day staleness threshold for all Critical-tier components in PC-Infrastructure compliance evidence bundles?

Level 3: 1. Are ≥90% of detections expressed as version-controlled, CI/CD-deployed code artifacts with automated test coverage against realistic synthetic log data, and is detection coverage auto-verified for 100% of new or re-tiered SM-Infrastructure inventory entries within 24 hours of the inventory change event? 2. Are ≥90% of Critical/High-tier components running anomaly detection on request, access, and utilization corpora, with anomaly models retrained monthly on production log data and anomaly-model alerts feeding the IM-Infrastructure incident backlog through the same detection-to-ticket pipeline as rule-based detections? 3. Has the program contributed ≥2 telemetry-standard artifacts per year to CNCF observability working group or OpenSSF AI Infrastructure and ≥12 anonymized detection signatures per year to sector ISACs, and has it proposed or validated ≥2 MITRE ATLAS AML.M00xx detection-mitigation entries, with contributions maintained current and external adoption tracked?

Document Version: HAIAMM v3.0, 2026-05-14, Verifhai Practice: Monitoring & Logging (ML) Domain: Infrastructure Last Updated: 2026-05-14 Author: Verifhai