Monitoring & Logging (ML) - Vendors Assessment

Assessment questionnaire for measuring maturity. Answer each question honestly based on current, implemented practices.

v3.0 canonical: This questionnaire is authored against the v3.0 source-of-truth one-pager ../practices/ML-Vendors-OnePager.md and the §10.2 priority compliance map in ../HAIAMM-v3.0-Framing.md. Through-lines: EU AI Act Art. 26 deployer duties · GDPR Art. 30 records of processing · ISO/IEC 42001 AIMS operational evidence · shadow AI prevention as the primary L1 outcome.

Monitoring & Logging (ML) - Vendors Domain

HAIAMM Assessment Questionnaire v3.0

Practice: Monitoring & Logging (ML) Domain: Vendors Purpose: Assess organizational maturity in establishing the AI-vendor logging baseline per archetype, operating a high-signal detection set including shadow-AI detections targeting TA-Vendors threats, and producing the evidence trail that satisfies EU AI Act Art. 26 deployer duties and GDPR processor obligations on demand.

Instructions

Answer each question honestly based on current, implemented practices (not plans or aspirations)
Each question has two components: Evidence (what you have done) and Outcome Metrics (how well it is working)
Scoring uses 4 tiers: Fully Mature (1.0), Implemented (0.67), Partial (0.33), Not Implemented (0.0)
Answer progressively, Complete all Level 1 questions before Level 2
Level progression, Achieve ALL questions at a lower level before advancing
Baseline first, Record current metric values before setting targets

Scoring Methodology

Score	Label	Criteria
1.0	Fully Mature	Evidence complete + ≥3 outcome metrics meet targets
0.67	Implemented	Evidence complete + 2 outcome metrics meet targets
0.33	Partial	Evidence partially complete + <2 outcome metrics meet targets
0.0	Not Implemented	No evidence of the practice

Level score = average of the three question scores for that level. Overall ML-Vendors score = weighted average: L1 × 0.5 + L2 × 0.3 + L3 × 0.2.

Maturity Level 1

Objective: Establish the AI-vendor logging baseline per archetype, operate a small high-signal detection set including shadow-AI detections, and ensure evidence retention meets deployer-duty requirements, providing visibility into what sanctioned AI vendors are doing and what shadow AI is attempting.

Question 1: Per-Archetype AI-Vendor Logging Baseline

Q1.1: Is a per-archetype logging baseline published specifying the minimum events, fields, retention period, and export path for each of the five AI vendor archetypes, and has compliance of each active integration been measured against it within the last quarter?

Evidence Required: - [ ] Published baseline for each AI vendor archetype with common cross-archetype fields: user identity (SSO subject), timestamp, vendor, integration identifier, action class (access/data-flow/config-change/admin-audit), data-class tag where applicable, trace/correlation ID - [ ] Consumer GenAI: org-tenant session events, admin-audit events, content-filter actions - [ ] AI-embedded SaaS: AI-feature activation/deactivation, per-workspace usage, admin-audit for AI-feature toggles, parent-vendor audit feed ingested - [ ] AI coding assistant: IDE policy-match events, prohibited-path/data-marker blocks, license-usage attribution - [ ] AI API / model: prompt/response logging at the internal proxy with PII scrubbing per SR-Vendors; model-version, region, latency, error; admin-level key-use attribution - [ ] AI agent / automation platform: session start/end, tool-call events (tool/parameters/outcome), HITL gate invocations, permission-denials, session outcome - [ ] Retention meets or exceeds the longest applicable requirement per §10.2 priority compliance map (EU AI Act Art. 26 high-risk logs, GDPR records-of-processing, ISO/IEC 42001 AIMS) per data-class; export path tested at least annually - [ ] Compliance audit within the last quarter; gaps on backlog with named owner

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | % active AI vendor integrations meeting per-archetype logging baseline | % | % | ≥90% | ☐ | | | Export path test result, on-demand pull time | d | d | ≤2 business days | ☐ | | | % integrations with retention meeting longest applicable regulation | % | % | 100% | ☐ | | | Archetype-baseline gap count on backlog with named owner | ___ | ___ | 0 unowned gaps | ☐ | |

Metric Collection Guidance: - Logging baseline compliance: Cross-reference each active AI vendor integration against the published baseline checklist for its archetype. Count integrations meeting all required fields divided by total integrations. Source: logging configuration audit. - Export path test: Time the annual export path test for each archetype. Measure request to assembled package. Source: export path test records. - Retention compliance: Compare configured retention against longest applicable regulation per §10.2 for each integration. Source: retention policy audit. - Gap count: Count open backlog findings tagged "logging-baseline-gap" with named owner. Source: IM-Vendors or backlog.

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No per-archetype logging baseline)

Evidence Location: _____ Validation Date: ____ Notes: ______

Question 2: High-Signal Detection Set Including Shadow-AI Detections

Q1.2: Is a high-signal detection set of ≤12 detections active, each tied to a TA-Vendors archetype threat, including shadow-AI egress, no-train flag change, agent tool-call violations, HITL bypass attempt, vendor breach advisory, and AI-feature toggle change, with false-positive rates tracked per detection and monthly tuning reviews occurring?

Evidence Required: - [ ] Detection registry: each entry includes owner, detection query, SLA, archetype tag, last-tuned date, and false-positive rate - [ ] Shadow-AI egress detection active: traffic to unsanctioned AI vendor domains from managed endpoints or networks - [ ] Shadow-AI SaaS sign-in detection active: SSO or IdP activity against an AI SaaS not in the sanctioned catalog - [ ] Consumer AI personal-account sign-in from org endpoints detection active: domain/email pattern anomaly - [ ] Bulk content paste/upload to AI vendor domains detection active: DLP rule-set match on volume and data-class - [ ] API-proxy anomalies detection active: prompt/response volume spikes, model-version unexpected change, PII scrubbing failures - [ ] Agent tool-call violations detection active: tool calls outside allowlist or outside scoped parameters - [ ] Agent HITL bypass attempt detection active: HITL gate invoked and declined followed by retry patterns in same session - [ ] Parent-SaaS AI-feature toggle change detection active: unexpected enablement of an AI feature in an approved parent SaaS - [ ] No-train flag change detection active: vendor admin-audit event where the training-toggle state changes - [ ] AI-vendor breach / advisory detection active: external-intel feed match against vendor inventory - [ ] Egress to new AI vendor domain first-seen detection active: discovery signal for SM-Vendors inventory - [ ] Monthly tuning review log with review dates and changes; false-positive rate tracked per detection

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | Total active detections in detection registry | ___ | ___ | ≤12 | ☐ | | | Median detection-to-IM-ticket time for critical detections | h | h | ≤1h | ☐ | | | % detections with FP rate tracked and last-tuned date ≤30 days | % | % | 100% | ☐ | | | Monthly tuning reviews completed (last 12 months) | /12 | /12 | 12/12 | ☐ | |

Metric Collection Guidance: - Active detections: Count entries in detection registry with status = active. Source: detection registry. - Detection-to-ticket time: Measure from detection alert to IM-Vendors ticket creation for critical-tier detections. Source: alert → ticket telemetry (P50). - FP rate tracked: Count detections with FP rate populated and last-tuned date ≤30 days divided by total active detections. Source: detection tuning log. - Monthly reviews: Count monthly tuning review sessions with documented output in the last 12 calendar months. Source: detection tuning log.

Evidence Location: _____ Validation Date: ____ Notes: ______

Question 3: Deployer-Duty Evidence Trail

Q1.3: Has a deployer-duty evidence view been produced for every high-risk AI vendor integration, and has a quarterly drill confirmed it can be assembled within 2 business days on request, with EU AI Act Art. 26, GDPR Art. 30, and ISO/IEC 42001 AIMS obligations traceable to the ML log store?

Evidence Required: - [ ] Deployer-duty evidence view per high-risk AI vendor integration: pulls intake approval, REM record, DR decision, IR config records, ST pass records, ML logs, incident records, and AUP coverage for associated users - [ ] GDPR Art. 30 records-of-processing entries reference ML retention evidence for each integration processing personal data - [ ] ISO/IEC 42001 AIMS evidence assets linked from ML storage (or identified as gaps with IM-Vendors findings) - [ ] EU AI Act Art. 26 deployer-duty obligations: for every high-risk AI vendor integration, logging and human-oversight obligations traceable to ML log store retention records - [ ] Quarterly deployer-duty drill records showing: integration selected, drill start time, evidence assembly time, gaps found, disposition - [ ] §10.2 priority compliance map referenced in retention evidence requirements for each high-risk integration

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | Quarterly deployer-duty drills completed (last 4 quarters) | /4 | /4 | 4/4 | ☐ | | | Evidence assembly time in most recent drill | d | d | ≤2 business days | ☐ | | | % high-risk AI vendor integrations with deployer-duty evidence view produced | % | % | 100% | ☐ | | | Drill gaps routed to IM-Vendors with named owner | ___ | ___ | 0 unowned | ☐ | |

Metric Collection Guidance: - Drill completion: Count quarterly drill sessions in the last 4 quarters with documented output. Source: drill records. - Assembly time: Record the most recent drill's evidence assembly time. Source: drill records. - Evidence view coverage: Count high-risk AI vendor integrations with a deployer-duty evidence view on file divided by total high-risk integrations. Source: evidence registry × SM-Vendors inventory. - Unowned gaps: Count drill-gap findings without an owner assigned. Source: IM-Vendors backlog.

Evidence Location: _____ Validation Date: ____ Notes: ______

Maturity Level 2

Objective: Add anomaly detection on AI-vendor behavior, correlate across vendors in a multi-vendor integration graph, and automate deployer-duty evidence generation for Critical-tier integrations.

Question 4: Anomaly Detection on AI-Vendor Behavior

Q2.1: Are ≥90% of Critical-tier AI vendor integrations running anomaly detection with established behavioral baselines, a tuned FP rate trending downward, and escalation to IM-Vendors with baseline-vs.-observed context when anomalies fire?

Evidence Required: - [ ] Behavioral baselines established for Critical-tier integrations: prompt volume, tool-call patterns, egress volume, time-of-day patterns, per integration and archetype - [ ] Monthly baseline refresh cadence honored; last-refresh date per integration - [ ] Anomaly detection active for Critical-tier integrations; anomaly alerts escalate to IM-Vendors with baseline-vs.-observed context (not just a threshold breach flag) - [ ] FP rate tracked and tuned per integration; evidence of FP rate trending downward quarter-over-quarter - [ ] Detections exceeding 20% FP reviewed at quarterly cycle; stale baselines (not refreshed in >30 days) flagged as a process gap

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | % Critical-tier integrations with anomaly-detection baselines established | % | % | ≥90% | ☐ | | | Anomaly-detection FP rate for Critical-tier (trend direction) | ___ | ___ | trending down | ☐ | | | % anomaly baselines refreshed monthly, Critical-tier (last 3 months) | % | % | 100% | ☐ | | | % anomaly alerts escalated to IM-Vendors with baseline-vs.-observed context | % | % | 100% | ☐ | |

Metric Collection Guidance: - Baseline establishment: Count Critical-tier integrations with documented behavioral baseline divided by total Critical-tier integrations. Source: detection telemetry. - FP rate trend: Compare anomaly FP rate current quarter vs. prior quarter for Critical-tier integrations. Source: alert telemetry. - Refresh cadence: For each Critical-tier integration baseline, verify last-refresh date ≤30 days. Source: anomaly baseline refresh records. - Context in alerts: Count anomaly alerts with baseline-vs.-observed context field populated divided by total anomaly alerts. Source: IM-Vendors ticket records.

Evidence Location: _____ Validation Date: ____ Notes: ______

Question 5: Cross-Vendor Graph Analysis and Automated Deployer-Duty Evidence

Q2.2: Is the cross-vendor integration graph refreshed at least weekly with anomalous edges and unexpected vendor intermediaries automatically surfacing as detections, and is automated deployer-duty evidence assembled for 100% of Critical-tier integrations, enabling regulator inquiry turnaround within 3 business days?

Evidence Required: - [ ] Multi-vendor integration chains mapped as a graph; graph refreshed at least weekly - [ ] Graph anomalies: unexpected edges (new vendor-to-vendor data flows), changing centralities, new intermediate vendors surface automatically as detections routed to IM-Vendors - [ ] Shadow-AI detection graph signals: unexpected AI-vendor egress from previously non-AI endpoints surfaces from graph analysis - [ ] Automated deployer-duty evidence pipeline: for Critical-tier integrations, the evidence view (logs, human-oversight assignments, disclosures, Art. 26 checklist attestations) auto-assembles on a schedule; pipeline health monitored - [ ] Evidence pipeline health: % Critical integrations producing a fresh evidence artifact within the last scheduled cycle; pipeline alerts when a Critical integration's evidence is stale - [ ] Documented evidence of a regulator or audit inquiry turned around within 3 business days using auto-assembled evidence

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | Cross-vendor integration graph refreshed weekly | ___ | ___ | yes, weekly | ☐ | | | % Critical-tier integrations with automated deployer-duty evidence assembled | % | % | 100% | ☐ | | | Regulator-inquiry turnaround time (last 4 quarters) | d | d | ≤3 business days | ☐ | | | Graph anomaly detections routed to IM-Vendors (last 90 days, or no applicable events) | ___ | ___ | active routing | ☐ | |

Metric Collection Guidance: - Graph refresh: Verify graph telemetry shows weekly refresh cadence honored. Source: graph telemetry. - Automated evidence: Count Critical-tier integrations with an auto-assembled deployer-duty evidence artifact (log citations, oversight assignments, disclosure records) produced within the last scheduled cycle divided by total Critical-tier integrations. Source: evidence telemetry. - Regulator turnaround: For each regulator or auditor inquiry in the last 4 quarters, measure calendar days from request to evidence package delivery. Source: inquiry log. - Graph anomaly routing: Verify graph anomaly detections produce IM-Vendors tickets or document no applicable events occurred. Source: IM-Vendors backlog.

Evidence Location: _____ Validation Date: ____ Notes: ______

Q2.3: Are EU AI Act Art. 26 and GDPR Art. 30 obligations for Critical-tier AI vendor integrations traceable to machine-generated log evidence, with no manual compilation required for Critical tier, and is the ML logging-baseline validation element completing inside the published staleness threshold?

Evidence Required: - [ ] EU AI Act Art. 26 obligations: for each Critical-tier high-risk integration, human-oversight duty, logging duty, and disclosure duty are each linked to a specific ML log store retention record and auto-assembled evidence artifact - [ ] GDPR Art. 30 obligations: for each Critical-tier integration processing personal data, the records-of-processing entry references the ML log store citation automatically; no manual log assembly required for regulatory inquiries - [ ] Evidence pipeline audit showing a Critical-tier integration's Art. 26 and Art. 30 evidence assembled automatically, documented with a sample evidence artifact - [ ] ML logging-baseline validation element fresh within the published staleness threshold for all Critical-tier integrations in any PC-Vendors compliance evidence bundle

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | % Critical-tier integrations with Art. 26 obligations traceable to machine-generated evidence | % | % | 100% | ☐ | | | % Critical-tier integrations with Art. 30 evidence requiring no manual log assembly | % | % | 100% | ☐ | | | ML logging-baseline validation freshness, Critical-tier (days) | d | d | ≤30d | ☐ | | | Evidence pipeline failures (stale Critical-tier evidence artifacts, last 90 days) | ___ | ___ | 0 | ☐ | |

Metric Collection Guidance: - Art. 26 traceability: Count Critical-tier high-risk integrations with Art. 26 obligation records that link to ML log store evidence divided by total such integrations. Source: evidence registry. - Art. 30 no-manual-assembly: Count Critical-tier integrations where the last regulator or audit inquiry was fulfilled from auto-assembled evidence without manual log reconstruction divided by total Critical-tier integrations with at least one inquiry. Source: inquiry fulfillment log. - Validation freshness: Check date of most recent ML logging-baseline validation in PC-Vendors compliance evidence bundle per Critical-tier integration. Source: evidence registry. - Pipeline failures: Count Critical-tier integration evidence artifacts with a freshness age exceeding the staleness threshold in the last 90 days. Source: evidence pipeline monitoring.

Evidence Location: _____ Validation Date: ____ Notes: ______

Maturity Level 3

Objective: Real-time AI-vendor attestation queryable by regulators and auditors; contribute to industry AI-vendor telemetry standards; share anonymized detection signatures with ISACs.

Question 7: Real-Time Deployer-Duty Attestation

Q3.1: Are ≥90% of Critical-tier AI vendor integrations covered by a live-queryable deployer-duty posture, logs, oversight assignments, disclosure records, compliance attestations, that a regulator or auditor can query directly without manual log assembly?

Evidence Required: - [ ] Live-queryable evidence view for Critical-tier integrations: real-time deployer-duty posture showing current log retention status, human-oversight assignments, disclosure records, and Art. 26 checklist attestations - [ ] Live attestation pipeline health monitored: % Critical integrations producing a fresh attestation artifact within the last 1 hour; pipeline alerts when an integration's attestation is stale - [ ] Cryptographic signing of attestation artifacts where applicable; regulators and auditors can verify attestation chain independently - [ ] Live attestation view covers: EU AI Act Art. 26 deployer duties, GDPR Art. 30 records-of-processing currency, ISO/IEC 42001 AIMS operational evidence currency - [ ] Documented case of a regulator or auditor querying the live attestation view directly, or a simulation drill confirming end-to-end query capability

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | % Critical-tier integrations with live attestation view producing fresh attestation artifact (last 1 hour) | % | % | ≥90% | ☐ | | | Regulator/auditor evidence requests resolved from live attestation view within 1 business day | % | % | ≥90% | ☐ | | | % Critical attestation artifacts cryptographically signed and verifiable | % | % | ≥90% | ☐ | | | Live attestation pipeline health, pipeline failures per week (last 4 weeks) | ___ | ___ | 0 | ☐ | |

Metric Collection Guidance: - Live attestation coverage: Count Critical-tier integrations with an attestation artifact produced within the last 1 hour divided by total Critical-tier integrations. Source: attestation telemetry. - Regulator resolution from live view: Count regulator or auditor inquiries resolved from the live attestation view within 1 business day divided by total such inquiries. Source: inquiry log. - Cryptographic signing: Count Critical-tier attestation artifacts with a valid cryptographic signature divided by total Critical-tier attestation artifacts. Source: attestation signing telemetry. - Pipeline failures: Count attestation pipeline failures (producing stale or missing artifacts) per week in the last 4 weeks. Source: pipeline monitoring.

Evidence Location: _____ Validation Date: ____ Notes: ______

Question 8: Telemetry Standard Contributions

Q3.2: Has the program contributed ≥2 telemetry-standard artifacts per year to OpenTelemetry AI workgroup, CSA AI Safety Initiative, or equivalent, with adoption tracked, and has it contributed to at least one additional industry body such as OpenSSF AI telemetry efforts?

Evidence Required: - [ ] OpenTelemetry AI workgroup contribution records: schemas, semantic conventions, or required-field definitions for AI-vendor event types; at least one contribution per annual cycle - [ ] CSA AI Safety Initiative contribution records: AI-vendor logging schemas or detection patterns contributed; at least one per annual cycle - [ ] Contribution maintenance evidence: schema versioning showing contributions updated when internal practice changes; not point-in-time submissions - [ ] Adoption tracking: records of external organizations citing or adopting contributed telemetry schemas - [ ] Contributions in-flight: at least one contribution item (draft, in-review, or submitted) active at all times

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | Telemetry-standard contributions per year (OpenTelemetry AI, CSA AI Safety, or equivalent) | ___ | ___ | ≥2 | ☐ | | | Contributions in-flight (draft, in-review, or submitted) at any time | ___ | ___ | ≥1 | ☐ | | | Contributions with evidence of external adoption (citations, integrations) | ___ | ___ | ≥1 | ☐ | | | Contributed schemas maintained current (within 90 days of internal practice change) | % | % | 100% | ☐ | |

Metric Collection Guidance: - Telemetry contributions: Count schema or convention contributions to OpenTelemetry AI, CSA AI Safety, or equivalent in the last 12 months. Source: contribution log. - In-flight items: Count contribution items in draft, in-review, or submitted status. Source: contribution pipeline. - External adoption: Count external citations or integrations of contributed schemas. Source: contribution tracking record. - Schema currency: For each contributed schema, verify last-update date is within 90 days of the most recent internal practice change. Source: contribution version log.

Evidence Location: _____ Validation Date: ____ Notes: ______

Question 9: Shared Detection Signatures for AI-Vendor Threats

Q3.3: Has the program contributed ≥12 anonymized detection signatures per year to ISAC AI-vendor feeds or OpenSSF AI, with at least one ISAC partner citing adoption, and are signatures implementable by partner organizations without significant adaptation?

Evidence Required: - [ ] Sector ISAC submission records (FS-ISAC, H-ISAC, IT-ISAC AI working groups): anonymized, generalized AI-vendor detection signatures; at least one per month - [ ] OpenSSF AI contribution records for AI-vendor detection signatures where applicable - [ ] Legal-vet records for each signature submission confirming anonymization and no competitive or proprietary information - [ ] Signatures include sufficient context for implementation: vendor archetype, detection logic, required log fields, threshold guidance, without requiring reconstruction of the context removed for anonymization - [ ] Evidence of at least one ISAC partner citing or adopting a contributed AI-vendor detection signature

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | AI-vendor detection signatures contributed to ISACs per year | ___ | ___ | ≥12 | ☐ | | | ISAC partner organizations citing contributed signatures | ___ | ___ | ≥1 | ☐ | | | Signatures with legal-vet record confirming anonymization | % | % | 100% | ☐ | | | Submission cadence, at least one signature per month (last 12 months) | /12 | /12 | 12/12 | ☐ | |

Metric Collection Guidance: - ISAC signatures: Count AI-vendor detection signatures submitted to sector ISACs or OpenSSF AI in the last 12 months. Source: contribution log × ISAC submission receipts. - Partner adoption: Count ISAC partner organizations that have cited or listed a contributed AI-vendor signature in their own detection catalogs. Source: contribution tracking record × ISAC partner feedback. - Legal vet coverage: Count submitted signatures with a legal-vet record on file divided by total signatures submitted. Source: legal review log. - Monthly cadence: Count months in the last 12 where at least one signature was submitted. Source: contribution log.

Evidence Location: _____ Validation Date: ____ Notes: ______

Summary Scorecard

Question	Level	Activity
Q1: Per-Archetype AI-Vendor Logging Baseline	L1	A
Q2: High-Signal Detection Set Including Shadow-AI	L1	B
Q3: Deployer-Duty Evidence Trail	L1	C
Q4: Anomaly Detection on AI-Vendor Behavior	L2	A
Q5: Cross-Vendor Graph and Automated Evidence	L2	B
Q6: Art. 26 / Art. 30 Traceability	L2	C
Q7: Real-Time Deployer-Duty Attestation	L3	A
Q8: Telemetry Standard Contributions	L3	B
Q9: Shared Detection Signatures	L3	C

L1 Score (avg Q1–Q3): _ / 1.0 L2 Score (avg Q4–Q6): / 1.0 L3 Score (avg Q7–Q9): __ / 1.0 Overall ML-Vendors Score (L1×0.5 + L2×0.3 + L3×0.2): _____ / 1.0

Assessor: _____ Assessment Date: ____ Next Review Date: ______

Document Version: HAIAMM v3.0, 2026-05-15, Verifhai Practice: Monitoring & Logging (ML) Domain: Vendors Source of Truth: docs/practices/ML-Vendors-OnePager.md Compliance Map: docs/HAIAMM-v3.0-Framing.md §10.2

Instructions:

Answer based on current practices, not plans
“Yes” requires documented evidence
Complete all Level 1 questions before Level 2
Partial implementation = “No”

↓ Download as Markdown

Monitoring & Logging (ML) - Vendors Assessment

Monitoring & Logging (ML) - Vendors Domain

HAIAMM Assessment Questionnaire v3.0

Instructions

Scoring Methodology

Maturity Level 1

Question 1: Per-Archetype AI-Vendor Logging Baseline

Question 2: High-Signal Detection Set Including Shadow-AI Detections

Question 3: Deployer-Duty Evidence Trail

Maturity Level 2

Question 4: Anomaly Detection on AI-Vendor Behavior

Question 5: Cross-Vendor Graph Analysis and Automated Deployer-Duty Evidence

Question 6: EU AI Act Art. 26 and GDPR Art. 30 Traceability

Maturity Level 3

Question 7: Real-Time Deployer-Duty Attestation

Question 8: Telemetry Standard Contributions

Question 9: Shared Detection Signatures for AI-Vendor Threats

Summary Scorecard