Assessment questionnaire for measuring maturity. Answer each question honestly based on current, implemented practices.
Canonical source-of-truth:
../practices/SM-Endpoints-OnePager.md. This questionnaire's questions, evidence requirements, and outcome metrics are derived from that one-pager. The canonical v3.0 model:../HAIAMM-v3.0-Framing.md.
Practice: Strategy & Metrics (SM) Domain: Endpoints Purpose: Stand up an AI/HAI Endpoint Assurance program that discovers, inventories, and strategically governs all AI/HAI-enabled endpoints and user-facing AI interfaces the organization operates, with shadow endpoint AI prevention as the primary L1 outcome and a defensible risk-tier rubric as the primary L2 deliverable. Scoring Model: Evidence + Outcome Metrics (see Scoring Methodology below)
| Tier | Score | Criteria |
|---|---|---|
| Fully Mature | 1.0 | Evidence complete + ≥3 outcome metrics meet targets |
| Implemented | 0.67 | Evidence complete + 2 outcome metrics meet targets |
| Partial | 0.33 | Evidence partially complete + <2 outcome metrics meet targets |
| Not Implemented | 0.0 | No substantive evidence of practice |
Practice maturity level achieved = the highest level where all 3 questions score ≥ 0.67.
Objective: Stand up the AI/HAI Endpoint Assurance program, build an inventory of all seven endpoint AI archetypes, and establish baseline metrics that prove shadow endpoint AI is decreasing
Q1.1: Do you have a published AI/HAI Endpoint Assurance program charter that names the problem (shadow AI at the endpoint layer, AI features silently enabled in SaaS productivity suites, own-built customer-facing AI surfaces without security review, AI browser extensions exfiltrating sensitive content, edge devices running unreviewed on-device models), defines the seven in-scope endpoint AI archetypes, names an executive sponsor (CISO + CTO / Head of IT / CPTO), establishes a cross-functional working group, and defines decision rights for approval, block, exception, and go-live across all seven archetypes?
Evidence Required: - [ ] Published program charter with named executive sponsor (CISO + CTO / Head of IT) and Privacy/Legal co-signature for customer-facing archetypes - [ ] Problem statement covering EU AI Act Art. 50 transparency obligations (own-built customer-facing AI interfaces), GDPR Art. 32 (securing endpoints processing personal data through AI), and the data-egress risk when endpoint AI sends user-provided or observed data to vendor AI backends - [ ] Seven in-scope endpoint AI archetypes listed: AI assistant/copilot on managed endpoint, browser-based AI tool, chatbot/conversational UI, multi-modal AI interface, AI-augmented productivity, mobile AI app, edge AI device - [ ] Domain boundary operationalized: own-built endpoint AI surfaces (chatbots, mobile AI apps, edge AI devices) governed here; vendor AI tools consumed on endpoints cross-referenced with Vendors domain - [ ] Working group roster: Security, IT (endpoint management, MDM, EDR), Engineering (own-built chatbot and mobile AI), Product, Privacy/Legal, HR (acceptable use), application-architect reviewer for own-built surfaces - [ ] Decision rights defined: who approves a new AI/HAI endpoint asset, who blocks, who handles exceptions, who owns go-live for customer-facing AI surfaces
Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |---|---|---|---|---|---| | AI/HAI endpoint inventory coverage (% of discovered endpoint AI assets in inventory) | ___ | ___ | ≥90% within 12 months | ☐ | | | Shadow endpoint AI ratio (unsanctioned endpoint AI assets ÷ total endpoint AI assets) | ___ | ___ | ≤15% and trending down | ☐ | | | % managed-endpoint user population covered by an acknowledged Endpoint AI AUP | ___ | ___ | ≥95% of managed-endpoint users | ☐ | | | % AI/HAI endpoint assets with a named owning team or responsible owner | ___ | ___ | 100% | ☐ | | | Known data-exposure events from AI/HAI endpoint assets (per quarter) | ___ | ___ | trending down QoQ | ☐ | |
Metric Collection Guidance:
- Inventory coverage: Reconcile inventory count against all discovery signals (MDM/UEM telemetry, EDR/endpoint security telemetry, browser-extension inventory, SaaS admin consoles, identity/OAuth events, customer-facing AI inventory, mobile app store and MDM mobile inventory, IoT/kiosk asset registry). Formula: inventory_count / discovered_count × 100
- Shadow endpoint AI ratio: From inventory status field, count endpoint AI assets in use with status not "Sanctioned"; divide by total endpoint AI assets identified across all discovery sources
- Endpoint AI AUP attestation: HR/LMS acknowledgment records for Endpoint AI AUP filtered to managed-endpoint user population; denominator is that headcount
- Named owning team: Count inventory records with non-null owning_team or responsible_owner field vs. total; automated check at record creation
- Data-exposure events from endpoints: Aggregate DLP alerts for AI-tool data-sharing events, MDM/EDR egress review findings, and incident-tracker entries; count per quarter and trend
Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)
Evidence Location: _____ Metric Validation Date: ____ Notes: __________
Q1.2: Do you maintain a single AI/HAI endpoint inventory seeded from MDM/UEM telemetry, EDR/endpoint security telemetry, browser-extension admin policies, SaaS admin consoles (M365, Slack, Notion, Google Workspace), identity/OAuth events, customer-facing AI inventory from product and engineering, mobile app store and MDM mobile inventory, and IoT/kiosk asset registry, covering all seven archetypes with a minimum field set including archetype, endpoint population affected, own-built vs. vendor-provided, data classes accessible, action capability, customer-data egress potential, and approval status?
Evidence Required: - [ ] Single authoritative endpoint inventory with minimum fields: asset name, owning team/responsible owner, archetype, endpoint population affected (number + user roles), own-built or vendor-provided (vendor cross-reference ID if applicable), data classes accessible, action capability (read-only vs. action-taking), customer-data egress potential (yes/no + DPA status if applicable), deployment scale, regulatory scope, approval status, linked artifacts - [ ] MDM/UEM telemetry active (Jamf, Intune, Kandji, VMware Workspace ONE): app inventory on managed endpoints filtered for AI application categories; app install and update events captured - [ ] EDR/endpoint security telemetry reviewed (CrowdStrike, SentinelOne, Microsoft Defender): process execution and network-egress events to AI provider domains monitored - [ ] Browser extension inventory pulled (Chrome Enterprise Admin, Edge Admin Center, Firefox Enterprise Policy): extension listings filtered for AI capabilities; per-user and per-device extension presence - [ ] SaaS admin consoles audited for AI feature enablement: M365 Admin Center (Copilot license/feature enablement), Slack Admin (Slack AI), Notion Admin (Notion AI), Google Workspace Admin (Gemini), and similar - [ ] Amnesty window publicized to engineering, product, and IT; AI tools already installed or shipped can be disclosed without penalty
Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |---|---|---|---|---|---| | AI/HAI endpoint inventory coverage (% of discovered endpoint AI assets in inventory) | ___ | ___ | ≥90% within 12 months | ☐ | | | Shadow endpoint AI ratio (unsanctioned endpoint AI assets ÷ total endpoint AI assets) | ___ | ___ | ≤15% and trending down | ☐ | | | % managed-endpoint user population covered by an acknowledged Endpoint AI AUP | ___ | ___ | ≥95% of managed-endpoint users | ☐ | | | % AI/HAI endpoint assets with a named owning team or responsible owner | ___ | ___ | 100% | ☐ | | | Known data-exposure events from AI/HAI endpoint assets (per quarter) | ___ | ___ | trending down QoQ | ☐ | |
Metric Collection Guidance: - Inventory coverage: Monthly reconciliation comparing inventory records to MDM app telemetry + EDR process/egress events + browser-extension admin reports + SaaS-admin AI-feature dashboards + OAuth event logs + mobile app store + IoT asset registry; unmatched signals are shadow-AI candidates - Shadow endpoint AI ratio: Filter inventory for assets in use with approval status != "Sanctioned"; divide by total in-use assets across all sources; trend quarterly - Named owning team: Automated check, assets with null owning_team or responsible_owner flagged and assigned to triage owner within 5 BD - Data-exposure events from endpoints: DLP alerts for AI-tool data-sharing events + MDM/EDR egress review findings + incident-tracker entries combined per quarter; trend over time - Endpoint AI AUP attestation: HR/LMS query for managed-endpoint user population who completed Endpoint AI AUP acknowledgment; denominator from HR system
Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)
Evidence Location: _____ Metric Validation Date: ____ Notes: __________
Q1.3: Do you baseline and report quarterly to the executive sponsor a shadow endpoint AI scoreboard covering inventory state by archetype, new endpoint AI assets discovered and their intake status, shadow endpoint AI ratio trend over the last four quarters, Endpoint AI AUP attestation coverage, and the top five unmitigated AI endpoint risks with named owners and remediation status?
Evidence Required: - [ ] Quarterly shadow endpoint AI scoreboard published and delivered to executive sponsor, at least two consecutive quarters on record - [ ] Scoreboard includes archetype-level breakdown (AI assistant/copilot on managed endpoint, browser-based AI tool, chatbot/conversational UI, multi-modal AI interface, AI-augmented productivity, mobile AI app, edge AI device) - [ ] Shadow endpoint AI ratio trended over last 4 quarters with commentary on direction - [ ] Endpoint AI AUP attestation percentage reported with managed-endpoint user population denominator - [ ] Top 5 unmitigated AI endpoint risks listed with named owner and remediation status (TA-flagged, MDM/EDR-flagged, customer-facing-surface-flagged, or external-advisory-flagged) - [ ] Intake SLA tracked: new AI/HAI endpoint asset intake triaged within 5 BD; provisional approval within 10 BD for Low-tier archetypes (developer-only coding assistant, no regulated data, read-only)
Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |---|---|---|---|---|---| | AI/HAI endpoint inventory coverage (% of discovered endpoint AI assets in inventory) | ___ | ___ | ≥90% within 12 months | ☐ | | | Shadow endpoint AI ratio (unsanctioned endpoint AI assets ÷ total endpoint AI assets) | ___ | ___ | ≤15% and trending down | ☐ | | | % managed-endpoint user population covered by an acknowledged Endpoint AI AUP | ___ | ___ | ≥95% of managed-endpoint users | ☐ | | | % AI/HAI endpoint assets with a named owning team or responsible owner | ___ | ___ | 100% | ☐ | | | Known data-exposure events from AI/HAI endpoint assets (per quarter) | ___ | ___ | trending down QoQ | ☐ | |
Metric Collection Guidance: - Scoreboard delivery cadence: Confirm last two quarters have a dated scoreboard delivered to exec sponsor with acknowledgment on record - Archetype breakdown: Scoreboard section shows counts per archetype (sanctioned / provisional / prohibited / awaiting intake); detects which archetype classes are growing unchecked - Shadow endpoint AI ratio trend: Four-quarter chart or table; downward trend is the L1 success signal; source is inventory status field reconciled monthly against all discovery signals - AUP attestation: Percentage with managed-endpoint user population denominator explicitly stated; HR/LMS is the authoritative source; updated each quarter - Top-5 risks: Each entry lists risk description, source, named owner, and current remediation status (open / in-progress / mitigated); customer-facing surface risks given priority
Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)
Evidence Location: _____ Metric Validation Date: ____ Notes: __________
Objective: Risk-tier the AI/HAI endpoint inventory using a seven-dimension rubric, calibrate program intensity per tier, and measure practice maturity and shadow endpoint AI reduction per tier, establishing the rubric every other Endpoints-domain L2 practice depends on
Q2.1: Do you have a published risk-tier rubric (Critical / High / Medium / Low) assigning a tier to every AI/HAI endpoint asset based on seven auditable dimensions, user population, data classes accessible through the endpoint, action capability, customer-data egress potential, deployment scale, regulatory scope (including EU AI Act Art. 50 transparency obligation), and AI-content disclosure obligation, with tier derivation deterministic, human overrides recorded with rationale, and 100% of inventory records carrying a current tier?
Evidence Required: - [ ] Published tier-rubric document listing all seven auditable dimensions with deterministic assignment logic - [ ] 100% of inventory records carry a current tier assignment derived from the rubric - [ ] User population dimension: customer-facing public (no authentication required) → Critical; authenticated customer → High; employee-general → Medium; developer-only or IT-only → Low - [ ] Data classes dimension: regulated data (PHI/PCI/regulated PII/source code/customer confidential) accessible to or processable by the endpoint AI → elevate to Critical or High - [ ] Action capability dimension: AI can write, send, submit, or modify data on org or customer systems → elevate; read-only, informational-only → no elevation from this dimension alone - [ ] Customer-data egress potential dimension: endpoint AI sends user-provided or observed data to a vendor AI backend → elevate; on-device or local processing only → no elevation - [ ] Regulatory scope dimension: EU AI Act Art. 50 transparency obligation triggered (own-built customer-facing AI interfaces) → elevate; sector-specific triggers (HIPAA endpoint, PCI-DSS CDE endpoint, FERPA, COPPA) → Critical - [ ] Human override log maintained: overrides recorded with rationale and reviewed by working group
Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |---|---|---|---|---|---| | % of inventory with a current tier assignment | ___ | ___ | 100% | ☐ | | | Tier-treatment matrix adherence, % Critical assets with full-scope treatment completed in last 12 months | ___ | ___ | ≥95% | ☐ | | | Tier-weighted shadow endpoint AI ratio (Critical-weighted) | ___ | ___ | Critical = 0 unsanctioned in production; overall trending down | ☐ | | | Per-tier SLA adherence across practices (intake, DR, IR, ST, ML, IM) | ___ | ___ | ≥90% per tier | ☐ | | | Tier drift rate (tier changes per year) | ___ | ___ | tracked; unexplained changes = 0 | ☐ | |
Metric Collection Guidance: - % inventory with tier assignment: Automated check, records with null risk_tier flagged for remediation within 5 BD; stale tier (no re-confirmation after a material change) also flagged - Tier-treatment matrix adherence: Cross-reference Critical-tier endpoint assets against evidence of full-scope treatment: consent/disclosure UX review completed, per-asset TA deep model, DR completed, IR within cadence, full ST battery (data-egress canaries, prompt-injection corpus, action-scope boundary, disclosure-UX verification, kill-switch), all detections tuned - Tier-weighted shadow endpoint AI ratio: Critical-tier in-use assets not in "Sanctioned" status must be 0; overall ratio should trend down - Per-tier SLA adherence: From intake, DR, IR, ST, and IM trackers; % on-time per tier; report monthly - Tier drift rate: Governance log reviewed at each working-group meeting; changes without dimension-change rationale are unexplained; target = 0
Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)
Evidence Location: _____ Metric Validation Date: ____ Notes: __________
Q2.2: Do you have a published tier-treatment matrix defining differential program intensity across all downstream Endpoints-domain practices (PC, TA, SR, SA, DR, IR, ST, EH, ML, IM) for each tier, and is this matrix enforced, with Critical-tier endpoint AI assets receiving full SR pack + REM + explicit consent/disclosure UX review + executive sign-off at intake, per-asset deep TA, full-lane DR with Art. 50 UX review, semi-annual IR, full ST battery including quarterly adversarial exercise, all detections tuned, and mandatory re-review within 14 days of material change?
Evidence Required: - [ ] Tier-treatment matrix published covering all downstream practices with explicit controls per tier - [ ] Critical-tier treatment documented: full SR pack + REM + explicit consent/disclosure UX review + executive sign-off; per-asset deep TA with customer-data-egress and action-scope overlay; full-lane DR with named architect including Art. 50 UX review; semi-annual IR + on material change (model swap, new data class, new action capability, scope change); full ST battery (data-egress canaries, prompt-injection corpus, action-scope boundary, jailbreak regression, disclosure-UX verification, kill-switch) + quarterly adversarial exercise; MDM policy enforced + browser-extension DLP + SaaS-admin AI restricted to approved population + egress allowlist; all detections tuned per asset; IM SLA ack ≤4h/mitigate ≤48h; mandatory re-review within 14 days of material change - [ ] Low-tier fast-track documented: base SR pack only; archetype TA model; no DR required; go-live IR only; spot-check ST; MDM policy (basic); baseline logging; IM SLA ack ≤5BD/mitigate ≤30d; re-review at annual review - [ ] Evidence that Critical-tier assets are actually receiving full-scope treatment (assets cross-referenced to treatment evidence in program tracker) - [ ] Downstream practices acknowledged calibration via working-group decision record
Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |---|---|---|---|---|---| | % of inventory with a current tier assignment | ___ | ___ | 100% | ☐ | | | Tier-treatment matrix adherence, % Critical assets with full-scope treatment completed in last 12 months | ___ | ___ | ≥95% | ☐ | | | Tier-weighted shadow endpoint AI ratio (Critical-weighted) | ___ | ___ | Critical = 0 unsanctioned in production; overall trending down | ☐ | | | Per-tier SLA adherence across practices (intake, DR, IR, ST, ML, IM) | ___ | ___ | ≥90% per tier | ☐ | | | Tier drift rate (tier changes per year) | ___ | ___ | tracked; unexplained changes = 0 | ☐ | |
Metric Collection Guidance: - Tier-treatment matrix adherence: For each Critical-tier endpoint asset, verify: consent/disclosure UX review on file, TA report exists, DR completed, IR within cadence, ST battery complete, ML detections configured, EH controls confirmed; ≥95% must show all treatments in last 12 months - Art. 50 disclosure UX: For own-built customer-facing AI interfaces at Critical tier, verify a disclosure-UX review was completed and the interface meets Art. 50 transparency requirements before go-live - Per-tier SLA adherence: Aggregated from intake, DR, IR, ST, and IM trackers; % on-time per tier; reported monthly - Tier-weighted shadow endpoint AI ratio: Critical-tier unsanctioned assets must be 0; overall ratio should decrease; both reported in quarterly scoreboard - Tier drift rate: Governance log reviewed at each working-group meeting; unexplained tier changes flagged for sponsor visibility; target = 0
Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)
Evidence Location: _____ Metric Validation Date: ____ Notes: __________
Q2.3: Does the quarterly shadow endpoint AI scoreboard report inventory state per tier and per archetype with Critical-tier unsanctioned endpoint AI in production explicitly tracked at zero, include a tier-movement log with rationale, report per-tier SLA adherence, and is it reviewed by the executive sponsor who discusses tier-balance?
Evidence Required: - [ ] Quarterly scoreboard includes a tier × archetype breakdown table (Critical/High/Medium/Low rows by archetype columns) - [ ] Critical-tier unsanctioned endpoint AI in production is a named metric; target is 0; any non-zero value is a headline finding requiring sponsor action - [ ] Tier-movement log included: assets that moved up or down in the quarter, with dimension(s) that changed and rationale for each move - [ ] SLA adherence per tier reported for intake, DR, IR, ST, ML, and IM - [ ] Quarterly executive review documented (agenda + minutes) showing tier-balance discussion and sponsor sign-off; IT and security planning references tier-level dashboards - [ ] Tier calibration exercise documented: at least quarterly, a sample of 20 endpoint AI assets re-tiered by a second reviewer; drift tracked
Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |---|---|---|---|---|---| | % of inventory with a current tier assignment | ___ | ___ | 100% | ☐ | | | Tier-treatment matrix adherence, % Critical assets with full-scope treatment completed in last 12 months | ___ | ___ | ≥95% | ☐ | | | Tier-weighted shadow endpoint AI ratio (Critical-weighted) | ___ | ___ | Critical = 0 unsanctioned in production; overall trending down | ☐ | | | Per-tier SLA adherence across practices (intake, DR, IR, ST, ML, IM) | ___ | ___ | ≥90% per tier | ☐ | | | Tier drift rate (tier changes per year) | ___ | ___ | tracked; unexplained changes = 0 | ☐ | |
Metric Collection Guidance: - Per-tier scoreboard delivery: Last two consecutive quarterly scoreboards must include tier × archetype table; each shows delta from prior quarter - Critical-tier unsanctioned count: Named metric; source is inventory filtered on tier=Critical AND status != Sanctioned AND in-use; target = 0 - Tier-movement log completeness: Each entry must have asset name, prior tier, new tier, dimension(s) that changed, reviewer name, and date; unexplained changes target = 0 - SLA adherence per tier: Pulled from intake, DR, IR, ST, ML, and IM systems; aggregated per tier; reported as % on-time per tier per quarter - Executive review: Filed governance document confirming exec sponsor reviewed tier-balance section and issued follow-up actions or signed off
Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)
Evidence Location: _____ Metric Validation Date: ____ Notes: __________
Objective: Automate inventory and tier maintenance from MDM, browser, SaaS-admin, and identity signals; benchmark against external endpoint-AI peers; and contribute to industry endpoint-AI assurance standards
Q3.1: Does the AI/HAI endpoint inventory auto-update from live MDM app catalog events, browser-extension admin policy changes, SaaS-admin AI-feature enablement events, identity/OAuth events, own-built app release events, and edge-device firmware events, with tier assignments rule-based and replayable, tier changes auto-triggering downstream practice obligations within 24 hours, and a published data-quality SLO of ≥99% correctly tiered within 48 hours of a material change?
Evidence Required: - [ ] Published data-quality SLO: ≥99% of active AI/HAI endpoint assets correctly tiered within 48 hours of a material change; ≥95% inventory completeness against discovery-source reconciliation - [ ] Automated feeds operational: MDM app catalog events (new AI app installed or updated on managed endpoints), browser-extension admin policy changes (new AI extension approved or added), SaaS-admin AI-feature enablement events (M365 Copilot license assigned, Slack AI feature enabled for a team, Gemini for Workspace feature activated), identity/OAuth events (new AI service OAuth authorization from managed endpoint), own-built app release events (mobile app version with new AI features, chatbot deployment events), edge-device firmware events (on-device model updated or new inference capability added), self-attestation and intake - [ ] Tier rules documented as versioned, replayable logic; rule changes change-logged and replayable against historical inventory state - [ ] Tier-change events auto-trigger downstream obligations (e.g., Medium→Critical triggers DR, ST, ML reconfiguration, and Art. 50 disclosure UX review) within 24h; monitored via workflow telemetry - [ ] Human curation queue defined for: new archetypes, ambiguous discoveries (shared productivity tool sometimes used in customer-facing context), dimensional-input conflicts - [ ] Automation health dashboard: on-call paged when any feed exceeds staleness threshold
Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |---|---|---|---|---|---| | Inventory auto-update latency | ___ | ___ | ≤48h for material changes | ☐ | | | % inventory entries auto-curated vs. human-curated | ___ | ___ | ≥80% auto | ☐ | | | Inventory completeness against discovery-source reconciliation | ___ | ___ | ≥99% | ☐ | | | Tier-rule auto-trigger of downstream obligations on tier change | ___ | ___ | 100% within 24h | ☐ | | | External benchmarks tracked | ___ | ___ | ≥5 peer-comparable metrics | ☐ | | | Industry contributions per year | ___ | ___ | ≥4 substantive | ☐ | | | Executive ROI narrative refreshed with external benchmarks | ___ | ___ | semi-annual | ☐ | |
Metric Collection Guidance: - Auto-update latency: Measure time from a known material change event (new AI app installed on managed endpoint via MDM, new AI feature enabled in SaaS admin console) to the corresponding inventory record update; P95 across 20 sampled events per quarter - % auto-curated: From the curation log, count records updated by automated feeds vs. human-initiated edits; report as a ratio per quarter - Inventory completeness: Full discovery-source reconciliation across all seven archetype signal sources; report completeness % and list archetypes below 99% - Downstream obligation auto-trigger: Workflow telemetry showing each tier-change event produced a DR ticket, ST job, ML reconfiguration, or Art. 50 disclosure UX review task within 24h; report % within SLO - External benchmarks tracked: Count distinct benchmark data points in the semi-annual brief; each traceable to CSA, OASIS, OWASP MASVS, or sector ISAC source
Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)
Evidence Location: _____ Metric Validation Date: ____ Notes: __________
Q3.2: Do you publish a semi-annual external-benchmarking brief comparing the program against at least five peer-comparable metrics via CSA Endpoint Security Working Group and CSA AI Safety Initiative, OASIS AI governance working groups, mobile-AI ISACs and sector-specific groups, OWASP MASVS for own-built mobile AI apps, BSIMM observational data, and formal peer roundtables, and do benchmark deltas explicitly inform program investment decisions?
Evidence Required: - [ ] Semi-annual benchmarking brief published, two most recent on file with dates, each containing ≥5 peer-comparable metrics from named external sources - [ ] Benchmarking sources include at least two of: CSA Endpoint Security Working Group / CSA AI Safety Initiative / OASIS AI governance working groups / mobile-AI ISACs and sector groups (FS-ISAC, H-ISAC, IT-ISAC mobile and endpoint tracks) / OWASP MASVS (for own-built mobile AI apps) / BSIMM observational data / formal CISO peer roundtables - [ ] Metrics benchmarked cover: inventory coverage, shadow endpoint AI ratio, per-tier SLA adherence, automation level, Art. 50 disclosure UX compliance rate, endpoint-AI data-egress control coverage - [ ] Benchmark deltas explicitly referenced in a program investment or prioritization decision; documentation filed within 90 days of each brief - [ ] Peer selection rationale documented, peers chosen to stretch the program, not flatter it
Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |---|---|---|---|---|---| | Inventory auto-update latency | ___ | ___ | ≤48h for material changes | ☐ | | | % inventory entries auto-curated vs. human-curated | ___ | ___ | ≥80% auto | ☐ | | | Inventory completeness against discovery-source reconciliation | ___ | ___ | ≥99% | ☐ | | | Tier-rule auto-trigger of downstream obligations on tier change | ___ | ___ | 100% within 24h | ☐ | | | External benchmarks tracked | ___ | ___ | ≥5 peer-comparable metrics | ☐ | | | Industry contributions per year | ___ | ___ | ≥4 substantive | ☐ | | | Executive ROI narrative refreshed with external benchmarks | ___ | ___ | semi-annual | ☐ | |
Metric Collection Guidance: - External benchmarks tracked: Each brief lists ≥5 named benchmark data points; each traceable to a CSA/OASIS/OWASP MASVS/ISAC/BSIMM source or named peer roundtable - Benchmark-driven investment: Program planning or budget document explicitly citing a benchmark delta as rationale; filed within 90 days of each brief - Semi-annual cadence: Two briefs within a 12-month window; no gap > 7 months between consecutive briefs - Executive ROI narrative: Annual exec/board deck includes benchmark comparisons, Art. 50 disclosure UX compliance status, and avoided-loss examples (Critical-tier own-built chatbot caught at DR before customer-data-egress exposure; High-tier mobile AI app flagged for missing consent UX before public release)
Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)
Evidence Location: _____ Metric Validation Date: ____ Notes: __________
Q3.3: Does the program contribute at least four substantive, anonymized artifacts per year to the endpoint-AI assurance ecosystem through CSA AI Safety Initiative, OWASP MASVS and MASTG for own-built mobile AI apps, OASIS conversational AI and chatbot security standards, NIST AI RMF Playbook deployer-duty evidence patterns, EU AI Act Art. 50 transparency implementation guidance, or sector ISACs, with each contribution anonymized, legally vetted, and traceable to a published working-group output?
Evidence Required: - [ ] Contribution log maintained listing all submissions: target body (CSA, OWASP MASVS/MASTG, OASIS, NIST AI RMF, EU AI Act guidance bodies, sector ISACs), submission type (endpoint-AI controls matrix, browser-extension governance guidance, SaaS AI feature governance reference, mobile AI app security pattern, Art. 50 transparency UX reference pattern), date submitted, anonymization review completed, status - [ ] At least 4 substantive contributions per year in the most recent 12-month window; each is a technical artifact accepted or in active review by the named body; conference talks and press releases do not count - [ ] Each contribution has a legal/privacy review sign-off confirming anonymization before submission - [ ] Contributions traceable to published outputs: CSA AI Safety Initiative controls matrix updates, OWASP MASVS/MASTG publications, OASIS working-group deliverables, NIST AI RMF Playbook references, EU AI Act transparency implementation guides, sector ISAC advisory feeds - [ ] Contribution pipeline shows ≥2 items in-flight (draft, in-review, or being prepared) at any working-group review
Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |---|---|---|---|---|---| | Inventory auto-update latency | ___ | ___ | ≤48h for material changes | ☐ | | | % inventory entries auto-curated vs. human-curated | ___ | ___ | ≥80% auto | ☐ | | | Inventory completeness against discovery-source reconciliation | ___ | ___ | ≥99% | ☐ | | | Tier-rule auto-trigger of downstream obligations on tier change | ___ | ___ | 100% within 24h | ☐ | | | External benchmarks tracked | ___ | ___ | ≥5 peer-comparable metrics | ☐ | | | Industry contributions per year | ___ | ___ | ≥4 substantive | ☐ | | | Executive ROI narrative refreshed with external benchmarks | ___ | ___ | semi-annual | ☐ | |
Metric Collection Guidance: - Industry contributions per year: Count entries in the contribution log for trailing 12 months where status = submitted or accepted to a named body; only substantive technical artifacts (endpoint-AI controls matrix, browser-extension governance guidance, Art. 50 UX reference patterns) count - Contribution pipeline health: At any working-group meeting, pipeline log shows ≥2 items not yet in submitted status; noted in working-group minutes - Legal/privacy review: Each contribution log entry must have reviewer name and date; no contribution submitted without this sign-off - Executive ROI narrative: Filed annually to exec/board; references external benchmarks, Art. 50 disclosure UX compliance status, and avoided-loss examples; faster sanctioned endpoint AI adoption time (from "user requests an AI tool" to "provisional approval") is a key effectiveness metric
Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)
Evidence Location: _____ Metric Validation Date: ____ Notes: __________
| Level | Q1 | Q2 | Q3 | Avg | Achieved? |
|---|---|---|---|---|---|
| L1 | __ | __ | __ | __ | ☐ |
| L2 | __ | __ | __ | __ | ☐ |
| L3 | __ | __ | __ | __ | ☐ |
Practice maturity level achieved: ___ (highest level where all 3 questions score ≥ 0.67)
Document Version: HAIAMM v3.0 Practice: Strategy & Metrics (SM) Domain: Endpoints Last Updated: 2026-05-15 Author: Verifhai
Instructions: