Strategy & Metrics (SM)

Endpoints Domain - HAIAMM v3.0

Practice Overview

Objective: Stand up an AI/HAI Endpoint Assurance program that discovers, inventories, and strategically governs all AI/HAI-enabled endpoints and user-facing AI interfaces the organization operates, with shadow endpoint AI prevention as the primary L1 outcome and a defensible risk-tier rubric as the primary L2 deliverable.

Description: The Endpoints domain governs the AI/HAI capabilities that live at the endpoint layer, the seven canonical archetypes: AI assistants and copilots on managed endpoints (Cursor, Copilot, Claude Code, internal AI assistants on engineering laptops), browser-based AI tools (web UIs, browser extensions like Grammarly AI, screenshot and page AI tools), chatbots and conversational UIs (own-built customer-facing chat surfaces), multi-modal AI interfaces (voice, image, video AI on endpoints), AI-augmented productivity (M365 Copilot, Slack AI, Notion AI, Google Workspace Gemini on managed endpoints), mobile AI apps (own-built mobile apps with AI features or vendor mobile AI assistants on managed mobile), and edge AI devices (kiosks, IoT, on-device inference). SM-Endpoints establishes the program charter, an authoritative inventory of these endpoint AI assets, and the practice-maturity metrics that prove the program is working. SM-Endpoints L2 produces the risk-tier rubric every other Endpoints-domain L2 practice depends on.

Context: AI capabilities arrive at endpoints from every direction simultaneously. An engineer installs a coding copilot from the extension marketplace. A product manager enables M365 Copilot for a team and it quietly gains access to SharePoint. A customer-facing chatbot goes live on the marketing site without a security review. A sales team starts using a mobile AI assistant that sends call transcripts to an offshore vendor server. A kiosk in a retail location begins running on-device image recognition that collects biometric-adjacent data. None of this is coordinated, each arrives through a different channel (MDM app catalog, browser-extension policy, SaaS admin console, app store, IoT provisioning), each creates a different data-exposure and AI-decision surface, and none of it is visible to the team responsible for endpoint security or AI governance unless the program looks for it deliberately. The AI/HAI Endpoint Assurance program makes this surface visible, attaches accountable ownership, and establishes the governance layer that distinguishes own-built endpoint AI (in scope here) from vendor AI consumed on endpoints (cross-referenced with Vendors domain).

Maturity Level 1

Objective: Stand up the AI/HAI Endpoint Assurance program, build an inventory of all seven endpoint AI archetypes, and establish baseline metrics that prove shadow endpoint AI is decreasing

At this level, the organization makes its AI/HAI endpoint surface visible across all seven archetypes, assigns accountability, and begins measuring the reduction of shadow endpoint AI, unmanaged AI tools installed on endpoints, ungoverned browser extensions with AI capabilities, AI features silently enabled in SaaS productivity suites, and own-built customer-facing AI surfaces launched without security review.

Dependencies

• None, entry-point practice for the Endpoints domain. SM-Endpoints L1 precedes all other Endpoints-domain L1s.
• Alignment (not a hard dependency): enterprise-wide SM strategy, CISO and CTO / CISO / Head of IT governance structure, existing endpoint management program (MDM / EDR), existing AppSec program for own-built surfaces, so the AI/HAI endpoint program plugs into existing risk committees rather than forming a parallel stack.
• Supports / unblocks: PC-Endpoints L1 (policies need the inventory and archetypes), TA-Endpoints L1 (threat modeling needs the asset list), SR-Endpoints L1 (requirements packs key on archetype), SA-Endpoints L1 (reference patterns need the archetype list), IM-Endpoints L1 (incident routing needs the owner and sponsor structure), ML-Endpoints L1 (logging baseline needs the inventory).

Desired Outcomes

• Shadow endpoint AI is visible, attributed to a named owning team or user population, and trending down quarter-over-quarter.
• A single AI/HAI endpoint inventory is the authoritative source of truth across Security, IT, Engineering, Product, Legal/Privacy, and HR.
• An accountable executive owns AI/HAI endpoint risk; decision rights for approval, block, exception, and go-live are unambiguous across all seven archetypes.
• The distinction between own-built endpoint AI (chatbots, mobile AI apps, edge AI devices, governed here) and vendor AI consumed on endpoints (M365 Copilot, coding copilots, cross-referenced with Vendors domain) is operationalized in the inventory.
• Practice maturity is measurable from a small, automatable metric set rather than from activity counts.
• The program is an enabler, fast-track for Low-tier developer-only AI assistants, full review for Critical-tier customer-facing chatbots with data-egress potential.

Activities

A) Charter the AI/HAI Endpoint Assurance program

Publish a short program charter that names the problem (shadow AI at the endpoint layer, AI features silently enabled in SaaS productivity suites, own-built customer-facing AI surfaces without security review, AI browser extensions exfiltrating sensitive content, edge devices running unreviewed on-device models), defines scope across all seven archetypes, and assigns accountable ownership.

Charter elements: - Problem statement, why AI/HAI at endpoints is a distinct risk category: endpoints are the closest layer to regulated data, customer interactions, and user identity; AI capabilities at this layer can exfiltrate content through model APIs, make decisions that affect customers without disclosure, and operate autonomously on data that engineers and employees paste or speak without understanding the downstream flow; EU AI Act Art. 50 transparency obligations attach to own-built customer-facing AI interfaces; GDPR Art. 32 requires securing endpoints processing personal data through AI. - In-scope AI/HAI endpoint archetypes, the seven canonical types: 1. AI assistant / copilot on managed endpoint, coding assistants (Cursor, GitHub Copilot, Claude Code), internal AI assistants on engineering or employee laptops. 2. Browser-based AI tool, web UIs for AI services accessed from managed endpoints, browser extensions with AI capabilities (Grammarly AI, screenshot tools, page summarizers). 3. Chatbot / conversational UI, own-built customer-facing chat surfaces deployed on web or mobile properties. 4. Multi-modal AI interface, voice, image, or video AI capabilities running on endpoints (voice assistants, image recognition, video analysis). 5. AI-augmented productivity, M365 Copilot, Slack AI, Notion AI, Google Workspace Gemini, and similar AI features enabled within productivity SaaS on managed endpoints. 6. Mobile AI app, own-built mobile apps with AI features; vendor mobile AI assistants installed on managed mobile devices. 7. Edge AI device, kiosks, IoT devices, embedded systems running on-device inference. - Domain boundary: own-built endpoint AI surfaces (chatbots, mobile AI apps, edge AI devices) are governed primarily here; vendor AI tools consumed on endpoints (copilots, productivity AI, browser extensions from third parties) are cross-referenced with the Vendors domain; the Endpoints domain owns the endpoint-layer controls regardless of build origin. - Executive sponsor, typically the CISO co-sponsored by the CTO / Head of IT / CPTO; co-signed by Privacy/Legal where customer-facing archetypes are in scope. - Working group, Security, IT (endpoint management, MDM, EDR), Engineering (own-built chatbot and mobile AI), Product, Privacy/Legal, HR (acceptable use), one application-architect reviewer for own-built surfaces. - Decision rights, who can approve a new AI/HAI endpoint artifact; who can block one; who handles exceptions; who owns go-live for customer-facing AI surfaces. - Success definition for year one, a numerical target for the L1 outcome metrics below.

B) Build the AI/HAI endpoint inventory and discover shadow endpoint AI

Establish a single AI/HAI endpoint inventory as the program's source of truth. Seed it from authoritative endpoint management signals, then actively discover shadow AI using signals already available to IT, security, and SaaS admin teams, no new tooling required at L1.

Inventory fields (minimum): - Asset name, owning team or responsible owner, archetype (one of the seven above). - Endpoint population affected: number of endpoints, user roles (customer-facing, employee-general, developer-only, executive). - Own-built or vendor-provided; if vendor-provided, cross-reference ID to Vendors domain inventory. - Data classes accessible through the endpoint AI: regulated (PII / PHI / PCI / source code / customer confidential) vs. internal vs. public. - Action capability: read-only (the AI reads content and generates output) vs. action-taking (the AI can write, send, submit, or modify data on org or customer systems). - Customer-data egress potential: does the endpoint AI send user-provided or observed data to a vendor AI backend? (yes/no, if yes, cross-reference Vendors domain and DPA status). - Deployment scale: number of endpoints; whether concentrated in critical roles (executive, finance, legal, customer support, clinical). - Regulatory scope: EU AI Act Art. 50 transparency obligation triggered (own-built customer-facing AI interface, yes/no); sector-specific triggers (HIPAA, PCI-DSS, FERPA, COPPA). - Approval status: Sanctioned / Provisional / Under review / Prohibited / Awaiting Intake. - Risk tier assignment (populated at L2, see SM-Endpoints L2 Activity A). - Linked artifacts: TA threat snapshot, PC policy map entry, latest DR decision for own-built surfaces, ML logging-baseline status.

Discovery sources (at L1, use what IT, security, and SaaS admin teams already have): - MDM / UEM telemetry, Jamf, Intune, Kandji, VMware Workspace ONE: app inventory on managed endpoints filtered for AI application categories (productivity AI, coding assistants, AI utilities). App install and update events. - EDR / endpoint security telemetry, CrowdStrike, SentinelOne, Microsoft Defender for Endpoint: process execution and network-egress events to AI provider domains (openai.com, anthropic.com, api.github.com/copilot, .azure.com AI endpoints, .google.com AI endpoints, grammarly.com, notion.so, slack.com AI features, etc.). - Browser extension inventory, Chrome Enterprise Admin, Edge Admin Center, Firefox Enterprise Policy: extension listings filtered for AI capabilities; per-user and per-device extension presence. - SaaS admin consoles, M365 Admin Center (Copilot for M365 license and feature enablement), Slack Admin (Slack AI feature enablement, Slackbot AI features), Notion Admin (AI features enabled by workspace), Google Workspace Admin (Gemini for Workspace enablement, third-party AI apps authorized via OAuth). - Identity and access signals, IdP (Okta, Azure AD / Entra ID, Google Workspace SSO): OAuth app authorizations and sign-ins to AI service consoles from managed endpoints; new app authorizations by users. - Customer-facing AI inventory, Product owners and engineering managers self-attest own-built chatbot, conversational UI, and multi-modal AI surfaces deployed to customers; product roadmap reviews flag AI features in customer-facing apps. - Mobile app store and MDM mobile inventory, MDM mobile app catalog for managed mobile (iOS / Android) AI apps; own-built mobile app release notes flagging AI feature additions; public app store listing if own-built apps are consumer-facing. - IoT / kiosk asset registry, IT asset management system entries for edge devices, kiosks, and IoT deployments; firmware and model inventory for on-device inference devices. - Self-attestation, a 60-second intake form publicized to engineering, product, and IT; amnesty window for disclosing AI tools already installed or already shipped.

C) Establish foundational metrics that measure practice maturity and shadow endpoint AI reduction

Baseline and track a small set of outcome, process, and effectiveness metrics. Keep L1 metrics simple, automatable, and tied to the L1 outcome (shadow endpoint AI reduction and inventory coverage across all seven archetypes).

Shadow endpoint AI scoreboard (published quarterly to the executive sponsor): 1. AI/HAI endpoint assets in inventory (total / sanctioned / provisional / prohibited / awaiting intake), broken out by archetype. 2. New AI/HAI endpoint assets discovered this quarter and their intake status. 3. Shadow endpoint AI ratio trend (last 4 quarters): AI endpoint tools and surfaces without a known owner or intake record. 4. Endpoint AI Acceptable Use Policy attestation coverage across managed-endpoint user population. 5. Top 5 unmitigated AI endpoint risks (TA-flagged, MDM/EDR-flagged, customer-facing-surface-flagged, or external-advisory-flagged) with owners and remediation status.

Outcome Metrics (L1)

Process Metrics (leading)

• Discovery cadence, shadow endpoint AI discovery sweeps (MDM + EDR + browser-extension + SaaS-admin + identity-OAuth + self-attestation) run at least monthly.
• Intake SLA, new AI/HAI endpoint asset intake triaged within 5 business days; provisional approval within 10 BD for Low-tier archetypes (developer-only coding assistant, no regulated data, read-only).
• Inventory freshness, ≥80% of inventory records reviewed or updated in the last 90 days.
• Working-group cadence, at least monthly; minutes published.

Effectiveness Metrics (business value)

• Engineering and IT cycle-time impact, time from "user requests an AI tool for their endpoint" to "provisional approval or policy confirmation" decreases as the program matures.
• Reuse rate, % of endpoint AI asset requests resolved via the sanctioned-archetype catalog (already approved tool or reference configuration) vs. net-new intake; rising reuse indicates the program scales.
• Avoided-incident stories, documented cases where early discovery caught an endpoint AI risk before a data-exposure event (browser extension sending customer-support transcripts to an unvetted AI backend, own-built chatbot missing Art. 50 disclosure, mobile AI app requesting camera/microphone without consent basis).

Success Criteria

• Program charter published and sponsored by an accountable executive (CISO + CTO / Head of IT), with a cross-functional working group (Security, IT, Engineering, Product, Privacy/Legal, HR).
• AI/HAI endpoint inventory exists as a single source of truth covering all seven archetypes with ≥90% coverage of discovered endpoint AI assets within 12 months.
• Shadow endpoint AI ratio baselined and trending down for two consecutive quarters.
• ≥95% of managed-endpoint users have acknowledged the Endpoint AI AUP.
• Quarterly shadow endpoint AI scoreboard delivered to the executive sponsor with archetype-level breakdown.

Maturity Level 2

Objective: Risk-tier the AI/HAI endpoint inventory using a seven-dimension rubric, calibrate program intensity per tier, and measure practice maturity and shadow endpoint AI reduction per tier, establishing the rubric every other Endpoints-domain L2 practice depends on

At this level, the AI/HAI Endpoint Assurance program stops treating every endpoint AI asset the same. Risk tiers drive how deep intake goes, how often reviews happen, which detections fire, and what the sponsor sees on the scoreboard. A Critical-tier customer-facing chatbot processing customer PII with action-taking capability is not equivalent to a Low-tier developer-only read-only coding assistant. Per §9.3 of the v3.0 framing, the rubric established here is the prerequisite for L2 at PC, TA, SR, SA, DR, IR, ST, EH, ML, and IM in the Endpoints domain.

Dependencies

• SM-Endpoints L1 (required): inventory, charter, working group, and baseline metrics are the substrate L2 tiers and calibrates.
• PC-Endpoints L1 (required): the priority compliance map provides tier dimensions (EU AI Act Art. 50 transparency obligation, Art. 26 deployer duties, Art. 9 risk management, GDPR Art. 22/32/25, sector-specific endpoint scope).
• TA-Endpoints L1 (required): the threat library provides threat dimensions (data-egress surface, action-taking scope, multi-modal data capture, customer-data exfiltration vectors, browser-extension privilege scope).
• Supports / unblocks: PC-Endpoints L2 (tier-driven policy depth), TA-Endpoints L2 (per-artifact deep threat models for Critical/High), SR-Endpoints L2 (per-tier requirements packs), SA-Endpoints L2 (tier-conditional reference patterns), DR/IR/ST/EH/ML/IM-Endpoints L2 (all per-tier calibrated).

Desired Outcomes

• Every AI/HAI endpoint asset in the inventory carries a risk-tier assignment tied to seven explicit, auditable dimensions, not reviewer intuition.
• Program intensity is visibly differentiated: Critical customer-facing chatbots processing regulated PII with action-taking capability get the full program; Low developer-only coding assistants with read-only, public-data access get the fast track.
• The quarterly shadow endpoint AI scoreboard splits by tier and archetype; the sponsor can see which tiers are healthy and which are drifting.
• Tier movements (an asset upgraded when it gains customer exposure, action-taking capability, or regulated-data access; downgraded when scope shrinks) are tracked, rationale-logged, and sponsor-visible.
• Practice maturity is defensible per tier: "we are mature at Critical customer-facing surfaces and still building at Medium productivity AI" is a real, evidenced statement.

Activities

A) Define the AI/HAI endpoint risk-tier rubric

Four tiers, Critical / High / Medium / Low, assigned from seven auditable dimensions specific to AI/HAI endpoint assets:

1. User population, customer-facing public (no authentication required to interact with the AI) → Critical; authenticated customer (logged-in users of own products) → High; employee-general → Medium; developer-only or IT-only → Low.
2. Data classes accessible through the endpoint, regulated data (PHI / PCI / regulated PII / source code / customer confidential) accessible to or processable by the AI at the endpoint → elevate to Critical or High; internal-only data → Medium; public data only → Low.
3. Action capability, AI can take actions on org or customer systems (write to records, send messages, execute commands, submit forms, modify files, initiate workflows) → elevate; read-only, informational-only AI → no elevation from this dimension alone.
4. Customer-data egress potential, the endpoint AI sends data entered by or observed from users to a vendor AI backend or third-party model → elevate; purely local or on-device processing with no external data transmission → no elevation.
5. Deployment scale, large managed-endpoint population or concentration in critical roles (finance, legal, clinical, executive, customer support) → elevate; small population or non-critical roles → neutral.
6. Regulatory scope, EU AI Act Art. 50 transparency obligation triggered (own-built customer-facing AI interfaces) → elevate; sector-specific trigger (HIPAA endpoint, PCI-DSS cardholder-data-environment endpoint, FERPA for educational endpoints, COPPA for children-facing AI interfaces) → Critical; no sector trigger → neutral.
7. AI-content disclosure obligation, own-built AI interface visible to end users where Art. 50 notification is required → required at Critical; recommended at High.

The rubric is documented as a short table; tier is derived deterministically from inputs; human overrides are allowed but recorded with rationale and reviewed by the working group.

B) Calibrate program intensity per tier

Publish a tier-treatment matrix defining what each tier receives from the Endpoints-domain program. The matrix is the canonical reference every downstream Endpoints-domain practice inherits at L2.

Each downstream Endpoints-domain L2 practice inherits this calibration. The rubric and matrix are authored here, in SM-Endpoints L2, and changes flow through the SM working group.

C) Per-tier scoreboard and governance

The L1 shadow endpoint AI scoreboard becomes tier-aware at L2: - Inventory state reported by tier and by archetype. - Shadow endpoint AI ratio per tier, a Critical-tier unsanctioned customer-facing chatbot is a headline; a Low-tier unsanctioned coding assistant is a line item. - SLA adherence per tier (intake, DR, IR, ST, ML, IM) reported monthly. - Tier-movement log, assets that moved up this quarter (tighter treatment now applies) and those that moved down (with rationale). - Quarterly executive review explicitly discusses tier-balance, is program effort matching the risk profile of the endpoint AI surface?

Outcome Metrics (L2)

Process Metrics (leading)

• Tier-rubric review cadence, reviewed every 2 quarters; changes change-logged.
• Tier calibration exercise, at least quarterly, a sample of 20 endpoint AI assets re-tiered by a second reviewer; drift tracked.
• Per-tier queue depth monitored; no tier's backlog exceeds a published threshold.
• Working-group sprint to onboard the next downstream practice's L2 calibration.

Effectiveness Metrics (business value)

• Effort allocation match, % of reviewer hours on Critical+High tiers vs. Medium+Low; should rise relative to L1.
• IT and engineering throughput at Low/Medium tiers, sanctioned-archetype catalog reuse accelerates approval time vs. greenfield intake.
• Avoided-incident stories where tier-differentiation caught risk earlier (Critical-tier own-built chatbot caught at DR before customer-data-egress exposure went live; High-tier mobile AI app flagged for missing consent UX before public release).
• Scoreboard drives budget, tier-level dashboards referenced in quarterly IT and security planning.

Success Criteria

• Risk-tier rubric published and applied; tier assigned to 100% of inventory.
• Tier-treatment matrix published; downstream practices (PC, TA, SR, SA, DR, IR, ST, EH, ML, IM) calibrated to it.
• Per-tier shadow endpoint AI ratio reported quarterly; Critical-tier unsanctioned endpoint AI = 0.
• Per-tier SLA adherence ≥90% across practices.
• Tier-movement governance active, changes logged with rationale, reviewed by the sponsor.

Maturity Level 3

Objective: Automate inventory and tier maintenance from MDM, browser, SaaS-admin, and identity signals; benchmark against external endpoint-AI peers; and contribute to industry endpoint-AI assurance standards

At this level, the program is predominantly signal-driven rather than ticket-driven. Inventory and tiering update from MDM app catalogs, browser-extension admin feeds, SaaS-admin AI-feature telemetry, and identity-OAuth events; human review is exception-based. The program benchmarks against external peers through CSA Endpoint Working Group, OASIS, and mobile-AI ISACs, and contributes to the endpoint-AI assurance ecosystem through standards bodies and open-source reference configurations.

Dependencies

• SM-Endpoints L2 (required): tiering and calibration must be settled before automation is trustworthy.
• ML-Endpoints L2+ (required): signals (MDM events, browser-extension telemetry, SaaS-admin AI-feature events, identity-OAuth events, endpoint-AI egress logs) need the monitoring pipeline behind them.
• EG-Endpoints L2+ (required): the endpoint-user literacy that lets users and IT self-attest inventory accurately.
• Supports / unblocks: the other 11 Endpoints-domain practices can move to L3 automation patterns because SM now supplies automated inventory and tier data.

Desired Outcomes

• Inventory accuracy is measured in hours of latency, not months.
• Tier assignments adjust automatically when dimensional inputs change (a developer tool gains customer-facing access, a productivity AI feature is enabled for a regulated-data team, an edge device gains action-taking capability); humans intervene only on exceptions.
• External benchmarking is routine, the program sponsor can answer "how do we compare on endpoint AI governance?" with specific deltas.
• The organization is a net contributor to the endpoint-AI assurance ecosystem, CSA, OASIS, OWASP MASVS, and mobile-AI ISAC working groups reference program outputs.
• The program's strategic ROI is demonstrable: program investment mapped to avoided loss (incidents prevented, regulatory exposure mitigated, faster sanctioned endpoint AI adoption).

Activities

A) Continuous inventory and tier automation from MDM, browser, SaaS-admin, and identity signals

• Inventory auto-updates from: MDM app catalog events (new AI app installed or updated on managed endpoints), browser-extension admin policy changes (new AI extension approved, added, or removed from allowlist), SaaS-admin AI-feature enablement events (M365 Copilot license assigned, Slack AI feature enabled for a team, Gemini for Workspace feature activated), identity-OAuth events (new AI service OAuth authorization from managed endpoint), own-built app release events (mobile app version released with new AI features, chatbot deployment events), edge-device firmware events (on-device model updated or new inference capability added), self-attestation and intake queue.
• Tier assignments are rule-based on the L2 rubric inputs; rule changes are versioned and replayable; tier changes auto-trigger downstream practice obligations (a Medium→Critical upgrade triggers DR, ST, ML reconfiguration and Art. 50 disclosure UX review).
• Human curation handles: new archetypes (a new category of endpoint AI not covered by existing rubric), ambiguous discoveries (a shared productivity tool that sometimes has AI features enabled), dimensional-input conflicts (a tool classified as employee-only that is also used in a customer-facing workflow).
• A data-quality SLO is published: ≥99% of active AI/HAI endpoint assets correctly tiered within 48 hours of a material change; ≥95% inventory completeness against discovery-source reconciliation.

B) External benchmarking

• Program metrics compared against peer benchmarks via:
• CSA Endpoint Security Working Group and CSA AI Safety Initiative.
• OASIS AI governance working groups relevant to endpoint and conversational AI.
• Mobile-AI ISACs and sector-specific groups (FS-ISAC, H-ISAC, IT-ISAC mobile and endpoint tracks).
• OWASP MASVS (Mobile Application Security Verification Standard) for own-built mobile AI app benchmarking.
• BSIMM or comparable observational data on what peer organizations govern at the endpoint AI layer.
• Formal peer roundtables (CISO communities, endpoint security practitioners with AI workloads).
• A published "how we compare" brief refreshed semi-annually covers: inventory coverage, shadow endpoint AI ratio, per-tier SLA adherence, automation level, Art. 50 disclosure UX compliance rate, endpoint-AI data-egress control coverage.
• Benchmark deltas inform program investment, board-level narrative, and the next year's L2/L3 priorities.

C) Contribute anonymized endpoint-AI assurance intelligence

• Contribute to:
• CSA AI Safety Initiative (endpoint-AI controls matrix, browser-extension governance guidance, SaaS AI feature governance reference).
• OWASP MASVS and OWASP Mobile Application Security Testing Guide (MASTG) for own-built mobile AI app security patterns.
• OASIS conversational AI and chatbot security standards.
• NIST AI RMF Playbook (deployer-duty evidence patterns for endpoint-facing AI).
• EU AI Act Art. 50 transparency implementation guidance (reference UX patterns for own-built conversational and multi-modal AI interfaces).
• Sector ISACs where endpoint-AI working groups accept practitioner input.
• Target: minimum 4 substantive contributions per year; quality over volume; every contribution anonymized and legally vetted.

Outcome Metrics (L3)

Process Metrics (leading)

• Automation health, signal-feed freshness and error rate monitored; on-call paged when a feed staleness threshold is exceeded.
• Benchmarking cadence honored (semi-annual brief published on schedule).
• Contribution pipeline always has ≥2 items in-flight.
• Tier-rule change-log healthy, rule changes versioned, replayable, reviewed quarterly by the working group.

Effectiveness Metrics (business value)

• Sponsor decisions (budget, headcount, scope) citing benchmark data and tier-level metrics.
• Industry recognition, invitations to working groups, citations of program contributions, peer adoption of published reference configurations.
• Talent, the program attracts experienced endpoint security and product security engineers with AI governance experience.
• Faster sanctioned endpoint AI adoption, time from "user or team requests an AI tool" to "provisional approval" is industry-leading.

Success Criteria

• Inventory auto-update SLO published and met.
• Tier-assignment automation operational with published rules, replayable change-log, and exception-based human review.
• Semi-annual external-benchmarking brief published to the sponsor with ≥5 peer-comparable metrics.
• ≥4 substantive industry contributions per year, anonymized and cited.
• ROI narrative including external benchmarks delivered to exec/board at least annually.

Key Success Indicators

Level 1: - AI/HAI Endpoint Assurance program charter published and sponsored by an accountable executive (CISO + CTO / Head of IT), with a cross-functional working group (Security, IT, Engineering, Product, Privacy/Legal, HR). - AI/HAI endpoint inventory exists as a single source of truth, covering all seven archetypes (AI assistant/copilot on managed endpoint, browser-based AI tool, chatbot/conversational UI, multi-modal AI interface, AI-augmented productivity, mobile AI app, edge AI device). - Shadow endpoint AI actively discovered each month from MDM, EDR, browser-extension admin, SaaS-admin console, identity-OAuth, customer-facing-surface, mobile app store, and IoT asset-registry signals, reconciled against the inventory. - Endpoint AI Acceptable Use Policy acknowledged by ≥95% of managed-endpoint users. - Foundational metrics baselined: inventory coverage, shadow endpoint AI ratio, AUP attestation, intake SLA; quarterly shadow endpoint AI scoreboard delivered to the exec sponsor with archetype-level breakdown.

Level 2: - Risk-tier rubric published and applied, 100% of inventory carries a current tier from auditable dimensions (user population, data classes accessible, action capability, customer-data egress potential, deployment scale, regulatory scope, AI-content disclosure obligation). - Tier-treatment matrix published; downstream practices (PC, TA, SR, SA, DR, IR, ST, EH, ML, IM) calibrated to it. - Quarterly scoreboard reports per tier and per archetype; Critical-tier unsanctioned endpoint AI = 0. - Per-tier SLA adherence ≥90% across program activities. - Tier-movement governance operating with logged rationale and sponsor review.

Level 3: - Inventory auto-update latency ≤48 hours for material changes; ≥80% of curation is automated; ≥99% inventory completeness against discovery-source reconciliation. - Tier-assignment automation operates on a published, versioned rule set with exception-based human review; tier changes auto-trigger downstream practice obligations within 24 hours. - Semi-annual external-benchmarking brief published to the sponsor, citing at least five peer-comparable metrics from CSA / OASIS / OWASP MASVS / sector ISACs. - ≥4 substantive anonymized industry contributions per year (CSA AI Safety Initiative, OWASP MASVS, OASIS, NIST AI RMF, EU AI Act transparency guidance, sector ISACs). - Executive / board ROI narrative refreshed at least annually with external benchmarks and documented avoided-loss examples.

Common Pitfalls

Level 1: - ❌ Inventory is seeded only from "AI tools IT knows about", misses browser extensions installed by users without IT approval, AI features silently enabled by SaaS admins in productivity tools, own-built chatbots launched by product teams without security notification, and mobile AI apps installed on managed devices via personal App Store accounts. - ❌ Treating all endpoint AI as a Vendors-domain concern, own-built chatbots, mobile AI apps, and edge AI devices are first-party surfaces with deployer-duty obligations; the vendor question is a sub-concern for vendor-provided AI consumed at endpoints. - ❌ Discovery is passive (waiting for IT tickets) rather than active, MDM app telemetry, browser-extension admin reports, and SaaS-admin AI-feature dashboards exist but no one pulls them monthly. - ❌ Executive sponsor is IT-only; CISO and product/engineering leadership are not co-owners, so the program cannot govern own-built customer-facing AI surfaces. - ❌ Metrics count devices scanned rather than outcomes, shadow endpoint AI ratio never baselined; the program cannot demonstrate shadow AI is decreasing. - ❌ No amnesty window, employees hide AI tools they have installed rather than disclose them; shadow AI inventory stays permanently incomplete. - ❌ Inventory archetypes too coarse ("AI app on endpoint"), a Critical customer-facing chatbot and a Low developer coding assistant get conflated; the program cannot tier later without re-inventorying.

Level 2: - ❌ Tier-rubric inputs are subjective ("important endpoint," "sensitive data"), reviewers tier differently; auditors cannot verify; tier movements feel political rather than mechanical. - ❌ Tier-treatment matrix published but not enforced, Critical own-built chatbots routed through the same fast-lane as Low coding assistants; calibration exists on paper only. - ❌ Scoreboard still reported in aggregate, hiding that Critical-tier shadow endpoint AI is present because overall averages look acceptable. - ❌ Tier upgrades get resistance from product teams because they trigger more review, no governance on tier-movement leaves the program stuck at initial under-tiered assignments. - ❌ Downstream practices treat tier as advisory, DR/IR/ST/ML do not differentiate scope by tier, defeating the purpose of L2. - ❌ Rubric over-engineered, too many dimensions, tier derivation becomes an oracle ritual; IT and product teams cannot apply it without the security team acting as intermediary.

Level 3: - ❌ Automation runs without a data-quality SLO, signal-driven inventory silently drifts and IT teams stop trusting it. - ❌ Benchmarking chooses peers that flatter the program (comparing against startups without customer-facing AI when operating at enterprise scale with regulated data). - ❌ Industry "contributions" are conference talks about the program, not technical artifacts that land in CSA / OWASP / OASIS / NIST working group deliverables. - ❌ Automated tiering rules fire too noisily on every SaaS minor update, IT teams disable the signal-source rather than tune the rule sensitivity. - ❌ ROI narrative decouples from reality, external benchmarks cited but the program's own metrics are stale; the sponsor stops trusting the briefing deck.

Practice Maturity Questions

Level 1: 1. Is there a published AI/HAI Endpoint Assurance program charter with a named executive sponsor (CISO + CTO / Head of IT), a cross-functional working group, and clear decision rights for approval, block, exception, and go-live across all seven endpoint AI archetypes (AI assistant/copilot, browser-based AI tool, chatbot/conversational UI, multi-modal AI interface, AI-augmented productivity, mobile AI app, edge AI device)? 2. Does a single AI/HAI endpoint inventory exist, seeded from MDM, EDR, browser-extension admin, SaaS-admin console, identity-OAuth, customer-facing AI, mobile app store, and IoT asset-registry signals, covering all seven archetypes with ≥90% coverage of discovered endpoint AI assets within 12 months? 3. Are the L1 outcome metrics baselined and reported quarterly to the sponsor, inventory coverage, shadow endpoint AI ratio (≤15% and trending down), AUP attestation (≥95% of managed-endpoint users), assets with named owner (100%), and known data-exposure events from endpoint AI assets?

Level 2: 1. Is every AI/HAI endpoint asset in the inventory assigned a risk tier based on the seven auditable dimensions (user population, data classes accessible, action capability, customer-data egress potential, deployment scale, regulatory scope, AI-content disclosure obligation), with a published tier-treatment matrix driving differential program intensity? 2. Is there a published tier-treatment matrix driving differential controls across PC, TA, SR, SA, DR, IR, ST, EH, ML, IM, with ≥95% of Critical-tier endpoint AI assets receiving full-scope treatment in the last 12 months? 3. Does the quarterly scoreboard report per tier and per archetype (with Critical-tier unsanctioned endpoint AI explicitly tracked at zero), and does tier-movement get logged and reviewed by the program sponsor?

Level 3: 1. Does inventory and tier assignment auto-update from live MDM, browser-extension admin, SaaS-admin, identity-OAuth, own-built app release, and IoT telemetry signals with a published data-quality SLO, and is ≥80% of curation handled automatically with exception-based human review? 2. Do you publish a semi-annual external-benchmarking brief comparing the program against at least five peer-comparable metrics via CSA / OASIS / OWASP MASVS / sector ISACs, and does it drive program investment decisions? 3. Does the program contribute at least four substantive, anonymized artifacts per year to the endpoint-AI assurance ecosystem (CSA AI Safety Initiative, OWASP MASVS, OASIS, NIST AI RMF, EU AI Act transparency guidance, sector ISACs), and does the exec/board ROI narrative cite external benchmarks?

Document Version: HAIAMM v3.0 Practice: Strategy & Metrics (SM) Domain: Endpoints Last Updated: 2026-05-14 Author: Verifhai