Implementation Review (IR)

Endpoints Domain - HAIAMM v3.0

Practice Overview

Objective: Verify, at deployment and on a recurring cadence, that the actual configuration of AI/HAI-enabled endpoints and user-facing AI interfaces matches the design approved at DR, and that it stays there as the endpoint evolves.

Description: IR-Endpoints is the configuration check for AI/HAI endpoint deployments, the moment a reviewer opens the MDM console, the browser admin policy dashboard, the SaaS admin audit log, and the device attestation report and confirms that what is running matches the DR decision record. At L1 the review runs at deployment, at least annually, and on material change (new AI feature enabled tenant-wide, DLP scope changed, Art. 50 disclosure removed, extension-allowlist updated, model or firmware signature changed). At L2, IR-Endpoints consumes MDM webhook events, browser-policy state feeds, SaaS-admin webhooks, mobile MDM scan deltas, edge device attestation freshness, and vendor admin API probes to detect configuration drift continuously for High and Critical-tier endpoints. Findings are severity-tagged and SLA-bound per the SM-Endpoints L2 tier-treatment matrix; they feed IM-Endpoints for tracking and resolution. Vendor no-train flags, Art. 50 disclosure rendering, and logging flows are probed recurrently, not trusted from contract language or design text alone.

Context: The gap between the approved endpoint design and the deployed configuration is the primary source of silent security exposure in AI/HAI endpoint deployments. A chatbot's Art. 50 disclosure is present in the DR record but removed in a UX refresh. A SaaS-AI feature is scoped to the HR workspace in the DR checklist but enabled org-wide by a platform admin. A managed-endpoint AI assistant is approved for MDM-enrolled devices but also installed on personal BYOD by an undetected sideload. IR-Endpoints closes these gaps by making the implementation check systematic, evidence-based, and recurring, not a one-time pre-launch checkbox or a scramble when an incident reveals a configuration regression.

Maturity Level 1

Objective: Run per-archetype implementation reviews at deployment, annually, and on material change, verifying deployed endpoint configuration matches the SA-Endpoints pattern, the DR decision, and that the SR-Endpoints REM evidence is current

At this level, the gap between approved design and deployed endpoint configuration is systematically checked at the moments it matters most. Every review produces findings with severity tags, named owners, and SLA-bound resolution dates.

Dependencies

• DR-Endpoints L1 (required): the approved DR decision record is the specification IR checks against; without it there is no authoritative baseline.
• SR-Endpoints L1 (required): the REM defines which requirements must be evidenced; IR verifies the evidence is current and accurate.
• SA-Endpoints L1 (required): the SA reference pattern defines the intended configuration shape; IR checks adherence to the pattern's controls.
• EG-Endpoints L1 (required): reviewers must understand AI/HAI endpoint archetypes and know where to look for pattern controls in MDM consoles, browser admin dashboards, SaaS admin settings, and device attestation reports.
• Supports / unblocks: ST-Endpoints L1 (tests run against the verified configuration), EH-Endpoints L1 (hardening acts on IR findings), IM-Endpoints L1 (IR findings become issues in the backlog), ML-Endpoints L1 (logging configuration verified here feeds monitoring).

Desired Outcomes

• The gap between the DR-approved design and the live endpoint configuration stays small and short-lived.
• Material configuration drift is found by the program, not by an external auditor or an incident.
• Every finding has a named owner, a severity tag, and a SLA-bound resolution date; aging findings are visible to the program sponsor.
• Material changes to a production AI/HAI endpoint always trigger a review before the change goes live, SaaS feature enablements, extension-allowlist updates, firmware and model signature changes, and DLP scope changes do not bypass the gate.
• The SR REM is kept current, evidence cited in the REM is verified to still be accurate, not just present.

Activities

A) Publish the per-archetype implementation review checklist

One checklist per SM-Endpoints archetype, focused on the configuration points where production reality most commonly drifts from the approved design. Each item is a yes/no with a required evidence artifact (MDM policy export, admin-console screenshot, SaaS audit log entry, device attestation report, log sample).

Common spine across all archetypes: - MDM policy applies as designed, DLP rules active and match the approved scope; AI assistant or app allowlist enforced; policy deployment scope matches the approved endpoint population; confirm via MDM console export (Jamf / Intune / Kandji), not from design text alone. - Config matches DR decision, the DR-approved posture for this archetype's configuration points is reflected in the live admin state; deviations flagged with severity. - SR REM evidence is current, a stratified sample of REM rows verified against current observable reality: Art. 50 disclosure confirmed via live UX sample-check; vendor no-train flag verified via admin API (not from contract alone); logging confirmed via log-export test; kill-switch path confirmed to function with a test record on file. - Logging is actually flowing to SIEM, pull a sample of interaction, admin-audit, and identity events from the logging pipeline; confirm they are present and match the format and retention policy specified in the SR REM. - Kill-switch / disable path actually works when triggered, execute the kill-switch test for the endpoint archetype and record the result; confirm the AI feature or assistant becomes unavailable within the specified SLA.

Archetype-specific additions:

• AI assistant / copilot on managed endpoint: MDM policy applies as designed (DLP rules active, assistant allowlist enforced on enrolled devices); SSO enforcement confirmed (assistant authenticates through org SSO, test with an unauthenticated session and confirm rejection); tool-allowlist enforcement confirmed in the deployed configuration (not only declared in the DR checklist); audit-log completeness confirmed (pull a sample of invocation logs and verify user identity and timestamp fields are populated).

• Browser-based AI tool: browser admin policy applies (extension allowlist enforced, verify via Chrome / Edge / Safari admin policy export); per-extension scope honored (declared permissions match the live extension manifest, compare installed version with the DR-approved manifest); DLP integration active (browser DLP policy enforced for content flowing to or from the extension, verify via a DLP test event); backend SSO confirmed (extension authenticates through org SSO, verify via sign-in flow test).

• Chatbot / conversational UI: Art. 50 disclosure rendered in live UX (sample-check the live interface and confirm the AI disclosure is present on the first and every subsequent interaction, record screenshot as evidence); output filter active (test with a known-bad input and confirm the filter blocks or flags the response); prompt-injection defense present in the deployed prompt structure (review deployed system prompt to confirm instruction / user-input separation); escalation path functional (customer can reach a human within the declared SLA, test the escalation flow).

• Multi-modal AI interface: modality-specific input validation active for each declared modality (confirm image, voice, and document validation controls are wired in the deployed configuration); output safety filter applied to each modality's output path (test with a known-bad input for each modality); cross-modal consistency confirmed (same intent via different modalities produces consistent safe output, record test evidence); Art. 50 disclosure present for each interaction mode.

• AI-augmented productivity (SaaS-AI on endpoint): SaaS admin state matches design, which AI features are enabled per tenant / role matches the DR-approved posture (verify via M365 / Slack / Workspace / Notion admin console export); admin-audit log complete (enablement and configuration events for AI features are captured and flowing to SIEM); conditional enablement confirmed (feature is scoped to the DR-approved tenant / role / group, not silently expanded to all users); vendor no-train flag verified via admin API (not from contract alone, confirm via the platform's admin settings or API endpoint).

• Mobile AI app: app version and local-model signature current (MDM confirms the installed app version and local model hash match the DR-approved values, verify via mobile MDM report); permission minimization confirmed (installed app's declared permissions match the minimum declared in the DR checklist, compare via MDM app-detail report); on-device integrity attestation confirmed (attestation check is active and passing, verify via MDM device compliance report).

• Edge AI device: firmware and model signature current (device attestation report confirms firmware version and model hash match the DR-approved values); boot attestation active and passing (attestation report shows secure-boot passing, not only device enrollment status); physical-tamper detection functional (tamper-detection mechanism is active and reporting to the management plane, verify via device status report); uplink encryption active (confirm TLS certificate and cipher suite match the DR-approved specification); remote-disable path confirmed functional (test the remote-disable command and verify the device's AI capability becomes unavailable within the declared SLA, record the test date and result).

B) Perform reviews at the right moments

Three triggers at L1:

• Deployment review: before the endpoint AI capability goes live (or before a new version is rolled out), verify the deployed configuration against the DR-approved design. No production rollout with a blocker finding open.
• Annual review: every active AI/HAI-enabled endpoint reviewed at least annually; scheduled from the SM-Endpoints inventory (last-IR-date field linked to a review-due alert).
• Material-change review: any of the following triggers an ad-hoc review before the change goes live: SaaS-AI feature enabled tenant-wide; DLP scope changed; extension-allowlist updated; AI assistant or app version changed beyond the DR-approved version range; Art. 50 disclosure UX modified; local model or firmware signature changed; managed-endpoint requirement scope changed; vendor no-train or data-handling setting changed.

Reviews are evidence-based, MDM exports, admin-console screenshots, SaaS audit log entries, or device attestation reports stored with the IR record. Target timebox: 20–60 minutes per endpoint depending on archetype complexity.

C) Track findings to closure

Every review produces zero or more findings. Each finding carries: - Severity: Critical (e.g., Art. 50 disclosure absent from live customer-facing chatbot; DLP scope collapsed to allow regulated data to reach the AI vendor model) / High / Medium / Low. Severity calibrated to the SM-Endpoints L2 tier-treatment matrix; at L1 use a consistent judgment rubric pending SM L2 formalization. - Owner: named engineer, admin, or team owner; not "the IT team." - SLA from SM tier-treatment matrix: Critical blocker resolved before production cutover or rollback required; High ≤7 days; Medium ≤30 days; Low ≤90 days or accepted residual. - Evidence: after-fix evidence artifact (admin-console screenshot, MDM compliance report, SaaS audit log entry, test record) linked to the finding before closure.

Findings feed IM-Endpoints as issues (for tracking, aging, trending) and loop back to SR-Endpoints where a finding reveals that an REM row's cited evidence was inaccurate, the REM row is updated before the finding is closed.

Drift sources verified at L1 (without continuous tooling): - MDM telemetry: Jamf / Intune / Kandji policy compliance reports; device enrollment and compliance state; AI app version and permission reports. - Browser admin policy state: Chrome / Edge / Safari admin policy export; extension allowlist compliance report; per-extension permission scope. - SaaS admin audit feeds: M365 admin audit log; Slack admin event log; Google Workspace admin audit; Notion admin settings, AI feature enable / disable events, data-scope configuration changes, permission changes. - Mobile MDM: app version compliance, model hash, device attestation passing status. - Edge device attestation reports: firmware version, model hash, secure-boot status, tamper-detection status.

Outcome Metrics (L1)

Process Metrics (leading)

• Annual review calendar populated from the SM-Endpoints inventory; endpoints nearing review-due date visible in advance.
• Material-change trigger wired to SM-Endpoints inventory material-change events; reviews queued within 5 business days of a confirmed material change.
• Reviewer backlog aging, no single reviewer more than 3 endpoints overdue.
• SR REM update loop active, % of IR findings that trigger an REM row update for the affected requirement.

Effectiveness Metrics (business value)

• Drift-caught-early rate, findings closed before they reach an incident or an external audit.
• REM accuracy improvement, % of REM rows where the cited evidence was verified current vs. stale; stale-evidence rate trends down as the IR / SR feedback loop operates.
• Avoided-incident stories, documented cases where IR caught a configuration regression before it was exploited or caused a data event.

Success Criteria

• Per-archetype IR checklists published, owned, and linked from the SM-Endpoints inventory record and the DR decision record.
• Deployment, annual, and material-change review triggers wired to the SM-Endpoints inventory; 100% of new AI/HAI-enabled endpoints in the last 90 days have a deployment IR record.
• ≥90% of active AI/HAI-enabled endpoints carry a current-year IR record.
• All Critical / blocker findings resolved before production cutover; High findings closed within 7 days with evidence linked.
• Findings-aging dashboard reviewed at least monthly by the program sponsor.

Maturity Level 2

Objective: Detect configuration drift continuously for Critical and High-tier endpoints via MDM webhook events, browser-policy state monitoring, SaaS-admin webhooks, mobile MDM scan deltas, edge attestation freshness, and vendor admin API recurrent probes; calibrate IR cadence per SM-Endpoints tier

At this level, implementation review stops being a point-in-time check and becomes a continuous signal for Critical and High-tier endpoints. Drift sources are wired to automated detection. Vendor no-train flags are validated recurrently via admin APIs, not trusted from contract language. SaaS-AI feature enablements at the tenant level automatically flag an event. IR cadence is explicitly differentiated by SM-Endpoints L2 risk tier.

Dependencies

• IR-Endpoints L1 (required): per-archetype checklists and findings workflow must be established.
• SM-Endpoints L2 (required): the risk-tier rubric (Critical / High / Medium / Low) drives IR cadence and depth per the tier-treatment matrix.
• SA-Endpoints L2 (required): MDM-encoded and browser-policy-encoded patterns establish the "correct" baseline that continuous drift detection measures against.
• Supports / unblocks: ST-Endpoints L2 (tests run against the continuously verified configuration), EH-Endpoints L2, ML-Endpoints L2 (monitoring configuration verified here feeds detections).

Desired Outcomes

• Configuration drift on Critical-tier endpoints is detected within days, not months.
• Vendor no-train flags are verified via admin API state recurrently, not trusted from contract language, and a vendor-side change (setting reset by a product update) is detected before it becomes a data-handling incident.
• A SaaS-AI feature enabled tenant-wide without a DR approval automatically flags an event in the IR pipeline.
• IR cadence visibly differentiates by tier: Critical gets semi-annual reviews plus continuous drift detection; Low gets deployment only.
• All Critical-tier IR findings carry severity tags and SLAs matching the SM-Endpoints L2 tier-treatment matrix.

Activities

A) Continuous drift detection from MDM, browser-policy, SaaS-admin, and device-attestation sources

Wire the following signal sources to an automated drift-detection pipeline for Critical and High-tier endpoints:

• MDM webhook events: Jamf / Intune / Kandji policy compliance events (DLP rule active/inactive, AI app allowlist changes, device enrollment / unenrollment, compliance policy drift) trigger an automated comparison against the DR-approved MDM policy baseline; material deviations open an IR finding automatically.
• Browser-policy state monitoring: Chrome / Edge / Safari admin policy state checked on a scheduled basis against the DR-approved extension allowlist and per-extension permission scope; deviations from the approved state open IR findings.
• SaaS-admin webhook (new AI feature enabled tenant-wide): M365 / Slack / Workspace / Notion admin event webhooks configured to flag any AI feature enablement event at the tenant or broad-group level; a feature-enablement event without a corresponding open DR approval is a Critical finding.
• Mobile MDM scan deltas: scheduled mobile MDM scan compares installed app version and model hash against the DR-approved values; version or hash changes since the last scan open IR findings.
• Edge attestation freshness: edge device attestation report freshness monitored; a device that has not produced a current attestation within the declared window (e.g., ≤7 days for Critical-tier) opens an IR finding.
• Vendor admin API recurrent probes: vendor admin APIs probed recurrently for no-train state for Critical and High-tier endpoints (see §B below).

Detection latency targets: Critical-tier drift detection ≤7 days from change event to finding opened; High-tier ≤30 days.

B) Vendor admin API probing for no-train and data-handling settings

No-train and data-handling settings are probed recurrently via vendor admin APIs for Critical and High-tier endpoints, not trusted from contract language or one-time admin-console screenshots:

• Amazon Bedrock: AWS Service Control Policy and Bedrock configuration, confirm no model fine-tuning on customer data paths; confirm CloudTrail logging active.
• Google Vertex AI / Gemini: Google Cloud Organization Policy, confirm data logging settings and no training-data opt-in active.
• Azure OpenAI: Azure resource settings, confirm data-processing and abuse-monitoring settings match the DR-approved posture.
• OpenAI (API): Org Settings API, confirm data_controls.training_data_sharing is false for applicable API keys.
• Anthropic: Organization admin settings, confirm model training usage terms reflect no-train commitment.
• Other vendors (SaaS-AI productivity): M365 Copilot admin settings, Slack AI admin settings, Workspace AI admin settings, Notion AI admin settings, confirm data-scope and training-opt-out settings match the DR-approved posture; use admin API where available; UI-based verification with screenshot evidence as fallback.

Probing cadence: Critical-tier, monthly; High-tier, quarterly. Delta from the previous probe opens an IR finding with severity matching the data-class impact of the change.

C) Tier-calibrated IR cadence

Publish and enforce per the SM-Endpoints L2 tier-treatment matrix: - Critical: deployment + semi-annual + material-change-triggered + continuous drift detection. - High: deployment + annual + material-change-triggered. - Medium: deployment + annual. - Low: deployment + re-review on material change.

Every endpoint in the SM-Endpoints inventory has a last-IR-date and next-IR-due field; Critical-tier endpoints with no IR in the last 180 days are escalated to the program sponsor.

Outcome Metrics (L2)

Process Metrics (leading)

• Drift-detection pipeline health monitored, % Critical endpoints producing a fresh signal in the last 7 days; on-call alert if feed silent for >48 hours.
• Vendor admin API probing calendar maintained; missed probes tracked as process-metric failures.
• SaaS-admin webhook coverage tracked, % of in-scope SaaS platforms with webhook or polling integration configured.
• IR backlog tier-aware; Critical-tier findings never wait behind Low-tier queue items.

Effectiveness Metrics (business value)

• Drift caught before ST/ML detections or incidents, trend measured over quarters.
• Reduced audit findings on configuration claims, external auditors (SOC 2, ISO 42001) find IR evidence sufficient without supplemental screenshots or interviews.
• Vendor admin API probing reveals vendor-side changes (no-train setting reset by a product update, data-scope expanded by a feature launch) before they become data-handling incidents.

Success Criteria

• ≥90% of Critical-tier endpoints under continuous drift detection; median detection latency ≤7 days.
• Vendor no-train and data-handling settings verified via admin APIs for ≥80% of Critical/High-tier endpoints on a monthly (Critical) and quarterly (High) probing cadence, not from contract language alone.
• ≥95% of SaaS-AI tenant-wide feature enablements automatically flagged and routed to IR within 24 hours.
• Tier-cadence adherence ≥95%; Critical-tier findings aged per the SM-Endpoints L2 tier-treatment matrix SLAs.

Maturity Level 3

Objective: Continuous configuration attestation for Critical-tier endpoints with a daily attestation signal confirming pattern compliance and evidence freshness, automatic IM ticket on drift, and contribution to CSA endpoint, OWASP MASVS, and OASIS attestation standards

At this level, configuration for Critical-tier AI/HAI-enabled endpoints is not periodically reviewed, it is attested continuously. Every Critical endpoint produces a daily attestation signal confirming that pattern compliance, REM evidence freshness, and deployed configuration are within tolerance. Drift opens an IM-Endpoints ticket automatically. Per-archetype configuration baseline schemas are contributed to CSA endpoint working groups, OWASP MASVS, and OASIS standards bodies.

Dependencies

• IR-Endpoints L2 (required): continuous drift detection, vendor admin API probing, SaaS-admin webhook integration, and tier-calibrated cadence must be in place.
• SA-Endpoints L3 (required): externalized patterns supply the attestation frame for automated compliance scans.
• ML-Endpoints L2+ (required): runtime signals (MDM telemetry, logging completeness, SaaS-admin event anomalies) are evidence sources the attestation pipeline reads.
• SR-Endpoints L3 (alignment): machine-readable REM schema provides the evidence-freshness signals the attestation pipeline validates against.

Desired Outcomes

• Every Critical-tier AI/HAI-enabled endpoint produces a daily attestation signal, pattern compliance, evidence freshness, and deployed configuration are continuously within tolerance.
• Drift automatically opens an IM-Endpoints ticket; the program does not wait for the next scheduled review to act.
• Per-archetype configuration baseline schemas (what "correct" looks like for each endpoint archetype at each tier) are published to CSA endpoint working groups, OWASP MASVS, and OASIS, the program is a net contributor to the AI-assurance endpoint implementation-review ecosystem.
• IR reviewer-hours per endpoint per year trend down as attestation absorbs routine checks; reviewers focus on novel configurations and exception escalations.

Activities

A) Daily attestation signal for Critical-tier endpoints

Each Critical-tier AI/HAI-enabled endpoint produces a daily composite attestation signal covering three dimensions:

1. Pattern compliance: automated SA-Endpoints-pattern compliance scan confirms key controls are present and active in the deployed configuration (MDM DLP rules match approved scope; browser extension allowlist enforced; SaaS-AI feature state matches DR-approved posture; Art. 50 disclosure confirmed present in live UX via automated probe; device firmware and model signatures current). Uses the MDM webhook and SaaS-admin feeds from IR L2, now producing machine-readable output on a daily schedule.

2. Evidence freshness: the SR-Endpoints REM's evidence citations are checked for staleness, are the cited evidence artifacts (vendor no-train API probe result, Art. 50 disclosure screenshot, kill-switch test record, MDM compliance report) within their defined freshness window? Stale evidence opens a finding automatically. Evidence freshness windows (per SM-Endpoints tier-treatment matrix): vendor admin API probe ≤30 days (Critical) / ≤90 days (High); Art. 50 disclosure sample-check ≤7 days (Critical chatbots); kill-switch test ≤90 days; MDM policy compliance report ≤24 hours.

3. Configuration within tolerance: deployed configuration (SaaS-AI feature scope, DLP policy state, extension allowlist, device firmware and model hash) checked against the DR-approved baseline. Tolerances defined per-control (minor app version updates within the same approved version range are tolerated; a version change outside the range is not tolerated without a DR review; a SaaS-AI feature scope expansion is never tolerated without a DR re-review).

Attestation artifacts are machine-readable, signed, and stored in the SM-Endpoints inventory record. They are regulator-consumable for EU AI Act Art. 9 risk-management evidence and deployer-duty records per Art. 26.

Drift opens an IM-Endpoints ticket automatically; the ticket carries the drift dimension (pattern compliance / evidence freshness / configuration), the specific control that failed tolerance, and a link to the DR decision record.

B) Contribute per-archetype configuration baseline schemas

Publish per-archetype IR configuration baseline schemas, defining what "correct" implementation looks like for each AI/HAI endpoint archetype at each SM-Endpoints tier, to: - CSA endpoint working groups, reference attestation schema for AI/HAI-enabled endpoint configurations. - OWASP MASVS extensions, mobile AI app and browser-based AI tool configuration baselines; practitioner-level checklist items and evidence-type definitions. - OASIS AI assurance standards, per-archetype endpoint configuration controls mapped to OASIS control categories.

Internal practice remains aligned to the published external versions; internal-only deviations are proposed as upstream changes.

Adoption tracked: citations, forks, direct acknowledgment from peer organizations, inclusion in external tooling or assessment frameworks.

C) Automated drift-to-IM escalation and SLA enforcement

• All IR findings (whether from daily attestation or from periodic reviews) flow into IM-Endpoints automatically with severity and SLA pre-populated from the SM-Endpoints L2 tier-treatment matrix.
• IM-Endpoints SLA clock starts when the finding is opened; overdue Critical findings escalate to the program sponsor automatically at 50% and 100% of the SLA window.
• Post-incident reviews in IM-Endpoints that touch a configuration control automatically re-examine the IR record for the affected endpoint, was the drift detectable earlier? What attestation rule would have caught it? The answer updates the attestation rule and the IR checklist.

Outcome Metrics (L3)

Process Metrics (leading)

• Attestation-pipeline health monitored, % Critical endpoints producing a fresh attestation signal in the last 24 hours; on-call paged if any Critical endpoint silent for >24 hours.
• Schema publication pipeline, at least one schema in-draft, in-review, or published at any time.
• IM escalation automation tested quarterly, confirm Critical-finding auto-escalation fires correctly for a synthetic finding.
• Post-incident IR feedback loop active, % of IM post-incident reviews that produce an attestation rule update.

Effectiveness Metrics (business value)

• IR reviewer-hours per endpoint per year trending down as attestation absorbs routine checks.
• Zero Critical-tier go-live events where the DR-approved design and the deployed endpoint configuration are materially different.
• Auditor configuration-claim findings approaching zero as daily attestation corroborates internal records; regulator inquiries answered via machine-readable attestation artifacts without manual assembly.
• External recognition, configuration baseline schemas cited by peer organizations, security tooling vendors, or regulatory guidance documents.

Success Criteria

• Daily attestation operating for ≥90% of Critical-tier endpoints across all three dimensions (pattern compliance, evidence freshness, configuration tolerance); deviations auto-opening IM tickets within 1 hour.
• Zero stale-evidence violations for Critical-tier REMs; High-tier stale-evidence rate trending down.
• Per-archetype configuration baseline schemas published to CSA endpoint working groups, OWASP MASVS, or OASIS with documented external adoption.
• IR reviewer-hours per Critical endpoint per year trending down over two consecutive quarters.

Key Success Indicators

Level 1: - Per-archetype IR checklists published, one per SM-Endpoints archetype (AI assistant on managed endpoint, browser-based AI tool, chatbot / conversational UI, multi-modal AI interface, SaaS-AI productivity feature, mobile AI app, edge AI device), covering MDM-policy-matches-design verification, config-matches-DR verification, SR REM evidence currency check, logging-flow verification, and kill-switch test; chatbot checklist verifies Art. 50 disclosure in live UX; edge device checklist verifies firmware and model signature currency and remote-disable function. - Deployment, annual, and material-change review triggers wired to the SM-Endpoints inventory; 100% of new AI/HAI-enabled endpoints in the last 90 days have a deployment IR record; ≥90% of active endpoints carry a current-year IR record. - All Critical / blocker findings resolved before production cutover; High findings closed within 7 days with evidence linked; findings-aging dashboard reviewed monthly by the program sponsor. - SR REM update loop active, IR findings that reveal stale or inaccurate REM evidence trigger REM row updates before the finding is closed.

Level 2: - ≥90% of Critical-tier endpoints under continuous drift detection (MDM webhook events, browser-policy state monitoring, SaaS-admin webhook, mobile MDM scan deltas, edge attestation freshness); median detection latency ≤7 days. - Vendor no-train and data-handling settings verified via admin APIs (Bedrock / Vertex / Azure OpenAI / OpenAI / Anthropic / SaaS-AI platforms) for ≥80% of Critical/High-tier endpoints on a monthly (Critical) and quarterly (High) probing cadence, not from contract language alone. - ≥95% of SaaS-AI tenant-wide feature enablements automatically flagged and routed to IR within 24 hours. - Tier-cadence adherence ≥95%: Critical on semi-annual + continuous, High on annual, Medium on annual, Low on deployment + material-change.

Level 3: - ≥90% of Critical-tier endpoints producing a daily attestation signal across all three dimensions (pattern compliance, evidence freshness, configuration tolerance); deviations auto-opening IM tickets within 1 hour. - Zero stale-evidence violations for Critical-tier REMs; evidence freshness windows enforced per the attestation pipeline. - Per-archetype configuration baseline schemas published to CSA endpoint working groups, OWASP MASVS, or OASIS with documented adoption. - IR reviewer-hours per Critical endpoint per year trending down over two consecutive quarters as attestation absorbs routine checks.

Common Pitfalls

Level 1: - ❌ IR treated as a one-time deployment formality, no annual re-review and no material-change trigger; configuration drift accumulates silently for quarters until an audit or an incident surfaces it. - ❌ Reviewers take the DR decision record at face value without opening the MDM console or SaaS admin dashboard, the DLP rule is declared in the checklist but never confirmed active in the deployed MDM policy. - ❌ Art. 50 disclosure verified from the design mockup rather than the live UX, the disclosure was present in the prototype but removed in a UX refresh before launch. - ❌ Vendor no-train settings verified from contract language or DPA text without opening the vendor admin console, the setting can be reset by a vendor product update and the team does not know. - ❌ Kill-switch documented in the DR record but never tested, the IR checklist has a "kill-switch: yes" box checked without an actual test execution and a recorded result. - ❌ Material-change trigger is not wired to SM-Endpoints inventory events, SaaS-AI feature enablements and DLP scope changes ship without triggering an IR; the review calendar misses the changes entirely.

Level 2: - ❌ MDM webhook integration exists but generates no findings on policy drift, the webhook fires but automated finding creation was never configured; drift detection is manual in practice. - ❌ SaaS-admin webhook is configured for M365 but not for Slack, Workspace, or Notion, new AI feature enablements on those platforms are not detected until the next annual review. - ❌ Vendor admin API probing is configured once at onboarding and never re-run, a no-train setting reset by a vendor product update goes undetected for months. - ❌ Tier-calibrated cadence exists on paper but Critical and Low-tier endpoints sit in the same review queue with no prioritization, Critical-tier endpoints wait behind Low-tier backlogs. - ❌ Drift findings from automated detection dead-end in an alert dashboard rather than auto-opening IM tickets, findings age without owners.

Level 3: - ❌ Daily attestation signals show green across all Critical endpoints but the underlying checks cover only MDM enrollment status, DLP rule scope, extension-allowlist enforcement, Art. 50 disclosure, and vendor no-train state are not checked; attestation is cosmetic. - ❌ Configuration baseline schemas published externally reflect L1 checklist items only, internal practice has advanced to continuous API probing and SaaS-admin webhooks; external adopters build on a stale baseline. - ❌ Attestation-exception queue overwhelms the team because configuration tolerance thresholds are too tight, every minor MDM policy refresh triggers a deviation; reviewers suppress the signal source rather than tune the tolerance rules. - ❌ Post-incident IR feedback loop exists in policy but never fires in practice, IM post-incident reviews do not include the IR-record re-examination step; attestation rules never update from incident learning.

Practice Maturity Questions

Level 1: 1. Is there a published, per-archetype IR checklist, one per SM-Endpoints archetype (AI assistant on managed endpoint, browser-based AI tool, chatbot / conversational UI, multi-modal AI interface, SaaS-AI productivity feature, mobile AI app, edge AI device), covering MDM-policy-matches-design verification, config-matches-DR verification, SR REM evidence currency check, logging-flow verification to SIEM, and kill-switch test execution, with the chatbot checklist verifying Art. 50 disclosure in the live UX and the edge device checklist verifying firmware / model signature currency and remote-disable function? 2. Do 100% of new AI/HAI-enabled endpoints going to production in the last 90 days carry a deployment IR record, and do ≥90% of all active endpoints carry a current-year IR record, with material-change triggers wired to SM-Endpoints inventory events, Critical / blocker findings resolved before production, and High findings closed within 7 days with evidence linked? 3. Are findings severity-tagged and tracked in IM-Endpoints with named owners and SLA-bound closure dates, and does every IR finding that reveals stale or inaccurate REM evidence trigger an SR REM row update before the finding is closed?

Level 2: 1. Are ≥90% of Critical-tier AI/HAI-enabled endpoints under continuous drift detection, via MDM webhook events, browser-policy state monitoring, SaaS-admin webhooks, mobile MDM scan deltas, and edge device attestation freshness, with median detection latency ≤7 days and automated finding creation on material deviations? 2. Are vendor no-train and data-handling settings verified via vendor admin APIs (Bedrock / Vertex / Azure OpenAI / OpenAI / Anthropic / SaaS-AI platforms) on a monthly (Critical) and quarterly (High) probing cadence, not from contract language alone, covering ≥80% of Critical/High-tier endpoints, with deltas from the previous probe opening IR findings with severity matching the data-class impact? 3. Are ≥95% of SaaS-AI tenant-wide feature enablements automatically flagged and routed to IR within 24 hours, and is tier-cadence adherence ≥95% with Critical-tier findings aged per the SM-Endpoints L2 tier-treatment matrix SLAs?

Level 3: 1. Are ≥90% of Critical-tier AI/HAI-enabled endpoints producing a daily attestation signal across all three dimensions (pattern compliance, evidence freshness, configuration tolerance), with deviations auto-opening IM-Endpoints tickets within 1 hour and zero stale-evidence violations for Critical-tier REMs? 2. Has the program published per-archetype configuration baseline schemas to CSA endpoint working groups, OWASP MASVS, or OASIS, with documented adoption and internal practice aligned to the published versions, and is IR reviewer-hours per Critical endpoint per year trending down over two consecutive quarters? 3. Is the post-incident IR feedback loop operational, IM-Endpoints post-incident reviews include a mandatory IR-record re-examination step, and ≥1 attestation rule update is produced per material incident, ensuring incident learning continuously improves the attestation coverage?

Document Version: HAIAMM v3.0 Practice: Implementation Review (IR) Domain: Endpoints Last Updated: 2026-05-14 Author: Verifhai