Security Testing (ST) - Endpoints Assessment

Assessment questionnaire for measuring maturity. Answer each question honestly based on current, implemented practices.

v3.0 framing: The canonical source-of-truth for Security Testing (ST) in the Endpoints domain is ../practices/ST-Endpoints-OnePager.md. Outcome metrics, activities, and success criteria below are verbatim from that one-pager. Canonical subject and through-lines: ../HAIAMM-v3.0-Framing.md.

Security Testing (ST) - Endpoints Domain

HAIAMM Assessment Questionnaire v3.0

Practice: Security Testing (ST) Domain: Endpoints Purpose: Assess organizational maturity in running adversarial test batteries and regression corpora against AI/HAI-enabled endpoints and user-facing AI interfaces, from foundational per-archetype batteries through continuous automated adversarial testing Scoring Model: Evidence + Outcome Metrics (see Scoring Methodology below)

Instructions

Answer each question honestly based on current, implemented practices (not plans or aspirations)
Each question has two components: Evidence (what you did) and Outcome Metrics (how well it worked)
Scoring uses 4 tiers: Fully Mature (1.0), Implemented (0.67), Partial (0.33), Not Implemented (0.0)
Answer progressively - Complete all Level 1 questions before Level 2
Level progression - Achieve ALL questions at lower level before advancing
Baseline first - Record current metric values before setting targets

Scoring Methodology

Score	Label	Criteria
1.0	Fully Mature	Evidence complete + ≥3 outcome metrics meet targets
0.67	Implemented	Evidence complete + 2 outcome metrics meet targets
0.33	Partial	Evidence partially complete + <2 outcome metrics meet targets
0.0	Not Implemented	No evidence of the activity

Level Score = average of the 3 question scores for that level Overall ST-Endpoints Score = weighted average: L1 × 0.5 + L2 × 0.3 + L3 × 0.2

Maturity Level 1

Objective: Establish a foundational per-archetype test battery and regression corpora for AI/HAI-enabled endpoints, and verify that every endpoint reaches production with a passed go-deployment battery on record

Question 1: Per-Archetype Test Battery

Q1.1: Is a per-archetype foundational test battery published for all seven AI/HAI endpoint archetypes (AI assistant on managed endpoint, browser-based AI tool, chatbot/conversational UI, multi-modal AI interface, SaaS-AI productivity feature, mobile AI app, edge AI device), with each test class tied to a TA-Endpoints archetype threat (HAI TTP + ATLAS tactic ID) and an SR-Endpoints requirement, defined inputs/outputs/pass-fail criteria, and an evidence artifact?

Evidence Required: - [ ] Test battery document published per archetype; covers per-archetype probes including DLP-paste-block test, managed-endpoint requirement test, tool-allowlist test (ATLAS TA0004), extension-scope test (OWASP Browser-Extension Top 10), prompt-injection regression CI corpus (ATLAS TA0001/TA0003), jailbreak regression corpus, multi-modal image-injection corpus (steganographic prompts, ATLAS TA0001 indirect), voice-injection corpus, Art. 50 disclosure-presence test (EU AI Act Art. 50), silent-enablement detection test, app-signature verification test, biometric-bypass-via-synthetic-media test, firmware signature test, and remote-disable test as applicable per archetype - [ ] Each test class records: inputs, expected output, pass/fail criteria, evidence artifact (log snippet, screenshot, or CI run link) - [ ] Each test class mapped to a TA-Endpoints archetype threat (HAI TTP + ATLAS tactic ID) and an SR-Endpoints requirement - [ ] Battery linked from SM-Endpoints inventory record and DR-Endpoints/IR-Endpoints artifacts for each AI/HAI-enabled endpoint - [ ] Named battery owner per archetype documented; quarterly re-run scheduled in advance - [ ] 100% of AI/HAI-enabled endpoints reaching production in the last 90 days have a passed go-deployment battery on record

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | % of AI/HAI-enabled endpoints reaching production with a passed go-deployment battery on record | % | % | ≥90% within 12 months; 100% for Critical/High-tier | ☐ | | | % Art. 50 disclosure-presence tests passing on every deployment update for customer-facing chatbots | % | % | 100% | ☐ | | | % archetype threat library entries covered by at least one test or corpus entry | % | % | ≥80% by end of year 1 | ☐ | | | % test failures routed to IM within 1 business day | % | % | 100% | ☐ | |

Metric Collection Guidance: - Go-deployment battery coverage: Query SM-Endpoints inventory for endpoints promoted to production in the last 90 days; cross-reference with test-run registry. Formula: endpoints_with_passed_battery / endpoints_reaching_production × 100 - Art. 50 disclosure pass rate: For every deployment update of customer-facing chatbots and multi-modal interfaces, check whether a disclosure-presence test run and pass record exists. Formula: deployment_updates_with_disclosure_pass / total_customer_facing_deployment_updates × 100. Source: test-run registry - Threat coverage: TA-Endpoints library entries mapped to test classes or corpus entries vs. total archetype library entries. Source: TA-Endpoints library × test metadata - IM routing rate: Test failure log timestamps vs. IM ticket creation timestamps. Source: CI failure log × IM system

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence of per-archetype test battery)

Evidence Location: __ Validation Date: __ Notes: ___

Question 2: Regression Corpora on Deployment

Q2.1: Are six regression corpora (prompt-injection, jailbreak, multi-modal injection, DLP-paste-block, browser-extension-scope, Art. 50 disclosure) versioned in source control, with a named corpus owner, a monthly refresh cadence from internal and external sources, and runs triggered on deployment updates for Critical/High-tier endpoints, and is the Art. 50 disclosure corpus re-run on every UX release for customer-facing chatbots and multi-modal interfaces?

Evidence Required: - [ ] Six regression corpora published in source control: prompt-injection corpus (30–100 direct and indirect inputs covering system-prompt extraction, instruction-override, role-manipulation, multi-turn injection, ATLAS TA0001/TA0003), jailbreak corpus (30–100 role-override, persona-switch, authority-claim, encoding-bypass inputs), multi-modal injection corpus (20–60 image and audio inputs embedding encoded prompt-injection instructions, steganographic payloads, supersonic embedding), DLP-paste-block corpus (20–50 regulated-data canary inputs, credit card patterns, SSN-format strings, synthetic PHI, synthetic source-code fragments), browser-extension-scope corpus (20–50 crafted page configurations testing host-permission scope enforcement, OWASP Browser-Extension Top 10), and Art. 50 disclosure corpus (10–20 opening-interaction test fixtures verifying disclosure presence at first interaction, EU AI Act Art. 50) - [ ] Each corpus entry includes: input, expected safe output pattern, threat tag (HAI TTP + ATLAS tactic ID), OWASP reference, source, date added - [ ] Deployment triggers confirmed: corpus runs triggered on deployment updates for Critical/High-tier endpoints; failure blocks go-live for Critical/High-tier - [ ] Art. 50 disclosure corpus re-run on every UX release for customer-facing chatbots and multi-modal interfaces; evidence: UX release records × disclosure test run records - [ ] Named corpus owner per corpus; corpus changes go through review with named owner - [ ] Monthly refresh cadence evidenced: change-log showing updates from internal observations (IR-Endpoints findings, IM-Endpoints incidents, red-team results), external sources (OWASP LLM Top 10, ATLAS examples, OWASP MASVS test cases, public jailbreak research), and sector-specific advisory content

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | Regression corpora published (prompt-injection, jailbreak, multi-modal injection, DLP-paste-block, browser-extension-scope, Art. 50 disclosure) | /6 | /6 | 6/6 | ☐ | | | % Art. 50 disclosure corpus re-run on every UX release for customer-facing chatbots | % | % | 100% | ☐ | | | Corpus refresh cadence, months since last update per corpus | ___ | ___ | ≤1 month | ☐ | | | % archetype threat library entries covered by at least one corpus entry | % | % | ≥80% by end of year 1 | ☐ | |

Metric Collection Guidance: - Corpus publication: Count corpora present in source control with versioned entries and a named owner. Source: corpus registry in VCS - Art. 50 re-run rate: For UX releases of customer-facing chatbots and multi-modal interfaces in the last 6 months, check whether a disclosure corpus run and pass record exists. Formula: UX_releases_with_disclosure_test / total_UX_releases × 100. Source: UX release log × test-run registry - Refresh cadence: Inspect git log for each corpus; compute days since last commit adding new entries. Source: VCS change-log - Threat coverage: TA-Endpoints library entries mapped to corpus entries vs. total archetype library entries. Source: TA-Endpoints library × corpus metadata

Evidence Location: __ Validation Date: __ Notes: ___

Question 3: Go-Deployment Battery and IM Wire

Q3.1: Is the go-deployment battery operated with defined re-run triggers (pre-production, post-model-update, post-incident, quarterly), and are all test failures routed to IM-Endpoints within 1 business day with a severity tag and named owner?

Evidence Required: - [ ] Go-deployment battery process document defines re-run triggers: pre-production (blocks Sanctioned status in SM-Endpoints), post-model-update/firmware-update/app-version-change within 14 days (Critical-tier: 7 days), post-incident before incident closure, and quarterly for all active endpoints - [ ] Go-deployment test records linked from SM-Endpoints inventory for endpoints promoted in the last 90 days - [ ] Test failure to IM-Endpoints routing confirmed: sample of recent failures shows IM-Endpoints tickets created within 1 business day with severity tag, named owner, and test battery reference - [ ] Severity rubric published and applied consistently to test failure triage - [ ] Battery re-run cadence evidenced: quarterly re-run records on file for active endpoints with named battery owner - [ ] Post-model-update re-run triggered and completed within SLA for any model version change, firmware update, or app version change in the last 6 months

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | % test failures routed to IM within 1 business day | % | % | 100% | ☐ | | | % post-model/firmware/app-update re-runs completed within declared SLA (7/14 days by tier) | % | % | ≥90% | ☐ | | | % active endpoints with a quarterly battery re-run completed in last 90 days | % | % | ≥90% | ☐ | | | % Art. 50 disclosure-presence tests passing on every deployment update for customer-facing chatbots | % | % | 100% | ☐ | |

Metric Collection Guidance: - IM routing rate: Compare test failure log timestamps to IM-Endpoints ticket creation timestamps. Source: test failure log × IM-Endpoints system - Post-update SLA: For model/firmware/app version changes in last 6 months, check re-run date vs. change date by tier. Source: endpoint change log × test-run registry - Quarterly re-run rate: Count active endpoints with a battery run record dated within the last 90 days. Source: battery run registry × SM-Endpoints active endpoint list - Art. 50 disclosure rate: Deployment update records × disclosure test run records. Source: test-run registry filtered to Art. 50 disclosure test class

Evidence Location: __ Validation Date: __ Notes: ___

Maturity Level 2

Objective: Calibrate test depth per risk tier using the SM-Endpoints L2 tier-treatment matrix, run scheduled per-tier red-team exercises for Critical (quarterly) and High (semi-annual) endpoints, and test cross-archetype compositions for Critical-tier endpoints

Question 4: Tier-Calibrated Battery and Corpus Depth

Q4.1: Is per-tier corpus calibration enforced (Critical-tier: all 6 corpora on every deployment update; Low-tier: prompt-injection corpus on deployment), and are Art. 50 disclosure and DLP-paste-block tests differentiated by tier?

Evidence Required: - [ ] Per-tier test treatment published and aligned to SM-Endpoints L2 tier-treatment matrix: Critical (full battery + all 6 corpora on every deployment update + Art. 50 automated probe on every UX release as P1 + DLP-paste-block quarterly), High (full battery + all 6 corpora on deployment + periodic + Art. 50 probe on every UX release + DLP-paste-block semi-annually), Medium (subset battery + prompt-injection + jailbreak corpora on deployment + Art. 50 verified annually + DLP verified annually), Low (spot-check + prompt-injection corpus on deployment + Art. 50 verified at go-deployment) - [ ] Deployment pipeline configuration confirms tier-differentiated corpus assignments, not the same corpus applied to all tiers - [ ] Art. 50 disclosure automated probe configured as a P1 failure for Critical-tier endpoints on every UX release; evidence: CI/CD configuration - [ ] DLP-paste-block test verified on quarterly cadence for Critical-tier endpoints; evidence: DLP test run records - [ ] Per-tier SLA adherence tracked and reported to program sponsor

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | % Critical-tier endpoints with all 6 corpora running on every deployment update | % | % | 100% | ☐ | | | Per-tier SLA adherence for testing activities (go-deployment battery, post-model-update re-run, red-team cadence) | % | % | ≥90% per tier | ☐ | | | % Critical-tier customer-facing chatbots with Art. 50 automated probe on every UX release | % | % | 100% | ☐ | | | % Critical-tier endpoints with DLP-paste-block verified quarterly | % | % | 100% | ☐ | |

Metric Collection Guidance: - Corpus per-deployment rate: Deployment pipeline telemetry for Critical-tier endpoints; verify all 6 corpus runs appear in each deployment's check suite. Source: CI/CD telemetry - SLA adherence: Program telemetry across go-deployment battery turnaround, post-update re-run dates, and red-team calendar adherence. Formula: activities_completed_within_SLA / total_activities × 100 - Art. 50 probe on UX release: UX release log × Art. 50 disclosure test run records; count UX releases with a disclosure probe run. Source: deployment pipeline telemetry - DLP-paste-block quarterly: Count Critical-tier endpoints with a DLP-paste-block test record in the last 90 days. Source: DLP test run registry

Evidence Location: __ Validation Date: __ Notes: ___

Question 5: Scheduled Red-Team Exercises

Q5.1: Are 100% of Critical-tier AI/HAI-enabled endpoints red-teamed at least quarterly, and 100% of High-tier semi-annually, with scope derived from TA-Endpoints L2 per-artifact deep threat models covering prompt-injection chains, multi-modal injection, DLP bypass, Art. 50 disclosure circumvention, tool-allowlist escape, extension-scope violation, data-exfiltration probes, and physical-interface attacks for applicable archetypes?

Evidence Required: - [ ] Red-team schedule on calendar covering all Critical-tier (quarterly) and High-tier (semi-annual) endpoints; no Critical-tier endpoint skipped in the last 12 months - [ ] Red-team scope documented per exercise: written rules of engagement, test plan reviewed with endpoint owner, scope derived from TA-Endpoints L2 per-artifact deep threat model (not archetype snapshot); covers prompt-injection chains (ATLAS TA0001/TA0003), multi-modal injection (image/voice, ATLAS TA0001 indirect), DLP bypass, Art. 50 disclosure circumvention, tool-allowlist escape (ATLAS TA0004), extension-scope violation (OWASP Browser-Extension Top 10), data-exfiltration probes (ATLAS TA0013), physical-interface attacks for edge device and mobile app archetypes, and biometric-bypass via synthetic media - [ ] ATLAS tactic IDs referenced in scope: TA0001 Reconnaissance, TA0003 Initial Access, TA0004 ML Model Access, TA0013 Exfiltration - [ ] Red-team execution log and structured findings report on file: severity, root cause, ATLAS tactic ID, SR-Endpoints requirement traced, remediation pairing - [ ] Findings routed to IM-Endpoints with severity tag and named endpoint owner as assignee; remediation tracked - [ ] Scheduled red-team exercises confirmed: quarterly for Critical, semi-annual for High

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | % Critical-tier endpoints red-teamed in last 90 days | % | % | 100% | ☐ | | | % High-tier endpoints red-teamed in last 180 days | % | % | 100% | ☐ | | | % red-team findings (Critical/High severity) converted to corpus entries within 30 days | % | % | ≥90% | ☐ | | | Per-tier SLA adherence for red-team cadence | % | % | ≥90% | ☐ | |

Metric Collection Guidance: - Critical red-team rate: Count Critical-tier endpoints with a red-team report dated within the last 90 days vs. total Critical-tier endpoints. Source: ST records - High red-team rate: Same logic for 180 days. Source: ST records - Finding to corpus conversion: For Critical/High severity findings, count those with a corresponding corpus entry committed within 30 days. Formula: findings_with_corpus_entry / total_Critical_High_findings × 100. Source: finding → corpus pipeline telemetry - SLA adherence: Compare scheduled red-team dates to actual execution dates. Source: program calendar × ST records

Evidence Location: __ Validation Date: __ Notes: ___

Question 6: Cross-Archetype Composition Tests

Q6.1: Are cross-archetype composition tests (AI assistant + browser extension, chatbot + multi-modal) documented and executed for all Critical-tier composite endpoints, with each Critical/High-severity red-team finding producing a corpus entry within 30 days?

Evidence Required: - [ ] Cross-archetype composition test plan published for each Critical-tier endpoint with a composite archetype; reviewed by named architect - [ ] AI assistant + browser extension composition test: browser extension passes content to the AI assistant containing injected instructions; verify the assistant does not follow injected instructions; verify DLP policy covers the combined data-flow path; test documented with execution record - [ ] Chatbot + multi-modal composition test: multi-modal input (image + text) submitted to chatbot interface; verify cross-modal safety filter applies consistently; verify Art. 50 disclosure present regardless of input modality; test for indirect prompt injection via image content flowing into text-response generation path (ATLAS TA0001 indirect); test documented with execution record - [ ] Regression corpus growth rate evidenced: Critical-tier corpora show ≥1 new entry per month from red-team or incident findings - [ ] Finding to TA-Endpoints library-gap pipeline: Critical gaps closed within 30 days; High within 60 days

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | % Critical-tier composite endpoints with documented cross-archetype composition tests | % | % | 100% | ☐ | | | Regression corpus growth rate, Critical-tier corpora (new entries per month) | ___ | ___ | ≥1 per month | ☐ | | | % red-team findings (Critical/High severity) converted to corpus entries within 30 days | % | % | ≥90% | ☐ | | | % TA-Endpoints library gaps (Critical tier) closed within 30 days | % | % | 100% | ☐ | |

Metric Collection Guidance: - Composition test coverage: Count Critical-tier endpoints with a composite archetype that have a documented test plan + execution record vs. total composite Critical-tier endpoints. Source: ST records - Corpus growth: Git log of Critical-tier corpus files; count commits adding new entries per calendar month. Source: VCS change-log per corpus file - Finding to corpus conversion: Finding log × corpus commit-log cross-reference. Source: IM-Endpoints findings × corpus change-log - TA gap closure: TA-Endpoints library-gap ticket backlog filtered to Critical tier; measure open-to-close duration. Source: issue tracker

Evidence Location: __ Validation Date: __ Notes: ___

Maturity Level 3

Objective: Operate continuous automated adversarial testing for Critical-tier endpoints, publish regression corpora and findings as open artifacts, and contribute discovered TTPs to MITRE ATLAS, AVID, OWASP MASVS, OWASP Browser-Extension Top 10, and CSA endpoint working groups

Question 7: Continuous Automated Adversarial Testing Harness

Q7.1: Are ≥80% of Critical-tier AI/HAI-enabled endpoints under continuous automated adversarial testing with daily probe execution, covering prompt-injection generation, multi-modal injection seeding, DLP-bypass generation, extension-scope probing, and Art. 50 disclosure monitoring, with novel TTPs triaged into the TA-Endpoints library within 14 days and high-severity automated findings routed to IM within 24 hours?

Evidence Required: - [ ] Automated adversarial testing harness deployed and producing daily probe results against Critical-tier endpoints: prompt-injection generator (mutation of regression corpus + template-based variation + jailbreak-ladder generation, ATLAS TA0001/TA0003), multi-modal injection seeder (steganographic payload generation, supersonic audio embedding, adversarial overlay generation, ATLAS TA0001 indirect), DLP-bypass generator (regulated-data canary variants with encoding variations, tokenization-splitting, homoglyph substitution), extension-scope probe (crafted page configurations and cross-origin access attempts testing browser extension host-permission enforcement, OWASP Browser-Extension Top 10), and Art. 50 disclosure monitor (automated daily probe of all live customer-facing chatbot and multi-modal interfaces confirming disclosure at first interaction; absence generates P1 finding within 1 hour) - [ ] Continuous adversarial testing telemetry: harness health dashboard showing % Critical-tier endpoints with a fresh probe result within the last 24 hours; on-call paged when feed goes stale >24 hours - [ ] Finding triage process: named ST owner reviewing automated findings at least weekly; novel TTP patterns forwarded to TA-Endpoints L3 auto-proposal pipeline within 14 days - [ ] High-severity automated findings route to IM-Endpoints within 24 hours; evidence: IM ticket timestamps vs. harness alert timestamps - [ ] ATLAS tactic IDs covered by harness daily: TA0001, TA0003, TA0004, TA0013

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | % Critical-tier endpoints under continuous automated adversarial testing (daily probe execution) | % | % | ≥80% | ☐ | | | New-TTP ingestion lead time (automated finding to TA-Endpoints library entry) | ___ days | ___ days | ≤14 days | ☐ | | | Art. 50 disclosure probe, % of live customer-facing chatbot and multi-modal interfaces passing daily disclosure check | % | % | 100% | ☐ | | | % high-severity automated findings routed to IM within 24 hours | % | % | 100% | ☐ | |

Metric Collection Guidance: - Continuous coverage: Harness telemetry, count Critical-tier endpoints with a probe result in the last 24 hours vs. total Critical-tier. Source: ST harness telemetry - TTP ingestion lead time: For novel patterns identified by the harness, measure time from harness alert to TA-Endpoints library entry. Source: harness to TA pipeline telemetry - Art. 50 disclosure probe: Automated probe telemetry; count in-scope customer-facing interfaces passing disclosure check daily. Source: automated probe telemetry - IM routing within 24h: Compare harness alert timestamp to IM-Endpoints ticket creation timestamp for high-severity findings. Source: harness alert log × IM-Endpoints system

Evidence Location: __ Validation Date: __ Notes: ___

Question 8: Industry Contributions

Q8.1: Has the program contributed ≥4 anonymized, legally-vetted findings per year to MITRE ATLAS, AVID, OWASP MASVS, or OWASP Browser-Extension Top 10, and are all six open regression corpora published under a permissive license and maintained upstream with documented external adoption?

Evidence Required: - [ ] Contribution log on file: ≥4 submissions per year to MITRE ATLAS (novel prompt-injection technique observations for endpoint-specific surfaces: voice injection, image embedding, browser-extension injection chains, with ATLAS tactic IDs TA0001/TA0003), AVID (structured disclosure submissions for novel vulnerabilities in AI/HAI endpoint deployments), OWASP MASVS (real-world test evidence for mobile AI app security requirements), OWASP Browser-Extension Top 10 (real-world telemetry for browser-based AI tool attack surface), or CSA endpoint working groups (anonymized findings and test patterns for managed-endpoint and SaaS-AI feature attack surfaces) - [ ] At least one submission accepted as a new or refined technique; evidence: ATLAS, AVID, OWASP, or CSA acknowledgment or technique ID assignment - [ ] Legal-vetting record for each contribution: org identity scrubbed; coordinated disclosure completed for any third-party component involvement - [ ] All 6 open regression corpora published under a permissive license (prompt-injection, jailbreak, multi-modal injection, DLP-paste-block, browser-extension-scope, Art. 50 disclosure); published versions scrubbed of org-specific endpoint names, data classes, and identifiers - [ ] Published corpora maintained upstream; internal corpora are a superset of published versions - [ ] Industry-contribution pipeline: at least one anonymized finding in-preparation, in-legal-review, or submitted at all times

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | Industry contributions per year (MITRE ATLAS / AVID / OWASP MASVS / Browser-Extension Top 10 / CSA) | ___ | ___ | ≥4 | ☐ | | | Open regression corpora published and maintained upstream | /6 | /6 | ≥6 corpora published | ☐ | | | Contributions accepted as new or refined techniques | ___ | ___ | ≥1 per year | ☐ | | | Published corpus recency, months since last upstream update | ___ | ___ | ≤1 month | ☐ | |

Metric Collection Guidance: - Contribution count: Contribution log filtered to the last 12 months; count submissions with submission confirmation. Source: contribution log - Open corpus publication: Confirm all 6 corpora exist in the public repository under a permissive license. Source: external repository - Acceptance: Cross-reference contribution log with ATLAS/AVID/OWASP/CSA acknowledgment records. Source: contribution log × external acknowledgments - Corpus recency: Last commit date to the public repository per corpus. Source: public VCS

Evidence Location: __ Validation Date: __ Notes: ___

Question 9: Industry-Shared Exercises

Q9.1: Has the program hosted at least 1 industry endpoint AI red-team benchmark per year and participated in ≥2 additional cross-org exercises, with documented cross-org detection-benchmark improvement data from participants?

Evidence Required: - [ ] Exercise log on file: ≥1 hosted endpoint AI red-team benchmark per year (OWASP AI chapter, CSA endpoint working group, or sector ISAC AI red-team exercise); ≥2 additional cross-org exercises participated - [ ] Hosted exercise documented: agenda, participant list, benchmark methodology, measurement of detection improvement before and after the exercise - [ ] Cross-org detection-benchmark improvement data collected from participants and documented in exercise report - [ ] Art. 50 disclosure daily probe covering 100% of in-scope customer-facing interfaces confirmed; probe health reported to program sponsor - [ ] Industry-exercise calendar: next hosted or co-hosted exercise scheduled at least 60 days in advance - [ ] Published regression corpora cited or adopted by at least one external organization; adoption documented

Outcome Metrics: | Metric | Baseline | Current | Target | Met? | Notes | |--------|----------|---------|--------|------|-------| | Industry-shared exercises per year | ___ | ___ | ≥1 hosted + ≥2 participated | ☐ | | | Art. 50 disclosure daily probe, % in-scope customer-facing interfaces passing daily check | % | % | 100% | ☐ | | | Open regression corpora published and maintained upstream | /6 | /6 | ≥6 corpora published | ☐ | | | Cross-org detection-benchmark improvement documented | No / Yes | No / Yes | Yes | ☐ | |

Metric Collection Guidance: - Exercise count: Exercise log filtered to the last 12 months; count hosted and participated exercises. Source: exercise log - Art. 50 disclosure probe coverage: Automated probe telemetry; count in-scope customer-facing interfaces with a disclosure check run in the last 24 hours vs. total in-scope interfaces. Source: automated probe telemetry - Improvement documentation: Hosted exercise report; participant pre/post detection benchmark data. Source: exercise reports - External adoption: External organizations that have cited or forked the public corpora. Source: public repository analytics + community references

Evidence Location: __ Validation Date: __ Notes: ___

Summary Scorecard

Level	Q#	Question	Score	Weight
L1	Q1	Per-Archetype Test Battery	___
L1	Q2	Regression Corpora on Deployment	___
L1	Q3	Go-Deployment Battery and IM Wire	___
L1 Score			___	0.5
L2	Q4	Tier-Calibrated Battery and Corpus Depth	___
L2	Q5	Scheduled Red-Team Exercises	___
L2	Q6	Cross-Archetype Composition Tests	___
L2 Score			___	0.3
L3	Q7	Continuous Automated Adversarial Testing Harness	___
L3	Q8	Industry Contributions	___
L3	Q9	Industry-Shared Exercises	___
L3 Score			___	0.2
Overall ST-Endpoints Score			___

Maturity Level Achieved: ☐ L1 ☐ L2 ☐ L3

Assessment Date: __ Assessor: __ Next Review Date: ___

Document Version: HAIAMM v3.0 Practice: Security Testing (ST) Domain: Endpoints Questionnaire Authored: 2026-05-15 Author: Verifhai

Instructions:

Answer based on current practices, not plans
“Yes” requires documented evidence
Complete all Level 1 questions before Level 2
Partial implementation = “No”

↓ Download as Markdown