HAIAMM v3.0, Vendors Domain Handbook

Vendor AI Assurance, security of the AI/HAI tools the organization consumes from third parties

Version: 3.0 Domain: Vendors Audience: Security, Procurement, Legal/Privacy, IT, Data Governance, Third-Party Risk Management Use: Conduct a maturity assessment of the Vendor AI Assurance program, and build the practices that move it from Foundational to Industry-Leading.

Preface

This handbook is a self-contained, practitioner-facing document. Read it end to end to understand the Vendors domain of HAIAMM, or jump to Part IV to perform an assessment.

The handbook makes three commitments to the reader:

1. Fundamentals first. It teaches the load-bearing practices an organization must have to claim mastery of AI vendor security, not a catalog of everything one could do.
2. Measurable by default. Every activity prescribed in these pages is paired with at least one outcome metric (with a baseline, a target, and a source). Activity counts are not metrics; outcomes are.
3. Self-contained. The document does not require the reader to follow links, open companion files, or chase references. Every concept used in the assessment is defined inside these pages.

If a statement in this handbook treats AI as a tool performing security rather than the subject being secured, that statement is wrong. Flag it.

Table of Contents

Part I, Domain Overview

1. About this handbook
2. The Vendors domain in v3.0 terms
3. Why a domain-specific handbook
4. The five AI vendor archetypes
5. Domain boundary rules
6. Stakeholders and roles
7. How to use this handbook

Part II, Foundations

1. The four Business Functions in this domain
2. The three maturity levels
3. HAI-specific threat tactics (EA, AGH, TM, RA)
4. The priority compliance map
5. Shadow AI in the Vendors domain (unsanctioned vendor AI consumption)
6. Metrics taxonomy

Part III, The Twelve Practices in the Vendors Domain

1. Strategy & Metrics (SM)
2. Policy & Compliance (PC)
3. Education & Guidance (EG)
4. Threat Assessment (TA)
5. Security Requirements (SR)
6. Secure Architecture (SA)
7. Design Review (DR)
8. Implementation Review (IR)
9. Security Testing (ST)
10. Environment Hardening (EH)
11. Issue Management (IM)
12. Monitoring & Logging (ML)

Part IV, Maturity Assessment Workbook

1. How the assessment works
2. Scoring methodology
3. The questionnaire (108 questions)
4. Practice-level rollup
5. Domain-level rollup
6. Improvement roadmap template

Part V, Reference

1. Glossary
2. Reference frameworks
3. Change log

Part I, Domain Overview

1. About this handbook

HAIAMM is the Human-Assisted Intelligence Assurance Maturity Model. It is an AI assurance maturity model, structured after OWASP SAMM and BSIMM in shape, and scoped to AI/HAI in content. HAIAMM has six domains, Software, Data, Endpoints, Infrastructure, Vendors, Processes, and twelve practices that apply across all six.

This handbook covers the Vendors domain. It contains:

• A definition of what the Vendors domain is and is not.
• The twelve practices, each described in Vendors-domain terms with three maturity levels (Foundational, Comprehensive, Industry-Leading).
• A complete maturity assessment workbook with 108 yes/no questions and a scoring methodology.
• A reference section with a glossary and the major frameworks HAIAMM aligns with.

This is one of six domain handbooks. Vendor-specific assessment questions live only in this handbook; the Software handbook contains only Software questions, the Data handbook only Data questions, and so on.

2. The Vendors domain in v3.0 terms

The Vendors domain governs the AI/HAI tools and services the organization consumes from third parties, not the AI tools the organization builds, and not the use of AI to perform security work.

In scope:

• Consumer-grade generative AI (ChatGPT, Claude, Gemini, Copilot, etc.) accessed through enterprise plans, free tiers, or personal accounts.
• AI features embedded inside SaaS platforms the organization already uses (Notion AI, Slack AI, Zoom AI Companion, Microsoft 365 Copilot, Salesforce Einstein, CRM and ticketing AI add-ons, Workspace Gemini).
• AI coding assistants (Cursor, Windsurf, GitHub Copilot, Tabnine, Cody) used inside the organization's IDEs and developer endpoints.
• AI APIs and foundation-model vendors (OpenAI, Anthropic, AWS Bedrock, Google Vertex AI, self-hosted open-source served behind a vendor API).
• AI agent and automation platforms (LangChain-based products, Zapier AI, Make AI, custom agents calling external APIs, multi-agent platforms).

Out of scope of the Vendors domain:

• AI software the organization builds itself, that is the Software domain.
• Data flowing into and out of AI systems, that is the Data domain (data is special and frequently cross-references Vendors).
• AI inference infrastructure the organization hosts, that is the Infrastructure domain.
• Business workflows that embed AI, that is the Processes domain.
• AI-enabled endpoints and user interfaces in scope of endpoint management, that is the Endpoints domain (a Vendors-supplied AI assistant deployed on managed endpoints is a Vendors artifact; the endpoint policy that constrains it is an Endpoints artifact).

The subject of every cell in this handbook is the AI/HAI tool we consume. The vendor is the actor; the consuming organization is the deployer. The tool is what is being secured.

3. Why a domain-specific handbook

A Vendor AI Assurance program requires a different operating model from classic third-party risk management. Five reasons motivate the standalone handbook:

• Adoption outpaces procurement. Employees adopt AI vendors faster than procurement, legal, and security can review them. Free tiers, credit-card SaaS, and "turn on AI" toggles inside already-approved vendors create a shadow footprint that bypasses data-handling, contractual, and regulatory controls.
• Defaults are unsafe. Many AI vendors train on customer data by default. Many AI features inside approved SaaS turn on silently with vendor product updates. Many AI agent platforms request a permission model broader than any individual user's. The classic vendor process never asked these questions.
• Regulators have caught up. EU AI Act deployer duties (Articles 26 and 50), GDPR Article 28 processor obligations applied to AI vendors, ISO/IEC 42001 supplier controls, SOC 2 CC9.2 vendor management, and sector-specific rules (HIPAA BAA, PCI-DSS 12.8, FINRA model risk, HHS/FDA clinical AI, NYDFS Part 500) now apply to AI vendors specifically. The program must produce evidence on demand.
• Shadow AI prevention is the program's primary L1 outcome. Shadow AI is not a footnote; it is the central problem the L1 program exists to solve. This is unusual among maturity model domains and is structurally embedded in this handbook.
• Five archetypes, one program. The five AI vendor archetypes (consumer GenAI, AI-embedded SaaS, AI coding assistant, AI API/model vendor, AI agent/automation platform) behave differently enough that threats, requirements, reference architectures, and tests are archetype-keyed throughout the handbook.

4. The five AI vendor archetypes

Most of the practices in this handbook key their content to five archetypes. Knowing the archetypes well is a prerequisite for using the handbook.

1. Consumer GenAI. Standalone generative AI products used directly by employees. Examples: ChatGPT, Claude, Gemini, Copilot (consumer), Perplexity. Risk shape: data egress on every prompt; default training-on-data behavior depends on plan tier; personal-account usage extremely common.

2. AI-embedded SaaS feature. AI capabilities surfaced inside platforms the organization already uses for non-AI reasons. Examples: Notion AI, Slack AI, Zoom AI Companion, Microsoft 365 Copilot, Salesforce Einstein, Workspace Gemini, CRM/ticketing AI add-ons. Risk shape: parent vendor already approved, but the AI feature has its own data flow, its own subprocessor chain, and often a different DPA addendum. The biggest blind spot in most programs.

3. AI coding assistant. AI integrated into the developer's IDE or workflow. Examples: GitHub Copilot, Cursor, Windsurf, Tabnine, Cody. Risk shape: source-code egress; training-on-code defaults vary by plan; telemetry scope often broader than developers expect; regulated repositories require an explicit no-train path.

4. AI API / foundation-model vendor. Programmatic access to models, used inside applications the organization builds. Examples: OpenAI API, Anthropic API, AWS Bedrock, Google Vertex AI, self-hosted open-source served behind an API. Risk shape: programmatic data egress in volume; model-version churn; subprocessor chain (vendor → cloud → silicon); rate-limit and abuse-throttle behavior.

5. AI agent / automation platform. Vendor platforms that run agents acting on the organization's systems. Examples: LangChain-based products, Zapier AI, Make AI, custom-built agents calling external APIs, multi-agent platforms. Risk shape: tool-scope often unbounded; permission model broader than any single user's; indirect prompt injection via retrieved or tool-returned content; agent-loop and goal-hijack hazards specific to autonomous behavior.

A single vendor product can be more than one archetype at the same time, for example, Cursor is both an AI coding assistant and an AI agent platform. Threat libraries, requirements packs, reference architectures, and tests in this handbook accommodate that.

5. Domain boundary rules

When in doubt about whether something belongs in the Vendors domain, ask: who is responsible for the security of the inside of this thing?

• If the vendor is responsible: it is a Vendors artifact (and possibly an Endpoints artifact, when the consumption point is a managed endpoint).
• If the organization is responsible: it is a Software, Data, Infrastructure, or Processes artifact, in that approximate order of likelihood.

Common boundary cases:

• An AI coding assistant deployed on engineer endpoints is a Vendors artifact (consumed from a third party). The endpoint policy that governs it is an Endpoints artifact.
• An internal LLM-powered feature shipped in the organization's product is a Software artifact, even if its inference runs on a vendor API. The vendor API itself, separately, is a Vendors artifact.
• A shared vector store hosted in the organization's cloud and used by multiple internal apps is an Infrastructure artifact. The embeddings stored there may also be a Data concern.
• Data flowing through an organization-built AI service is a Software concern at the service level and a Data concern at the corpus, log, or embedding level. Cross-references are expected.

6. Stakeholders and roles

The Vendor AI Assurance program is cross-functional by design. The following roles appear throughout this handbook:

• Executive sponsor. Typically the CISO, CIO, or Chief Procurement Officer; co-sponsorship by Legal/Privacy is common. Owns budget, scope, and decision rights for the program.
• Program lead. Operationally accountable for the program day-to-day. Maintains the inventory, runs the working group, owns the metrics.
• Cross-functional working group. Security, Procurement, Legal/Privacy, IT, Data Governance, and one business-unit representative. Meets at least monthly.
• Intake reviewers. A small population trained to assess AI vendors against the threat library, the requirements pack, and the priority compliance map. Drawn from Security, Procurement, Legal/Privacy, and TPRM.
• Architect reviewers. Senior engineers with sign-off authority on design reviews for AI vendor integrations.
• Integration owners. The business-side or engineering-side owner of a specific AI vendor's use inside the organization. Named in the inventory and accountable for maintaining the integration's posture.
• Admin governance owner. A named owner for the organization's parent SaaS vendors with AI features (Microsoft 365, Workspace, Slack, Notion, Zoom, Salesforce). Runs the quarterly review of AI feature toggles.
• Program sponsor's deputy. Receives the quarterly shadow AI scoreboard and the post-incident review summaries.

7. How to use this handbook

Three modes of use are supported:

• Read it linearly. Parts I and II ground the reader in the domain and foundations. Part III walks the twelve practices in the Vendors-domain context. Part IV provides the assessment instrument. Part V is reference.
• Run an assessment. Skim Parts I and II for context (one to two hours), then go directly to Part IV. The 108 questions are organized by practice and by maturity level, with explicit evidence prompts. Scoring methodology and rollup tables follow.
• Build a program from scratch. Read Part II carefully, then implement the twelve practices' Level 1 in the order described in Part II's dependency text. Use the Level 1 questions in Part IV as a self-check for completeness.

The questions in Part IV are duplicated as a per-practice "Practice Maturity Questions" section at the end of each practice in Part III. They are the same questions; the duplication is deliberate so the practice-by-practice reader sees the assessment instrument inline.

Part II, Foundations

8. The four Business Functions in this domain

The twelve practices group into four Business Functions. Each function exists for a distinct intent. Every practice belongs to exactly one function.

Governance, Strategy & Metrics (SM), Policy & Compliance (PC), Education & Guidance (EG). Establish why, what, who, and how: the program's strategic frame, its enforceable rules, and the workforce literacy that makes everything downstream possible. In this domain, Governance answers: who owns AI vendor risk, what policies apply, what training every employee and every reviewer must complete.

Building, Threat Assessment (TA), Security Requirements (SR), Secure Architecture (SA). Decide what could go wrong, what the integration must do about it, and how the integration is shaped to do it, before code is written or admin consoles are configured. In this domain, Building answers: what threats AI vendor archetypes carry, what requirements every AI vendor must meet, what reference patterns teams should reach for.

Verification, Design Review (DR), Implementation Review (IR), Security Testing (ST). Prove that the designed integration, the implemented integration, and the running integration actually meet the Building-function outputs. In this domain, Verification answers: did the team pick the reference pattern, do the live admin-console settings match the design, and does the integration actually behave correctly under adversarial probes.

Operations, Environment Hardening (EH), Issue Management (IM), Monitoring & Logging (ML). Run the program safely in production, harden the perimeter against AI vendors, manage the issues, and watch what is actually happening. In this domain, Operations answers: which controls keep sanctioned AI use frictionless and unsanctioned use observable, where AI vendor issues go, and what telemetry produces deployer-duty evidence on demand.

Cross-function rule: progress in one function without the others is unstable. A mature Verification function on top of weak Governance yields reviews nobody honors. A mature Operations function on top of weak Building chases effects without causes. The handbook is balanced across the four by design.

9. The three maturity levels

Every cell in this handbook (each combination of practice and level in the Vendors domain) is one of three maturity levels. The levels are cumulative, Level 2 assumes Level 1 is in place; Level 3 assumes Level 2 is in place.

Level 1, Foundational. Stand up the minimum viable capability. Discover what exists, publish the core policies, run the first version of the controls, baseline the metrics. Typical outputs: inventories, short published policies, checklists, baseline metrics, first detections, first reference patterns, first requirements pack, first tests, first reviews. Reality check: if the program cannot answer "what AI vendors do we have, what rules apply to them, and who is accountable" within a week, it is not at Level 1.

Level 2, Comprehensive. Calibrate the program's intensity by risk tier. Move from one-size-fits-all to differentiated depth. Replace point-in-time activities with continuous validation. Typical outputs: a published risk-tier rubric, a tier-treatment matrix specifying what each tier receives from each practice, per-tier calibrated activities, continuous validation for high tiers, deeper evidence assembly, post-incident learning loops. Reality check: if the same review effort goes to a consumer GenAI used by two employees and to a Critical-tier agent platform acting on customer systems, the program is not at Level 2.

Level 3, Industry-Leading. Automate the substrate. Benchmark externally against peers. Contribute back to the AI-assurance ecosystem. Typical outputs: signal-driven inventory and tier updates, telemetry-driven policy refresh, continuous attestation, external benchmarking briefs, contributions to MITRE ATLAS, OWASP LLM/Agentic Top 10, NIST AI RMF Playbook, AI Vulnerability Database, CSA AI Safety Initiative, OpenSSF AI, sector ISACs. Reality check: if all activity is still internally generated, no external contributions, no benchmarking deltas, no automation replacing routine ticket work, the program is mature for its own purposes but is not industry-leading.

10. HAI-specific threat tactics (EA, AGH, TM, RA)

Four AI-specific threat-tactic categories appear throughout this handbook. Threat libraries, requirements, reference architectures, tests, and detections are tagged to them.

EA, Excessive Agency. The AI or agent has more capability than its use case requires. Tool scopes too broad, permission model wider than any individual human's, effects reaching systems not in scope. The classic example: an AI agent platform issued an API key with the union of permissions every team's user has, a level of authority no single human in the organization holds.

AGH, Agent Goal Hijack. The agent's benign goal is redirected into an attacker's goal via content injected along a trusted-looking path. The injection might arrive in a retrieved document, a tool response, multi-turn history, or any input the agent treats as authoritative.

TM, Tool Misuse. Tools available to the AI or agent are invoked for attacker purposes, argument smuggling, unexpected combinations, crafted parameters, recursive invocation. Differs from Excessive Agency in that the scope of the tool may be appropriate; the tactic is the misuse of a legitimate tool.

RA, Rogue Agents. Autonomous agents drift from intended behavior across long sessions, reflective loops, or multi-agent miscoordination, producing harmful effects nobody explicitly instructed. Drift is the through-line; the trigger may be model update, prompt staleness, or emergent multi-agent interaction.

The four categories are not exhaustive of AI threat surface, they sit alongside classic prompt injection, training-data leakage, output integrity, and supply-chain concerns, but they are the categories most under-represented in classic third-party risk frameworks and most useful for organizing the Vendors domain's threat library.

11. The priority compliance map

Every Vendors-domain Policy & Compliance practice at Level 1 publishes (and downstream practices reference) a one-page priority compliance map. The set below is the priority set for the Vendors domain. Sector-specific items are added as applicable.

The map's purpose is traceability: an auditor or regulator asking "how is Article 26 addressed?" should reach a single cell in the map and from there one policy and from there one evidence artifact.

12. Shadow AI in the Vendors domain (unsanctioned vendor AI consumption)

Shadow AI, AI/HAI adopted outside the program's visibility, attribution, and governance, is the central problem the Level 1 Vendor AI Assurance program exists to solve. Three observations make it primary:

• Shadow AI compounds. Every month of unobserved adoption increases the data-class footprint, the user count, and the regulatory exposure inside the organization. Programs that defer shadow AI work to "later" find later is far more expensive than now.
• Shadow AI is observable today. The signals already exist in most enterprises, expense and procurement data, SSO and IdP app catalogs, DNS and proxy logs, SaaS admin audit feeds, endpoint inventories, employee self-disclosure under amnesty. No new tooling is required at L1.
• Shadow AI manifests through more than one domain. The handbook treats it primarily in Vendors but acknowledges its appearance in Endpoints (browser-extension AI), Software (unsanctioned internal AI builds), Data (data leaving governed stores into AI services), Infrastructure (unsanctioned model hosting), and Processes (AI embedded in business workflows without governance). Cross-domain coordination matters.

Every Level 1 activity in this handbook contributes to making shadow AI visible, attributable, and trending down. The Level 1 outcome metric "shadow AI ratio" appears in Strategy & Metrics, Policy & Compliance, Threat Assessment, Education & Guidance, Environment Hardening, and Monitoring & Logging, six of the twelve practices. That is intentional.

13. Metrics taxonomy

Every level block in this handbook carries three metric types. The taxonomy is the canonical vocabulary; examples and targets are practice-specific.

• Outcome metrics (lagging). Directly measure whether the level's goal was achieved. Reported monthly or quarterly. Stated in a four-column table: Metric · Baseline · Target · Source.
• Process metrics (leading). Predict outcome metrics by measuring execution. Reported weekly or at the cadence of the underlying activity.
• Effectiveness metrics (business value). Measure what the outcome means to the business. Reported quarterly, often qualitative supported by quantitative.

Metric selection in this handbook follows two rules:

• SMART: specific, measurable, achievable, relevant, time-bound.
• Outcome over output: results are preferred to activity counts. "Reviews completed" is an output. "Issues caught at design rather than production" is an outcome.

If a metric in this handbook does not have a baseline column, the baseline is the value the program records on first measurement. The first cycle of measurement is itself an L1 activity.

Part III, The Twelve Practices in the Vendors Domain

Each practice section follows the same shape:

• Practice Overview. Objective, description, context.
• Maturity Level 1. Objective, activities (A, B, C), outcome metrics, success criteria.
• Maturity Level 2. Same structure.
• Maturity Level 3. Same structure.
• Common Pitfalls. Three to four per level.
• Practice Maturity Questions. Three yes/no questions per level, the same questions also appear in the Part IV assessment workbook.

14. Strategy & Metrics (SM)

Practice Overview

Objective: Stand up a Vendor AI Assurance program that discovers, inventories, and strategically governs all AI/HAI tools and services provided by vendors, with shadow AI prevention as the primary L1 outcome.

Description: SM-Vendors establishes the program charter, the AI vendor inventory, and the practice-maturity metrics that prove the program is working. The Vendors domain governs AI capabilities the organization consumes from third parties: consumer GenAI, AI features inside existing SaaS, AI coding assistants, AI APIs, downloaded open-source models, and AI agent platforms.

Context: Employees and teams adopt AI vendors faster than procurement, legal, and security can review them. Free tiers, credit-card SaaS, and "turn on AI" toggles inside already-approved vendors create a shadow AI footprint that bypasses data-handling, contractual, and regulatory controls. The Vendor AI Assurance program makes this footprint visible and puts a light-touch intake on the path to adoption, sanctioned AI accelerates and unsanctioned AI cannot quietly accumulate risk.

Maturity Level 1

Objective: Stand up the program, build an AI vendor inventory, and establish baseline metrics that prove shadow AI is decreasing.

Activities.

A) Charter the Vendor AI Assurance program. Publish a short program charter that names the problem (shadow AI, uncontrolled data sharing with AI vendors, unassessed third-party AI risk), defines scope, and assigns accountable ownership. Charter elements include a problem statement, in-scope AI vendor categories (consumer GenAI, AI-embedded SaaS, AI coding assistants, AI APIs/models, AI agent platforms, AI-native point solutions), an executive sponsor (typically CISO, CIO, or CPO; co-sponsored by Legal/Privacy), a working group (Security, Procurement, Legal/Privacy, IT, Data Governance, business representative), decision rights for approval/block/exception, and a numerical success target for the L1 outcome metrics.

B) Build the AI vendor inventory and discover shadow AI. Establish a single AI vendor inventory as the program's source of truth. Minimum inventory fields: vendor name, product, AI capability, business owner, user count, data classes sent to the vendor, contract and DPA status, training-on-data posture, risk tier, approval status (Sanctioned / Provisional / Under review / Prohibited). Discovery sources at L1 use signals already available: expense and procurement data, identity and SSO logs, network egress logs, endpoint app inventory, existing SaaS admin consoles for AI features in already-approved vendors, and a self-attestation amnesty form.

C) Establish foundational metrics that measure practice maturity and shadow AI reduction. Baseline and track a small set of outcome, process, and effectiveness metrics. Publish a quarterly shadow AI scoreboard to the executive sponsor: total inventory by approval status, new AI vendors discovered this quarter and their intake status, shadow AI ratio trend, AUP attestation coverage, top five data-exposure risks to unsanctioned AI tools.

Outcome Metrics (L1).

Success Criteria.

• Program charter published and sponsored by an accountable executive.
• AI vendor inventory exists as a single source of truth with ≥90% coverage of discovered AI vendors within 12 months.
• Shadow AI ratio baselined and trending down for two consecutive quarters.
• ≥95% of employees have acknowledged an AI Acceptable Use Policy.
• Quarterly shadow AI scoreboard delivered to the executive sponsor.

Maturity Level 2

Objective: Risk-tier the AI vendor inventory, calibrate the program's intensity per tier, and measure practice maturity and shadow-AI reduction per tier, not only in aggregate.

Activities.

A) Define the AI vendor risk-tier rubric. Four tiers, Critical, High, Medium, Low, assigned from a small set of auditable dimensions: data sensitivity reaching the vendor, decision-affecting use (GDPR Article 22, EU AI Act Annex III), agentic capability, user exposure (customer-facing vs. internal), regulatory scope (HIPAA, PCI, FINRA/SEC, HHS/FDA), and concentration/criticality. Rubric is documented as a short table; tier is derived deterministically; human overrides are allowed but recorded with rationale.

B) Calibrate program intensity per tier. Publish a tier-treatment matrix specifying what each tier receives from each downstream practice, intake depth, threat-snapshot type, design review lane, implementation review cadence, security testing scope, monitoring detection set, and re-review on vendor material change. Each downstream practice's L2 inherits this calibration.

C) Per-tier scoreboard and governance. The L1 shadow AI scoreboard becomes tier-aware. A Critical-tier unsanctioned AI vendor is its own headline; a Low-tier one is a line item. SLA adherence reported per tier. Tier movements (upgrades and downgrades) are logged with rationale and reviewed by the program sponsor.

Outcome Metrics (L2).

Success Criteria.

• Risk-tier rubric published; tier assigned to 100% of inventory.
• Tier-treatment matrix published; downstream practices calibrated to it.
• Per-tier shadow AI ratio reported quarterly; Critical-tier unsanctioned AI count = 0.
• Per-tier SLA adherence ≥90% across practices.
• Tier-movement governance active.

Maturity Level 3

Objective: Automate inventory and tier maintenance from live signals, benchmark the program against external peers, and contribute anonymized AI-vendor ecosystem intelligence back to the industry.

Activities.

A) Continuous inventory and tier automation. Inventory auto-updates from expense/procurement feeds, SSO/IdP app catalog, DNS/egress telemetry, SaaS admin audit feeds, endpoint AI-tool inventory, intake system, and self-attestation. Tier assignments are rule-based on the L2 rubric; rule changes are versioned and replayable. Human curation handles new archetypes, ambiguous discoveries, and dimensional-input conflicts. A data-quality SLO is published (e.g., ≥99% of active AI vendors correctly tiered within 48 hours of a material change).

B) External benchmarking. Program metrics compared against peer benchmarks via sector ISACs (FS-ISAC, H-ISAC, IT-ISAC), AI-vendor standards bodies (ISO/IEC 42001 community, NIST AI RMF implementations, CSA AI Safety Initiative), Shared Assessments AI-vendor track, and formal peer roundtables. A semi-annual "how we compare" brief covers shadow AI ratio, AUP coverage, intake SLA, per-tier depth, automation level. Benchmark deltas inform program investment.

C) Contribute anonymized AI-vendor ecosystem intelligence. Contribute to MITRE ATLAS (new AI-vendor TTPs), OWASP LLM/Agentic Top 10, NIST AI RMF Playbook, AI Vulnerability Database, and ISAC feeds. Target minimum four substantive contributions per year; quality over volume; every contribution anonymized and legally vetted.

Outcome Metrics (L3).

Success Criteria.

• Inventory auto-update SLO published and met.
• Tier-assignment automation operational with published rules and human-exception handling.
• Semi-annual external-benchmarking brief published.
• ≥4 substantive industry contributions per year, anonymized and cited.
• Executive ROI narrative including external benchmarks delivered annually.

Common Pitfalls

Level 1. - Inventory seeded only from procurement records, misses free-tier, credit-card, and personal-account AI use. - Treating AI features in existing SaaS as out of scope because the parent vendor is already approved. - Program positioned as a blocker, intake SLA unpublished, procurement cycle time balloons, business units route around it. - Metrics count activity (tickets processed) instead of outcome (shadow AI ratio, AUP coverage, exposure events).

Level 2. - Tier-rubric inputs are subjective ("important vendor," "sensitive data"), reviewers tier differently. - Tier-treatment matrix published but not enforced, calibration exists on paper only. - Scoreboard still reported in aggregate, hiding Critical-tier shadow AI. - Tier upgrades resisted because they trigger more work, no governance on tier-movement.

Level 3. - Automation runs without a data-quality SLO, signal-driven inventory silently drifts. - Benchmarking chooses peers that flatter the program rather than stretch it. - Industry "contributions" are press releases, not technical artifacts that land in MITRE/OWASP/NIST/AIVD. - ROI narrative decouples from reality, external benchmarks cited but the program's own metrics are stale.

Practice Maturity Questions

Level 1. 1. Is there a published Vendor AI Assurance program charter with a named executive sponsor and a cross-functional working group (Security, Procurement, Legal/Privacy, IT, Data Governance, business representative)? 2. Does a single AI vendor inventory exist, seeded from expense, SSO, egress, endpoint, and SaaS-admin signals, covering consumer GenAI, AI-embedded SaaS, AI coding assistants, AI APIs/models, and AI agent platforms, with ≥90% coverage of discovered AI vendors? 3. Are the L1 outcome metrics baselined and reported quarterly to the sponsor, inventory coverage, shadow AI ratio, AUP attestation (≥95%), and known data-exposure events to unsanctioned AI tools?

Level 2. 1. Is every AI vendor in the inventory assigned a risk tier based on an auditable rubric covering data sensitivity, decision-affecting use, agentic capability, user exposure, regulatory scope, and concentration? 2. Is there a published tier-treatment matrix driving differential intensity across PC, TA, DR, IR, ST, and ML, with ≥95% of Critical-tier vendors receiving full-scope treatment in the last 12 months? 3. Does the quarterly shadow AI scoreboard report per tier (with Critical-tier unsanctioned AI explicitly tracked at zero), and does tier-movement get logged and reviewed by the program sponsor?

Level 3. 1. Does inventory and tier assignment auto-update from live signals (expense, SSO, egress, SaaS admin, endpoint, intake, self-attestation) with a published data-quality SLO, and is ≥80% of curation handled automatically with exception-based human review? 2. Does the program publish a semi-annual external-benchmarking brief comparing itself against at least five peer-comparable metrics via ISACs, standards bodies, or industry roundtables, and does it drive program investment decisions? 3. Does the program contribute at least four substantive, anonymized artifacts per year to the AI-vendor assurance ecosystem (MITRE ATLAS, OWASP LLM/Agentic, NIST AI RMF, AI Vulnerability Database, sector ISACs), and does the executive ROI narrative cite external benchmarks?

15. Policy & Compliance (PC)

Practice Overview

Objective: Publish the priority policies and compliance map that make a Vendor AI Assurance program enforceable, so adoption of AI/HAI tools from vendors is gated, attributable, and defensible to auditors, regulators, and customers. Shadow AI prevention is the primary L1 outcome.

Description: PC-Vendors codifies three priority policies (AI Acceptable Use, AI Procurement & Intake, Data-Sharing with AI Vendors) and maps them to the compliance regimes that explicitly apply to AI vendors. At L1 the goal is not exhaustive control coverage, it is the minimum set of policies and compliance mappings needed to stop unsanctioned AI vendors from silently entering the environment.

Context: Most organizations already have a generic Acceptable Use Policy and a vendor-risk process, neither answers the AI-specific questions: Can the vendor train on our data? Where does inference happen? What is the subprocessor chain? Does the AI feature inside an existing SaaS trigger a new DPA or AI-specific addendum? Without AI-specific policies and an explicit compliance map, shadow AI grows unchecked and the organization cannot demonstrate deployer duties under the EU AI Act or processor due diligence under GDPR.

Maturity Level 1

Objective: Publish the three priority AI vendor policies, map them to priority compliance requirements, and gate new AI vendor adoption through a lightweight intake.

Activities.

A) Publish the three priority AI vendor policies. Ship each in its smallest useful form, short, readable, enforceable. The AI Acceptable Use Policy specifies the approved AI vendor list, prohibited data classes, personal-account prohibition, output-review duty, disclosure obligation, and attestation cadence. The AI Procurement & Intake Policy requires intake for any AI tool, including AI features in already-approved SaaS, and publishes the intake SLA, fast-track path, and amnesty path for previously undisclosed shadow AI. The AI Vendor Data-Sharing Policy sets default training-on-data posture to off, requires DPA / AI addendum for any vendor receiving non-public data, and specifies data residency, retention, deletion, and breach-notification SLA terms.

B) Map the three policies to the priority AI vendor compliance requirements. Build a one-page compliance map covering EU AI Act Articles 26 and 50, NIST AI RMF GOVERN, GDPR Articles 28, 22, and 44–49, SOC 2 CC9.2, ISO/IEC 42001, ISO/IEC 27001 A.5.19–A.5.23, and any sector-specific (HIPAA BAA, PCI-DSS 12.8, FINRA/SEC, HHS/FDA, NYDFS Part 500). Each requirement traces to the specific L1 policy that carries it. The map tells an auditor in 60 seconds which policy answers which requirement.

C) Operate the intake gate and track foundational compliance outcomes. One intake form, one ticket queue, one SLA. Pre-approved AI vendors expose a fast-track path. Integration with the AI vendor inventory (approval creates or updates the inventory record) and with procurement (no PO or SaaS expense reimbursement without intake ID). Exceptions logged with owner and review date.

Outcome Metrics (L1).

Success Criteria.

• Three priority policies published, approved by Legal/Privacy + Security, communicated org-wide.
• One-page priority compliance map published and linked from each policy.
• Intake gate operating with a published SLA and an amnesty path for previously undisclosed AI use.
• ≥95% AUP attestation across the workforce.
• 100% of AI vendors handling non-public data have an executed DPA / AI addendum on file.

Maturity Level 2

Objective: Deepen policies and compliance evidence per AI vendor risk tier, bind intake decisions to ongoing vendor attestations, and produce audit-ready evidence trails automatically.

Activities.

A) Tier-aware policy depth and contractual controls. Extend the three priority policies with tier-specific addenda. Critical: contractual right-to-audit, right-to-test, model-version change notification ≥30 days, subprocessor add-notice ≥30 days with opt-out, incident-notification SLA ≤24 hours, training-data provenance attestation. High: subprocessor add-notice, incident-notification SLA ≤72 hours, annual SOC 2 / ISO 27001 evidence refresh. Medium: annual attestation refresh, subprocessor list update. Low: annual attestation only.

B) Continuous vendor attestation and compliance evidence assembly. Each Critical or High AI vendor maintains a "compliance evidence view" auto-assembling current DPA / AI addendum, subprocessor list, training-data posture statement, current SOC 2 Type II / ISO 27001, model-version log, vendor-side incident-notification history, and deployer-duty evidence. Staleness rules raise findings to IM. Sector-specific bundles (HIPAA BAA, PCI SP, clinical-AI, financial-services) generated from the compliance view.

C) Exception management and tier-aware enforcement. Exception register integrated with intake; no exception approved without tier-appropriate compensating-control definition. Monthly exception aging review; exceptions >90 days past expiry auto-escalate to the program sponsor. Unsanctioned AI in Critical-tier use cases triggers a blocking finding (no amnesty); other tiers continue to use the amnesty path.

Outcome Metrics (L2).

Success Criteria.

• Three priority policies extended with tier-specific addenda; tier-appropriate contractual controls in place for ≥90% of Critical/High AI vendors.
• Compliance evidence view live for every Critical/High AI vendor; staleness inside target.
• Exception register comprehensive, reviewed monthly; no exception >90 days past expiry un-escalated.
• Sector-specific evidence bundles complete for in-scope vendors.
• Regulatory or auditor inquiry SLA met in the last 12 months.

Maturity Level 3

Objective: Automate compliance attestation, drive policy updates from monitoring telemetry, and contribute to AI-vendor regulatory and standards development.

Activities.

A) Continuous compliance attestation and on-demand evidence packs. Evidence packs (by regulation, by sector, by use case) are generated on demand from the live compliance view. Vendor-side changes (product-update RSS, trust-center deltas, subprocessor changes, incident disclosures) auto-open tickets that refresh the view. SLA: any regulator or customer evidence request turned around inside 3 business days with complete provenance.

B) Telemetry-driven policy refresh. Quarterly policy-refresh cycle driven by ML-Vendors detection trends, IM-Vendors incident learnings, tier-movement data, and external regulatory and standards updates. Refresh output: versioned AUP / Intake / Data-Sharing changelog, communicated org-wide; EG-Vendors training content refreshed in the same cycle.

C) Regulatory and standards contribution. Participate in AI-vendor regulatory forums: EU AI Act deployer-guidance consultations, GDPR EDPB AI guidance, NIST AI RMF working groups, ISO/IEC 42001 community, sector regulators. Contribute AI-vendor clause templates, evidence-view schemas, and incident taxonomy to public standards (Shared Assessments, CSA AI Safety Initiative, OpenSSF). Target: at least two substantive public comments or standards contributions per year.

Outcome Metrics (L3).

Success Criteria.

• On-demand evidence pack generation inside 3 business days; SLA met in last 12 months.
• Quarterly telemetry-driven policy-refresh cycle operating with a versioned changelog.
• ≥2 public regulatory or standards contributions per year.
• External recognition documented (invitations, citations, adoption of contributed artifacts).
• Zero material audit findings on AI-vendor controls in the last 12 months.

Common Pitfalls

Level 1. - Relying on the generic AUP without AI-specific clauses. - No separate intake for AI features inside already-approved SaaS, the parent-vendor loophole. - Compliance map is aspirational, lists frameworks but does not say which policy carries which requirement. - Policy published but not operational, no intake queue, no named owner, no SLA.

Level 2. - Tier-specific addenda published but contract templates never updated, Critical-tier controls exist in policy only. - Evidence view is a folder of PDFs that only the compliance lead understands. - Exception register lacks expiry dates, stale exceptions become the norm. - Sector-specific bundles treated as "covered by the DPA" rather than operationalized.

Level 3. - Automation generates evidence packs that are technically complete but narratively thin, the regulator still needs a human walkthrough. - Policy refresh is cadence-only, quarterly ritual without real telemetry input. - "Regulatory contributions" are comments-on-deadline, not technical artifacts that regulators cite. - Contributed templates get adoption but are not maintained.

Practice Maturity Questions

Level 1. 1. Have you published and formally approved the three priority AI vendor policies (AI Acceptable Use, AI Procurement & Intake, AI Vendor Data-Sharing) with AI-specific clauses (prohibited data classes, personal-account prohibition, AI-feature-inside-SaaS intake requirement, training-on-data default-off, DPA/AI addendum requirement)? 2. Is there a one-page priority compliance map that ties each priority requirement (EU AI Act Art. 26/50, NIST AI RMF GOVERN, GDPR Art. 28/22/44–49, SOC 2 CC9.2, ISO/IEC 42001, ISO/IEC 27001 A.5.19–A.5.23, plus any sector-specific) to the specific L1 policy that carries the control? 3. Is the AI vendor intake gate operational with a published SLA, an amnesty path for previously undisclosed use, ≥95% AUP attestation, and 100% DPA/AI-addendum coverage for AI vendors handling non-public data?

Level 2. 1. Have the three priority AI vendor policies been extended with tier-specific addenda, and do Critical/High AI vendors carry tier-appropriate contractual controls (audit rights, model-version change notice, subprocessor add-notice, incident-notification SLA, training-data provenance attestation)? 2. Does every Critical/High AI vendor have a live, continuously-assembled compliance evidence view covering DPA / AI addendum, subprocessor list, training-data posture, SOC 2/ISO evidence, model-version log, incident history, and deployer-duty evidence, with staleness inside tier-specific targets? 3. Is an exception register operated with named owners, compensating controls, and expiry dates, reviewed monthly, with unsanctioned AI in Critical-tier use cases subject to blocking enforcement and sector-specific evidence bundles complete for in-scope vendors?

Level 3. 1. Can on-demand, regulator-grade evidence packs for any active AI vendor be generated inside 3 business days, with vendor-side changes auto-refreshing the evidence view and full provenance traceable? 2. Does the program operate a quarterly, telemetry-driven policy-refresh cycle (ML detection trends, IM incident learnings, tier-movement, external regulatory updates) with a versioned changelog, and are the changes reflected in EG training within 30 days? 3. Does the program contribute at least two substantive public comments or standards artifacts per year on AI-vendor topics (EU AI Act deployer guidance, GDPR EDPB AI guidance, NIST AI RMF, ISO/IEC 42001, sector regulators, or community standards bodies), with documented external recognition?

16. Education & Guidance (EG)

Practice Overview

Objective: Give the workforce the literacy to recognize AI/HAI vendors in their day-to-day tools and route them through the intake gate, and give the small reviewer population (Security, Procurement, Legal/Privacy, TPRM) the specific skills to run AI-vendor reviews consistently. Shadow AI reduction via awareness is the primary L1 outcome.

Description: EG-Vendors delivers two training tracks at L1: workforce-level AI vendor literacy for every employee, and role-based training for the intake reviewers who must make consistent judgments on training-data posture, DPA adequacy, model provenance, and EU AI Act deployer duties. It also runs the communication campaigns that make shadow AI uncomfortable to hide and sanctioned AI easy to adopt.

Context: Shadow AI is usually a literacy problem first and a policy problem second. Employees adopt ChatGPT, Cursor, or a vendor's freshly-shipped AI assistant without realizing it is a new third party receiving organizational data, and reviewer roles were trained on classic vendor assessment, not on questions like "does the vendor train on our data by default?" or "who is the named deployer oversight?" L1 closes that gap at both levels with minimum viable training, not a full curriculum.

Maturity Level 1

Objective: Deliver foundational AI-vendor literacy to ≥95% of the workforce and role-based AI-vendor-review training to 100% of intake reviewer roles, with an active shadow AI awareness campaign.

Activities.

A) Ship workforce-level AI vendor literacy training. A single short course (≤20 minutes) every employee takes on hire and refreshes annually, tied to the AUP attestation from PC-Vendors. Content covers what counts as an AI vendor (with concrete examples), the AUP in five rules, how to submit intake, the amnesty path, recognizing AI features inside existing tools, and a "before you paste" decision aid. Delivery: LMS module, one-page reference card on the intranet, channel-pinned summary in team chat. No role gating.

B) Deliver role-based training for the AI-vendor intake reviewers. A deeper module (~2 hours) for reviewers only. Content covers reading an AI vendor's training-data posture, DPA and AI addendum adequacy, model provenance and subprocessor chain, EU AI Act Article 26 checklist, the priority compliance map applied to specific vendors, the risk-tier rubric and fast-track path, and a calibration exercise where reviewers independently score sample intakes. Completion is a prerequisite to approving intakes, not optional.

C) Run the shadow AI awareness campaign. An always-on communication program. Elements include an executive-sponsor launch moment that publishes the sanctioned catalog and amnesty window; recurring monthly stories (new AI vendor approved, fast-track win, anonymized intake the team caught, external incident reframed); an "Is this AI?" series calling out AI features quietly turning on inside known SaaS; visible amnesty path; an employee feedback loop to nominate AI tools; and deployer-duty micro-content for regulated or customer-facing AI uses.

Outcome Metrics (L1).

Success Criteria.

• Workforce AI-vendor literacy module launched and ≥95% current-year completion sustained.
• Role-based reviewer training launched, gated on intake approval permissions, and reviewer calibration drift inside target for two consecutive quarters.
• Shadow AI awareness campaign running with at least monthly content cadence.
• Deployer-duty micro-content deployed for every regulated or customer-facing AI vendor use case.
• Training content owner named, content updated within 30 days of policy changes.

Maturity Level 2

Objective: Move from foundational literacy to scenario-based reviewer training with depth calibrated per AI-vendor risk tier, and ship targeted training to product/engineering teams building on AI vendors.

Activities.

A) Scenario-based reviewer training. Per-archetype scenario library built from anonymized real intakes; each scenario includes the as-submitted vendor, the original reviewer decision, the disagreement (if any), and the eventual outcome. Paired calibration exercises with instructor-facilitated debrief on deltas. Tier-weighted curriculum: Critical-tier scenarios dominate advanced modules. Reviewers graduate by running three live intakes end-to-end with senior-reviewer shadow.

B) Product-team and engineering-team AI-vendor training. A distinct track for teams that build on AI vendors: deployer-duty walkthroughs, Article 50 disclosure in user experience, output-integrity patterns, kill-switch design, logging obligations. Paired with SA reference-pattern walkthroughs. Required for any team owning a Critical- or High-tier AI vendor integration; target ≥1 attendee per integration.

C) Behavior-driven shadow AI campaigns. Campaigns tied to observed risk windows (year-end OKR rush, hiring surges, post-industry-incident moments). Each campaign has a pre-measured behavior target and a post-campaign measurement. Amnesty windows run alongside; disclosure volume monitored and attributed back to campaign channels.

Outcome Metrics (L2).

Success Criteria.

• Scenario library of ≥30 real-sourced scenarios across archetypes; reviewer calibration drift inside target on two consecutive quarters.
• Product-team training delivered to ≥1 member of every Critical/High AI vendor integration.
• At least 2 behavior-driven campaigns run in the last 12 months with measured outcomes.
• Content refresh cadence met; ≥80% of content refreshed in last 90 days.

Maturity Level 3

Objective: Operate continuous calibration at scale, publish the AI-vendor reviewer curriculum and rubric as an industry-shared artifact, and contribute to emerging AI-vendor reviewer certification.

Activities.

A) Publish the curriculum and reviewer rubric. Curriculum, scenario library (anonymized), rubrics, and assessment harness published under a permissive license or as a consortium deliverable (CSA AI Safety Initiative, OpenSSF AI, Shared Assessments AI-vendor track). Community contributions welcomed; changes flow back into internal content.

B) Continuous live calibration. Monthly calibration round using a current anonymized intake sampled from the program's live queue; reviewer cohort answers; drift reported. Individual reviewer drift is a development signal, not a performance lever, with coaching hand-offs.

C) Industry-certification contribution. Contribute to AI-vendor reviewer certification pathways (Shared Assessments, CSA, ISC2, ISACA, sector-specific ISAC credentials). Align internal reviewer capstone with certification-grade rubric; support reviewers pursuing external credentials.

Outcome Metrics (L3).

Success Criteria.

• Curriculum and rubric published externally with documented adoption.
• Monthly live calibration operating; drift inside target two consecutive quarters.
• ≥50% of Critical-tier reviewers credentialed (where a credential exists).
• ≥2 substantive contributions to industry certification / curriculum per year.

Common Pitfalls

Level 1. - Vendor security training only for the security team, procurement and legal not trained, security becomes the bottleneck. - Training focuses on process not risk, questionnaire mechanics taught without explaining why vendor security matters. - No business stakeholder awareness, business doesn't understand vendor risk and pressures to skip security. - Training is fear-based, risk aversion blocks necessary vendor relationships.

Level 2. - Scenario library built from invented scenarios rather than anonymized real intakes, reviewers learn the shape but not the real edge cases. - Product-team training is optional; integration owners skip it and produce poorly-scoped designs that DR catches late. - Campaigns launched without a pre-measured behavior target, "awareness" claimed without data. - Calibration drift measured but not acted on, reviewers with persistent drift never receive coaching.

Level 3. - External publication without ongoing maintenance, third parties find a stale artifact and stop trusting the program. - Credentialing becomes performative, reviewers chase credentials that don't map to the program's actual rubric. - Live calibration becomes a "gotcha" instead of a development signal; reviewers game it. - Contributions to working groups don't loop back, what's published externally differs from what reviewers actually use internally.

Practice Maturity Questions

Level 1. 1. Have all procurement, legal, vendor management, and development teams received foundational training on vendor security and supply chain risks? 2. Are awareness campaigns actively communicating vendor security risks, supply chain threats, and real-world vendor breach impacts? 3. Is basic vendor security guidance available (assessment procedures, security questionnaires, contract security clauses)?

Level 2. 1. Is there a scenario library of anonymized real AI-vendor intakes powering reviewer training, with quarterly calibration exercises that show Critical-tier drift inside target? 2. Have you delivered a product/engineering team training track, covering deployer duties (EU AI Act Art. 26/50), output integrity, and SA reference-pattern adherence, to at least one member of every Critical/High AI-vendor integration team? 3. Are shadow AI awareness campaigns running on a behavior-driven cadence with pre-measured targets, and is training content refreshed at least quarterly from program telemetry?

Level 3. 1. Has the curriculum, scenario library, and reviewer rubric been published externally (CSA, OpenSSF AI, Shared Assessments, sector ISAC) with documented adoption, citations, or contributions back? 2. Is a continuous live-calibration cadence operating (monthly anonymized live-intake exercise) with reviewer drift tracked as a development signal, and do ≥50% of Critical-tier reviewers hold an external AI-vendor reviewer credential (where one exists)? 3. Does the program contribute at least two substantive artifacts per year to industry AI-vendor reviewer certification or curriculum working groups, with a traceable loop back into internal content?

17. Threat Assessment (TA)

Practice Overview

Objective: Build the foundational AI-vendor threat library that lets every intake produce a fast, consistent threat snapshot, and give shadow AI its own explicit threat surface. Primary L1 outcome: no AI vendor enters the environment without a documented threat view in under 30 minutes.

Description: TA-Vendors catalogs the threats specific to AI/HAI tools the organization consumes, not the threats of using AI to assess vendors. At L1 the library is category-level (one threat model per AI vendor archetype), mapped to HAIAMM's HAI-specific TTPs (EA, AGH, TM, RA), to top LLM/agentic risks (prompt injection, training-data leakage, output misuse, tool/function abuse, subprocessor exposure), and to shadow AI vectors. Each intake pulls the relevant archetype snapshot and adapts it.

Context: Classic third-party risk threat models miss AI-specific failure modes: a vendor's default train-on-your-data setting turns every prompt into model training data; an AI coding assistant silently egresses source code; an AI agent platform receives a permission model broader than any individual employee's; an AI feature quietly enabled inside an already-approved SaaS inherits parent trust without ever being threat-modeled. L1 closes that gap by making AI-category threats a first-class library reviewers pull from and by naming shadow AI as a distinct category with its own threats.

Maturity Level 1

Objective: Build the AI-vendor threat library, produce a threat snapshot at every intake, and explicitly model the shadow AI threat surface.

Activities.

A) Build the AI-vendor threat library. Author one threat model per AI vendor archetype, five at L1 (consumer GenAI, AI-embedded SaaS, AI coding assistant, AI API/foundation-model, AI agent platform). Each archetype model is short (≤2 pages), explicitly scoped, and maps threats to HAI TTPs and to the priority compliance map. Per-archetype content includes core threats (data egress, prompt injection, output misuse, tool/function abuse, subprocessor exposure, model-family changes, vendor lock-in), HAI TTP tags, the shadow-AI variant of each archetype, reference incidents, and compliance linkage. Owner: named TA-Vendors library steward; cadence: reviewed quarterly.

B) Produce a per-intake threat snapshot inside the intake gate. Bind TA into PC, every intake emits a snapshot; no snapshot, no approval. Snapshot contents (designed to fit one screen): which archetypes apply (a vendor can be more than one), data classes at stake, archetype threats elevated/reduced/n-a for this vendor with justification, top-3 threats with HAI TTP tag and compliance linkage, controls covered by the vendor vs. gaps needing SR/SA follow-up, reviewer name, date, expiry. Time target: ≤30 minutes per intake with the library available.

C) Model the shadow AI threat surface explicitly. Shadow AI gets its own standalone threat document. Content: entry vectors (free tier + personal email, AI assistant toggled inside existing SaaS, browser-extension AI tools, credit-card SaaS under expense thresholds, developer-downloaded OSS models, agent platforms wired without review), threats specific to shadow AI (no DPA, no logging, no human-oversight assignment, no subprocessor visibility, regulated data leaving compliance scope), amplifiers (auto-enabled AI features, parent-DPA inheritance, broad permission models), detections available at L1 (egress/DNS, expense data, SaaS-admin audit, SSO app catalog, endpoint inventory, amnesty self-disclosure), and a triage rubric distinguishing incident from retroactive intake.

Outcome Metrics (L1).

Success Criteria.

• Five archetype threat models published, each tagged to HAI TTPs and linked to the priority compliance map.
• Threat-snapshot gate live inside the intake gate, 100% of new approvals in last 90 days have a snapshot.
• Shadow AI threat doc published and feeding the ML/IM backlogs.
• Named library steward and quarterly refresh cadence operating.
• ≥90% of active AI vendors in the inventory carry a current-year snapshot.

Maturity Level 2

Objective: Layer per-vendor deep threat models on top of archetype snapshots for Critical/High tiers, integrate external AI-vendor threat intelligence, and red-team the threat library itself quarterly.

Activities.

A) Per-vendor deep threat modeling for Critical/High tiers. Attack trees beyond the archetype snapshot: vendor-specific exposure (custom features, specific subprocessors, model-family choice, API surface), org-specific data flow. Abuse-case catalog per vendor, who, with what access, could achieve what harm. Deployer-duty mapping per vendor reflecting Article 26 obligations. Refresh cadence: Critical semi-annual + change-driven; High annual + change-driven.

B) External AI-vendor threat intelligence integration. Subscribe to and operationalize MITRE ATLAS, the AI Vulnerability Database, OWASP LLM/Agentic Top 10 revisions, sector ISAC AI-vendor intelligence, academic security venues, and direct vendor-advisory feeds for Critical-tier vendors. Quarterly triage decides which new intel items change the archetype library or per-vendor models.

C) Red-team the threat library itself. Quarterly exercise: ST-Vendors runs against an in-scope integration using only threats from the library; what's missed is a library gap. Gap closure is a governance activity, misses are tickets with owners and expiries. External purple-team participation where possible (industry tabletops, ISAC exercises).

Outcome Metrics (L2).

Success Criteria.

• Per-vendor deep models live for 100% Critical and ≥90% High tier.
• External intel integrated with quarterly triage and change-log.
• Quarterly red-team-the-library exercise operating; gaps closed with owners and expiries.
• Lead time from intel signal to library update ≤30 days on Critical-impact items.

Maturity Level 3

Objective: Automate threat-library maintenance from telemetry and external feeds, and publish anonymized AI-vendor threat patterns back to the industry.

Activities.

A) Telemetry-driven library updates. ML detections, IM incidents, and external feeds auto-propose library changes; human curators approve. Change-log fully machine-readable; downstream practices (SR, SA, ST) consume updates automatically.

B) Industry contribution. Contribute emerging AI-vendor TTPs to MITRE ATLAS cycles, AI Vulnerability Database, OWASP LLM / Agentic Top 10 revisions. Target ≥4 substantive contributions per year; quality-graded, legally vetted.

C) Shared threat-model artifacts. Publish anonymized archetype threat models under permissive license. Host or co-host industry tabletops tied to the library.

Outcome Metrics (L3).

Success Criteria.

• Library auto-update pipeline operating with ≤14-day lead time.
• ≥4 industry contributions per year; ≥2 recognized externally.
• Anonymized archetype models published with tracked adoption.
• Industry tabletop hosted or co-hosted in last 12 months.

Common Pitfalls

Level 1. - Threat scenarios are generic, not AI-vendor-specific. - Inventory is incomplete, missing AI features inside existing SaaS, missing developer endpoints, missing agent platforms. - Threats documented but not shared, security knows; procurement and legal do not. - Vendor contracts lack security provisions (no breach SLA, no audit rights, no subprocessor approval).

Level 2. - Per-vendor deep models written once and never refreshed, staleness hidden because age isn't tracked. - External intel integrated but never triaged, feeds pile up, library unchanged. - "Red-team the library" is a presentation, not an exercise. - Deep modeling stops at Critical tier; High tier stays on archetype snapshot only.

Level 3. - Automated proposals accepted without curation, library contaminated by false-signal telemetry. - "Contributions" to MITRE/AIVD/OWASP are cosmetic, names on PRs, not substantive technical artifacts. - Published archetype models are unmaintained; external users find stale versions. - Program leans on external feeds and stops investing in its own per-vendor deep modeling.

Practice Maturity Questions

Level 1. 1. Have you documented threat scenarios specific to the five AI vendor archetypes (consumer GenAI, AI-embedded SaaS, AI coding assistant, AI API/model, AI agent platform) tagged to HAI TTPs (EA, AGH, TM, RA) and linked to the priority compliance map? 2. Is a per-intake threat snapshot wired into the intake gate (≤30 minutes per intake, ≥90% inventory coverage), and is the shadow AI threat surface documented as a standalone artifact reviewed by the program sponsor? 3. Is there a named library steward and a quarterly refresh cadence, with downstream practices (SR, SA, ST, IM, ML) consuming the library rather than re-deriving threats?

Level 2. 1. Does every Critical AI vendor have a current-year per-vendor deep threat model (not only an archetype snapshot), and ≥90% of High AI vendors the same? 2. Is external AI-vendor threat intelligence (MITRE ATLAS, AI Vulnerability Database, OWASP LLM/Agentic Top 10, sector ISACs, vendor advisories) integrated with a quarterly triage cadence and a documented change-log? 3. Do you run a quarterly red-team-the-library exercise, track and close library gaps with named owners and expiries, and keep intel-to-library lead time ≤30 days on Critical-impact items?

Level 3. 1. Does the threat library auto-update from telemetry (ML detections, IM incidents, external feeds) with human curation, and is the change lead time from signal to library update ≤14 days? 2. Does the program contribute at least four substantive AI-vendor threat artifacts per year to MITRE ATLAS / AIVD / OWASP cycles, with at least two externally recognized? 3. Are anonymized archetype threat models published under permissive license with documented peer-org adoption, and does the program host or co-host industry tabletops tied to the library?

18. Security Requirements (SR)

Practice Overview

Objective: Define the minimum, reusable security requirements pack for AI vendors that the intake gate enforces, translating the threats from TA and the policies from PC into specific, testable requirements the vendor must meet (or the organization must compensate for) before approval.

Description: SR-Vendors authors a small, archetype-keyed AI Vendor Requirements Pack: one base requirement set that applies to every AI vendor, plus per-archetype deltas. Each requirement is stated as a testable condition with an evidence source (DPA clause, vendor docs, admin-console setting, SOC 2 control, demonstration) so the intake gate produces a decision that is defensible to auditors and customers.

Context: Without an explicit AI-vendor requirements pack, each intake invents the bar from scratch. Two reviewers score the same vendor differently, contracts miss clauses that would have been obvious, and the program cannot evidence EU AI Act deployer duties or GDPR processor adequacy across its AI vendor footprint. L1 closes that gap with the minimum viable pack, not a checklist sprawl, but the 20-ish requirements that matter for every AI vendor plus archetype-specific additions.

Maturity Level 1

Objective: Publish the AI Vendor Requirements Pack (base plus per-archetype), wire it into the intake gate, and produce a Requirements-Evidence Map (REM) for every new AI vendor.

Activities.

A) Author the base AI Vendor Requirements Pack. Target ≤20 base requirements. Each has an ID, statement, rationale (threat plus compliance tag), evidence source, test method, and acceptance criteria. Minimum base categories: data handling (no-training default, retention cap, deletion on termination, data residency), contracting (executed DPA / AI addendum, current subprocessor list, breach-notification SLA ≤72 hours, audit or attestation right, exit data-return clause), identity and access (SSO/SAML or OIDC, admin-role separation, API-key rotation, least-privilege roles), logging and observability (per-user and per-prompt audit logs available, export mechanism, retention meets policy), transparency and oversight (vendor discloses model family and material changes, human-in-the-loop options, Article 50 disclosure where applicable), and security posture (current SOC 2 Type II / ISO 27001, vulnerability-handling process, recent pen-test summary).

B) Author per-archetype requirement deltas. Each archetype has 3–8 additional requirements on top of the base pack. Consumer GenAI: enterprise tenant required, personal-account prohibition enforceable via SSO, content-filtering configuration, no memory across org boundaries unless scoped. AI-embedded SaaS: AI feature explicitly in parent DPA or new addendum, feature toggleable per workspace, training-on-data contractually off by default, admin visibility into users. AI coding assistant: no training on org code by default, on-prem or no-retention path for regulated workloads, IDE/endpoint policy control, telemetry scoped. AI API / foundation-model: no-train commitment, region pinning, model-version pinning or change-notification, rate-limit and abuse-throttle behavior, PII-handling controls. AI agent / automation platform: explicit tool allowlist, per-tool scope and rate caps, human-in-the-loop gates for destructive actions, agent-session audit logs, indirect-prompt-injection defenses.

C) Wire the pack into the intake gate and produce a Requirements-Evidence Map per vendor. Every new AI vendor approval produces a REM. Each applicable pack requirement is marked Met / Met-with-compensating-control / Gap-accepted / Not-applicable (with justification). Each "Met" row cites the evidence (DPA section, admin-console screenshot, SOC 2 control ID, vendor doc URL, demonstration note). Each "Gap-accepted" row names a compensating control, an owner, and a re-review date. The REM is stored with the intake ticket and linked from the inventory record. Renewal, contract change, or material AI-feature change triggers REM re-review.

Outcome Metrics (L1).

Success Criteria.

• Base pack plus five archetype deltas published, tagged to TA threats and the priority compliance map.
• 100% of new AI vendor approvals in the last 90 days have a REM on file.
• ≥90% of active AI vendors carry a current-year REM.
• Named pack owner and quarterly refresh cadence operating.
• Accepted-gap backlog tracked with median age inside target.

Maturity Level 2

Objective: Replace qualitative requirements with quantitative, SLA-bound ones; deepen per-tier requirement deltas; validate requirement-evidence continuously.

Activities.

A) Quantitative requirement pack. Every base requirement gets a measurable condition: incident-notification SLA in hours, retention in days, availability in percent, log exportability in format, subprocessor notice in days. Archetype deltas similarly quantified: AI API proxy latency SLO, agent tool-scope enumeration, AI-embedded SaaS per-workspace toggle state. Training-data attestation requires explicit no-train contract language and technical-control confirmation.

B) Per-tier requirement depth. Critical tier adds: red-team right, FRIA support, model-version pinning with 30-day advance notice, provenance attestation, EU AI Act Article 26 full checklist. High tier: SOC 2 Type II + ISO 27001 annual refresh, model-version change-notification, incident-notification ≤72h. Medium: annual attestation refresh, subprocessor-list update. Low: annual attestation only.

C) Continuous REM-evidence validation. Quarterly sampling for Critical, semi-annual for High, pick N REM rows, verify evidence against current reality (admin console, vendor API, IR findings, ML logs). Deltas raise findings routed to IM. Accepted-gap aging review monthly; blocker escalations to sponsor.

Outcome Metrics (L2).

Success Criteria.

• 100% requirements quantitative or binary.
• ≥95% Critical REMs re-validated in last 90 days.
• Accepted-gap backlog inside aging target.
• Full Article 26 evidence for 100% Critical AI vendors.

Maturity Level 3

Objective: Publish the AI-vendor requirements pack as an industry-shared artifact, automate REM-evidence validation, and contribute to AI-vendor procurement language standards.

Activities.

A) Publish the pack and REM schema. The AI-vendor requirement pack (base + archetypes) and REM schema published as permissive-license artifacts. Contribute to Shared Assessments AI-vendor track, CSA AI Safety Initiative, sector standards bodies.

B) Automated REM validation. Vendor API ingestion for admin-console state, subprocessor lists, SOC 2 bridge letters. Internal telemetry (ML detections, IR findings) auto-corroborates evidence claims. Human review reserved for exceptions and novel clauses.

C) Standards contribution. Contribute model DPA / AI addendum clauses to standards bodies (Shared Assessments, IAPP, CSA). Comment on regulatory guidance where appropriate (EDPB AI, EU AI Act deployer practice, NIST Playbook).

Outcome Metrics (L3).

Success Criteria.

• Pack and REM schema published under permissive license with tracked adoption.
• ≥70% REM auto-validation.
• ≥2 industry-standard contributions per year.
• Vendor redline success rate ≥80% on contributed clauses.

Common Pitfalls

Level 1. - Pack treated as a one-time artifact, published, then never refreshed. - Requirements stated qualitatively ("appropriate logging") rather than as testable conditions. - REM is a tickbox exercise, every row marked "Met" with no evidence citation. - Accepted gaps lack expiry dates, they quietly become permanent exceptions.

Level 2. - Quantitative requirements adopted in policy but not enforced in contract, vendors negotiate them out. - Continuous validation runs but findings don't route to IM, drift visible to compliance, invisible to engineering. - Tier-aware deltas are advisory, not operational, the same evidence depth is collected for Critical and Low. - Article 26 checklist treated as documentation rather than evidence-bearing artifacts.

Level 3. - Published pack is unmaintained; external users find stale clauses. - Auto-validation accepts vendor self-reports as ground truth, no independent corroboration. - Standards contributions are press-release artifacts, not technical clauses regulators or peers cite. - Vendor redline patterns ignored, recurring redlines signal a pack that doesn't reflect market reality.

Practice Maturity Questions

Level 1. 1. Have you published the AI Vendor Requirements Pack (base ≤20 requirements + five archetype deltas) with each requirement tagged to a TA archetype threat and a priority-compliance item, and is a named owner running quarterly refresh? 2. Is the pack wired into the intake gate, with 100% of new AI vendor approvals in the last 90 days carrying a Requirements-Evidence Map (REM) and ≥90% of active vendors carrying a current-year REM? 3. Are accepted gaps tracked with named owner, compensating control, and expiry date, and is the median accepted-gap age ≤90 days?

Level 2. 1. Are 100% of pack requirements expressed as quantitative or binary testable conditions (SLA hours, retention days, availability percent), with tier-specific deltas operationalized (Critical = red-team right, FRIA support, Art. 26 full checklist)? 2. Are ≥95% of Critical REMs re-validated against observed reality (admin console, vendor API, IR findings, ML logs) in the last 90 days, with deltas routed to IM as findings? 3. Is the accepted-gap backlog inside the aging target by tier (Critical ≤60 days median), and does every Critical AI vendor carry full EU AI Act Article 26 checklist evidence?

Level 3. 1. Are the requirements pack and REM schema published under a permissive license with tracked external adoption, and does the program contribute ≥2 industry-standard contributions per year (Shared Assessments AI-vendor track, CSA, IAPP)? 2. Are ≥70% of REM rows auto-validated via vendor API ingestion (admin-console state, subprocessor lists, SOC 2 bridge letters) and internal telemetry corroboration, with human review reserved for exceptions? 3. Is the vendor redline success rate ≥80% on program-contributed clauses, and does the program participate in EDPB AI, EU AI Act deployer practice, and NIST Playbook regulatory comment cycles?

19. Secure Architecture (SA)

Practice Overview

Objective: Publish the reference architectures for safely consuming each AI vendor archetype, so teams integrating an AI vendor have a vetted "green path" that already implements the SR requirements, and teams deviating from it do so knowingly and explicitly.

Description: SA-Vendors ships a small catalog of reference integration patterns, one per AI vendor archetype, showing how to place the data boundary, enforce identity, route traffic (and keep it routable), log activity, and contain agentic behavior. Each pattern is a block diagram plus a decision-log that ties the choices back to SR-Vendors requirements and TA-Vendors threats. Teams reach for the pattern first; deviations require design review.

Context: Without reference patterns, every team reinvents the integration. Some pipe prompts direct to a vendor API with no logging, some paste regulated data into consumer GenAI, some grant an AI agent platform credentials that exceed any human's permission set. At L1 SA-Vendors makes the safe path the easy path, not by blocking integrations but by publishing the shape of one that already satisfies the requirements.

Maturity Level 1

Objective: Publish reference architectures per AI vendor archetype, an anti-patterns catalog, and a deviation-review path; link each pattern to SR requirements and TA threats.

Activities.

A) Publish reference architectures per AI vendor archetype. Five patterns at L1, one per archetype. Each pattern is short (≤2 pages), includes a labeled diagram, and covers a common skeleton: scope (what the pattern covers and explicitly does not), data boundary (where org data meets vendor data, DLP inspection points), identity and authentication (SSO-backed access, service-principal model for API use, secret management), traffic path (egress through monitored network, optional proxy with API-level inspection, region pinning), logging (what is logged where, retention, exportability, deployer-duty evidence trail), controls mapped to SR requirements (row-by-row mapping with gaps acknowledged), and threats mitigated (which TA archetype threats the pattern addresses and which remain residual). Archetype-specific emphasis includes: enterprise tenant + SSO + content-filtering for consumer GenAI; toggle + per-workspace scope + parent-DPA addendum for AI-embedded SaaS; IDE policy + regulated-repo no-train path for AI coding assistants; an internal API proxy (the single most load-bearing pattern element) with prompt/response logging, PII scrubbing, rate-limit shaping, model-version pinning, kill-switch for AI API/foundation-model; tool allowlist + per-tool scope + human-in-the-loop wrapper + indirect-prompt-injection defense for AI agent platforms.

B) Publish the anti-patterns catalog. Name, describe, and prohibit AI-vendor integrations that reliably cause incidents. The L1 set: copy-paste workflow (round-tripping regulated data into a consumer GenAI tab), shadow API key (a single shared API key in a vault with no attribution or revocation plan), unscoped agent (agent platform with unrestricted tool access or broad network/filesystem reach), trust the embedded feature (AI feature inside approved SaaS turned on org-wide with no incremental review), pull-without-label (RAG consuming untrusted content with no provenance tagging or sanitization), direct-to-vendor call (apps calling vendor APIs directly without the internal proxy). Each anti-pattern includes description, why it's dangerous, real-incident flavor, and the reference pattern that replaces it.

C) Publish the deviation-review path and integrate patterns into intake/inventory flow. Teams adopting an approved AI vendor land on the inventory record, see the recommended pattern, and pick "using pattern" or "deviating." Deviations require a short design review (handled in DR L1) with a named architect reviewer and documented rationale stored with the integration. Repeat deviations in the same direction signal the need to update the pattern, not to keep approving exceptions.

Outcome Metrics (L1).

Success Criteria.

• Five reference patterns published, each mapped to SR requirements and TA threats.
• Anti-patterns catalog published, linked from the AUP and intake gate, referenced in EG training.
• Deviation-review path operational with a named architect reviewer population.
• ≥85% of active AI vendor integrations classified as "on pattern" or "deviation with review."
• 100% of API/model archetype integrations with non-public data flowing through the internal API proxy.

Maturity Level 2

Objective: Extend reference patterns to multi-region, multi-tenant, and agent-platform complexity; publish an incident-informed anti-pattern catalog; encode patterns as IaC templates teams fork.

Activities.

A) Extended reference patterns. Multi-region AI-vendor integration pattern (residency enforcement, cross-region failover, GDPR transfer mechanism). Multi-tenant parent-SaaS AI-feature pattern (per-tenant key scope, per-tenant data scope, admin-governance integration). Agent-platform pattern (tool-sandboxing, human-in-the-loop gates, session-isolation, indirect-prompt-injection defense in RAG-shaped agents).

B) Incident-driven anti-pattern catalog. Every IM-Vendors incident classified to an anti-pattern (existing or new). Catalog refreshed monthly; teams see new anti-patterns before new features.

C) Pattern-as-IaC. Reference patterns encoded as Terraform modules, Kubernetes manifests, Crossplane compositions, or Pulumi stacks. Teams fork rather than handcraft; deviations flagged at plan time.

Outcome Metrics (L2).

Success Criteria.

• Three extended patterns published.
• ≥80% Critical/High integrations on IaC patterns.
• Anti-pattern catalog updated from ≥3 incidents in last 12 months.
• Pattern-drift detection covering 100% of IaC integrations.

Maturity Level 3

Objective: Contribute reference patterns to industry; implement zero-trust AI-vendor access; formally specify agent-tool-scope boundaries for Critical-tier agents.

Activities.

A) Contribute reference patterns externally. Patterns published to CNCF AI SIG, OpenSSF AI, CSA, or sector-specific bodies. Maintained upstream; internal use aligns with external version.

B) Zero-trust AI-vendor access. Continuous session verification for human access. Just-in-time tool scope for agents (scopes issued per session/per task). Device-trust integration, endpoint posture required for AI-vendor access.

C) Formal tool-scope specification. Critical-tier agent integrations carry a formal tool-scope spec (parameter types, rate, data-class) enforced at runtime. Spec changes pass a DR L3 review gate.

Outcome Metrics (L3).

Success Criteria.

• ≥2 patterns externally adopted.
• ≥90% human sessions under zero-trust.
• ≥70% machine calls under JIT-scoped credentials.
• 100% Critical-tier agents with formal tool-scope specs.

Common Pitfalls

Level 1. - Reference patterns published without diagrams, text-only patterns are harder to follow than the integrations they replace. - Patterns published but not linked from the inventory, teams never find them. - Anti-patterns catalog is theoretical, no reference incidents, so it's ignored as alarmism. - Direct-to-vendor calls allowed because "the proxy is too slow", single biggest hidden risk.

Level 2. - Extended patterns ship but the original five are not maintained, the Library splits. - Anti-patterns added from incidents, but the originating IM incidents never close, pattern updates without remediation. - IaC encoding is partial, half the controls in the pattern have IaC equivalents, the other half are documented prose. - Pattern-drift detection raises noise, false positives from legitimate variation.

Level 3. - External pattern contributions diverge from internal use, the upstream pattern reflects a previous internal version. - Zero-trust enforcement is bypassed for senior users or break-glass scenarios that quickly become routine. - Formal tool-scope specs are written but not runtime-enforced, they're documentation, not control. - Industry contributions chase novelty rather than stabilizing the most-used patterns.

Practice Maturity Questions

Level 1. 1. Are five reference architectures published (one per archetype: consumer GenAI, AI-embedded SaaS, AI coding assistant, AI API/foundation-model, AI agent platform) with each mapped to SR requirements and TA threats? 2. Is an anti-patterns catalog published, linked from the AUP and intake gate, referenced in EG training, with each anti-pattern carrying a description, why-dangerous, real-incident flavor, and the replacement pattern? 3. Are ≥85% of active AI vendor integrations classified as "on pattern" or "deviation with review," and are 100% of API/model archetype integrations with non-public data flowing through the internal API proxy?

Level 2. 1. Are extended patterns published for multi-region, multi-tenant, and agent-platform complexity, and is the anti-pattern catalog updated from at least three IM incidents in the last 12 months? 2. Are ≥80% of Critical/High integrations using IaC-encoded patterns (Terraform, Kubernetes manifests, Crossplane, Pulumi), with pattern-drift detection covering 100% of IaC-encoded integrations? 3. Are pattern updates traceable to a specific incident, threat, or requirement change, and is the change log refreshed monthly?

Level 3. 1. Are at least two reference patterns externally adopted (CNCF AI SIG, OpenSSF AI, CSA AI Safety Initiative, sector bodies), with maintained upstream parity to internal versions? 2. Are ≥90% of AI-vendor human sessions under continuous zero-trust verification (device trust, session evaluation), and are ≥70% of machine calls under JIT-scoped credentials? 3. Do 100% of Critical-tier agent integrations carry runtime-enforced formal tool-scope specifications (parameter types, rate, data-class), with spec changes gated through a DR L3 review?

20. Design Review (DR)

Practice Overview

Objective: Operate a lightweight design checkpoint between intake approval and production rollout for every AI vendor integration, confirming the team picked a reference pattern, covered the SR requirements, and accepted only the residual risks the program can live with.

Description: DR-Vendors is the single moment where architecture (SA), requirements (SR), and threats (TA) meet an actual planned integration. At L1 the review is deliberately small: a structured design checkpoint with a named reviewer, a standard checklist keyed to the archetype, and a written decision (approve / approve-with-conditions / send back). It sits before implementation begins, catching issues when they cost hours to fix, not weeks.

Context: Without a design checkpoint, AI vendor integrations get discovered already in production. The pattern is skipped, the logging omitted, the permission boundary too broad, and retrofitting after launch is expensive and visible. L1 DR-Vendors puts a small, predictable gate in front of rollout, timeboxed so it does not slow teams down.

Maturity Level 1

Objective: Run a standard design checkpoint per AI vendor integration before production, producing a written decision with traceability to SR, SA, and TA.

Activities.

A) Publish the AI Vendor Integration Design Checklist. One checklist per archetype, derived from the SA reference pattern. Common spine: pattern adherence (using the reference pattern, or documented deviation with rationale), data boundary (which data classes cross to the vendor, DLP/proxy inspection points), identity (SSO-backed human access, service-principal model for machine access, secret management), logging (prompt/completion logs captured, human-oversight evidence trail present, retention meets policy), failure modes (vendor outage, model change, rate limit, fallback or kill-switch defined), permissions for agent archetype (tool allowlist, per-tool scope, human-in-the-loop gates), disclosure (Article 50 disclosure where AI interaction is user-visible), residual risk (explicit list of residual risks, compensating controls, owners, expiry).

B) Triage and route reviews by risk tier. Two-lane model aligned to the risk tier from intake. Fast-lane (Low/Medium): async checklist review, target SLA ≤2 business days. Full-lane (High/Critical): 45–60 minute architect review with the team walking the reference pattern, target SLA ≤5 business days. Trigger for full-lane regardless of tier: deviation from the reference pattern, agent archetype, regulated-data involvement, external customer exposure. Both lanes produce one structured decision record (approve / approve-with-conditions / send-back) stored against the integration.

C) Close the loop with SA, SR, and IM. Three deviations in the same direction for the same archetype auto-queue a pattern review with SA. A requirement repeatedly waived with a compensating control auto-queues an SR pack-revision review. Every IM-Vendors incident triggers a re-look at the design checkpoint that approved the integration: was the issue visible? what would catch it earlier?

Outcome Metrics (L1).

Success Criteria.

• Design checklist per archetype published and versioned.
• Two-lane review model operational with published SLAs.
• ≥95% of AI vendor integrations going to production in last 90 days have a completed checkpoint record.
• Pattern-update and pack-update triggers wired to SA and SR.
• Named reviewer population trained and active.

Maturity Level 2

Objective: Move design reviews from checklist to scenario-based walkthroughs; include vendor participation for Critical-tier; detect design drift between reviews.

Activities.

A) Scenario-based reviews for Critical/High tiers. Reviewer walks 3–5 specific threat scenarios against the proposed design. Scenarios sourced from the TA library plus anonymized industry incidents. Review decision tied explicitly to how the design handles each scenario.

B) Vendor participation in Critical-tier reviews. Pre-established communication channels with Critical-tier vendor security and architecture teams. Joint design discussion for novel integrations; vendor-side FRIA cooperation for EU AI Act high-risk uses. Template NDA and sharing-agreement ready; Legal pre-approval.

C) Design-drift detection. Compare live integration vs. approved design, quarterly for Critical, annually for High. Material drift (pattern change, new tools, new data classes, new regions) auto-routes back to DR.

Outcome Metrics (L2).

Success Criteria.

• 100% Critical reviews scenario-based.
• ≥70% Critical-tier vendors participating in joint reviews.
• Drift detection operating; 100% material drifts return to DR.

Maturity Level 3

Objective: Continuous design attestation via automated pattern-compliance telemetry, and contribute architecture-review patterns to industry.

Activities.

A) Continuous design attestation. Integrations declare compliance monthly via automated pattern-compliance scans and configuration audits. Deviations open a DR-exception ticket automatically.

B) Contribute review patterns to industry. Publish review rubrics, scenario templates, vendor-cooperation frameworks to OpenSSF AI, CSA, Shared Assessments.

C) Pattern evolution driven by external + internal data. External incident patterns (from ISACs, MITRE ATLAS) plus internal IM incidents plus ML telemetry drive design-pattern updates quarterly.

Outcome Metrics (L3).

Success Criteria.

• Monthly auto-attestation for ≥90% of Critical integrations.
• ≥2 externally contributed review artifacts per year.
• Review backlog age inside target.

Common Pitfalls

Level 1. - Design review skipped for "small" integrations that turn out not to be small. - Reviewers default to approving, the rejection bar is undefined, every integration passes. - Decisions written in chat instead of stored as records, the audit trail is private message history. - The design checklist is published but not linked from the intake artifact, so teams never see it until DR raises a finding.

Level 2. - Scenario walkthroughs become recitations, the team reads scenarios aloud, no probing happens. - Vendor participation requested but never honored, pre-established channels go cold. - Drift detection finds drift, but the program lacks SLA discipline to bring drifted integrations back through DR. - The two-lane model devolves to one-lane (everything fast-lane or everything full-lane).

Level 3. - Auto-attestation accepts integration self-reports without independent corroboration. - Industry contributions are slide decks rather than rubrics, scenario templates, or vendor-cooperation frameworks. - Continuous attestation creates noise the integration owners cannot absorb. - The review backlog age stays inside target by closing tickets, not by resolving findings.

Practice Maturity Questions

Level 1. 1. Is a per-archetype Design Checklist published and versioned, with reviews routed via a two-lane model (fast-lane Low/Medium, full-lane High/Critical or any deviation/agent/regulated-data case) with published SLAs? 2. Do ≥95% of AI vendor integrations going to production in the last 90 days have a completed design-checkpoint record that references the applicable SA pattern and SR requirement pack? 3. Are pattern-update and pack-update triggers wired to SA and SR (three deviations in the same direction queue a pattern review; repeated requirement waivers queue a pack-revision review), and is a named reviewer population trained and active?

Level 2. 1. Are 100% of Critical-tier reviews using scenario-based walkthrough (3–5 specific threat scenarios from the TA library plus anonymized industry incidents)? 2. Do ≥70% of eligible Critical-tier vendors participate in joint reviews via pre-established communication channels with template NDA and Legal pre-approval? 3. Is design-drift detection running on cadence (quarterly Critical, annual High), with 100% of material drifts returning to DR?

Level 3. 1. Do ≥90% of Critical integrations carry monthly auto-attestation via automated pattern-compliance scans and configuration audits, with deviations opening DR-exception tickets automatically? 2. Does the program contribute at least two externally-published artifacts per year to OpenSSF AI, CSA, or Shared Assessments (review rubrics, scenario templates, vendor-cooperation frameworks)? 3. Is the mean review backlog age ≤7 days, and is pattern evolution driven by a pipeline of external incident patterns (ISACs, MITRE ATLAS) plus internal IM incidents plus ML telemetry refreshed quarterly?

21. Implementation Review (IR)

Practice Overview

Objective: Verify, at the point of deployment and on a recurring cadence, that the actual AI vendor integration configuration matches the design approved at DR, closing the gap between what was designed and what is running.

Description: IR-Vendors is the configuration check for AI vendor integrations, the moment someone opens the vendor's admin console, the app's config, the proxy rules, and the SaaS admin settings and confirms they match the approved design. At L1 the review is a short checklist per archetype, focused on the handful of settings where production reality most commonly drifts from the design.

Context: AI vendor admin consoles are deep and fast-moving. Defaults change; vendors quietly add new AI features to existing panels; API-key permissions drift; logging toggles revert to off after a product update. Without an implementation review, "we turned off training" decided at DR becomes "we thought we turned off training" six months later. L1 IR-Vendors makes the config check a regular, lightweight activity rather than a periodic audit scramble.

Maturity Level 1

Objective: Run an implementation review at go-live and at least annually per AI vendor integration, using a per-archetype config checklist; track findings to closure.

Activities.

A) Publish the per-archetype implementation review checklist. Common spine: no-train and retention settings (verify admin-console toggles match DPA commitments), SSO and identity binding (SSO enforced, local-auth disabled, admin-role separation), logging enabled and exported (logs cover required events, retention is policy-compliant, export path works), API key/token scope (least-privilege, owner-attributed, rotation schedule), rate and abuse limits (caps align with DR-approved assumptions), region and residency (region setting matches DPA), new AI features surfaced (vendor panel scanned for features added since last review). Archetype-specific: org-tenant binding and content-filter for consumer GenAI; per-workspace toggle state and users-with-access for AI-embedded SaaS; IDE-policy enforcement and regulated-repo no-train path for AI coding assistants; proxy presence and health, model-version pin, PII scrubbing, kill-switch wired for AI API/model; tool allowlist, per-tool scope, HITL gates, session logging, untrusted-content provenance for AI agent platforms.

B) Perform reviews at the right moments. Three triggers at L1: go-live (before production cutover, reviewing as-built config against the DR-approved design), annual (every active integration reviewed at least annually, scheduled from the inventory), change-triggered (vendor major product update, pricing-plan change, org-plan migration, or material admin-console redesign triggers an ad-hoc review). Reviews are short (target 20–45 minutes per integration) and evidence-based, screenshots or config exports stored with the record.

C) Track findings to closure. Every review produces zero or more findings with severity (blocker / high / medium / low), named owner, SLA (blocker fix before production or rollback; high ≤14 days; medium ≤45 days; low ≤90 days or accepted residual), and evidence (after-fix screenshot or config export linked to close). Findings feed IM as issues and EH as hardening work where the pattern itself needs changes.

Outcome Metrics (L1).

Success Criteria.

• Per-archetype checklists published and owned.
• Go-live, annual, and change-triggered review triggers wired to the inventory and integration tracker.
• 100% of new AI vendor integrations in the last 90 days have a go-live IR record.
• ≥90% of active AI vendor integrations carry a current-year IR record.
• Findings-aging dashboard reviewed at least monthly by the program sponsor.

Maturity Level 2

Objective: Continuous configuration validation via vendor APIs; automated no-train and retention verification; tier-calibrated IR cadence.

Activities.

A) API-based config monitoring. Critical vendors' admin APIs consumed by the program to produce a live config snapshot. Change events flow into the IR queue; material changes auto-generate findings. Where vendor APIs do not exist, scheduled UI-scraping or attestation confirmations fill in.

B) Automated no-train and retention verification. Recurring synthetic canaries for vendors where probing is contract-permitted. Admin-console state polled; deltas vs. DPA commitments raise findings. Retention-window sampling: prompt/completion logs older than agreed retention trigger evidence request.

C) Tier-calibrated cadence. Critical: go-live + semi-annual + change-triggered + continuous drift detection. High: go-live + annual + change-triggered. Medium: go-live + annual. Low: go-live only + re-review on change.

Outcome Metrics (L2).

Success Criteria.

• ≥90% Critical vendors under API-based config monitoring.
• Critical drift detection ≤7 days.
• Automated no-train verification for ≥80% Critical.
• Tier-cadence adherence ≥95%.

Maturity Level 3

Objective: Real-time configuration attestation; contribute config-baseline schemas to industry; integrate with vendor trust-center attestations.

Activities.

A) Real-time attestation. Config state evaluated continuously; any deviation raises an instant finding. Attestation artifacts machine-readable and regulator-consumable.

B) Contribute config-baseline schemas. Per-archetype IR config-baseline schemas (what "correct" looks like) published to Shared Assessments, CSA AI Safety Initiative.

C) Trust-center integration. Ingest vendor trust-center artifacts (SIG Lite, CAIQ, custom) into IR evidence automatically. Deltas from DPA / AI addendum auto-flagged.

Outcome Metrics (L3).

Success Criteria.

• Real-time attestation for ≥90% Critical.
• Config-baseline schemas published with external adoption.
• Trust-center feeds live for ≥70% Critical/High.

Common Pitfalls

Level 1. - Annual reviews calendared but not enforced, review-due dates pass quietly. - Findings recorded without evidence, reviewers note "no-train confirmed" without a screenshot. - Change-triggered reviews skipped because the program never sees the change, vendor changelogs aren't watched. - Closure SLAs missed without escalation, high-severity findings age into months.

Level 2. - API-based monitoring ingests vendor admin API state but the data isn't compared to DR-approved baselines. - No-train verification trusts the admin-console toggle without behavioral probing. - Tier-calibrated cadence drifts to "everyone gets the same cadence we have time for." - Drift findings raised but then accepted without re-review at DR.

Level 3. - Real-time attestation is high-frequency but low-fidelity, config drift signals are buried in noise. - Config-baseline schemas published but unmaintained. - Trust-center feeds ingested without delta-detection, automated work that misses the point. - The program treats automation as a substitute for an inventory rather than a complement to it.

Practice Maturity Questions

Level 1. 1. Are per-archetype implementation review checklists published and owned, with go-live, annual, and change-triggered review triggers wired to the inventory and integration tracker? 2. Do 100% of new AI vendor integrations in the last 90 days have a go-live IR record, and do ≥90% of active integrations carry a current-year IR record? 3. Are findings tracked with severity, named owner, SLA, and evidence, with a findings-aging dashboard reviewed monthly by the program sponsor and zero blocker findings open at go-live?

Level 2. 1. Are ≥90% of Critical vendors under API-based configuration monitoring (admin APIs ingested, with material changes auto-generating findings, scheduled UI-scraping where APIs do not exist)? 2. Are ≥80% of Critical vendors under automated no-train and retention verification (admin-console state polled, synthetic canaries where contract-permitted, retention-window sampling)? 3. Is the tier-calibrated IR cadence adhered to ≥95% (Critical: go-live + semi-annual + change + continuous; High: go-live + annual + change; Medium: go-live + annual; Low: go-live only)?

Level 3. 1. Do ≥90% of Critical vendors have real-time configuration attestation (continuous evaluation, instant findings on deviation, machine-readable regulator-consumable artifacts)? 2. Are per-archetype IR config-baseline schemas published externally (Shared Assessments, CSA AI Safety Initiative) with documented external adoption? 3. Are vendor trust-center artifacts (SIG Lite, CAIQ, custom) ingested for ≥70% of Critical/High vendors with deltas from DPA and AI addendum auto-flagged?

22. Security Testing (ST)

Practice Overview

Objective: Exercise the AI vendor integration end-to-end with foundational tests that directly target the top archetype threats (data egress, prompt injection, permission-boundary abuse, logging completeness, shadow-AI discovery), so reviewed configurations are not only correct on paper but observed to behave correctly.

Description: ST-Vendors operates at two levels at L1: per-integration acceptance tests that every approved AI vendor integration must pass before production, and program-level shadow AI discovery tests that exercise the organization's ability to detect unsanctioned AI use. The test scope is deliberately narrow at L1, five or six recurring test classes, automated where possible, tied directly to threats in the TA library.

Context: Most AI vendor integrations are tested only against the happy path. Prompt injection, tool-abuse, data-exfil via helpful behavior, and "feature silently enabled" scenarios are rarely exercised. L1 ST-Vendors installs the minimum repeatable test battery that turns these from theoretical threats into observed pass/fail outcomes.

Maturity Level 1

Objective: Run a foundational AI-vendor test battery at go-live and quarterly; operate program-level shadow-AI discovery tests at least quarterly; feed findings into IM.

Activities.

A) Publish the foundational AI-vendor test battery per archetype. Target ≤6 tests per archetype, each tied to a TA threat and an SR requirement. Common test classes: data egress test (synthetic canary payload, verify logging, vendor-side retention, DLP interception), no-train verification (admin-console state, DPA reference, behavioral probe where testable), prompt-injection resilience probe (curated set of injection strings, verify system prompts and tool permissions hold), permission-boundary / tool-scope test for agent archetype (attempt actions outside the allowlist, verify deny + log), logging completeness test (verify every required event type produces a log line that reaches the org-side store with correct attribution), kill-switch / rate-limit test (exercise the kill-switch path, verify behavior), AI-embedded SaaS toggle-drift test (re-check toggle state and users-with-access against approved scope). Each test specifies inputs, expected output, pass/fail criteria, and evidence artifact.

B) Run the battery at go-live and at least quarterly. Triggers: go-live (all applicable tests pass before production cutover), quarterly (all active integrations re-run the battery; failures route to IM), post-change (vendor major version bump, model-family change, admin-console redesign, org-plan migration), post-incident (any IM-Vendors incident re-runs the relevant subset before the incident is closed).

C) Operate shadow-AI discovery tests at program level. ST also tests the program's ability to detect unsanctioned AI use. Quarterly synthetic scenarios with sponsor approval and scope limits: a test account attempts to pay for an unsanctioned consumer GenAI subscription via expense pathways, a test endpoint downloads a known-AI-tool installer, a test SaaS admin toggles on an AI feature in a sandbox workspace, a test egress to a known AI vendor domain from an unmanaged path. Each scenario produces pass/fail plus time-to-detect; failures feed ML's detection backlog.

Outcome Metrics (L1).

Success Criteria.

• Foundational battery published per archetype and linked from DR/IR artifacts.
• 100% of AI vendor integrations reaching production in the last 90 days have a passed go-live battery.
• Shadow-AI discovery exercise run at least once in the last 90 days with results reviewed by the program sponsor.
• Named battery owner per archetype.
• Test failures routed to IM with 1-day handoff SLA.

Maturity Level 2

Objective: Dedicated AI-vendor red team for Critical tier quarterly; maintained regression corpus for jailbreaks and prompt-injection; bug-bounty integration where applicable.

Activities.

A) Quarterly red-team for Critical-tier integrations. Scope: prompt-injection chains, indirect-prompt-injection via RAG, agent tool abuse, jailbreak regression, data-egress canaries. Deliverable: red-team report with findings, reproducibility notes, remediation recommendations; routes to IM.

B) Maintained regression corpus. Versioned jailbreak / prompt-injection corpus (internal + external sources); expands monthly. Runs in CI against Critical/High integrations weekly. Pass/fail trend reported.

C) Bug-bounty integration. Participate in vendor-side bug bounties where AI-vendor programs exist. Run internal bug-bounty for custom AI integrations. Findings feed the program's test library and TA updates.

Outcome Metrics (L2).

Success Criteria.

• Quarterly Critical-tier red-team for 100% of integrations.
• Regression corpus running weekly in CI for ≥90% of Critical/High.
• ≥4 bug-bounty-sourced findings consumed per quarter.

Maturity Level 3

Objective: Continuous automated red-teaming for Critical-tier; publish anonymized findings to industry; host industry-shared red-team exercises.

Activities.

A) Continuous automated red-team. Automated probes (prompt-injection generators, jailbreak ladders, indirect-injection seeded content) run daily against Critical-tier. Findings triaged; new TTPs feed the TA library.

B) Contribute findings to industry. Anonymized, legally-vetted contributions to MITRE ATLAS, AI Vulnerability Database, OWASP LLM / Agentic Top 10. Target ≥4 contributions/year.

C) Industry-shared exercises. Host or co-host cross-org red-team benchmarks. Participate in ISAC AI-vendor tabletops.

Outcome Metrics (L3).

Success Criteria.

• ≥80% Critical under continuous automated red-team.
• ≥4 industry contributions/year.
• ≥1 hosted plus ≥2 participated industry exercises per year.

Common Pitfalls

Level 1. - Test battery published but not run at go-live, go-live tests treated as optional under release pressure. - Tests written without evidence-collection design, pass/fail recorded but the artifact behind it is missing. - Shadow-AI discovery exercises are theoretical, synthetic scenarios designed but never executed. - Test failures don't reach IM, the test owner closes the failure quietly.

Level 2. - Red-team becomes a vendor demonstration, the same scenarios run every quarter, no novel adversarial work. - Regression corpus grows but its CI runtime cost causes teams to skip it. - Bug-bounty findings consumed without attribution to the test library, the library never improves. - Critical-tier red-teaming runs without coordination with vendor security, unnecessary friction.

Level 3. - Continuous automated red-team produces noise that masks the high-signal findings. - Industry contributions are write-ups that don't include reproducible artifacts. - Hosted industry exercises drift to thought leadership without measurable cross-org outcomes. - Automated probes are tuned for the program's existing controls, missing the threats outside known patterns.

Practice Maturity Questions

Level 1. 1. Is a foundational test battery published per archetype (≤6 tests each) tied to TA threats and SR requirements, with a named battery owner per archetype, and is it run at go-live, quarterly, post-change, and post-incident? 2. Do 100% of AI vendor integrations reaching production in the last 90 days have a passed go-live battery, and do ≥90% of active integrations have a current-quarter test-battery pass? 3. Has a shadow-AI discovery exercise been run in the last 90 days (synthetic scenarios across expense, endpoint, SaaS-admin, and egress paths) with sponsor review of pass rate and time-to-detect, and are 100% of test failures converted to IM issues within one business day?

Level 2. 1. Are 100% of Critical-tier integrations red-teamed in the last 90 days (prompt-injection chains, indirect-injection via RAG, agent tool abuse, jailbreak regression, data-egress canaries), with reports routed to IM? 2. Is a versioned jailbreak / prompt-injection regression corpus maintained (≥1 update/month) and running weekly in CI against ≥90% of Critical/High integrations with pass/fail trend reported? 3. Are ≥4 bug-bounty-sourced findings consumed into the test library and TA updates per quarter, drawn from vendor-side and internal programs?

Level 3. 1. Are ≥80% of Critical-tier integrations under continuous automated red-team (daily probes via prompt-injection generators, jailbreak ladders, indirect-injection seeded content) with new TTPs feeding the TA library? 2. Does the program contribute ≥4 anonymized, legally-vetted artifacts per year to MITRE ATLAS, AI Vulnerability Database, or OWASP LLM/Agentic Top 10? 3. Does the program host ≥1 industry-shared red-team exercise and participate in ≥2 ISAC AI-vendor tabletops per year?

23. Environment Hardening (EH)

Practice Overview

Objective: Harden the organization's perimeter against AI-vendor data leakage and shadow AI, using controls already present in most enterprise stacks (SSO/IdP, DLP, browser management, egress control, endpoint management, SaaS admin governance) tuned specifically for AI vendor behavior.

Description: EH-Vendors does not harden the vendors, that is not possible. It hardens the paths to them. At L1 the organization tunes its existing egress, identity, and endpoint controls for AI-vendor-specific behavior: known-AI-vendor domain lists, DLP policies that understand prompt and content exfil patterns, browser policies that constrain consumer GenAI use, SSO enforcement for all approved AI SaaS, and SaaS admin-governance for AI-feature toggles inside approved parent vendors.

Context: Most controls needed are already deployed for other reasons. They have not been tuned for AI vendors, DLP rules don't see "prompt text," egress lists don't distinguish AI vendor domains from general SaaS, browser policies don't recognize "AI assistant" extensions. L1 closes that tuning gap with low-effort, high-leverage configuration changes.

Maturity Level 1

Objective: Tune SSO/IdP, DLP, browser, endpoint, and SaaS-admin controls to enforce AI-vendor policies and raise the cost of shadow AI.

Activities.

A) Identity and SSO tuning for AI vendors. Every sanctioned AI SaaS vendor behind SSO/SAML/OIDC; local auth disabled on admin tenants where possible. Org-tenant binding enforced for consumer GenAI enterprise plans. Service principals / machine identities for AI API use are scoped, owner-attributed, and rotated; shared static keys tracked and aged out. Conditional access rules for AI vendor SaaS match the risk tier (MFA always; device-trust where applicable).

B) DLP, browser, and endpoint tuning for AI vendor paths. Egress / DNS / proxy: known-AI-vendor domain list curated from the sanctioned catalog plus a prohibition list for prohibited consumer AI services; alerting on unexpected traffic to unclassified AI-vendor domains. DLP for AI paths: rules that recognize prompt-like bulk text pastes and file uploads into browser tabs of AI vendor domains; block or warn based on data class. Browser management: managed-browser policy limits AI-extension installation; separate-profile model for work/personal; tab-level data-loss alerts on AI vendor domains. Endpoint inventory: MDM/EDR watches for known AI desktop apps and browser extensions; alerts on install. Developer-endpoint specifics: AI coding assistant client policy enforced.

C) SaaS admin governance for AI-embedded features. A published "AI feature registry" per approved parent vendor, which AI features exist, which are on, who has access. Default posture: AI features off org-wide until the PC intake has covered them; turned on via change-managed action. Admin-audit feeds from parent SaaS vendors captured to detect silent new AI features. Quarterly review of every parent vendor's AI-feature panel by a named admin-governance owner.

Outcome Metrics (L1).

Success Criteria.

• 100% of sanctioned AI SaaS behind SSO; prohibited consumer AI services blocked at egress.
• AI-aware DLP rules deployed and actively tuned.
• Managed-browser and endpoint AI-tool policies active on ≥95% of managed endpoints.
• AI-feature registry published for every approved parent SaaS vendor with active AI features.
• Named admin-governance owner operating quarterly reviews.

Maturity Level 2

Objective: Extend hardening with CASB/SSPM tuned for AI vendors, SASE-level egress governance, and per-tenant isolation in multi-tenant AI vendor deployments.

Activities.

A) CASB/SSPM enforcement tuned for AI vendors. CASB platforms (Netskope, Defender for Cloud Apps, Zscaler, Cisco Umbrella) policies tuned for AI-vendor behaviors: bulk-paste detection, file-upload controls, tool-invocation detection where visible. SSPM platforms (Obsidian, Adaptive Shield, AppOmni) focused on parent-SaaS AI-feature config drift and admin-console changes.

B) SASE egress governance. Per-user, per-app, per-device rules for AI-vendor domains. Policy-based access requiring device-trust and current session for Critical-tier AI SaaS. AI-vendor-specific inspection where permissible.

C) Per-tenant isolation for multi-tenant AI vendors. Per-tenant key scope for AI APIs at vendor boundaries. Per-tenant data scope in shared AI-vendor environments. Admin-governance model distinguishes tenant-scoped vs. org-wide settings.

Outcome Metrics (L2).

Success Criteria.

• AI-vendor-tuned CASB/SSPM policies live with named owner.
• 100% Critical-tier AI SaaS under SASE per-user policy.
• ≥90% multi-tenant AI-vendor deployments with per-tenant scope.

Maturity Level 3

Objective: Adaptive policy from ML detections and IM incidents; hardening-as-code; contribute AI-vendor hardening baselines to industry.

Activities.

A) Adaptive policy. ML detections plus IM incidents generate policy-tightening proposals; human-approved before deploy. Change-log machine-readable; downstream teams notified.

B) Hardening-as-code. All EH controls (CASB policies, DLP rules, SASE rules, SSO policies, MDM/EDR config for AI tooling) expressed as IaC (Terraform, OPA, Rego, Kyverno). Drift detected continuously; auto-remediation for low-risk drift.

C) Contribute AI-vendor hardening baselines. Contribute baselines to CIS (AI workloads), CSA AI Safety Initiative, sector ISACs. Maintained upstream; internal use aligns with external version.

Outcome Metrics (L3).

Success Criteria.

• ≥90% EH controls as IaC.
• Adaptive-policy pipeline operating.
• ≥2 industry baseline contributions per year.

Common Pitfalls

Level 1. - Domain list curated once and never refreshed, new AI vendors quietly miss the list. - DLP rules are too aggressive, generate false positives, and get disabled by IT without informing the program. - Browser management policies don't distinguish AI-vendor tabs from general SaaS, controls are coarse. - Parent-SaaS feature reviews calendared but the admin-audit feed is never read.

Level 2. - CASB policies tuned for AI vendors but only against known categories, new categories miss the policy until next refresh. - SASE per-user rules deployed but device-trust integration is incomplete, leaving an unscoped path. - Per-tenant isolation works for new deployments but legacy multi-tenant ones are grandfathered. - False-positive rates trend down on paper but support tickets rise, users route around the controls.

Level 3. - Adaptive policy generates proposals nobody reviews, automation accumulates, the human queue grows stale. - IaC encoding is partial, half the controls are IaC, the other half are manual changes that drift. - Industry baselines are published but treated as marketing; the maintained version internal use diverges. - Auto-remediation closes drift tickets without recording the root cause, the program loses the learning.

Practice Maturity Questions

Level 1. 1. Are 100% of sanctioned AI SaaS vendors behind SSO/SAML/OIDC, with org-tenant binding for consumer GenAI enterprise plans, scoped service principals for AI API use, and explicit egress block rules for prohibited consumer AI services? 2. Are AI-aware DLP rules deployed and tuned (recognizing prompt-like bulk text pastes and file uploads into AI vendor domains), with managed-browser and endpoint AI-tool policies on ≥95% of managed endpoints? 3. Is an "AI feature registry" published for 100% of approved parent SaaS vendors with active AI features, with default-off posture, admin-audit feeds captured, and a named admin-governance owner running quarterly reviews?

Level 2. 1. Are AI-vendor-tuned CASB/SSPM policies live (Netskope/Defender/Zscaler/Umbrella for CASB; Obsidian/Adaptive Shield/AppOmni for SSPM) with a named owner and AI-specific signal (bulk-paste detection, file-upload controls, tool-invocation detection)? 2. Are 100% of Critical-tier AI SaaS under SASE per-user policy with device-trust requirement, and is the false-positive rate on AI-specific DLP/CASB signals actively tuned and trending down? 3. Do ≥90% of multi-tenant AI-vendor deployments enforce per-tenant key and data scope, with the admin-governance model distinguishing tenant-scoped vs. org-wide settings?

Level 3. 1. Are ≥90% of EH controls expressed as Infrastructure-as-Code (Terraform, OPA, Rego, Kyverno) with continuous drift detection and ≥70% auto-remediation rate for low-risk drift? 2. Is an adaptive-policy pipeline operating that generates tightening proposals from ML detections and IM incidents, with human approval before deploy and a machine-readable change log? 3. Does the program contribute ≥2 AI-vendor hardening baselines per year to CIS (AI workloads), CSA AI Safety Initiative, or sector ISACs, with maintained upstream parity?

24. Issue Management (IM)

Practice Overview

Objective: Run a single backlog and a single incident playbook for AI-vendor issues, so that findings from TA snapshots, SR requirement gaps, DR conditions, IR drifts, ST failures, and ML detections all flow into one prioritized queue with named owners, SLAs, and a clear path to vendor-breach notification.

Description: IM-Vendors is the clearinghouse for everything the other Vendors practices produce. At L1 it maintains one issue backlog tagged by source, one triage rubric, one incident playbook specifically for AI-vendor events, and the vendor-breach-notification SLA tracker tied to contractual commitments.

Context: Without a unified backlog, AI-vendor issues scatter across project trackers, engineer dashboards, legal trackers, and TPRM folders, so nothing ages correctly, nothing gets prioritized against anything else, and a vendor breach notification can sit while individual teams wait on each other. L1 IM-Vendors centralizes the queue and standardizes the playbook.

Maturity Level 1

Objective: Operate a single AI-vendor issue backlog with a standard triage rubric, an AI-vendor incident playbook, and vendor-breach-notification tracking.

Activities.

A) Stand up the AI-vendor issue backlog and triage rubric. One backlog with standardized metadata: source (TA / SR / DR / IR / ST / ML / External), affected vendor and integration (linked to inventory), archetype, severity (blocker / critical / high / medium / low, anchored to data-class exposure × regulated posture × number of users × whether vendor breach SLA is active), owner, SLA (severity-based: blocker immediate; critical ≤72h to containment / ≤14d closure; high ≤14d / ≤45d; medium ≤45d; low ≤90d), and evidence link. Triage cadence: daily for new critical/blocker, weekly for high/medium, monthly for full backlog aging review.

B) Publish the AI-vendor incident playbook. Common incident classes: vendor breach notification received (confirm scope, classify affected data, trigger GDPR Article 33 / HIPAA / contract SLAs, coordinate Legal/Privacy, notify affected users, log for deployer-duty evidence), vendor outage or degraded service (fallback or kill-switch activation, user communication, post-event SA pattern review), prompt-injection or output-integrity incident (containment via feature toggle or tool-scope shrink, scope assessment, customer-communication decision, ML detection tuning), shadow-AI data-exposure incident (containment, amnesty-path reinforcement, policy-violation routing, deployer-duty evidence capture), agent runaway / tool-abuse incident (kill-switch, scope-reduction, session-log capture, human-in-the-loop reinforcement), vendor material change (model-family swap, plan change, subprocessor addition, triggers re-review in DR/IR, REM update). Each entry: trigger, named roles, step-by-step, artifacts to collect, closure criteria, SLA targets.

C) Track vendor-breach-notification SLAs and run post-incident reviews. Breach-SLA tracker: contractual SLA per vendor (from DPA/AI addendum) + regulatory SLAs (GDPR Article 33 72-hour, HIPAA 60-day, sector-specific). Every critical/blocker incident gets a post-incident review within 14 days. Outputs feed: SA pattern update queue, SR pack update queue, EG training-content queue, ML detection-backlog, shadow-AI threat doc refresh.

Outcome Metrics (L1).

Success Criteria.

• Single AI-vendor issue backlog established with standardized metadata.
• AI-vendor incident playbook published with at least 5 named incident classes and assigned roles.
• Vendor-breach-notification SLA tracker live; 100% adherence in last 90 days.
• Post-incident review loop wired to SA, SR, EG, and ML.
• Program-sponsor dashboard showing backlog aging, SLA adherence, and post-incident learnings refreshed monthly.

Maturity Level 2

Objective: Tier-calibrated incident response, formal vendor-coordination playbook for Critical-tier, and supply-chain-style orchestration when an AI vendor breach affects multiple org integrations.

Activities.

A) Tiered incident playbook. Critical: full IM team plus Legal, Privacy, Communications, Executive Sponsor; ≤1h acknowledgement; ≤4h containment target. High: scoped response team; ≤4h acknowledgement; ≤24h containment. Medium: standard response; ≤1 business day. Low: tracked and trended; aggregated handling.

B) Vendor-coordination playbook. Pre-established communication channels with Critical-tier vendor security/incident teams. Template communications, NDA, joint-IR coordination protocol. Annual joint tabletop with top-5 Critical vendors.

C) Supply-chain-style orchestration. When a single vendor breach affects multiple org integrations, an Incident Commander coordinates across integration owners. Shared status board, shared communications, shared remediation tracking. Post-incident review spans all affected integrations.

Outcome Metrics (L2).

Success Criteria.

• Critical MTTA ≤1h; MTTC ≤4h.
• ≥90% Critical vendors with coordination channel.
• ≥1 joint tabletop per year with top-5 Critical.

Maturity Level 3

Objective: Industry-coordinated incident response; contribute to AI-vendor incident taxonomy and response playbooks; automated SLA enforcement.

Activities.

A) Industry-coordinated response. Contribute and consume vendor-incident intelligence via sector ISACs (FS-ISAC, H-ISAC, IT-ISAC). Participate in ISAC AI-vendor incident exercises.

B) Contribute to AI-vendor incident taxonomy. Classification schemes, severity anchors, response playbook templates contributed to standards (CSA, Shared Assessments, OpenSSF AI).

C) Automated SLA enforcement. Vendor-side SLA adherence (notification-time, RCA-delivery, remediation-time) monitored from incident records. Patterns of breach trigger PC contract review or non-renewal recommendation automatically.

Outcome Metrics (L3).

Success Criteria.

• ≥4 ISAC contributions/year.
• ≥1 industry-taxonomy contribution/year.
• 100% Critical vendors under automated SLA tracking.

Common Pitfalls

Level 1. - Multiple backlogs persist, Security keeps its own, TPRM keeps its own, engineering keeps its own. - Severity rubric is subjective, same finding gets graded differently by different reviewers. - Vendor-breach SLAs tracked but not exercised, the first breach is the first time the playbook runs. - Post-incident reviews happen but the outputs never reach SA, SR, EG, or ML.

Level 2. - Tiered playbook exists but Critical and High collapse to the same response in practice. - Vendor-coordination channels established but personnel turn over and channels go stale. - Supply-chain orchestration runs only the first time it is needed; the second multi-integration breach reverts to ad-hoc. - Joint tabletops happen but findings are not converted into playbook updates.

Level 3. - ISAC contributions become bulletin-style summaries rather than reusable artifacts. - Industry-taxonomy contributions reflect a previous version of the program's internal taxonomy. - Automated SLA enforcement triggers contract action that contradicts a strategic relationship, without escalation guard. - Automated tracking generates noise, frequent SLA violations on Low/Medium vendors crowd out the Critical view.

Practice Maturity Questions

Level 1. 1. Is a single AI-vendor issue backlog operating with standardized metadata (source, affected vendor/integration, archetype, severity, owner, SLA, evidence) and ≥95% of AI-vendor issues recorded in it (vs. scattered across other trackers)? 2. Is an AI-vendor incident playbook published with at least five named incident classes (vendor breach notification, vendor outage, prompt-injection/output-integrity, shadow-AI exposure, agent runaway, vendor material change) and assigned roles per class? 3. Is the vendor-breach-notification SLA tracker live (contractual + GDPR Art. 33 + HIPAA + sector-specific) with 100% adherence in last 90 days, and is the post-incident review loop wired to SA, SR, EG, and ML?

Level 2. 1. Is the incident playbook tier-calibrated (Critical: ≤1h MTTA, ≤4h MTTC, full team + Legal + Privacy + Communications + Sponsor; High: ≤4h MTTA, ≤24h MTTC; Medium and Low standardized) with measured per-tier adherence? 2. Are pre-established vendor-coordination channels in place for ≥90% of Critical-tier vendors (template comms, NDA, joint-IR protocol), with at least one joint tabletop per year against the top-5 Critical? 3. When a single vendor breach affects multiple org integrations, is supply-chain-style orchestration used 100% of the time (one Incident Commander, shared status board, shared remediation tracking, cross-integration post-incident review)?

Level 3. 1. Does the program contribute ≥4 vendor-incident intelligence artifacts per year via sector ISACs (FS-ISAC, H-ISAC, IT-ISAC) and ≥1 substantive incident-taxonomy contribution per year (classification schemes, severity anchors, response playbook templates) to standards bodies? 2. Are 100% of Critical vendors under automated SLA tracking (notification-time, RCA-delivery, remediation-time) with patterns of breach surfacing for PC contract review or non-renewal recommendation? 3. Is the SLA-breach-to-contract-action lead time automated and ≤5 business days, with appropriate escalation guards to avoid contradicting a strategic vendor relationship?

25. Monitoring & Logging (ML)

Practice Overview

Objective: Establish the logging baseline per AI/HAI vendor archetype and operate a small high-signal detection set so AI-vendor activity produces evidence and detectable signals on demand, including the deployer-duty evidence regulators may request.

Description: ML-Vendors defines what gets logged for each archetype, where it goes, how long it is retained, and how it can be exported. It operates a small set of high-signal detections targeted at the top threats from TA, including shadow-AI egress, consumer AI personal-account sign-in, bulk paste/upload, API-proxy anomalies, agent tool-call violations, and parent-SaaS AI-feature toggle changes. It produces the "deployer-duty evidence view" for high-risk AI vendor integrations.

Context: Without an AI-specific logging baseline, classic SIEMs receive vendor logs but nothing is tagged to make AI activity searchable. Without high-signal detections, the program runs blind against the threats it most needs to see. L1 closes both gaps without buying new tools.

Maturity Level 1

Objective: Establish the AI-vendor logging baseline per archetype, operate a small high-signal detection set including shadow-AI detections, and ensure evidence retention meets deployer-duty requirements.

Activities.

A) Establish the AI-vendor logging baseline per archetype. Per-archetype event minimums: consumer GenAI (sign-in event, conversation-creation event, bulk paste/upload event, admin-audit event, retention setting change), AI-embedded SaaS (feature-on/off event, per-user invocation, admin-audit event for feature toggles), AI coding assistant (sign-in, regulated-repo guard, telemetry consent event), AI API/foundation-model (prompt request, completion response, model-version field, error rate, kill-switch invocation), AI agent platform (session start/end, tool-call events covering tool, parameters, outcome, HITL gate invocations, permission denials). Baseline retention meets or exceeds the longest applicable requirement (EU AI Act high-risk logs, GDPR records-of-processing) per data class. Export path tested at least annually.

B) Operate a small high-signal detection set. L1 target ≤12 detections, each tied to a TA archetype threat with owner, query, SLA, last-tuned date. Core detections: shadow-AI egress (traffic to unsanctioned AI vendor domains), shadow-AI SaaS sign-in (SSO/IdP activity against unclassified AI SaaS), consumer AI personal-account sign-in from org endpoints, bulk content paste/upload to AI vendor domains, API-proxy anomalies (volume spikes, model-version change, PII scrubbing failures), agent tool-call violations (calls outside allowlist), agent HITL bypass attempt, parent-SaaS AI-feature toggle change, admin-key anomaly, no-train setting change, AI-vendor breach/advisory match against inventory, egress to new AI vendor domain first-seen. Each: false-positive rate tracked, monthly tuning review.

C) Prove the evidence trail for deployer duties and processor obligations. A single "deployer-duty evidence view" per high-risk AI vendor integration pulling intake approval, REM, DR decision, IR config records, ST pass records, ML logs, incident records, AUP coverage for associated users. GDPR Article 30 records-of-processing entries reference ML retention evidence. ISO/IEC 42001 AIMS evidence assets linked or identified as gaps. At least one quarterly drill: pull deployer-duty evidence for a random high-risk vendor within 2 business days.

Outcome Metrics (L1).

Success Criteria.

• Per-archetype logging baseline published and operated.
• ≤12-detection high-signal set live, each with owner and tuning record.
• Deployer-duty evidence view produced for every high-risk AI vendor integration.
• Quarterly deployer-duty drill executed and inside target SLA.
• Retention and export path tested at least annually.

Maturity Level 2

Objective: Add anomaly detection on AI-vendor behavior, correlate across vendors, and automate deployer-duty evidence generation.

Activities.

A) Anomaly detection on AI-vendor behavior. Baseline normal per-integration behavior: prompt volume, tool-call patterns, egress volume, time-of-day patterns. Detect anomalies, tune for false-positive rate, escalate to IM with context (baseline vs. observed).

B) Cross-vendor correlation. Multi-vendor integration chains mapped as graph; graph anomalies (unexpected edges, changing centralities, new intermediate vendors). Shadow-AI graph signals: unexpected AI-vendor egress from previously non-AI endpoints.

C) Automated deployer-duty evidence. For Critical-tier integrations, evidence view (logs, human-oversight assignments, disclosures, Article 26 checklist attestations) auto-assembles on schedule. Regulator inquiry turnaround SLA ≤3 business days.

Outcome Metrics (L2).

Success Criteria.

• ≥90% Critical with anomaly baselines.
• Cross-vendor graph active.
• Automated deployer-duty evidence for 100% Critical.
• Regulator inquiry ≤3 business days.

Maturity Level 3

Objective: Real-time AI-vendor attestation; contribute to industry AI-vendor telemetry standards; share anonymized detection signatures.

Activities.

A) Real-time attestation. Live-queryable evidence view for Critical integrations; cryptographic signing where applicable.

B) Contribute telemetry standards. OpenTelemetry AI workgroup, CSA AI Safety Initiative, OpenSSF AI telemetry efforts: schemas, semantic conventions, required fields.

C) Shared detection signatures. Anonymized detection signatures contributed to ISAC AI-vendor feeds and OpenSSF AI; target ≥12 signatures/year.

Outcome Metrics (L3).

Success Criteria.

• ≥90% Critical with live attestation.
• ≥2 telemetry-standard contributions per year.
• ≥12 shared detection signatures per year.

Common Pitfalls

Level 1. - Logging baseline published but not enforced, integrations skip logging setup. - Detections generate noise, false positives crowd out signal, teams ignore alerts. - Evidence view is manual, pulling deployer-duty evidence takes days instead of hours. - Retention policy unclear, logs deleted before audit period ends.

Level 2. - Anomaly baselines are brittle, legitimate spikes interpreted as anomalies. - Graph analysis runs but nobody acts on results, cross-vendor risks unmeasured. - Evidence generation is one-way, automation produces artifacts but does not validate. - Regulator inquiry process not streamlined, even with automation, turnaround misses SLA.

Level 3. - Real-time attestation is stale data, lag between actual state and queryable view. - Telemetry-standard contributions are academic, not adopted by OpenTelemetry or CSA. - Shared detection signatures lack context, other organizations cannot integrate them without explanation. - Cryptographic signing is implemented but the key-management plan does not match the retention period.

Practice Maturity Questions

Level 1. 1. Are per-archetype logging baselines established and operated for ≥90% of active AI vendor integrations (consumer GenAI, AI-embedded SaaS, AI coding assistant, AI API/model, AI agent platform), with retention meeting the longest applicable requirement (EU AI Act high-risk logs, GDPR records-of-processing) and export paths tested at least annually? 2. Is a high-signal detection set (≤12 detections) live with named owners, queries, SLAs, and last-tuned dates, covering shadow-AI egress, shadow-AI SaaS sign-in, consumer AI personal accounts, bulk paste/upload, API-proxy anomalies, agent tool-call violations, HITL bypass, parent-SaaS feature toggles, admin-key anomalies, no-train settings, vendor breaches, and new AI vendor domains? 3. Is a deployer-duty evidence view produced for every high-risk AI vendor integration, and has a quarterly drill completed inside ≤2 business days SLA in the last 90 days?

Level 2. 1. Are ≥90% of Critical integrations under anomaly detection (baseline per-integration normal behavior, prompt volume, tool-call patterns, egress volume, time-of-day; deviations escalated to IM with context), with FP rate actively tuned and trending down? 2. Is multi-vendor cross-correlation operational (graph of vendor chains refreshed weekly, detecting unexpected edges, changing centralities, new intermediaries, shadow-AI egress from non-AI endpoints)? 3. Do 100% of Critical integrations have automated deployer-duty evidence generation, with regulator-inquiry turnaround ≤3 business days?

Level 3. 1. Are ≥90% of Critical integrations under live-queryable evidence attestation with cryptographic signing where applicable? 2. Does the program contribute ≥2 telemetry-standard artifacts per year to OpenTelemetry AI workgroup, CSA AI Safety Initiative, or OpenSSF AI (schemas, semantic conventions, required fields)? 3. Does the program contribute ≥12 anonymized detection signatures per year to ISAC AI-vendor feeds or OpenSSF AI with documented adoption context?

Part IV, Maturity Assessment Workbook

This part is the assessment instrument for the Vendors domain. The 108 questions here are the same ones that close each practice section in Part III. Use this part to score the program; use Part III to understand what each question is testing.

The workbook is self-contained. An assessor can perform a full Vendors-domain assessment by working through Part IV alone, referring back to Part III only as needed for clarification.

26. How the assessment works

Scope. A single assessment covers all 12 practices in the Vendors domain, with 3 questions per maturity level per practice, 108 questions total.

Cumulative levels. Maturity levels are cumulative. A practice cannot be at Level 2 unless it is at Level 1; it cannot be at Level 3 unless it is at Level 2. Score Level 1 questions before answering Level 2 questions for the same practice.

Answers. Each question accepts one of three answers:

• Yes, fully implemented, evidence-backed, sustained over time. A "Yes" requires evidence (an artifact, telemetry pull, or process record) the assessor has actually seen.
• Partial, partially implemented, or implemented but not sustained, or evidence is incomplete. A "Partial" is recorded as half credit in scoring.
• No, not implemented, or implemented inconsistently to the point that no evidence supports it.

Evidence. Every "Yes" requires a citation in the Evidence box. "It's documented in the wiki" is not evidence. "Wiki page X dated Y, screenshot of admin console Z" is.

Honesty. The assessment is for the program, not for the assessor. A "No" honestly recorded is more useful than a "Yes" that does not survive auditor scrutiny.

Cadence. Run the full assessment at least annually. Run a Level 1 self-check quarterly during the first year of program operation.

Roles. The assessor should be independent of day-to-day program operations. A peer assessor from another team or function works well; an external assessor works better. The program lead should not assess their own program.

27. Scoring methodology

Two scoring approaches are supported. Use the simplified scoring for self-assessments and quarterly check-ins. Use the precise scoring for formal audits and external benchmarking.

Simplified scoring (recommended for self-assessment)

For each practice:

Level 1 achieved (all 3 Level 1 questions = Yes): 1.0 point
Level 2 achieved (all 3 Level 2 questions = Yes, AND Level 1 achieved): +1.0 (total 2.0)
Level 3 achieved (all 3 Level 3 questions = Yes, AND Level 2 achieved): +1.0 (total 3.0)

A "Partial" answer counts as half toward the level, but the level is only achieved when all three questions are at full Yes. Partial credit shows up in the precise score.

Precise scoring (recommended for formal audits)

For each practice, with Y = Yes (1.0), P = Partial (0.5), N = No (0):

L1_score = (sum of L1 answers) / 3
L2_score = (sum of L2 answers) / 3 × L1_score
L3_score = (sum of L3 answers) / 3 × L2_score

Practice Score = L1_score + L2_score + L3_score    (max 3.0)

The L2 and L3 multipliers enforce the cumulative rule, a practice cannot earn full L2 credit if L1 is incomplete.

Domain rollup.

Domain Maturity = (sum of all 12 Practice Scores) / 12    (max 3.0)

Maturity bands.

• 0.0 – 0.9, Ad-hoc. No Vendor AI Assurance program in operational use.
• 1.0 – 1.9, Foundational. L1 in place across most practices; some L2 progress.
• 2.0 – 2.9, Comprehensive. L2 calibrated by tier across most practices; some L3 contributions.
• 3.0, Industry-Leading. L3 automation, benchmarking, and contribution sustained across all practices.

28. The questionnaire

The 108 questions follow. Each question has the same workbook layout: question text, answer field, evidence box, and notes box. Practice and level headings are repeated so the workbook is usable as a printout.

28.1 Strategy & Metrics (SM)

SM Level 1.

Q-SM-L1-1. Is there a published Vendor AI Assurance program charter with a named executive sponsor and a cross-functional working group (Security, Procurement, Legal/Privacy, IT, Data Governance, business representative)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SM-L1-2. Does a single AI vendor inventory exist, seeded from expense, SSO, egress, endpoint, and SaaS-admin signals, covering consumer GenAI, AI-embedded SaaS, AI coding assistants, AI APIs/models, and AI agent platforms, with ≥90% coverage of discovered AI vendors?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SM-L1-3. Are the L1 outcome metrics baselined and reported quarterly to the sponsor, inventory coverage, shadow AI ratio, AUP attestation (≥95%), and known data-exposure events to unsanctioned AI tools?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

SM Level 2.

Q-SM-L2-1. Is every AI vendor in the inventory assigned a risk tier based on an auditable rubric covering data sensitivity, decision-affecting use, agentic capability, user exposure, regulatory scope, and concentration?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SM-L2-2. Is there a published tier-treatment matrix driving differential intensity across PC, TA, DR, IR, ST, and ML, with ≥95% of Critical-tier vendors receiving full-scope treatment in the last 12 months?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SM-L2-3. Does the quarterly shadow AI scoreboard report per tier (with Critical-tier unsanctioned AI explicitly tracked at zero), and does tier-movement get logged and reviewed by the program sponsor?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

SM Level 3.

Q-SM-L3-1. Does inventory and tier assignment auto-update from live signals (expense, SSO, egress, SaaS admin, endpoint, intake, self-attestation) with a published data-quality SLO, and is ≥80% of curation handled automatically with exception-based human review?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SM-L3-2. Does the program publish a semi-annual external-benchmarking brief comparing itself against at least five peer-comparable metrics via ISACs, standards bodies, or industry roundtables, and does it drive program investment decisions?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SM-L3-3. Does the program contribute at least four substantive, anonymized artifacts per year to the AI-vendor assurance ecosystem (MITRE ATLAS, OWASP LLM/Agentic, NIST AI RMF, AI Vulnerability Database, sector ISACs), and does the executive ROI narrative cite external benchmarks?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.2 Policy & Compliance (PC)

PC Level 1.

Q-PC-L1-1. Have you published and formally approved the three priority AI vendor policies (AI Acceptable Use, AI Procurement & Intake, AI Vendor Data-Sharing) with AI-specific clauses (prohibited data classes, personal-account prohibition, AI-feature-inside-SaaS intake requirement, training-on-data default-off, DPA/AI addendum requirement)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-PC-L1-2. Is there a one-page priority compliance map that ties each priority requirement (EU AI Act Art. 26/50, NIST AI RMF GOVERN, GDPR Art. 28/22/44–49, SOC 2 CC9.2, ISO/IEC 42001, ISO/IEC 27001 A.5.19–A.5.23, plus any sector-specific) to the specific L1 policy that carries the control?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-PC-L1-3. Is the AI vendor intake gate operational with a published SLA, an amnesty path for previously undisclosed use, ≥95% AUP attestation, and 100% DPA/AI-addendum coverage for AI vendors handling non-public data?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

PC Level 2.

Q-PC-L2-1. Have the three priority AI vendor policies been extended with tier-specific addenda, and do Critical/High AI vendors carry tier-appropriate contractual controls (audit rights, model-version change notice, subprocessor add-notice, incident-notification SLA, training-data provenance attestation)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-PC-L2-2. Does every Critical/High AI vendor have a live, continuously-assembled compliance evidence view covering DPA / AI addendum, subprocessor list, training-data posture, SOC 2/ISO evidence, model-version log, incident history, and deployer-duty evidence, with staleness inside tier-specific targets?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-PC-L2-3. Is an exception register operated with named owners, compensating controls, and expiry dates, reviewed monthly, with unsanctioned AI in Critical-tier use cases subject to blocking enforcement and sector-specific evidence bundles complete for in-scope vendors?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

PC Level 3.

Q-PC-L3-1. Can on-demand, regulator-grade evidence packs for any active AI vendor be generated inside 3 business days, with vendor-side changes auto-refreshing the evidence view and full provenance traceable?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-PC-L3-2. Does the program operate a quarterly, telemetry-driven policy-refresh cycle (ML detection trends, IM incident learnings, tier-movement, external regulatory updates) with a versioned changelog, and are the changes reflected in EG training within 30 days?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-PC-L3-3. Does the program contribute at least two substantive public comments or standards artifacts per year on AI-vendor topics (EU AI Act deployer guidance, GDPR EDPB AI guidance, NIST AI RMF, ISO/IEC 42001, sector regulators, or community standards bodies), with documented external recognition?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.3 Education & Guidance (EG)

EG Level 1.

Q-EG-L1-1. Have all procurement, legal, vendor management, and development teams received foundational training on vendor security and supply chain risks?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EG-L1-2. Are awareness campaigns actively communicating vendor security risks, supply chain threats, and real-world vendor breach impacts?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EG-L1-3. Is basic vendor security guidance available (assessment procedures, security questionnaires, contract security clauses)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

EG Level 2.

Q-EG-L2-1. Is there a scenario library of anonymized real AI-vendor intakes powering reviewer training, with quarterly calibration exercises that show Critical-tier drift inside target?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EG-L2-2. Have you delivered a product/engineering team training track, covering deployer duties (EU AI Act Art. 26/50), output integrity, and SA reference-pattern adherence, to at least one member of every Critical/High AI-vendor integration team?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EG-L2-3. Are shadow AI awareness campaigns running on a behavior-driven cadence with pre-measured targets, and is training content refreshed at least quarterly from program telemetry?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

EG Level 3.

Q-EG-L3-1. Has the curriculum, scenario library, and reviewer rubric been published externally (CSA, OpenSSF AI, Shared Assessments, sector ISAC) with documented adoption, citations, or contributions back?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EG-L3-2. Is a continuous live-calibration cadence operating (monthly anonymized live-intake exercise) with reviewer drift tracked as a development signal, and do ≥50% of Critical-tier reviewers hold an external AI-vendor reviewer credential (where one exists)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EG-L3-3. Does the program contribute at least two substantive artifacts per year to industry AI-vendor reviewer certification or curriculum working groups, with a traceable loop back into internal content?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.4 Threat Assessment (TA)

TA Level 1.

Q-TA-L1-1. Have you documented threat scenarios specific to the five AI vendor archetypes (consumer GenAI, AI-embedded SaaS, AI coding assistant, AI API/model, AI agent platform) tagged to HAI TTPs (EA, AGH, TM, RA) and linked to the priority compliance map?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-TA-L1-2. Is a per-intake threat snapshot wired into the intake gate (≤30 minutes per intake, ≥90% inventory coverage), and is the shadow AI threat surface documented as a standalone artifact reviewed by the program sponsor?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-TA-L1-3. Is there a named library steward and a quarterly refresh cadence, with downstream practices (SR, SA, ST, IM, ML) consuming the library rather than re-deriving threats?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

TA Level 2.

Q-TA-L2-1. Does every Critical AI vendor have a current-year per-vendor deep threat model (not only an archetype snapshot), and ≥90% of High AI vendors the same?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-TA-L2-2. Is external AI-vendor threat intelligence (MITRE ATLAS, AI Vulnerability Database, OWASP LLM/Agentic Top 10, sector ISACs, vendor advisories) integrated with a quarterly triage cadence and a documented change-log?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-TA-L2-3. Do you run a quarterly red-team-the-library exercise, track and close library gaps with named owners and expiries, and keep intel-to-library lead time ≤30 days on Critical-impact items?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

TA Level 3.

Q-TA-L3-1. Does the threat library auto-update from telemetry (ML detections, IM incidents, external feeds) with human curation, and is the change lead time from signal to library update ≤14 days?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-TA-L3-2. Does the program contribute at least four substantive AI-vendor threat artifacts per year to MITRE ATLAS / AIVD / OWASP cycles, with at least two externally recognized?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-TA-L3-3. Are anonymized archetype threat models published under permissive license with documented peer-org adoption, and does the program host or co-host industry tabletops tied to the library?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.5 Security Requirements (SR)

SR Level 1.

Q-SR-L1-1. Have you published the AI Vendor Requirements Pack (base ≤20 requirements + five archetype deltas) with each requirement tagged to a TA archetype threat and a priority-compliance item, and is a named owner running quarterly refresh?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SR-L1-2. Is the pack wired into the intake gate, with 100% of new AI vendor approvals in the last 90 days carrying a Requirements-Evidence Map (REM) and ≥90% of active vendors carrying a current-year REM?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SR-L1-3. Are accepted gaps tracked with named owner, compensating control, and expiry date, and is the median accepted-gap age ≤90 days?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

SR Level 2.

Q-SR-L2-1. Are 100% of pack requirements expressed as quantitative or binary testable conditions (SLA hours, retention days, availability percent), with tier-specific deltas operationalized (Critical = red-team right, FRIA support, Art. 26 full checklist)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SR-L2-2. Are ≥95% of Critical REMs re-validated against observed reality (admin console, vendor API, IR findings, ML logs) in the last 90 days, with deltas routed to IM as findings?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SR-L2-3. Is the accepted-gap backlog inside the aging target by tier (Critical ≤60 days median), and does every Critical AI vendor carry full EU AI Act Article 26 checklist evidence?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

SR Level 3.

Q-SR-L3-1. Are the requirements pack and REM schema published under a permissive license with tracked external adoption, and does the program contribute ≥2 industry-standard contributions per year (Shared Assessments AI-vendor track, CSA, IAPP)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SR-L3-2. Are ≥70% of REM rows auto-validated via vendor API ingestion (admin-console state, subprocessor lists, SOC 2 bridge letters) and internal telemetry corroboration, with human review reserved for exceptions?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SR-L3-3. Is the vendor redline success rate ≥80% on program-contributed clauses, and does the program participate in EDPB AI, EU AI Act deployer practice, and NIST Playbook regulatory comment cycles?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.6 Secure Architecture (SA)

SA Level 1.

Q-SA-L1-1. Are five reference architectures published (one per archetype: consumer GenAI, AI-embedded SaaS, AI coding assistant, AI API/foundation-model, AI agent platform) with each mapped to SR requirements and TA threats?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SA-L1-2. Is an anti-patterns catalog published, linked from the AUP and intake gate, referenced in EG training, with each anti-pattern carrying a description, why-dangerous, real-incident flavor, and the replacement pattern?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SA-L1-3. Are ≥85% of active AI vendor integrations classified as "on pattern" or "deviation with review," and are 100% of API/model archetype integrations with non-public data flowing through the internal API proxy?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

SA Level 2.

Q-SA-L2-1. Are extended patterns published for multi-region, multi-tenant, and agent-platform complexity, and is the anti-pattern catalog updated from at least three IM incidents in the last 12 months?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SA-L2-2. Are ≥80% of Critical/High integrations using IaC-encoded patterns (Terraform, Kubernetes manifests, Crossplane, Pulumi), with pattern-drift detection covering 100% of IaC-encoded integrations?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SA-L2-3. Are pattern updates traceable to a specific incident, threat, or requirement change, and is the change log refreshed monthly?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

SA Level 3.

Q-SA-L3-1. Are at least two reference patterns externally adopted (CNCF AI SIG, OpenSSF AI, CSA AI Safety Initiative, sector bodies), with maintained upstream parity to internal versions?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SA-L3-2. Are ≥90% of AI-vendor human sessions under continuous zero-trust verification (device trust, session evaluation), and are ≥70% of machine calls under JIT-scoped credentials?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-SA-L3-3. Do 100% of Critical-tier agent integrations carry runtime-enforced formal tool-scope specifications (parameter types, rate, data-class), with spec changes gated through a DR L3 review?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.7 Design Review (DR)

DR Level 1.

Q-DR-L1-1. Is a per-archetype Design Checklist published and versioned, with reviews routed via a two-lane model (fast-lane Low/Medium, full-lane High/Critical or any deviation/agent/regulated-data case) with published SLAs?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-DR-L1-2. Do ≥95% of AI vendor integrations going to production in the last 90 days have a completed design-checkpoint record that references the applicable SA pattern and SR requirement pack?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-DR-L1-3. Are pattern-update and pack-update triggers wired to SA and SR (three deviations in the same direction queue a pattern review; repeated requirement waivers queue a pack-revision review), and is a named reviewer population trained and active?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

DR Level 2.

Q-DR-L2-1. Are 100% of Critical-tier reviews using scenario-based walkthrough (3–5 specific threat scenarios from the TA library plus anonymized industry incidents)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-DR-L2-2. Do ≥70% of eligible Critical-tier vendors participate in joint reviews via pre-established communication channels with template NDA and Legal pre-approval?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-DR-L2-3. Is design-drift detection running on cadence (quarterly Critical, annual High), with 100% of material drifts returning to DR?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

DR Level 3.

Q-DR-L3-1. Do ≥90% of Critical integrations carry monthly auto-attestation via automated pattern-compliance scans and configuration audits, with deviations opening DR-exception tickets automatically?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-DR-L3-2. Does the program contribute at least two externally-published artifacts per year to OpenSSF AI, CSA, or Shared Assessments (review rubrics, scenario templates, vendor-cooperation frameworks)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-DR-L3-3. Is the mean review backlog age ≤7 days, and is pattern evolution driven by a pipeline of external incident patterns (ISACs, MITRE ATLAS) plus internal IM incidents plus ML telemetry refreshed quarterly?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.8 Implementation Review (IR)

IR Level 1.

Q-IR-L1-1. Are per-archetype implementation review checklists published and owned, with go-live, annual, and change-triggered review triggers wired to the inventory and integration tracker?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IR-L1-2. Do 100% of new AI vendor integrations in the last 90 days have a go-live IR record, and do ≥90% of active integrations carry a current-year IR record?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IR-L1-3. Are findings tracked with severity, named owner, SLA, and evidence, with a findings-aging dashboard reviewed monthly by the program sponsor and zero blocker findings open at go-live?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

IR Level 2.

Q-IR-L2-1. Are ≥90% of Critical vendors under API-based configuration monitoring (admin APIs ingested, with material changes auto-generating findings, scheduled UI-scraping where APIs do not exist)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IR-L2-2. Are ≥80% of Critical vendors under automated no-train and retention verification (admin-console state polled, synthetic canaries where contract-permitted, retention-window sampling)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IR-L2-3. Is the tier-calibrated IR cadence adhered to ≥95% (Critical: go-live + semi-annual + change + continuous; High: go-live + annual + change; Medium: go-live + annual; Low: go-live only)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

IR Level 3.

Q-IR-L3-1. Do ≥90% of Critical vendors have real-time configuration attestation (continuous evaluation, instant findings on deviation, machine-readable regulator-consumable artifacts)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IR-L3-2. Are per-archetype IR config-baseline schemas published externally (Shared Assessments, CSA AI Safety Initiative) with documented external adoption?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IR-L3-3. Are vendor trust-center artifacts (SIG Lite, CAIQ, custom) ingested for ≥70% of Critical/High vendors with deltas from DPA and AI addendum auto-flagged?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.9 Security Testing (ST)

ST Level 1.

Q-ST-L1-1. Is a foundational test battery published per archetype (≤6 tests each) tied to TA threats and SR requirements, with a named battery owner per archetype, and is it run at go-live, quarterly, post-change, and post-incident?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ST-L1-2. Do 100% of AI vendor integrations reaching production in the last 90 days have a passed go-live battery, and do ≥90% of active integrations have a current-quarter test-battery pass?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ST-L1-3. Has a shadow-AI discovery exercise been run in the last 90 days (synthetic scenarios across expense, endpoint, SaaS-admin, and egress paths) with sponsor review of pass rate and time-to-detect, and are 100% of test failures converted to IM issues within one business day?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

ST Level 2.

Q-ST-L2-1. Are 100% of Critical-tier integrations red-teamed in the last 90 days (prompt-injection chains, indirect-injection via RAG, agent tool abuse, jailbreak regression, data-egress canaries), with reports routed to IM?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ST-L2-2. Is a versioned jailbreak / prompt-injection regression corpus maintained (≥1 update/month) and running weekly in CI against ≥90% of Critical/High integrations with pass/fail trend reported?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ST-L2-3. Are ≥4 bug-bounty-sourced findings consumed into the test library and TA updates per quarter, drawn from vendor-side and internal programs?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

ST Level 3.

Q-ST-L3-1. Are ≥80% of Critical-tier integrations under continuous automated red-team (daily probes via prompt-injection generators, jailbreak ladders, indirect-injection seeded content) with new TTPs feeding the TA library?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ST-L3-2. Does the program contribute ≥4 anonymized, legally-vetted artifacts per year to MITRE ATLAS, AI Vulnerability Database, or OWASP LLM/Agentic Top 10?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ST-L3-3. Does the program host ≥1 industry-shared red-team exercise and participate in ≥2 ISAC AI-vendor tabletops per year?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.10 Environment Hardening (EH)

EH Level 1.

Q-EH-L1-1. Are 100% of sanctioned AI SaaS vendors behind SSO/SAML/OIDC, with org-tenant binding for consumer GenAI enterprise plans, scoped service principals for AI API use, and explicit egress block rules for prohibited consumer AI services?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EH-L1-2. Are AI-aware DLP rules deployed and tuned (recognizing prompt-like bulk text pastes and file uploads into AI vendor domains), with managed-browser and endpoint AI-tool policies on ≥95% of managed endpoints?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EH-L1-3. Is an "AI feature registry" published for 100% of approved parent SaaS vendors with active AI features, with default-off posture, admin-audit feeds captured, and a named admin-governance owner running quarterly reviews?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

EH Level 2.

Q-EH-L2-1. Are AI-vendor-tuned CASB/SSPM policies live (Netskope/Defender/Zscaler/Umbrella for CASB; Obsidian/Adaptive Shield/AppOmni for SSPM) with a named owner and AI-specific signal (bulk-paste detection, file-upload controls, tool-invocation detection)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EH-L2-2. Are 100% of Critical-tier AI SaaS under SASE per-user policy with device-trust requirement, and is the false-positive rate on AI-specific DLP/CASB signals actively tuned and trending down?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EH-L2-3. Do ≥90% of multi-tenant AI-vendor deployments enforce per-tenant key and data scope, with the admin-governance model distinguishing tenant-scoped vs. org-wide settings?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

EH Level 3.

Q-EH-L3-1. Are ≥90% of EH controls expressed as Infrastructure-as-Code (Terraform, OPA, Rego, Kyverno) with continuous drift detection and ≥70% auto-remediation rate for low-risk drift?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EH-L3-2. Is an adaptive-policy pipeline operating that generates tightening proposals from ML detections and IM incidents, with human approval before deploy and a machine-readable change log?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-EH-L3-3. Does the program contribute ≥2 AI-vendor hardening baselines per year to CIS (AI workloads), CSA AI Safety Initiative, or sector ISACs, with maintained upstream parity?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.11 Issue Management (IM)

IM Level 1.

Q-IM-L1-1. Is a single AI-vendor issue backlog operating with standardized metadata (source, affected vendor/integration, archetype, severity, owner, SLA, evidence) and ≥95% of AI-vendor issues recorded in it (vs. scattered across other trackers)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IM-L1-2. Is an AI-vendor incident playbook published with at least five named incident classes (vendor breach notification, vendor outage, prompt-injection/output-integrity, shadow-AI exposure, agent runaway, vendor material change) and assigned roles per class?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IM-L1-3. Is the vendor-breach-notification SLA tracker live (contractual + GDPR Art. 33 + HIPAA + sector-specific) with 100% adherence in last 90 days, and is the post-incident review loop wired to SA, SR, EG, and ML?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

IM Level 2.

Q-IM-L2-1. Is the incident playbook tier-calibrated (Critical: ≤1h MTTA, ≤4h MTTC, full team + Legal + Privacy + Communications + Sponsor; High: ≤4h MTTA, ≤24h MTTC; Medium and Low standardized) with measured per-tier adherence?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IM-L2-2. Are pre-established vendor-coordination channels in place for ≥90% of Critical-tier vendors (template comms, NDA, joint-IR protocol), with at least one joint tabletop per year against the top-5 Critical?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IM-L2-3. When a single vendor breach affects multiple org integrations, is supply-chain-style orchestration used 100% of the time (one Incident Commander, shared status board, shared remediation tracking, cross-integration post-incident review)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

IM Level 3.

Q-IM-L3-1. Does the program contribute ≥4 vendor-incident intelligence artifacts per year via sector ISACs (FS-ISAC, H-ISAC, IT-ISAC) and ≥1 substantive incident-taxonomy contribution per year (classification schemes, severity anchors, response playbook templates) to standards bodies?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IM-L3-2. Are 100% of Critical vendors under automated SLA tracking (notification-time, RCA-delivery, remediation-time) with patterns of breach surfacing for PC contract review or non-renewal recommendation?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-IM-L3-3. Is the SLA-breach-to-contract-action lead time automated and ≤5 business days, with appropriate escalation guards to avoid contradicting a strategic vendor relationship?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

28.12 Monitoring & Logging (ML)

ML Level 1.

Q-ML-L1-1. Are per-archetype logging baselines established and operated for ≥90% of active AI vendor integrations (consumer GenAI, AI-embedded SaaS, AI coding assistant, AI API/model, AI agent platform), with retention meeting the longest applicable requirement (EU AI Act high-risk logs, GDPR records-of-processing) and export paths tested at least annually?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ML-L1-2. Is a high-signal detection set (≤12 detections) live with named owners, queries, SLAs, and last-tuned dates, covering shadow-AI egress, shadow-AI SaaS sign-in, consumer AI personal accounts, bulk paste/upload, API-proxy anomalies, agent tool-call violations, HITL bypass, parent-SaaS feature toggles, admin-key anomalies, no-train settings, vendor breaches, and new AI vendor domains?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ML-L1-3. Is a deployer-duty evidence view produced for every high-risk AI vendor integration, and has a quarterly drill completed inside ≤2 business days SLA in the last 90 days?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

ML Level 2.

Q-ML-L2-1. Are ≥90% of Critical integrations under anomaly detection (baseline per-integration normal behavior, prompt volume, tool-call patterns, egress volume, time-of-day; deviations escalated to IM with context), with FP rate actively tuned and trending down?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ML-L2-2. Is multi-vendor cross-correlation operational (graph of vendor chains refreshed weekly, detecting unexpected edges, changing centralities, new intermediaries, shadow-AI egress from non-AI endpoints)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ML-L2-3. Do 100% of Critical integrations have automated deployer-duty evidence generation, with regulator-inquiry turnaround ≤3 business days?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

ML Level 3.

Q-ML-L3-1. Are ≥90% of Critical integrations under live-queryable evidence attestation with cryptographic signing where applicable?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ML-L3-2. Does the program contribute ≥2 telemetry-standard artifacts per year to OpenTelemetry AI workgroup, CSA AI Safety Initiative, or OpenSSF AI (schemas, semantic conventions, required fields)?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

Q-ML-L3-3. Does the program contribute ≥12 anonymized detection signatures per year to ISAC AI-vendor feeds or OpenSSF AI with documented adoption context?

Answer: ☐ Yes ☐ Partial ☐ No Evidence: _______ Notes: _________

29. Practice-level rollup

After completing all 108 questions, fill in the table below. For each practice, count Yes (Y), Partial (P), and No (N) answers per level. Compute the precise score as L1_score + L2_score + L3_score, where L1_score = (Y + 0.5P) / 3, L2_score = (Y + 0.5P) / 3 × L1_score, L3_score = (Y + 0.5P) / 3 × L2_score.

Notes column for assessor.

Use the space below to record per-practice observations: which questions were the hardest to answer, where evidence was thin, where Partial answers cluster.

30. Domain-level rollup

Domain Maturity = (sum of all 12 Practice Scores) / 12 = ____ / 3.0

Maturity band achieved: ☐ Ad-hoc (0.0–0.9) ☐ Foundational (1.0–1.9) ☐ Comprehensive (2.0–2.9) ☐ Industry-Leading (3.0)

Per-Business-Function rollup.

A domain is mature when all four Business Functions are at the same band. A domain whose Operations function trails the others by a band is not at the higher band, it has built well and verified well but cannot run. A domain whose Verification function trails has the opposite problem, it ships without proof.

Strengths.

Gaps.

Highest-priority remediation areas (top 5).

31. Improvement roadmap template

Use this template to convert assessment findings into a 12-month roadmap. Each entry names a target gap, the practice and level it addresses, the owner, the success metric, and the deadline.

Quarter 1 (months 1–3). Stabilize Level 1 across the four Business Functions. Priority practices: SM L1, PC L1, EG L1, TA L1.

Quarter 2 (months 4–6). Complete remaining L1 practices and begin L2 calibration. Priority practices: SR L1, SA L1, DR L1, IR L1, ST L1, EH L1, IM L1, ML L1; SM L2 risk-tier rubric.

Quarter 3 (months 7–9). Operationalize L2 across the priority practices. Priority practices: PC L2 evidence views, TA L2 per-vendor models, DR L2 scenario-based, IR L2 API-based monitoring, ML L2 anomaly detection.

Quarter 4 (months 10–12). Complete L2 across all 12 practices and prepare L3 entries. Priority practices: ST L2 red-team, EH L2 CASB/SSPM, IM L2 tier-calibrated playbook; begin L3 scope decisions for SM, TA, EG.

Reassessment date (12 months from this assessment): ____

Part V, Reference

32. Glossary

AI Acceptable Use Policy (AUP). The first of the three priority AI vendor policies. States what employees may and may not do with AI vendors, including approved tools, prohibited data classes, the personal-account prohibition, output-review duties, and disclosure obligations.

AI Procurement & Intake Policy. The second priority policy. Requires intake for any AI tool, including AI features inside already-approved SaaS, before adoption. Publishes the intake SLA, fast-track path, and amnesty path.

AI Vendor Data-Sharing Policy. The third priority policy. Sets default training-on-data posture to off and requires DPA / AI addendum for any AI vendor receiving non-public data, with explicit terms for residency, retention, deletion, and breach notification.

AGH, Agent Goal Hijack. One of the four HAI-specific TTPs. The agent's benign goal is redirected via content injected through a trusted-looking path (retrieved document, tool response, multi-turn history).

AI vendor archetype. One of five categories of AI/HAI tool the organization consumes from a third party: consumer GenAI, AI-embedded SaaS, AI coding assistant, AI API/foundation-model vendor, AI agent/automation platform. Threat libraries, requirements, reference architectures, and tests are archetype-keyed.

AI vendor inventory. The single source of truth for all AI vendors in use, sanctioned or not, owned by the program lead. Seeded from expense, SSO, egress, endpoint, and SaaS-admin signals.

Amnesty path. A documented path for employees to disclose previously undisclosed AI use without penalty, routed to retroactive intake. Specific to the L1 program; replaced by blocking enforcement at L2 for Critical-tier use cases.

Anti-pattern catalog. A catalog of AI-vendor integrations known to cause incidents, copy-paste workflow, shadow API key, unscoped agent, trust-the-embedded-feature, pull-without-label, direct-to-vendor call. Each anti-pattern is named, described, prohibited, and paired with the reference pattern that replaces it.

Archetype snapshot. A category-level threat model produced once per archetype at L1 and adapted per intake to produce a per-intake threat snapshot in under 30 minutes.

Compliance evidence view. A continuously-assembled view of compliance evidence per Critical/High AI vendor, covering DPA, subprocessor list, training-data posture, current SOC 2 / ISO 27001, model-version log, incident history, and deployer-duty evidence. Becomes the source for on-demand evidence packs at L3.

Critical / High / Medium / Low. The four risk tiers introduced at SM L2. Driven by data sensitivity, decision-affecting use, agentic capability, user exposure, regulatory scope, and concentration. Each tier carries different intake depth, review cadence, monitoring depth, and re-review triggers.

Deployer. The legal role assigned to an organization that uses an AI system, under EU AI Act terminology. The Vendors-domain program is the organization's apparatus for meeting deployer duties (Article 26) for AI systems consumed from vendors.

Deployer-duty evidence view. A single view per high-risk AI vendor integration consolidating intake approval, REM, DR decision, IR config records, ST pass records, ML logs, incident records, and AUP coverage. Tested via quarterly drill at L1; auto-assembled at L2; live-queryable at L3.

DPA / AI addendum. Data Processing Agreement, with an AI-specific addendum where applicable, executed between the organization and the AI vendor. Required for any AI vendor receiving non-public data.

EA, Excessive Agency. One of the four HAI-specific TTPs. The AI or agent has more capability than its use case requires.

FRIA. Fundamental Rights Impact Assessment, required for certain high-risk AI systems under the EU AI Act.

Intake gate. The single procurement gate through which all AI vendors must pass before adoption. Operates a published SLA, fast-track for pre-approved parent vendors, and the amnesty path for previously undisclosed use.

Priority compliance map. A one-page artifact that ties each priority regulatory or standards requirement (EU AI Act Articles 26 and 50, NIST AI RMF GOVERN, GDPR Articles 28, 22, and 44–49, SOC 2 CC9.2, ISO/IEC 42001, ISO/IEC 27001 A.5.19–A.5.23, sector-specific) to the specific organizational policy that carries it.

RA, Rogue Agents. One of the four HAI-specific TTPs. Autonomous agents drift from intended behavior across long sessions, reflective loops, or multi-agent miscoordination, producing harmful effects nobody explicitly instructed.

REM, Requirements-Evidence Map. A per-vendor map that records, for each applicable requirement in the AI Vendor Requirements Pack, whether the requirement is Met, Met-with-compensating-control, Gap-accepted, or Not-applicable, with a citation to evidence.

Reference pattern. A vetted "green path" architecture pattern published per AI vendor archetype. Teams reach for the pattern first; deviations require design review.

Risk tier rubric. A short table introduced at SM L2 that derives a deterministic risk tier (Critical / High / Medium / Low) from auditable inputs. Drives the differential intensity of every downstream practice's L2 and L3 work.

Shadow AI. AI/HAI adopted outside the program's visibility, attribution, and governance. The Vendors-domain program's primary L1 outcome is to make shadow AI visible, attributable, and trending down.

Shadow AI ratio. Unsanctioned AI vendors divided by total AI vendors in use. A primary outcome metric of the L1 program. Reported quarterly and trending down. Reported per tier at L2; Critical-tier shadow AI tracked at zero.

Shadow AI scoreboard. A quarterly artifact delivered to the executive sponsor summarizing inventory state, new discoveries, shadow AI ratio trend, AUP attestation, and top exposure risks. Becomes tier-aware at L2.

Tier-treatment matrix. A published matrix introduced at SM L2 specifying what each tier (Critical, High, Medium, Low) receives from each downstream practice (PC, TA, DR, IR, ST, ML).

TM, Tool Misuse. One of the four HAI-specific TTPs. Tools available to the AI or agent are invoked for attacker purposes, argument smuggling, unexpected combinations, crafted parameters, recursive invocation.

Tool allowlist. The explicit list of tools an AI agent may invoke, with per-tool scope (parameter types, rate, data class). At L1 the allowlist exists; at L3 it is runtime-enforced as a formal specification.

33. Reference frameworks

This handbook is one of six domain handbooks that, together with a master handbook, constitute HAIAMM v3.0. The frameworks named here are referenced throughout. They are listed by name only; consult the issuing body's current published version when running an assessment.

Maturity-model lineage.

• OWASP SAMM (Software Assurance Maturity Model). HAIAMM borrows SAMM's lifecycle shape (Governance, Building, Verification, Operations) and practice-per-function structure. SAMM covers classic AppSec maturity; HAIAMM extends into AI-specific territory rather than replacing it.
• BSIMM (Building Security In Maturity Model). HAIAMM borrows the observational "this is what organizations actually do" posture at higher maturity levels.

AI-governance frameworks (complementary).

• NIST AI RMF 1.0 + Playbook. The risk-management-framework counterpart to HAIAMM's maturity-model shape. An organization uses NIST AI RMF to decide what to govern, and HAIAMM to measure how mature its ability to govern is. The Playbook gives concrete actions for each function.
• ISO/IEC 42001 (AI Management System). A management-system standard for AI. HAIAMM practices supply the operational evidence an ISO 42001 AIMS requires.
• ISO/IEC 27001 / 27002. General Information Security Management System and controls. HAIAMM practices produce evidence that maps to relevant Annex A controls, especially A.5.19–A.5.23 on supplier relationships.

Regulations applicable to AI vendors.

• EU AI Act. Article 26 (deployer duties), Article 50 (transparency), Annex III (high-risk systems), Article 9 (risk management), Article 15 (accuracy/robustness/cybersecurity).
• GDPR. Article 28 (processor), Article 22 (automated decision-making), Articles 44–49 (international transfers), Article 33 (breach notification), Article 32 (security).
• SOC 2. CC9.2 (vendor management) and trust services criteria applicable to AI services.
• HIPAA. Business Associate Agreement requirements; subcontractor agreements; vendor breach notification.
• PCI-DSS 12.8. Service-provider management; written agreements; ongoing monitoring.
• FINRA / SEC. Third-party model risk management guidance.
• HHS / FDA. AI-enabled medical device guidance.
• NYDFS Part 500. Third-party service-provider security policy.
• OCC third-party risk guidance. Banking sector third-party risk lifecycle.

Threat taxonomies.

• MITRE ATLAS (Adversarial Threat Landscape for AI Systems). Canonical adversarial-ML reference. HAIAMM's TA practice consumes ATLAS technique IDs and contributes back at L3.
• OWASP Top 10 for LLM Applications. Threat reference for LLM-based AI systems. Consumed by TA; reviewed in EG curriculum.
• OWASP Top 10 for Agentic AI. Threat reference for agentic systems. Consumed by TA.
• AI Vulnerability Database. Catalog of disclosed AI-specific vulnerabilities. Consumed by TA; contributed to at L3.

Industry communities.

• CSA AI Safety Initiative / AI Controls Matrix. Cross-organization AI controls work. HAIAMM contributes to controls matrix at L3.
• OpenSSF AI. Open Source Security Foundation working group on AI. HAIAMM contributes telemetry standards and detection signatures at L3.
• Shared Assessments (AI-vendor track). Standardized vendor risk assessments adapted for AI vendors. HAIAMM contributes pack and REM schemas at L3.
• Sector ISACs. FS-ISAC (financial services), H-ISAC (health), IT-ISAC (IT), and others provide intelligence sharing for AI vendor incidents.

Threat-tactic categories specific to HAIAMM (reproduced for reference).

• EA, Excessive Agency. The AI or agent has more capability than its use case requires.
• AGH, Agent Goal Hijack. The agent's benign goal is redirected by content injected along a trusted-looking path.
• TM, Tool Misuse. Tools available to the AI or agent are invoked for attacker purposes.
• RA, Rogue Agents. Autonomous agents drift from intended behavior across long sessions, reflective loops, or multi-agent miscoordination.

34. Change log

End of HAIAMM v3.0 Vendors Domain Handbook.