# Secure Architecture (SA) - Vendors Domain
## HAIAMM Assessment Questionnaire v3.0

> **v3.0 canonical source:** [`../practices/SA-Vendors-OnePager.md`](../practices/SA-Vendors-OnePager.md). Outcome metrics, activities, and success criteria are verbatim from that document. Subject rule (§12.1): the AI is what is being secured, not a tool performing security tasks.

---

**Practice:** Secure Architecture (SA)
**Domain:** Vendors
**Purpose:** Assess organizational maturity in publishing and operationalizing reference architectures for safely consuming each AI vendor archetype the organization integrates
**Scoring Model:** Evidence + Outcome Metrics (see Scoring Methodology below)

---

## Instructions

- Answer each question honestly based on **current, implemented practices** (not plans or aspirations)
- Each question has two components: **Evidence** (what you did) and **Outcome Metrics** (how well it worked)
- **Scoring uses 4 tiers:** Fully Mature (1.0), Implemented (0.67), Partial (0.33), Not Implemented (0.0)
- **Answer progressively**, complete all Level 1 questions before Level 2
- **Level progression**, achieve ALL questions at a lower level before advancing
- **Baseline first**, record current metric values before setting targets

---

## Scoring Methodology

| Tier | Score | Criteria |
|------|-------|----------|
| Fully Mature | 1.0 | Evidence complete + 3 or more outcome metrics meet targets |
| Implemented | 0.67 | Evidence complete + 2 outcome metrics meet targets |
| Partial | 0.33 | Evidence partially complete + fewer than 2 metrics meet targets |
| Not Implemented | 0.0 | No evidence of the practice |

**Level Score** = average of the three question scores for that level.
**Overall SA-Vendors Score** = weighted average: L1 × 0.5 + L2 × 0.3 + L3 × 0.2.

---

## Maturity Level 1

**Objective:** Publish reference architectures per AI vendor archetype, an anti-patterns catalog, and a deviation-review path; link each pattern to SR-Vendors requirements and TA-Vendors threats.

---

### Question 1: Publish Reference Architectures per AI Vendor Archetype

**Q1.1:** Has the organization published a reference architecture pattern for each of the five AI vendor archetypes, consumer GenAI, AI-embedded SaaS, AI coding assistant, AI API / foundation-model, and AI agent / automation platform, with each pattern including a labeled diagram, data-boundary definition, identity and auth model, traffic path, logging specification, and explicit row-by-row mapping to SR-Vendors requirements and TA-Vendors threats, accessible within one click of the integration inventory record?

**Evidence Required:**
- [ ] Architecture registry listing all five vendor-archetype reference patterns with version and publication date
- [ ] Each pattern document includes a labeled diagram covering scope, data boundary, identity and auth model (SSO-backed access, service-principal model, secret management, token lifecycle), traffic path (monitored network egress, optional proxy, region pinning), logging spec, SR mapping, and threat mapping
- [ ] Archetype-specific emphasis documented: consumer GenAI (enterprise tenant, SSO, content filtering, personal-account prohibition); AI-embedded SaaS (DPA addendum, no-train verification); coding assistant (IDE policy, prohibited paths); AI API (internal proxy as the load-bearing element); AI agent (tool allowlist, HITL gate, indirect-prompt-injection defense)
- [ ] SM inventory records link to the applicable reference pattern within one click of the integration record
- [ ] Deviation-review path documented with a named architect-reviewer population and a stated SLA (target: ≤5 business days)
- [ ] 100% of API/model archetype integrations handling non-public data verified to route through the internal API proxy (confirmed by proxy routing configuration, not only policy declaration)

**Outcome Metrics:**
| Metric | Baseline | Target | Source | Met? | Notes |
|--------|----------|--------|--------|------|-------|
| Reference patterns published per archetype (consumer GenAI, AI-embedded SaaS, coding assistant, AI API / foundation-model, AI agent / automation platform) | 0 / 5 | 5 / 5 | Architecture registry | ☐ | |
| % active AI vendor integrations using a named reference pattern or documented deviation | measure | ≥85% | Inventory × integration metadata | ☐ | |
| % of high-risk-tier AI vendor integrations (API/model archetype) running behind the internal API proxy | measure | 100% | Proxy routing config | ☐ | |
| Pattern-to-SR requirement mapping coverage | measure | 100% of pattern controls tagged | Pattern metadata | ☐ | |

**Metric Collection Guidance:**
- **Patterns published**: Count published patterns with all required skeleton elements present for all five archetypes. Source: architecture registry. Reviewed quarterly.
- **Inventory integration adoption**: Query the integration inventory for each active AI vendor integration's pattern-adoption field. Count integrations classified as "on pattern" or "deviation with review" divided by total active integrations. Source: integration inventory export.
- **Internal API proxy adoption**: Query proxy routing configuration for all API/model archetype integrations handling non-public data. Count integrations confirmed routing through the proxy divided by total such integrations. Source: proxy routing configuration audit.
- **SR mapping coverage**: For each pattern, count controls with a SR requirement tag divided by total controls. Aggregate across all five patterns. Source: pattern metadata.

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No evidence)

**Evidence Location:** _______________
**Validation Date:** _______________
**Notes:** _______________

---

### Question 2: Publish the Anti-Patterns Catalog and Enforce the Internal API Proxy

**Q1.2:** Has the organization published an anti-patterns catalog covering the six L1 mandatory entries, copy-paste workflow, shadow API key, unscoped agent, trust-the-embedded-feature, pull-without-label, and direct-to-vendor call, linked from the AI Acceptable Use Policy and the intake gate, and is the internal API proxy the mandatory traffic path for all API/model-archetype integrations handling non-public data, verified against proxy routing configuration rather than only declared in the pattern?

**Evidence Required:**
- [ ] Anti-patterns catalog document with at least 6 named entries covering the L1 mandatory set (copy-paste workflow, shadow API key, unscoped agent, trust-the-embedded-feature, pull-without-label, direct-to-vendor call)
- [ ] Each entry includes: description, why dangerous, real-incident flavor, and the reference pattern that replaces it
- [ ] Catalog linked from the AI Acceptable Use Policy (with a dated link)
- [ ] Catalog linked from the SM intake gate (verified by IR spot-check)
- [ ] Catalog referenced in EG-Vendors training materials with a dated curriculum link
- [ ] Proxy routing configuration audit confirming 100% of API/model archetype integrations with non-public data route through the internal API proxy

**Outcome Metrics:**
| Metric | Baseline | Target | Source | Met? | Notes |
|--------|----------|--------|--------|------|-------|
| Anti-patterns catalog published and linked from AUP / intake | n/a | Yes | Document registry | ☐ | |
| % API/model-archetype integrations with non-public data confirmed routing through the internal API proxy | measure | 100% | Proxy routing config | ☐ | |
| Anti-pattern catalog entries with a real-incident or representative-incident citation | measure | 100% of entries | Catalog metadata | ☐ | |
| Time from new IM-Vendors incident classification to anti-pattern catalog entry | measure | ≤30 days | Catalog change log | ☐ | |

**Metric Collection Guidance:**
- **Catalog published**: Binary check, catalog exists, is versioned, and links are present from the three required touchpoints. Source: document registry audit.
- **Internal proxy adoption**: Query proxy routing configuration for all API/model archetype integrations. Count those confirmed routing through the proxy divided by total such integrations. Source: proxy routing configuration audit (not only a policy declaration).
- **Incident citation coverage**: Count anti-pattern entries with an incident citation field populated divided by total entries. Source: catalog metadata.
- **Catalog update lead time**: From IM-Vendors incident classification timestamp to catalog-entry publication timestamp. Source: IM-Vendors log and catalog change log.

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No evidence)

**Evidence Location:** _______________
**Validation Date:** _______________
**Notes:** _______________

---

### Question 3: Publish the Deviation-Review Path and Integrate Patterns into the Intake/Inventory Flow

**Q1.3:** Is a repeat-deviation signal operational, such that three deviations in the same direction for the same vendor archetype automatically queue a pattern-update review with SA ownership, and is the anti-patterns catalog linked from the AI Acceptable Use Policy, the intake gate, and EG-Vendors training?

**Evidence Required:**
- [ ] Integration inventory fields for pattern-adoption status populated for all active AI vendor integrations
- [ ] Repeat-deviation signal wired: a query, report, or automation that detects three or more deviations in the same direction for the same vendor archetype and generates a pattern-update queue item
- [ ] Pattern-update queue items traceable to deviation records with SA ownership assigned
- [ ] New-archetype lead-time SLA documented (target: 30 days from first intake in a new vendor archetype category to pattern publication)
- [ ] Pattern quarterly review schedule with change-log entries maintained
- [ ] Zero integrations with unreviewed/silent deviations confirmed by audit

**Outcome Metrics:**
| Metric | Baseline | Target | Source | Met? | Notes |
|--------|----------|--------|--------|------|-------|
| % active AI vendor integrations using a named reference pattern or documented deviation | measure | ≥85% | Inventory × integration metadata | ☐ | |
| Repeat-deviation signal operational (three deviations in same direction queue pattern-update review) | measure | Yes, operational and tested | Deviation-review log | ☐ | |
| New-archetype lead time (days from first intake to pattern publication) | measure | ≤30 days | Architecture registry change log | ☐ | |
| Silent-deviation count (integrations with no pattern classification) | measure | 0 | Integration inventory audit | ☐ | |

**Metric Collection Guidance:**
- **Inventory adoption**: Same query as Q1.1 outcome metric 2. Reported monthly.
- **Repeat-deviation signal**: Demonstrate by showing at least one instance of the trigger firing and a resulting pattern-update queue item. Source: deviation-review log and pattern-update queue.
- **New-archetype lead time**: For each new vendor archetype category added to the inventory in the review period, measure elapsed days from first intake record to published pattern date. Source: integration inventory and architecture registry.
- **Silent deviations**: Export integration inventory and count integrations where pattern-adoption field is null, empty, or unclassified. Target is zero. Source: integration inventory export.

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No evidence)

**Evidence Location:** _______________
**Validation Date:** _______________
**Notes:** _______________

---

## Maturity Level 2

**Objective:** Extend reference patterns to multi-region, multi-tenant, and agent-platform complexity; publish an incident-informed anti-pattern catalog; encode patterns as IaC templates teams fork.

---

### Question 4: Extended Reference Patterns

**Q2.1:** Are the three extended patterns, multi-region AI-vendor integration, multi-tenant parent-SaaS AI-feature, and agent-platform, published as forkable IaC modules with plan-time deviation flagging, and are 80% or more of Critical/High AI vendor integrations running on IaC-encoded patterns as confirmed by the IaC and integration registry?

**Evidence Required:**
- [ ] Three extended pattern variants documented and published: multi-region pattern (residency enforcement, cross-region failover, GDPR transfer mechanism), multi-tenant pattern (per-tenant key scope, per-tenant data scope, admin-governance integration), agent-platform pattern (tool-sandboxing, HITL gates, session isolation, indirect-prompt-injection defense)
- [ ] Each variant encoded as a forkable IaC module (Terraform / Kubernetes manifests / Crossplane / Pulumi or equivalent)
- [ ] Each IaC module includes plan-time deviation flagging (deviations surface at plan time, not only post-deployment)
- [ ] HITL gates wired in agent-platform IaC module (not only documented in the pattern diagram)
- [ ] Pattern-drift detection covering 100% of IaC-encoded integrations

**Outcome Metrics:**
| Metric | Baseline | Target | Source | Met? | Notes |
|--------|----------|--------|--------|------|-------|
| Extended patterns published (multi-region, multi-tenant, agent-platform) | 0 / 3 | 3 / 3 | Architecture registry | ☐ | |
| % Critical/High integrations using IaC-encoded pattern | measure | ≥80% | IaC / integration registry | ☐ | |
| Anti-patterns fed from IM-Vendors incidents in last 12 months | measure | ≥3 additions | Anti-pattern change log | ☐ | |
| Pattern-drift detection coverage | measure | 100% of IaC-encoded integrations | Drift telemetry | ☐ | |

**Metric Collection Guidance:**
- **Extended patterns published**: Count published extended variants with IaC module and plan-time deviation flagging present. Source: architecture registry. Reviewed quarterly.
- **IaC adoption rate**: Cross-reference IaC registry against integration inventory for all Critical and High-tier AI vendor integrations. Divide IaC-encoded count by total Critical/High count. Source: IaC registry and integration inventory.
- **Anti-pattern additions from incidents**: Count catalog entries added in the last 12 months that carry an IM-Vendors incident reference. Source: anti-pattern change log.
- **Drift detection coverage**: Count IaC-encoded integrations covered by drift telemetry divided by total IaC-encoded integrations. Source: drift telemetry report.

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No evidence)

**Evidence Location:** _______________
**Validation Date:** _______________
**Notes:** _______________

---

### Question 5: Incident-Driven Anti-Pattern Catalog

**Q2.2:** Has the anti-pattern catalog been updated from three or more real IM-Vendors incidents in the last 12 months, with new entries surfaced to teams at integration time rather than stored only in a reference document, and is pattern-drift detection covering 100% of IaC-encoded integrations with drift findings tracked to resolution?

**Evidence Required:**
- [ ] IM-Vendors incident log showing at least 3 incidents in the last 12 months classified to an anti-pattern (existing or new)
- [ ] Anti-pattern catalog change log showing entries added from IM-Vendors classifications with incident references
- [ ] Anti-patterns surfaced at integration time: intake gate or pre-integration checklist shows current anti-patterns alongside approved vendor/archetype selection
- [ ] Drift telemetry findings log for the last 90 days showing findings with assigned owners and resolution timestamps
- [ ] Monthly anti-pattern review cadence confirmed (review meeting records or automated report schedule)

**Outcome Metrics:**
| Metric | Baseline | Target | Source | Met? | Notes |
|--------|----------|--------|--------|------|-------|
| Anti-pattern catalog additions fed from IM-Vendors incidents in last 12 months | measure | ≥3 additions | Anti-pattern change log | ☐ | |
| Pattern-drift detection coverage across IaC-encoded integrations | measure | 100% of IaC-encoded integrations | Drift telemetry | ☐ | |
| Drift findings tracked to resolution (no open findings >30 days without an owner) | measure | 100% of findings have an owner and resolution timeline | Drift finding tracker | ☐ | |
| Anti-pattern review cadence (monthly from IM-Vendors findings) | measure | 12 of 12 months completed | Review cadence log | ☐ | |

**Metric Collection Guidance:**
- **Anti-pattern additions from incidents**: Count catalog entries added in the last 12 months that carry an IM-Vendors incident reference. Source: anti-pattern change log.
- **Drift detection coverage**: Same as Q2.1 outcome metric 4. Source: drift telemetry report.
- **Drift finding resolution**: Export drift telemetry findings. Count findings with no assigned owner or with age >30 days and no resolution timestamp. Target is zero. Source: drift finding tracker.
- **Anti-pattern review cadence**: Count monthly review records completed in the last 12 months. Target is 12. Source: review cadence log.

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No evidence)

**Evidence Location:** _______________
**Validation Date:** _______________
**Notes:** _______________

---

### Question 6: Pattern-as-IaC Enforcement and Sanctioned-Catalog Architecture

**Q2.3:** Is pattern-drift detection covering 100% of IaC-encoded integrations with drift findings tracked to resolution, and is a change-log maintained for quarterly pattern refreshes?

**Evidence Required:**
- [ ] Sanctioned-catalog architecture published: the approved AI vendor catalog is structured so teams find the reference pattern and anti-patterns at the same time they find the approved vendor
- [ ] IaC modules with plan-time deviation flagging confirmed operational by at least one test-run record showing a deviation flagged at plan time
- [ ] Pattern change log maintained with dated entries covering at least 4 quarters
- [ ] Quarterly pattern refresh completed on schedule (at least 1 substantive pattern change per quarter, change-logged)
- [ ] Integration time-to-production for teams using IaC patterns tracked and compared to hand-crafted integrations (effectiveness measurement record on file)

**Outcome Metrics:**
| Metric | Baseline | Target | Source | Met? | Notes |
|--------|----------|--------|--------|------|-------|
| Sanctioned-catalog architecture operational (pattern and anti-patterns visible at vendor approval time) | measure | Yes, operational and tested | Sanctioned catalog structure audit | ☐ | |
| Quarterly pattern refresh completed on schedule | measure | 4 of 4 quarters completed | Pattern change log | ☐ | |
| Integration time-to-production delta (IaC pattern vs. hand-crafted) | measure | IaC pattern integrations ≤ hand-crafted by ≥10% | Integration time-to-production log | ☐ | |
| Incident rate on IaC-encoded integrations vs. hand-crafted integrations (rolling 12-month) | measure | IaC integrations show lower incident rate | IM-Vendors incident log × integration registry | ☐ | |

**Metric Collection Guidance:**
- **Sanctioned-catalog architecture**: Verify that the approved vendor catalog record includes links to the applicable reference pattern and anti-patterns. Count vendors in the catalog with both links present divided by total approved vendors. Source: sanctioned catalog structure audit.
- **Quarterly refresh cadence**: Count quarterly pattern refresh records in the pattern change log in the last 12 months. Target is 4. Source: pattern change log.
- **Integration time-to-production delta**: Compare average elapsed days from integration approval to production for IaC-pattern integrations vs. hand-crafted integrations over the last 12 months. Source: integration time-to-production log.
- **Incident rate comparison**: Count IM-Vendors incidents attributable to IaC-encoded integrations divided by total IaC-encoded integrations, and compare to the same ratio for hand-crafted integrations. Source: IM-Vendors incident log and integration registry.

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No evidence)

**Evidence Location:** _______________
**Validation Date:** _______________
**Notes:** _______________

---

## Maturity Level 3

**Objective:** Contribute reference patterns to industry; implement zero-trust AI-vendor access; formally specify agent-tool-scope boundaries for Critical-tier agents.

---

### Question 7: Contribute Reference Patterns Externally

**Q3.1:** Have two or more reference patterns been externally adopted, cited, forked, or incorporated, by recognized industry bodies such as CNCF AI SIG, OpenSSF AI, CSA AI Safety Initiative, or sector-specific bodies, with documented evidence of adoption and internal practice aligned to the published version?

**Evidence Required:**
- [ ] At least 2 patterns published under an open license (Apache 2.0 or equivalent) in a public repository maintained by or contributed to a recognized industry body (CNCF AI SIG, OpenSSF AI, CSA, or sector equivalent)
- [ ] Documented evidence of external adoption: GitHub fork count, citation in a published standards or guidance document, documented adopter organization, or formal inclusion in a community standard
- [ ] Internal practice aligned to the published version: quarterly internal-external alignment audit showing zero unexplained divergences
- [ ] New archetypes or overlays developed internally proposed for external inclusion within 90 days of internal publication
- [ ] External contribution pipeline with at least 2 items in-flight (draft, in-review, or in-publication) at all times

**Outcome Metrics:**
| Metric | Baseline | Target | Source | Met? | Notes |
|--------|----------|--------|--------|------|-------|
| Patterns externally adopted (cited, forked, or incorporated) | 0 | ≥2 cited/forked | External telemetry | ☐ | |
| Internal practice aligned to published external version | n/a | Zero unexplained internal deviations from published version | Internal-external alignment audit | ☐ | |
| External contribution pipeline items in-flight | measure | ≥2 items in-flight at all times | External contribution pipeline log | ☐ | |
| Static-key retirement progress for AI-vendor machine calls | measure | Tracked on a published calendar with target date | Static-key retirement tracker | ☐ | |

**Metric Collection Guidance:**
- **External adoption**: Count patterns with documented external adoption evidence (fork count, citation, or adopter record). Source: external telemetry and citation tracking.
- **Internal-external alignment**: Run a quarterly diff between internal pattern version and published external version. Count unexplained divergences. Source: internal-external alignment audit.
- **Contribution pipeline**: Count items in the external contribution pipeline with a status of draft, in-review, or in-publication. Source: external contribution pipeline log.
- **Static-key retirement**: Verify that a static-key retirement project is on the calendar with a target date and progress tracker. Source: static-key retirement tracker.

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No evidence)

**Evidence Location:** _______________
**Validation Date:** _______________
**Notes:** _______________

---

### Question 8: Implement Zero-Trust AI-Vendor Access

**Q3.2:** Are 90% or more of AI-vendor human sessions operating under continuous zero-trust verification (continuous session verification via IdP/CASB, device-trust integration) and 70% or more of AI-vendor machine calls using JIT-scoped credentials, with static-key retirement tracked on a published calendar?

**Evidence Required:**
- [ ] Continuous session verification operational for AI-vendor human sessions (IdP/CASB policy enforcing continuous verification, not only at login)
- [ ] Device-trust integration operational: endpoint posture required before AI-vendor access is granted for human sessions
- [ ] JIT tool scope operational for AI agent machine calls: scopes issued per session or per task, not persistent long-lived credentials
- [ ] Static-key retirement calendar published with target dates per integration
- [ ] Measurement of % human sessions under continuous zero-trust verification from IdP/CASB telemetry
- [ ] Measurement of % machine calls under JIT-scoped credentials from secret manager telemetry

**Outcome Metrics:**
| Metric | Baseline | Target | Source | Met? | Notes |
|--------|----------|--------|--------|------|-------|
| % AI-vendor human sessions under zero-trust (continuous verification) | measure | ≥90% | IdP / CASB | ☐ | |
| % AI-vendor machine calls under JIT-scoped credentials | measure | ≥70% | Secret manager | ☐ | |
| Static-key retirement calendar maintained with target dates per integration | measure | Yes, maintained and current | Static-key retirement tracker | ☐ | |
| Device-trust integration operational for AI-vendor human access | measure | Yes, operational and verified | IdP / MDM compliance report | ☐ | |

**Metric Collection Guidance:**
- **Human session zero-trust coverage**: Query IdP/CASB for AI-vendor access sessions. Count sessions with continuous verification active divided by total sessions in the review period. Source: IdP/CASB telemetry.
- **JIT-scoped machine calls**: Query secret manager for AI-vendor API calls. Count calls using JIT-scoped credentials divided by total machine calls in the review period. Source: secret manager telemetry.
- **Static-key retirement calendar**: Verify the calendar lists each AI-vendor integration using a static key with a target retirement date. Count integrations on the calendar divided by total integrations using static keys. Source: static-key retirement tracker.
- **Device-trust integration**: Query IdP/MDM for AI-vendor access sessions. Count sessions where device posture was verified divided by total human sessions. Source: IdP/MDM compliance report.

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No evidence)

**Evidence Location:** _______________
**Validation Date:** _______________
**Notes:** _______________

---

### Question 9: Formal Tool-Scope Specification for Critical-Tier Agents

**Q3.3:** Do 100% of Critical-tier agent deployments carry a formal tool-scope specification, covering parameter types, rate, and data-class, enforced at runtime, with specification changes gated through a DR-Vendors L3 review and a traceable change log?

**Evidence Required:**
- [ ] Agent registry listing all Critical-tier AI agent deployments with their formal tool-scope specification document
- [ ] Each formal tool-scope spec covers: parameter types, invocation rate limits, and data-class constraints for each declared tool
- [ ] Runtime enforcement confirmed: the tool-scope spec is enforced at the agent runtime layer (not only documented); a test record demonstrates that a tool invocation outside the declared spec is blocked
- [ ] DR-Vendors L3 review gate operational: specification changes for Critical-tier agents are gated through a formal design review with at least one named reviewer
- [ ] Traceable change log for tool-scope specifications covering the last 12 months
- [ ] Quarterly pattern-evolution cadence driven by external signals (ISAC, MITRE ATLAS) and internal signals (IM, ML) with a traceable change log

**Outcome Metrics:**
| Metric | Baseline | Target | Source | Met? | Notes |
|--------|----------|--------|--------|------|-------|
| Critical-tier agents with formal tool-scope specs | measure | 100% | Agent registry | ☐ | |
| % Critical-tier agent tool-scope specs with runtime enforcement confirmed | measure | 100% | Agent runtime enforcement test records | ☐ | |
| % tool-scope spec changes for Critical-tier agents gated through DR-Vendors L3 review | measure | 100% | DR-Vendors L3 review log | ☐ | |
| Quarterly pattern-evolution cadence completed on schedule | measure | 4 of 4 quarters completed | Pattern change log | ☐ | |

**Metric Collection Guidance:**
- **Formal tool-scope spec coverage**: Query agent registry for Critical-tier agents. Count those with a formal tool-scope specification document on file divided by total Critical-tier agents. Source: agent registry.
- **Runtime enforcement**: For each Critical-tier agent, verify that a runtime enforcement test record exists demonstrating that out-of-spec tool invocations are blocked. Count agents with a passing enforcement test divided by total Critical-tier agents. Source: agent runtime enforcement test records.
- **DR-Vendors L3 review gate**: Count tool-scope specification change events in the last 12 months that were processed through a DR-Vendors L3 review divided by total tool-scope change events. Source: DR-Vendors L3 review log.
- **Pattern-evolution cadence**: Count quarterly pattern-evolution review records in the last 12 months. Target is 4. Source: pattern change log.

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No evidence)

**Evidence Location:** _______________
**Validation Date:** _______________
**Notes:** _______________

---

## Summary Scorecard

| Level | Q1 Score | Q2 Score | Q3 Score | Level Score |
|-------|----------|----------|----------|-------------|
| L1, Publish reference architectures and anti-patterns catalog | ___/1.0 | ___/1.0 | ___/1.0 | ___/1.0 |
| L2, IaC-encoded extended patterns with drift detection | ___/1.0 | ___/1.0 | ___/1.0 | ___/1.0 |
| L3, Open artifacts, zero-trust access, formal tool-scope specs | ___/1.0 | ___/1.0 | ___/1.0 | ___/1.0 |

**Overall SA-Vendors Score (L1×0.5 + L2×0.3 + L3×0.2):** ___/1.0

**Maturity Statement:**
- **Score 0.0–0.32:** Pre-L1, reference patterns and anti-patterns catalog are not yet published; no vetted green path exists for AI vendor integration teams.
- **Score 0.33–0.65:** L1 Partial, some reference patterns published but internal API proxy enforcement, catalog linkage, or deviation tracking is incomplete.
- **Score 0.66–0.79:** L1 Achieved, all five vendor archetypes have reference patterns; anti-patterns catalog published; internal API proxy confirmed for 100% of API/model integrations; deviation-review path operational.
- **Score 0.80–0.89:** L2 Achieved, three extended IaC patterns operational; drift detection at target; incident-informed catalog updates in place; sanctioned-catalog architecture operational.
- **Score 0.90–1.0:** L3 Achieved, patterns externally adopted; zero-trust AI-vendor access operational; Critical-tier agents carry formal tool-scope specs enforced at runtime.

---

**Document Version:** HAIAMM v3.0
**Practice:** Secure Architecture (SA)
**Domain:** Vendors
**Questionnaire Version:** v3.0
**Publication Date:** 2026-05-15
**Author:** Verifhai