Security Architecture (SA) - Vendors Assessment

Assessment questionnaire for measuring maturity. Answer each question honestly based on current, implemented practices.

Secure Architecture (SA) - Vendors Domain

HAIAMM Assessment Questionnaire v3.0

v3.0 canonical source: ../practices/SA-Vendors-OnePager.md. Outcome metrics, activities, and success criteria are verbatim from that document. Subject rule (§12.1): the AI is what is being secured, not a tool performing security tasks.


Practice: Secure Architecture (SA) Domain: Vendors Purpose: Assess organizational maturity in publishing and operationalizing reference architectures for safely consuming each AI vendor archetype the organization integrates Scoring Model: Evidence + Outcome Metrics (see Scoring Methodology below)


Instructions

  • Answer each question honestly based on current, implemented practices (not plans or aspirations)
  • Each question has two components: Evidence (what you did) and Outcome Metrics (how well it worked)
  • Scoring uses 4 tiers: Fully Mature (1.0), Implemented (0.67), Partial (0.33), Not Implemented (0.0)
  • Answer progressively, complete all Level 1 questions before Level 2
  • Level progression, achieve ALL questions at a lower level before advancing
  • Baseline first, record current metric values before setting targets

Scoring Methodology

Tier Score Criteria
Fully Mature 1.0 Evidence complete + 3 or more outcome metrics meet targets
Implemented 0.67 Evidence complete + 2 outcome metrics meet targets
Partial 0.33 Evidence partially complete + fewer than 2 metrics meet targets
Not Implemented 0.0 No evidence of the practice

Level Score = average of the three question scores for that level. Overall SA-Vendors Score = weighted average: L1 × 0.5 + L2 × 0.3 + L3 × 0.2.


Maturity Level 1

Objective: Publish reference architectures per AI vendor archetype, an anti-patterns catalog, and a deviation-review path; link each pattern to SR-Vendors requirements and TA-Vendors threats.


Question 1: Publish Reference Architectures per AI Vendor Archetype

Q1.1: Has the organization published a reference architecture pattern for each of the five AI vendor archetypes, consumer GenAI, AI-embedded SaaS, AI coding assistant, AI API / foundation-model, and AI agent / automation platform, with each pattern including a labeled diagram, data-boundary definition, identity and auth model, traffic path, logging specification, and explicit row-by-row mapping to SR-Vendors requirements and TA-Vendors threats, accessible within one click of the integration inventory record?

Evidence Required: - [ ] Architecture registry listing all five vendor-archetype reference patterns with version and publication date - [ ] Each pattern document includes a labeled diagram covering scope, data boundary, identity and auth model (SSO-backed access, service-principal model, secret management, token lifecycle), traffic path (monitored network egress, optional proxy, region pinning), logging spec, SR mapping, and threat mapping - [ ] Archetype-specific emphasis documented: consumer GenAI (enterprise tenant, SSO, content filtering, personal-account prohibition); AI-embedded SaaS (DPA addendum, no-train verification); coding assistant (IDE policy, prohibited paths); AI API (internal proxy as the load-bearing element); AI agent (tool allowlist, HITL gate, indirect-prompt-injection defense) - [ ] SM inventory records link to the applicable reference pattern within one click of the integration record - [ ] Deviation-review path documented with a named architect-reviewer population and a stated SLA (target: ≤5 business days) - [ ] 100% of API/model archetype integrations handling non-public data verified to route through the internal API proxy (confirmed by proxy routing configuration, not only policy declaration)

Outcome Metrics: | Metric | Baseline | Target | Source | Met? | Notes | |--------|----------|--------|--------|------|-------| | Reference patterns published per archetype (consumer GenAI, AI-embedded SaaS, coding assistant, AI API / foundation-model, AI agent / automation platform) | 0 / 5 | 5 / 5 | Architecture registry | ☐ | | | % active AI vendor integrations using a named reference pattern or documented deviation | measure | ≥85% | Inventory × integration metadata | ☐ | | | % of high-risk-tier AI vendor integrations (API/model archetype) running behind the internal API proxy | measure | 100% | Proxy routing config | ☐ | | | Pattern-to-SR requirement mapping coverage | measure | 100% of pattern controls tagged | Pattern metadata | ☐ | |

Metric Collection Guidance: - Patterns published: Count published patterns with all required skeleton elements present for all five archetypes. Source: architecture registry. Reviewed quarterly. - Inventory integration adoption: Query the integration inventory for each active AI vendor integration's pattern-adoption field. Count integrations classified as "on pattern" or "deviation with review" divided by total active integrations. Source: integration inventory export. - Internal API proxy adoption: Query proxy routing configuration for all API/model archetype integrations handling non-public data. Count integrations confirmed routing through the proxy divided by total such integrations. Source: proxy routing configuration audit. - SR mapping coverage: For each pattern, count controls with a SR requirement tag divided by total controls. Aggregate across all five patterns. Source: pattern metadata.

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)

Evidence Location: __ Validation Date: __ Notes: ___


Question 2: Publish the Anti-Patterns Catalog and Enforce the Internal API Proxy

Q1.2: Has the organization published an anti-patterns catalog covering the six L1 mandatory entries, copy-paste workflow, shadow API key, unscoped agent, trust-the-embedded-feature, pull-without-label, and direct-to-vendor call, linked from the AI Acceptable Use Policy and the intake gate, and is the internal API proxy the mandatory traffic path for all API/model-archetype integrations handling non-public data, verified against proxy routing configuration rather than only declared in the pattern?

Evidence Required: - [ ] Anti-patterns catalog document with at least 6 named entries covering the L1 mandatory set (copy-paste workflow, shadow API key, unscoped agent, trust-the-embedded-feature, pull-without-label, direct-to-vendor call) - [ ] Each entry includes: description, why dangerous, real-incident flavor, and the reference pattern that replaces it - [ ] Catalog linked from the AI Acceptable Use Policy (with a dated link) - [ ] Catalog linked from the SM intake gate (verified by IR spot-check) - [ ] Catalog referenced in EG-Vendors training materials with a dated curriculum link - [ ] Proxy routing configuration audit confirming 100% of API/model archetype integrations with non-public data route through the internal API proxy

Outcome Metrics: | Metric | Baseline | Target | Source | Met? | Notes | |--------|----------|--------|--------|------|-------| | Anti-patterns catalog published and linked from AUP / intake | n/a | Yes | Document registry | ☐ | | | % API/model-archetype integrations with non-public data confirmed routing through the internal API proxy | measure | 100% | Proxy routing config | ☐ | | | Anti-pattern catalog entries with a real-incident or representative-incident citation | measure | 100% of entries | Catalog metadata | ☐ | | | Time from new IM-Vendors incident classification to anti-pattern catalog entry | measure | ≤30 days | Catalog change log | ☐ | |

Metric Collection Guidance: - Catalog published: Binary check, catalog exists, is versioned, and links are present from the three required touchpoints. Source: document registry audit. - Internal proxy adoption: Query proxy routing configuration for all API/model archetype integrations. Count those confirmed routing through the proxy divided by total such integrations. Source: proxy routing configuration audit (not only a policy declaration). - Incident citation coverage: Count anti-pattern entries with an incident citation field populated divided by total entries. Source: catalog metadata. - Catalog update lead time: From IM-Vendors incident classification timestamp to catalog-entry publication timestamp. Source: IM-Vendors log and catalog change log.

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)

Evidence Location: __ Validation Date: __ Notes: ___


Question 3: Publish the Deviation-Review Path and Integrate Patterns into the Intake/Inventory Flow

Q1.3: Is a repeat-deviation signal operational, such that three deviations in the same direction for the same vendor archetype automatically queue a pattern-update review with SA ownership, and is the anti-patterns catalog linked from the AI Acceptable Use Policy, the intake gate, and EG-Vendors training?

Evidence Required: - [ ] Integration inventory fields for pattern-adoption status populated for all active AI vendor integrations - [ ] Repeat-deviation signal wired: a query, report, or automation that detects three or more deviations in the same direction for the same vendor archetype and generates a pattern-update queue item - [ ] Pattern-update queue items traceable to deviation records with SA ownership assigned - [ ] New-archetype lead-time SLA documented (target: 30 days from first intake in a new vendor archetype category to pattern publication) - [ ] Pattern quarterly review schedule with change-log entries maintained - [ ] Zero integrations with unreviewed/silent deviations confirmed by audit

Outcome Metrics: | Metric | Baseline | Target | Source | Met? | Notes | |--------|----------|--------|--------|------|-------| | % active AI vendor integrations using a named reference pattern or documented deviation | measure | ≥85% | Inventory × integration metadata | ☐ | | | Repeat-deviation signal operational (three deviations in same direction queue pattern-update review) | measure | Yes, operational and tested | Deviation-review log | ☐ | | | New-archetype lead time (days from first intake to pattern publication) | measure | ≤30 days | Architecture registry change log | ☐ | | | Silent-deviation count (integrations with no pattern classification) | measure | 0 | Integration inventory audit | ☐ | |

Metric Collection Guidance: - Inventory adoption: Same query as Q1.1 outcome metric 2. Reported monthly. - Repeat-deviation signal: Demonstrate by showing at least one instance of the trigger firing and a resulting pattern-update queue item. Source: deviation-review log and pattern-update queue. - New-archetype lead time: For each new vendor archetype category added to the inventory in the review period, measure elapsed days from first intake record to published pattern date. Source: integration inventory and architecture registry. - Silent deviations: Export integration inventory and count integrations where pattern-adoption field is null, empty, or unclassified. Target is zero. Source: integration inventory export.

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)

Evidence Location: __ Validation Date: __ Notes: ___


Maturity Level 2

Objective: Extend reference patterns to multi-region, multi-tenant, and agent-platform complexity; publish an incident-informed anti-pattern catalog; encode patterns as IaC templates teams fork.


Question 4: Extended Reference Patterns

Q2.1: Are the three extended patterns, multi-region AI-vendor integration, multi-tenant parent-SaaS AI-feature, and agent-platform, published as forkable IaC modules with plan-time deviation flagging, and are 80% or more of Critical/High AI vendor integrations running on IaC-encoded patterns as confirmed by the IaC and integration registry?

Evidence Required: - [ ] Three extended pattern variants documented and published: multi-region pattern (residency enforcement, cross-region failover, GDPR transfer mechanism), multi-tenant pattern (per-tenant key scope, per-tenant data scope, admin-governance integration), agent-platform pattern (tool-sandboxing, HITL gates, session isolation, indirect-prompt-injection defense) - [ ] Each variant encoded as a forkable IaC module (Terraform / Kubernetes manifests / Crossplane / Pulumi or equivalent) - [ ] Each IaC module includes plan-time deviation flagging (deviations surface at plan time, not only post-deployment) - [ ] HITL gates wired in agent-platform IaC module (not only documented in the pattern diagram) - [ ] Pattern-drift detection covering 100% of IaC-encoded integrations

Outcome Metrics: | Metric | Baseline | Target | Source | Met? | Notes | |--------|----------|--------|--------|------|-------| | Extended patterns published (multi-region, multi-tenant, agent-platform) | 0 / 3 | 3 / 3 | Architecture registry | ☐ | | | % Critical/High integrations using IaC-encoded pattern | measure | ≥80% | IaC / integration registry | ☐ | | | Anti-patterns fed from IM-Vendors incidents in last 12 months | measure | ≥3 additions | Anti-pattern change log | ☐ | | | Pattern-drift detection coverage | measure | 100% of IaC-encoded integrations | Drift telemetry | ☐ | |

Metric Collection Guidance: - Extended patterns published: Count published extended variants with IaC module and plan-time deviation flagging present. Source: architecture registry. Reviewed quarterly. - IaC adoption rate: Cross-reference IaC registry against integration inventory for all Critical and High-tier AI vendor integrations. Divide IaC-encoded count by total Critical/High count. Source: IaC registry and integration inventory. - Anti-pattern additions from incidents: Count catalog entries added in the last 12 months that carry an IM-Vendors incident reference. Source: anti-pattern change log. - Drift detection coverage: Count IaC-encoded integrations covered by drift telemetry divided by total IaC-encoded integrations. Source: drift telemetry report.

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)

Evidence Location: __ Validation Date: __ Notes: ___


Question 5: Incident-Driven Anti-Pattern Catalog

Q2.2: Has the anti-pattern catalog been updated from three or more real IM-Vendors incidents in the last 12 months, with new entries surfaced to teams at integration time rather than stored only in a reference document, and is pattern-drift detection covering 100% of IaC-encoded integrations with drift findings tracked to resolution?

Evidence Required: - [ ] IM-Vendors incident log showing at least 3 incidents in the last 12 months classified to an anti-pattern (existing or new) - [ ] Anti-pattern catalog change log showing entries added from IM-Vendors classifications with incident references - [ ] Anti-patterns surfaced at integration time: intake gate or pre-integration checklist shows current anti-patterns alongside approved vendor/archetype selection - [ ] Drift telemetry findings log for the last 90 days showing findings with assigned owners and resolution timestamps - [ ] Monthly anti-pattern review cadence confirmed (review meeting records or automated report schedule)

Outcome Metrics: | Metric | Baseline | Target | Source | Met? | Notes | |--------|----------|--------|--------|------|-------| | Anti-pattern catalog additions fed from IM-Vendors incidents in last 12 months | measure | ≥3 additions | Anti-pattern change log | ☐ | | | Pattern-drift detection coverage across IaC-encoded integrations | measure | 100% of IaC-encoded integrations | Drift telemetry | ☐ | | | Drift findings tracked to resolution (no open findings >30 days without an owner) | measure | 100% of findings have an owner and resolution timeline | Drift finding tracker | ☐ | | | Anti-pattern review cadence (monthly from IM-Vendors findings) | measure | 12 of 12 months completed | Review cadence log | ☐ | |

Metric Collection Guidance: - Anti-pattern additions from incidents: Count catalog entries added in the last 12 months that carry an IM-Vendors incident reference. Source: anti-pattern change log. - Drift detection coverage: Same as Q2.1 outcome metric 4. Source: drift telemetry report. - Drift finding resolution: Export drift telemetry findings. Count findings with no assigned owner or with age >30 days and no resolution timestamp. Target is zero. Source: drift finding tracker. - Anti-pattern review cadence: Count monthly review records completed in the last 12 months. Target is 12. Source: review cadence log.

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)

Evidence Location: __ Validation Date: __ Notes: ___


Question 6: Pattern-as-IaC Enforcement and Sanctioned-Catalog Architecture

Q2.3: Is pattern-drift detection covering 100% of IaC-encoded integrations with drift findings tracked to resolution, and is a change-log maintained for quarterly pattern refreshes?

Evidence Required: - [ ] Sanctioned-catalog architecture published: the approved AI vendor catalog is structured so teams find the reference pattern and anti-patterns at the same time they find the approved vendor - [ ] IaC modules with plan-time deviation flagging confirmed operational by at least one test-run record showing a deviation flagged at plan time - [ ] Pattern change log maintained with dated entries covering at least 4 quarters - [ ] Quarterly pattern refresh completed on schedule (at least 1 substantive pattern change per quarter, change-logged) - [ ] Integration time-to-production for teams using IaC patterns tracked and compared to hand-crafted integrations (effectiveness measurement record on file)

Outcome Metrics: | Metric | Baseline | Target | Source | Met? | Notes | |--------|----------|--------|--------|------|-------| | Sanctioned-catalog architecture operational (pattern and anti-patterns visible at vendor approval time) | measure | Yes, operational and tested | Sanctioned catalog structure audit | ☐ | | | Quarterly pattern refresh completed on schedule | measure | 4 of 4 quarters completed | Pattern change log | ☐ | | | Integration time-to-production delta (IaC pattern vs. hand-crafted) | measure | IaC pattern integrations ≤ hand-crafted by ≥10% | Integration time-to-production log | ☐ | | | Incident rate on IaC-encoded integrations vs. hand-crafted integrations (rolling 12-month) | measure | IaC integrations show lower incident rate | IM-Vendors incident log × integration registry | ☐ | |

Metric Collection Guidance: - Sanctioned-catalog architecture: Verify that the approved vendor catalog record includes links to the applicable reference pattern and anti-patterns. Count vendors in the catalog with both links present divided by total approved vendors. Source: sanctioned catalog structure audit. - Quarterly refresh cadence: Count quarterly pattern refresh records in the pattern change log in the last 12 months. Target is 4. Source: pattern change log. - Integration time-to-production delta: Compare average elapsed days from integration approval to production for IaC-pattern integrations vs. hand-crafted integrations over the last 12 months. Source: integration time-to-production log. - Incident rate comparison: Count IM-Vendors incidents attributable to IaC-encoded integrations divided by total IaC-encoded integrations, and compare to the same ratio for hand-crafted integrations. Source: IM-Vendors incident log and integration registry.

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)

Evidence Location: __ Validation Date: __ Notes: ___


Maturity Level 3

Objective: Contribute reference patterns to industry; implement zero-trust AI-vendor access; formally specify agent-tool-scope boundaries for Critical-tier agents.


Question 7: Contribute Reference Patterns Externally

Q3.1: Have two or more reference patterns been externally adopted, cited, forked, or incorporated, by recognized industry bodies such as CNCF AI SIG, OpenSSF AI, CSA AI Safety Initiative, or sector-specific bodies, with documented evidence of adoption and internal practice aligned to the published version?

Evidence Required: - [ ] At least 2 patterns published under an open license (Apache 2.0 or equivalent) in a public repository maintained by or contributed to a recognized industry body (CNCF AI SIG, OpenSSF AI, CSA, or sector equivalent) - [ ] Documented evidence of external adoption: GitHub fork count, citation in a published standards or guidance document, documented adopter organization, or formal inclusion in a community standard - [ ] Internal practice aligned to the published version: quarterly internal-external alignment audit showing zero unexplained divergences - [ ] New archetypes or overlays developed internally proposed for external inclusion within 90 days of internal publication - [ ] External contribution pipeline with at least 2 items in-flight (draft, in-review, or in-publication) at all times

Outcome Metrics: | Metric | Baseline | Target | Source | Met? | Notes | |--------|----------|--------|--------|------|-------| | Patterns externally adopted (cited, forked, or incorporated) | 0 | ≥2 cited/forked | External telemetry | ☐ | | | Internal practice aligned to published external version | n/a | Zero unexplained internal deviations from published version | Internal-external alignment audit | ☐ | | | External contribution pipeline items in-flight | measure | ≥2 items in-flight at all times | External contribution pipeline log | ☐ | | | Static-key retirement progress for AI-vendor machine calls | measure | Tracked on a published calendar with target date | Static-key retirement tracker | ☐ | |

Metric Collection Guidance: - External adoption: Count patterns with documented external adoption evidence (fork count, citation, or adopter record). Source: external telemetry and citation tracking. - Internal-external alignment: Run a quarterly diff between internal pattern version and published external version. Count unexplained divergences. Source: internal-external alignment audit. - Contribution pipeline: Count items in the external contribution pipeline with a status of draft, in-review, or in-publication. Source: external contribution pipeline log. - Static-key retirement: Verify that a static-key retirement project is on the calendar with a target date and progress tracker. Source: static-key retirement tracker.

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)

Evidence Location: __ Validation Date: __ Notes: ___


Question 8: Implement Zero-Trust AI-Vendor Access

Q3.2: Are 90% or more of AI-vendor human sessions operating under continuous zero-trust verification (continuous session verification via IdP/CASB, device-trust integration) and 70% or more of AI-vendor machine calls using JIT-scoped credentials, with static-key retirement tracked on a published calendar?

Evidence Required: - [ ] Continuous session verification operational for AI-vendor human sessions (IdP/CASB policy enforcing continuous verification, not only at login) - [ ] Device-trust integration operational: endpoint posture required before AI-vendor access is granted for human sessions - [ ] JIT tool scope operational for AI agent machine calls: scopes issued per session or per task, not persistent long-lived credentials - [ ] Static-key retirement calendar published with target dates per integration - [ ] Measurement of % human sessions under continuous zero-trust verification from IdP/CASB telemetry - [ ] Measurement of % machine calls under JIT-scoped credentials from secret manager telemetry

Outcome Metrics: | Metric | Baseline | Target | Source | Met? | Notes | |--------|----------|--------|--------|------|-------| | % AI-vendor human sessions under zero-trust (continuous verification) | measure | ≥90% | IdP / CASB | ☐ | | | % AI-vendor machine calls under JIT-scoped credentials | measure | ≥70% | Secret manager | ☐ | | | Static-key retirement calendar maintained with target dates per integration | measure | Yes, maintained and current | Static-key retirement tracker | ☐ | | | Device-trust integration operational for AI-vendor human access | measure | Yes, operational and verified | IdP / MDM compliance report | ☐ | |

Metric Collection Guidance: - Human session zero-trust coverage: Query IdP/CASB for AI-vendor access sessions. Count sessions with continuous verification active divided by total sessions in the review period. Source: IdP/CASB telemetry. - JIT-scoped machine calls: Query secret manager for AI-vendor API calls. Count calls using JIT-scoped credentials divided by total machine calls in the review period. Source: secret manager telemetry. - Static-key retirement calendar: Verify the calendar lists each AI-vendor integration using a static key with a target retirement date. Count integrations on the calendar divided by total integrations using static keys. Source: static-key retirement tracker. - Device-trust integration: Query IdP/MDM for AI-vendor access sessions. Count sessions where device posture was verified divided by total human sessions. Source: IdP/MDM compliance report.

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)

Evidence Location: __ Validation Date: __ Notes: ___


Question 9: Formal Tool-Scope Specification for Critical-Tier Agents

Q3.3: Do 100% of Critical-tier agent deployments carry a formal tool-scope specification, covering parameter types, rate, and data-class, enforced at runtime, with specification changes gated through a DR-Vendors L3 review and a traceable change log?

Evidence Required: - [ ] Agent registry listing all Critical-tier AI agent deployments with their formal tool-scope specification document - [ ] Each formal tool-scope spec covers: parameter types, invocation rate limits, and data-class constraints for each declared tool - [ ] Runtime enforcement confirmed: the tool-scope spec is enforced at the agent runtime layer (not only documented); a test record demonstrates that a tool invocation outside the declared spec is blocked - [ ] DR-Vendors L3 review gate operational: specification changes for Critical-tier agents are gated through a formal design review with at least one named reviewer - [ ] Traceable change log for tool-scope specifications covering the last 12 months - [ ] Quarterly pattern-evolution cadence driven by external signals (ISAC, MITRE ATLAS) and internal signals (IM, ML) with a traceable change log

Outcome Metrics: | Metric | Baseline | Target | Source | Met? | Notes | |--------|----------|--------|--------|------|-------| | Critical-tier agents with formal tool-scope specs | measure | 100% | Agent registry | ☐ | | | % Critical-tier agent tool-scope specs with runtime enforcement confirmed | measure | 100% | Agent runtime enforcement test records | ☐ | | | % tool-scope spec changes for Critical-tier agents gated through DR-Vendors L3 review | measure | 100% | DR-Vendors L3 review log | ☐ | | | Quarterly pattern-evolution cadence completed on schedule | measure | 4 of 4 quarters completed | Pattern change log | ☐ | |

Metric Collection Guidance: - Formal tool-scope spec coverage: Query agent registry for Critical-tier agents. Count those with a formal tool-scope specification document on file divided by total Critical-tier agents. Source: agent registry. - Runtime enforcement: For each Critical-tier agent, verify that a runtime enforcement test record exists demonstrating that out-of-spec tool invocations are blocked. Count agents with a passing enforcement test divided by total Critical-tier agents. Source: agent runtime enforcement test records. - DR-Vendors L3 review gate: Count tool-scope specification change events in the last 12 months that were processed through a DR-Vendors L3 review divided by total tool-scope change events. Source: DR-Vendors L3 review log. - Pattern-evolution cadence: Count quarterly pattern-evolution review records in the last 12 months. Target is 4. Source: pattern change log.

Answer: - ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets) - ☐ Implemented (Evidence complete + 2 metrics meet targets) - ☐ Partial (Evidence partially complete + <2 metrics meet targets) - ☐ Not Implemented (No evidence)

Evidence Location: __ Validation Date: __ Notes: ___


Summary Scorecard

Level Q1 Score Q2 Score Q3 Score Level Score
L1, Publish reference architectures and anti-patterns catalog ___/1.0 ___/1.0 ___/1.0 ___/1.0
L2, IaC-encoded extended patterns with drift detection ___/1.0 ___/1.0 ___/1.0 ___/1.0
L3, Open artifacts, zero-trust access, formal tool-scope specs ___/1.0 ___/1.0 ___/1.0 ___/1.0

Overall SA-Vendors Score (L1×0.5 + L2×0.3 + L3×0.2): ___/1.0

Maturity Statement: - Score 0.0–0.32: Pre-L1, reference patterns and anti-patterns catalog are not yet published; no vetted green path exists for AI vendor integration teams. - Score 0.33–0.65: L1 Partial, some reference patterns published but internal API proxy enforcement, catalog linkage, or deviation tracking is incomplete. - Score 0.66–0.79: L1 Achieved, all five vendor archetypes have reference patterns; anti-patterns catalog published; internal API proxy confirmed for 100% of API/model integrations; deviation-review path operational. - Score 0.80–0.89: L2 Achieved, three extended IaC patterns operational; drift detection at target; incident-informed catalog updates in place; sanctioned-catalog architecture operational. - Score 0.90–1.0: L3 Achieved, patterns externally adopted; zero-trust AI-vendor access operational; Critical-tier agents carry formal tool-scope specs enforced at runtime.


Document Version: HAIAMM v3.0 Practice: Secure Architecture (SA) Domain: Vendors Questionnaire Version: v3.0 Publication Date: 2026-05-15 Author: Verifhai

Instructions:

  • Answer based on current practices, not plans
  • “Yes” requires documented evidence
  • Complete all Level 1 questions before Level 2
  • Partial implementation = “No”

↓ Download as Markdown