AI Vendor Risk & Procurement Lab
Evaluate the security posture of a realistic AI model provider using their trust center, data processing addendum, and security questionnaire responses. Identify gaps, flag procurement blockers, and produce a vendor risk summary for a fictional enterprise customer.
Progress
0/100 points
Status
not-started
Steps
0/4
Mission
Primary objective
Review all three vendor artifacts. Identify: (1) claims that lack evidence, (2) DPA gaps specific to AI data handling, (3) questionnaire responses that are evasive or incomplete. Produce a risk summary with a pass/conditional pass/block recommendation and the specific remediation required.
Brief
Scenario
Enterprise AI vendor security evaluation
Your company is evaluating 'NeuralForge', a fictional enterprise LLM API provider. The security review team has collected three artifacts: a trust center summary document, a data processing addendum, and a completed security questionnaire. You must assess these for procurement readiness — can you sign a contract and move to production? If not, what must change?
Objectives
- Read an AI vendor trust center document and identify what is claimed versus what is evidenced.
- Evaluate a data processing addendum for AI-specific terms: training opt-out, data retention, subprocessor controls, and breach notification.
- Score a vendor security questionnaire response for completeness, evasion patterns, and missing controls.
- Identify procurement blockers: which gaps require contractual remediation before a purchasing decision.
- Produce a vendor risk summary usable in a procurement review or security committee presentation.
Prerequisites
- Understand what a data processing addendum (DPA) is and why it matters for AI vendors.
- Know the key AI-specific data risk concerns: training data use, prompt logging, model fine-tuning, and data residency.
- Review the AIPSA customer trust domain for what evidence AI vendors should be providing.
Expected signals
- trust center claim without evidence
- training opt-out absent from DPA
- retention period too long
- subprocessor list incomplete
- SOC 2 report not shared
- prompt logging not addressed
- data residency gap
Prepare
Reading materials
AIPSA Field Guide · Ch 13 · Ch 13
Vendor Risk and AI Procurement
Evaluating AI vendors, model providers, subprocessors, data processing terms, security questionnaires, contract controls, trust center claims, and procurement decisions.
~2 MB
Do not rebuild
Reusable source assets
NeuralForge trust center summary
Trust center claims: SOC 2 Type II (in progress), ISO 27001 certified, data not used for training (enterprise tier), 30-day retention. Gaps: no audit report links, 'enterprise tier' training opt-out is a plan not a DPA term, vague incident response SLA.
llm-attack-range/ai-security-range/fixtures/vendor-risk/neuralforge-trust-center.md
NeuralForge data processing addendum
DPA fixture: missing training data opt-out clause, 90-day retention (conflicts with 30-day trust center claim), subprocessor list references 'Annex B' which is empty, breach notification is 72 hours to NeuralForge only (not to customer), no data residency commitment.
llm-attack-range/ai-security-range/fixtures/vendor-risk/neuralforge-dpa.md
Security questionnaire responses
SIG Lite-style questionnaire. Notable responses: pen test 'conducted annually by internal team', MFA 'enabled for admin accounts', incident response plan 'available on request', encryption 'AES-256 at rest', prompt logs 'retained for abuse detection purposes' with no defined period.
llm-attack-range/ai-security-range/fixtures/vendor-risk/neuralforge-security-questionnaire.json
Sample inputs
DPA — Training data clause (absent) · text
The 90-day retention conflicts with the 30-day figure on the trust center. The absence of a training opt-out clause is a procurement blocker for most enterprise customers with data governance requirements.
Section 4 — Data Use NeuralForge processes Customer Data solely for the purpose of providing the Services as described in the Order Form. [No clause addressing model training, fine-tuning, or use of Customer Data for model improvement appears in this DPA.] Section 6 — Retention NeuralForge will retain Customer Data for a period of ninety (90) days following termination of the applicable Order Form, after which Customer Data will be deleted in accordance with NeuralForge's standard deletion procedures.
Questionnaire — key responses · json
Red flags: internal pen test (not third-party), incident response plan not provided upfront, prompt log retention has no defined period and includes 'service improvement' language, SOC 2 in progress means no report available, Annex B subprocessor list is empty.
{
"penetration_testing": "Conducted annually by NeuralForge internal security team",
"mfa": "Enforced for all administrative accounts",
"incident_response_plan": "Available upon request under NDA",
"encryption_at_rest": "AES-256",
"encryption_in_transit": "TLS 1.2 or higher",
"prompt_log_retention": "Retained for abuse detection and service improvement purposes",
"subprocessors": "See Annex B",
"soc2_report": "SOC 2 Type II audit currently in progress; report expected Q4",
"data_residency": "Data stored in US-East-1; additional regions available via enterprise add-on"
}Track progress
Lab steps
Audit the trust center claims for evidential backing
Go through each claim in the trust center document. For each: classify it as (A) claim with evidence, (B) claim without evidence, or (C) claim that conflicts with another source. The SOC 2 status, training opt-out, retention period, and incident SLA are the key areas. Flag any claim-evidence gap as a risk item.
Evidence prompt: List each trust center claim, its category (A/B/C), and the specific evidence gap or conflict.
Identify DPA gaps specific to AI data handling
Review the DPA fixture for five AI-specific items: (1) training data opt-out, (2) prompt/response log handling, (3) data retention period, (4) subprocessor disclosure, (5) breach notification path. For each missing or inadequate term, write the specific DPA language that should be added or corrected.
Evidence prompt: For each of the five items: current DPA state → gap → required contractual language.
Score the security questionnaire for evasion patterns
Rate each questionnaire response on a 1-3 scale: 1 = specific and evidenced, 2 = vague or partially answered, 3 = evasive or a procurement blocker. Flag the three most concerning responses and explain why each is insufficient. Note that 'available on request' and 'in progress' answers require follow-up.
Evidence prompt: Rate each response 1-3, identify the three highest-risk responses, and specify what evidence or commitment would move them to a 1.
Produce the vendor risk summary with recommendation
Write the vendor risk summary: (1) pass / conditional pass / block recommendation, (2) the specific procurement blockers that must be resolved before signing, (3) the acceptable risk items that can proceed with standard monitoring, (4) any items that require contractual addenda. This summary should be usable in a security committee presentation.
Evidence prompt: Recommendation (pass/conditional/block), list of blockers, list of acceptable risks, and required contractual changes.
Submission draft
Evidence artifact builder
AI Vendor Risk Assessment
Structured vendor risk output for procurement review. The recommendation and blocker list fields are the key artifacts for a security committee.
Reference
Framework mappings
NIST AI RMF
GOVERN · AI governance and third-party risk
ISO 42001
6.1.2 · AI risk assessment — third-party AI systems
OWASP LLM Top 10
LLM05 · Supply-Chain Vulnerabilities
EU AI Act
Art. 28 · Obligations of deployers
Self-assessment
Scoring checklist
Score estimate: 0/100
Explore
Related tools
Directory
Ecosystem tools
Export
Submit or export your lab evidence
Save a local progress draft, submit the self-scored artifact, or export Markdown for evidence portfolio use.