AI Product Security Assessment Sample

# AI Product Security Assessment Sample

AI Product Security Assessment

Executive Summary

The reviewed AI product is commercially promising, but its control evidence is behind its product ambition. The largest gaps are retrieval authorization, tool permission clarity, sensitive AI trace handling, and buyer-ready model provider language.

Decision · conditional

Recommended launch decision

Proceed with a controlled customer pilot, but do not expand enterprise rollout until retrieval authorization tests, the agent permission matrix, and AI trace retention controls are complete.

Metrics

Assessment snapshot

Critical risks

High risks

Evidence gaps

Release blockers

executive

Commercial meaning

The blockers are not abstract AI ethics concerns. They are procurement friction, trust review friction, and product-security execution gaps.

## System reviewed The assessment covered a customer-facing AI copilot using RAG, a third-party model provider, workflow tools, approval screens, and AI trace logging.

Trust boundary map

AI Trust Boundary Map

The trust boundary review identified the AI gateway as the main control point and the tool layer as the highest-risk authority boundary.

content/deliverables/data/ai-trust-boundary-map.sample.json

Synthetic sample trust boundary data for a customer-facing AI copilot using retrieval, model-provider routing, workflow tools, approval screens, and AI trace logging.

Nodes

Boundaries

Flows

Controls

actor

Authenticated User

medium

application

SaaS Web Application

medium

control-point

AI Gateway

critical

data-store

Retrieval Index

critical

third-party-service

Model Provider

high

tool-surface

Workflow Tools

critical

human-review

Approval Console

high

observability-store

AI Trace Store

high

Prompt submitted

Session auth, tenant scope, request validation

medium

Prompt envelope created

Gateway-only model access, request classification, tenant binding

high

Retrieval query

Authorization-preserving retrieval filters, source ACL tests

critical

Model call

Data minimization, provider boundary, training exclusion statement

high

Tool plan prepared

Permission matrix, action class policy, tool allowlist

critical

Approval request

Human approval with evidence bundle and reviewer identity

high

Boundary

Tenant Boundary

Separates one customer tenant's data, retrieval results, logs, and tool actions from another tenant.

Boundary

Retrieval Authorization Boundary

Ensures source-system authorization survives indexing, chunking, retrieval, reranking, and prompt assembly.

Boundary

Model Provider Boundary

Controls what data leaves the product boundary and how model provider commitments are represented to buyers.

Boundary

Tool Authority Boundary

Separates text generation from state-changing tool authority.

## Top findings

Findings

Top validated findings

Finding · critical

Retrieval can expose restricted customer context

Evidence: rag-authz-regression

Semantic retrieval can return content the user could not access directly through the source system. This is the most important security and buyer-trust issue in scope.

Finding · critical

Sensitive actions can be queued without enough approval context

Evidence: approval-workflow-review

The agent does not directly execute the highest-risk action, but the approval screen does not always show the evidence, target, blast radius, and rollback path needed for meaningful human review.

Finding · high

AI traces may contain sensitive customer data

Evidence: trace-schema-review

Prompts, retrieved snippets, model outputs, and tool-call records may enter logs that do not yet have AI-specific retention, redaction, and access controls.

Finding · high

Model provider boundary is not buyer-ready

Evidence: questionnaire-review

The underlying provider position may be acceptable, but the current buyer-facing answer is too scattered and imprecise for serious enterprise security review.

## Agent authority review

Agent permission matrix

Agent Tool Permission Matrix

Agent authority must be separated into read, suggest, draft, queue, approve, and execute. Broad tool access is not a control model.

content/deliverables/data/agent-tool-permission-matrix.sample.json

Synthetic sample permission matrix for an AI copilot with retrieval, case-management, customer messaging, CRM, billing, and notification tool access.

Principle

Separate reading, suggesting, drafting, queuing, approving, and executing. Do not treat all tool access as one permission.

Default posture

deny-by-default

Approval model

Human approval required for customer-visible, billing-impacting, destructive, privileged, or cross-tenant actions.

ReadSuggestDraftQueueApproveExecute

Agent	Tool	Action	Scope	Approval	Risk	Owner
Support Copilot	Case Management API	read	tenant-scoped support cases visible to the authenticated user	no	medium	Support Platform
Support Copilot	Customer Messaging	draft	draft response text for the active case only	yes, before send	high	Product Operations
Support Copilot	Customer Messaging	execute	send customer-visible response	yes, human-only approval	critical	Product Operations
Support Copilot	Case Management API	queue	priority, category, routing tags, summary fields	yes for priority and routing changes	high	Support Platform
Support Copilot	CRM	read	account profile and entitlement fields needed for support context	no	medium	Revenue Operations
Support Copilot	CRM	execute	update account fields	yes, restricted to human operators	critical	Revenue Operations
Support Copilot	Billing System	read	plan, invoice status, entitlement flags	no for entitlement lookups	high	Finance Systems
Support Copilot	Billing System	execute	issue credits, refunds, plan changes	human-only approval and finance policy gate	critical	Finance Systems
Support Copilot	Notification Service	queue	internal team notification for escalation only	no for internal escalation templates	medium	Product Operations
Support Copilot	External Webhook	execute	third-party workflow triggers	yes, security-reviewed allowlist only	critical	Integration Platform

Approval requirement

Approval

Approval requirement

Approval

warning

Avoid approval theater

Human oversight only matters when the reviewer sees the evidence, the model basis, the action diff, the expected outcome, and the rollback path.

## Risk register

Risk register

AI Risk Register

The risk register converts the assessment into owned remediation work.

content/deliverables/data/ai-risk-register.sample.json

Synthetic sample AI risk register for a customer-facing AI copilot using retrieval, model routing, tool access, approval workflows, and AI trace logging.

Risks

Open

Critical

Decisions

Roadmap

Controls

Risk	Domain	Severity	Decision	Owner	Status
Retrieval can expose content the user cannot access directly The retrieval layer uses tenant and source filters, but the evidence does not yet prove authorization survives indexing, chunking, semantic retrieval, reranking, and prompt assembly.	RAG and data access	critical	mitigate	Search Platform	open
Agent tool authority can exceed the intended user action Tool access is not yet consistently separated into read, suggest, draft, queue, approve, and execute action classes.	Agentic workflow controls	critical	mitigate	AI Platform Engineering	open
Human approval lacks enough context to be meaningful Approval screens do not always show evidence, target object, before/after diff, model rationale, blast radius, and rollback path.	Oversight	high	mitigate	Product Operations	open
AI traces may store sensitive customer and operational data Prompts, retrieved snippets, model outputs, tool calls, and approval records may contain sensitive information but do not yet have AI-specific classification, retention, and access rules.	Logging and evidence	high	mitigate	Security Engineering	open
Model provider boundary is not expressed clearly enough for buyers The provider contract may be acceptable, but the current buyer-facing language is too scattered to answer procurement questions quickly.	Third-party risk	high	mitigate	Vendor Management	open
Prompt injection and retrieval abuse tests are not release gates AI abuse tests exist as a draft plan but are not enforced as release gates for prompt, retrieval, and tool changes.	Security testing	high	mitigate	Product Security	open
AI incident response is not yet operationalized The incident response process does not yet define AI-specific triggers, evidence preservation, user notification triggers, or trace reconstruction steps.	Operations	medium	mitigate	Security Operations	planned
Sales answers may drift from engineering reality AI security questionnaire answers are not yet controlled through a single evidence pack, creating risk of inconsistent customer-facing claims.	Enterprise review	medium	mitigate	Trust and Security	open

Prove retrieval authorization

Search Platform · 2026-06-15

Enforce agent action classes

AI Platform Engineering · 2026-06-20

Upgrade approval context

Product Operations · 2026-06-25

Classify AI traces

Security Engineering · 2026-06-30

## Evidence pack implications

Evidence pack

Enterprise AI Security Evidence Pack

The evidence pack should become the reusable buyer-facing artifact for procurement, security questionnaires, and trust review.

content/deliverables/data/evidence-pack-controls.sample.json

Synthetic sample evidence pack for answering enterprise AI security review, procurement, legal, and trust-center questions.

implemented

partial

missing

planned

retrieval authorization evidenceagent permission matrix completionAI trace retention and access policybuyer-ready model provider boundary statement

AI system inventory

implemented

Model provider boundary statement

partial

Gateway-only model access

implemented

Authorization-preserving retrieval

partial

Prompt injection and retrieval abuse testing

partial

Agent tool permission policy

partial

Human approval for sensitive actions

partial

AI trace logging

implemented

Buyer question

Is customer data used to train foundation models?

draft · Vendor Management

Buyer question

Can a user receive information through AI that they cannot access directly?

partial · Search Platform

Buyer question

Can the AI system take actions in customer environments?

partial · AI Platform Engineering

Buyer question

Can AI interactions be audited?

implemented · Security Engineering

Evidence

AI System Inventory Record

available · Product Security

Evidence

Model Routing Architecture

available · AI Platform Engineering

Evidence

RAG Authorization Test Plan

needs-validation · Search Platform

Evidence

Agent Tool Permission Matrix

draft · AI Platform Engineering

Evidence

AI Trace Schema

available · Security Engineering

## Recommended remediation

First remediation wave

✓Add authorization-preserving retrieval tests.

✓Complete the Agent Tool Permission Matrix.

✓Define approval context bundles for sensitive actions.

✓Classify AI traces as sensitive evidence.

✓Write the model provider boundary statement.

✓Add AI abuse tests to release gates.

Decision · planned

Next commercial step

Turn this assessment into a two-week remediation sprint focused on buyer blockers: retrieval evidence, agent authority, logging controls, and enterprise questionnaire answers.