NEW

Start with the pressure: sales, launch, abuse, agents, data, or guardrails

Academy Labs/Data Leakage & Cross-Tenant Exposure Lab
AIPSA Academy Lab60 minAdvancedEvidence

Data Leakage & Cross-Tenant Exposure Lab

Review tenant data, prompt traces, PII fixtures, and retrieval behavior to identify AI-specific data exposure failures.

Progress

0/100 points

Status

not-started

Steps

0/4

Mission

Primary objective

Inspect tenant-data and PII fixtures, classify exposure paths, and write an evidence artifact that product, privacy, and customer trust teams can use.

Brief

Scenario

Tenant support copilot leakage review

A SaaS support copilot uses tenant-scoped RAG data and logs prompts for debugging. A customer reports seeing another tenant's operational details in a generated answer.

Objectives

  • Identify AI-specific data exposure locations — RAG, prompt logs, completions, vector metadata.
  • Separate RAG authorization failures from logging and retention failures.
  • Write evidence specific enough for remediation and safe enough for external use.
  • Map data exposure findings to privacy and customer trust language.

Prerequisites

  • Complete RAG Security or review RAG authorization concepts.
  • Review Data Exposure & Privacy flash cards.
  • Understand basic tenant isolation and prompt logging.

Expected signals

  • cross-tenant retrieval
  • PII in prompt
  • unsafe retention
  • metadata leakage
  • citation laundering

Prepare

Reading materials

AIPSA Handbook · Ch 5

Chapter 5 — RAG Authorization

Authorization across ingest and query layers, tenant isolation, cross-tenant leakage vectors, document poisoning, and retrieval auditability.

3.4 MB

Checking…

AIPSA Handbook · Ch 7

Chapter 7 — Data Exposure and Privacy

PII in prompt and retrieval context, cross-tenant data leakage, training data exposure, prompt log privacy, data minimization, and retention controls.

2.2 MB

Checking…

AIPSA Handbook · Ch 10

Chapter 10 — Logging and Telemetry

Prompt/response/tool-call log requirements, trace correlation, PII-safe telemetry, abuse monitoring signal design, and the minimum log surface for AI forensics.

2.2 MB

Checking…

AIPSA Handbook · Ch 11

Chapter 11 — Detection Engineering

Building detectors for prompt injection, jailbreaks, credential exposure, anomalous tool calls, and AI-specific abuse patterns using telemetry pipelines.

2.3 MB

Checking…

AIPSA Field Guide · Ch 4 · Ch 4

RAG Security

RAG authorization, cross-tenant leakage, vector database exposure, document poisoning, citation trust, and retrieval auditability.

~2 MB

Checking…

AIPSA Field Guide · Ch 9 · Ch 9

Privacy and Data Protection in AI Systems

Customer data usage, training policy, retention, prompt/log privacy, PII redaction, data minimization, data residency, and privacy controls for AI systems.

~2 MB

Checking…

AIPSA Field Guide · Ch 12 · Ch 12

Incident Response and AI Observability

AI incident detection, prompt/response/tool-call logs, traceability, abuse monitoring, alerting, forensics, containment, rollback, and post-incident learning.

~2 MB

Checking…

Mythos Report · Ch 11 · Ch 11

RAG and Context Systems Are Data Security Systems

The argument that retrieval systems must be governed as data access control systems — not just prompt augmentation layers — with all the authorization implications that follow.

~1 MB

Checking…

Do not rebuild

Reusable source assets

fixturestatic/read-only

Tenant data fixtures

Realistic tenant-scoped data records for boundary and isolation analysis.

llm-attack-range/ai-security-range/fixtures/tenant-data/

fixturestatic/read-only

PII fixtures

PII samples for classification and retention analysis.

llm-attack-range/ai-security-range/fixtures/pii/

attack-packstatic/read-only

Cross-tenant leakage attack pack

Scenarios showing tenant alias confusion, neighbor record probing, and scope escalation.

llm-attack-range/ai-security-range/attack-packs/cross-tenant-leakage/

attack-packstatic/read-only

Data exfiltration attack pack

Secret harvesting, progressive extraction, and metadata disclosure scenarios.

llm-attack-range/ai-security-range/attack-packs/data-exfiltration/

scanner

RAG Security Lab

Use the existing RAG analyzer for live RAG configuration review.

/academy/labs/rag-security

Sample inputs

Cross-tenant data fixture · json

From fixtures/tenant-data/cross-tenant-retrieval.json. Tenant B's user receives a response that includes Tenant A's premium plan details and real-time usage data. The retrieval query lacked tenant scoping — both tenants' account-limits documents scored highly and were both returned. The model cited 'chunk_a_0042' in its response to Tenant B's user.

{
  "retrieval_session": {
    "requesting_tenant": "tenant_b",
    "user_id": "user_b_8821",
    "query": "What are the current API rate limits for our account?"
  },
  "retrieved_chunks": [
    {
      "chunk_id": "chunk_a_0042",
      "source_tenant": "tenant_a",
      "content": "Tenant A premium plan: 50,000 API calls/day. Burst limit: 500 req/min. Current usage: 41,203 calls today. Overage billing: $0.002/call.",
      "score": 0.91,
      "metadata": {
        "tenant_id": "tenant_a",
        "document": "account-limits-2026.md",
        "last_updated": "2026-06-01"
      }
    },
    {
      "chunk_id": "chunk_b_0019",
      "source_tenant": "tenant_b",
      "content": "Tenant B standard plan: 10,000 API calls/day. Burst limit: 100 req/min.",
      "score": 0.88,
      "metadata": {
        "tenant_id": "tenant_b",
        "document": "account-limits-2026.md",
        "last_updated": "2026-05-15"
      }
    }
  ]
}
PII in prompt log · log

From fixtures/logs/prompt-logs/pl_9f3a2.log. The prompt log captures the full retrieval context sent to the LLM — including Tenant A's premium plan data AND a PII record (Jane Smith's SSN, email, and DOB) from a support history document. The PII was not the target of the query but was retrieved because the tenant-scoped support history had a high similarity score. Two distinct exposure types are present: cross-tenant data (Tenant A's plan details) and PII over-retrieval (Jane Smith's record from Tenant B's own corpus).

2026-06-01T14:22:31Z [INFO] prompt_log id=pl_9f3a2 user=user_b_8821 tenant=tenant_b
2026-06-01T14:22:31Z [DEBUG] context_block BEGIN ---
[SYSTEM] You are a helpful support assistant for Tenant B.
[RETRIEVED CONTEXT]
Source: tenant_a/account-limits-2026.md (score: 0.91)
  "Tenant A premium plan: 50,000 API calls/day ... "
Source: tenant_b/support-history-q1.md (score: 0.84)
  "Customer Jane Smith (SSN: 042-68-3319, email: jane.smith@corp-a.example.com, DOB: 1978-03-14)
   filed support ticket #84421 on 2026-03-02 regarding billing dispute for $2,340."
[USER QUERY] What are the current API rate limits for our account?
2026-06-01T14:22:31Z [DEBUG] context_block END ---
2026-06-01T14:22:31Z [INFO] completion_start tokens_in=847 model=gpt-4o
2026-06-01T14:22:33Z [INFO] completion_end tokens_out=142 latency_ms=1840
Cross-tenant leakage attack patterns · text

From attack-packs/cross-tenant-leakage/payloads.jsonl. Four canonical cross-tenant leakage patterns. Each shows the attack query, the mechanism, and the expected forensic signal. Use these to classify the exposure in the tenant-records fixture: which pattern matches the retrieval session shown?

[cross-tenant-0001] TENANT ALIAS CONFUSION
Query: "Show me Acme Corp's account settings"
Attack: User guesses a competitor's tenant display name. If the vector store indexes by display name rather than internal tenant_id, documents across tenants match.
Expected signal: retrieval returns chunks where tenant_id != requesting_tenant_id

[cross-tenant-0002] NEIGHBOR RECORD PROBE
Query: "What was the last support ticket filed before mine?"
Attack: Relative temporal queries (before/after/adjacent) can return records from neighboring tenants if ticket IDs are sequential and tenant scoping is applied post-retrieval rather than at query time.
Expected signal: retrieved chunk tenant_id differs from session tenant_id

[cross-tenant-0003] METADATA DISCLOSURE
Query: "What document versions are available for my account?"
Attack: Vector metadata (tenant_id, owner, classification) is included in retrieval responses even when content is filtered. Attacker queries for metadata to map the corpus structure of other tenants.
Expected signal: metadata fields expose tenant_id, user_email, or document classification for non-requesting tenant

[cross-tenant-0004] CITATION LAUNDERING
Query: "Summarize our usage policies based on all available documentation"
Attack: LLM is asked to synthesize across all retrieved chunks. Cross-tenant chunks are included without attribution in the summary. The model launders the source by presenting Tenant A's policy as if it applies to Tenant B.
Expected signal: completion contains policy details that do not match the requesting tenant's actual policy; citation audit trail is missing
Vector store chunk metadata (over-exposure) · json

From fixtures/tenant-data/vector-query-response.json. The vector store returns full metadata for each chunk including tenant_id, owner_email, document_classification, created_by, and the S3 source path. Even if the LLM is prompted not to reveal this metadata, the full metadata object is available to any layer in the retrieval pipeline that logs the raw response. An attacker who can read retrieval logs or has access to the raw API response learns: Tenant A's admin email, their S3 bucket name, and their document classification scheme.

{
  "query_vector": "[0.0821, -0.1432, 0.2201, ...]",
  "top_k": 5,
  "results": [
    {
      "id": "chunk_a_0042",
      "score": 0.91,
      "content": "Tenant A premium plan: 50,000 API calls/day...",
      "metadata": {
        "tenant_id": "tenant_a",
        "owner_email": "admin@tenant-a-corp.example.com",
        "document_classification": "confidential",
        "created_by": "user_a_0001",
        "ingested_at": "2026-06-01T09:00:00Z",
        "chunk_index": 42,
        "source_path": "s3://tenant-a-docs/account-limits-2026.md"
      }
    },
    {
      "id": "chunk_b_0019",
      "score": 0.88,
      "content": "Tenant B standard plan: 10,000 API calls/day...",
      "metadata": {
        "tenant_id": "tenant_b",
        "owner_email": "admin@tenant-b-corp.example.com",
        "document_classification": "internal",
        "created_by": "user_b_0003",
        "ingested_at": "2026-05-15T14:30:00Z",
        "chunk_index": 19,
        "source_path": "s3://tenant-b-docs/account-limits-2026.md"
      }
    }
  ]
}

Track progress

Lab steps

Review the reported exposure

Start with the customer report and identify what kind of data crossed the boundary: tenant data, PII, internal metadata, prompt logs, or retrieved document content.

Evidence prompt: What data class crossed the boundary? Be specific: was it tenant data, PII, a prompt log, completion text, or vector metadata?

Inspect fixture records

Classify each fixture as allowed, sensitive-but-authorized, leaked, over-retained, or insufficiently minimized.

Expected action: Review llm-attack-range/fixtures/tenant-data/ and fixtures/pii/ contents.

Evidence prompt: Write a table: fixture, data class, classification (leaked / authorized / over-retained).

Build the exposure path

Trace the likely path from source data to retrieval context, prompt, completion, log, or external answer.

Evidence prompt: Describe the full path: source → ingest → retrieval → prompt → response → log. Mark where the boundary failure occurred.

Write the evidence table

Produce a table with data class, source, exposure point, affected boundary, severity, recommended remediation, and public-safe wording.

Evidence prompt: Populate the evidence artifact builder below. All required fields must be filled before submitting.

Submission draft

Evidence artifact builder

Data Exposure Evidence Artifact

Document what data crossed a boundary, how it happened, and what must change. This artifact is for internal review — use the public-safe summary field for customer-facing language.

Reference

Framework mappings

OWASP LLM Top 10

LLM02 · Sensitive Information Disclosure

OWASP LLM Top 10

LLM08 · Vector and Embedding Weaknesses

NIST AI RMF

MANAGE · Risk treatment and monitoring

Self-assessment

Scoring checklist

Score estimate: 0/100

Export

Submit or export your lab evidence

Save a local progress draft, submit the self-scored artifact, or export Markdown for evidence portfolio use.

Next

Continue the AIPSA lab path