SecEng TraceAI Forensics

by aisecurity.llc › privacy-first AI security engineering

Fixture-driven · ACME Corp repo demo

14 signalsVS Code · Chrome · RAG

Assessment Result

Fixture-driven ACME repo

84/100

Show the VS Code extension as a live scorecard, not a static screenshot.

This route uses the ACME Corp fixture to present the repository tree, the security exposure mix, the redaction and injection findings, and the Chrome side panel with the same confidence and hierarchy you’d expect from a polished benchmark result.

AI detected

Yes

Security score

62/100

Signals

112

Top strengths

4 AI stacks surfaced
14 signals normalized
15 repo paths mapped

Priority gaps

Prompt injection through support inbox transcripts
Tool authority exceeds the approval boundary
RAG responses are not yet tenant-isolated by policy

Open full demo Back to product page

Trace Coverage

AIPSA-style scorecard view

Coverage

173/199

Risk count

Maturity

Managed AI Engineering

Fetch budget

87%

Signal trend

Key context

Signals captured14
Repo paths15
Latest scan5/21/2026

VS Code Extension AI Goggles (Chrome)RAG Boundary Lens Evidence Controls Capabilities

VS Code Extension

Repo AI Forensics Dashboard

Full workspace view — AI surface explorer, live signals, risk analysis, and controls coverage.

Editor-native surface

AISecurity Studio

aisecurity.llc › SecEng Trace › AI Forensics Module

VS Code Extension

route.ts

rag-boundary.tsx

dashboard.tsx

policies.ts

vector-store.ts

acme-corp/acme-assistant-platformFixture DemoLast scan: May 21, 2026, 09:20 AM

AI Detected

Yes✓

112 signals found

Adoption Score

84/100

Frameworks, tools, RAG active

+6 vs last scan

Security Score

62/100

Policy gaps present

-4 vs last scan

Maturity

Managed AI Engineering

Operationalized

Risks

2 high, 2 medium, 2 low

AI Stack Overview

6 technologies detected

OpenAILangChainNext.jsVercel AI SDKQdrantPromptfoo

Current scanPrevious

Signal Breakdown

112 total signals

AI Frameworks

28(25%)

Tools & Agents

24(21%)

RAG / Vector Stores

18(16%)

Models / Artifacts

16(14%)

Routes & Integrations

12(11%)

Controls & Governance

14(13%)

SecEng RAG Test Harness

RAG Boundary Lens

Boundary planning, testcase generation, and evidence classification rendered from the same public-safe trace fixture.

RAG detectedClaim-ready preview

Boundary score

72/100

RAG detected

Yes

Affected paths

Top tests

classifyRagEvidence

Evidence scorecard

fixture-driven

AuthZ pass

Pass

green

Retrieval gates are mostly aligned.

Context leaks

green

No leak-shaped signals surfaced.

Policy violations

amber

Policy language needs stronger enforcement.

Poisoned chunks

green

No poisoned chunk patterns detected.

PII / secret hits

amber

Redaction surfaced one or more hits.

Source provenance

amber

Source attribution and retrieval lineage need follow-up.

Missing boundaries

What still needs to be enforced

Tenant-scoped retrieval authorizationChunk provenance taggingPoisoned context quarantine

Top 3 tests

Highest-priority harness checks

1Tenant boundary enforcement on retrieval

2Provenance-preserving answer assembly

3PII and secret bleed guardrail

Pipeline map

planRagBoundaries → generateRagTestcases → classifyRagEvidence

Surface inventory

3 RAG paths mapped

done

Boundary planning

72/100 boundary score

done

Testcase generation

3 top tests queued

ready

Evidence classification

3 control paths found

ready

Harness export

5 config files ready

ready

Suggested tests

4 items

Cross-tenant namespace escape regression

Poisoned chunk provenance rejection

Context leak after redaction and rerank

Prompt injection embedded in retrieved documents

Controls found

3 items

packages/governance/policies.ts

docs/ai/trace-runbook.md

apps/web/app/api/assistant/route.ts

Affected paths

3 items

packages/rag/index.ts

packages/rag/vector-store.ts

apps/web/app/api/assistant/stream.ts

seceng-rag / fixture-driven preview / claim-ready

• seceng-rag/seceng-rag.config.json

• seceng-rag/identities.json

• seceng-rag/documents.json

• seceng-rag/tests.json

The lens is public-safe and directional. It uses job-description intelligence and trace fixture signals to show where RAG boundaries need reinforcement, without exposing raw documents or private payloads.

Top Risks

Prompt injection through support inbox transcripts

high

Tool authority exceeds the approval boundary

high

RAG responses are not yet tenant-isolated by policy

medium

Controls Coverage

Evaluations / Testing80%

Guardrails75%

Redaction / Secrets70%

Approval Gates60%

Audit Logging50%

Rate Limiting60%

Scan Trend (30 Days)

Confidence-weighted

Apr 24May 2May 10Now

Adoption ScoreSecurity ScoreRisk Count

Recently Detected Signals

4 visible

TypePathSignalConf.Detected

Psupport.mdApproval Template94%1m ago

Tcalendar-create.tsApproval Required91%2m ago

Rroute.tsAssistant Route97%3m ago

Cpolicies.tsPolicy Gate93%5m ago

Fetch Budget

Files Planned75 / 100

Files Analyzed65

Bytes Read28.4 MB / 51 MB

Requests Used42 / 80

Large Artifacts Skipped3

Budget87% used

main*·⚠ 2·AI Forensics: ReadyLn 42, Col 1·Spaces: 2·UTF-8·LF·TypeScript

Chrome Extension

AI Goggles — GitHub Side Panel

Browser-native AI intelligence. Surfaces security signals directly in the GitHub repository view.

Browser-native surface

github.com/acme-corp/acme-assistant-platform

Chrome Extension – AI Goggles Side Panel (on GitHub)

acme-corp/acme-assistant-platformPublic

0 0

Code

Issues12

Pull requests4

Actions

Security

⎇main

Go to file

Signal indicatorsInjectionPromptToolRAGRouteControl

.github

apps

web

app

(chat)

page.tsx

api

assistant

route.ts

stream.ts

components

chat

chat-panel.tsx

packages

prompt-templates

onboarding.md

support.md

tools

slack-search.ts

calendar-create.ts

rag

vector-store.ts

index.ts

model-registry

models.ts

governance

policies.ts

docs

trace-runbook.md

README.md

ACME Corp Assistant Platform — AI-powered support and workflow automation. Built on OpenAI, LangChain, and Qdrant. Includes prompt injection mitigations and RAG safety controls.

AI Goggles

by aisecurity.llc

Product Surface

Repository Summary

acme-corp/acme-assistant-platform

public repo · fixture-driven trace context · 15 paths

AI DetectedYes

Adoption

84/100

Security

62/100

Risks

Signal trendApr 24 → Now

Signals by category

AI Stack X-Ray12

Prompt Surfaces8

Tool Surfaces12

RAG / Vector Stores12

Model Artifacts8

AI Routes4

Controls16

Risks7

SecEng RAG Test Harness

RAG Boundary Lens

Boundary planning, testcase generation, and evidence classification rendered from the same public-safe trace fixture.

RAG detectedClaim-ready preview

Boundary score

72/100

RAG detected

Yes

Affected paths

Top tests

AuthZ pass

Pass

green

Retrieval gates are mostly aligned.

Context leaks

green

No leak-shaped signals surfaced.

Policy violations

amber

Policy language needs stronger enforcement.

Pipeline snapshot

Surface inventoryBoundary planningTestcase generationEvidence classificationHarness export

Suggested tests

Cross-tenant namespace escape regressionPoisoned chunk provenance rejectionContext leak after redaction and rerank

Controls found

packages/governance/policies.tsdocs/ai/trace-runbook.mdapps/web/app/api/assistant/route.ts

Affected paths

packages/rag/index.tspackages/rag/vector-store.ts

Missing boundaries

Priority gaps

Tenant-scoped retrieval authorizationChunk provenance taggingPoisoned context quarantine

Top tests

Harness checks

1Tenant boundary enforcement on retrieval

2Provenance-preserving answer assembly

• seceng-rag/seceng-rag.config.json

• seceng-rag/identities.json

• seceng-rag/documents.json

• seceng-rag/tests.json

Top Risks

Prompt injection through support inbox transcriptshigh

Tool authority exceeds the approval boundaryhigh

RAG responses are not yet tenant-isolated by policymedium

SecEng RAG Test Harness

Boundary planning and testcase generation

The same fixture now drives a compact boundary lens in Chrome and a fuller planning view in the editor, so the public demo stays close to the actual product flow.

planRagBoundaries / generateRagTestcases / classifyRagEvidence

SecEng RAG Test Harness

RAG Boundary Lens

Boundary planning, testcase generation, and evidence classification rendered from the same public-safe trace fixture.

RAG detectedClaim-ready preview

Boundary score

72/100

RAG detected

Yes

Affected paths

Top tests

classifyRagEvidence

Evidence scorecard

fixture-driven

AuthZ pass

Pass

green

Retrieval gates are mostly aligned.

Context leaks

green

No leak-shaped signals surfaced.

Policy violations

amber

Policy language needs stronger enforcement.

Poisoned chunks

green

No poisoned chunk patterns detected.

PII / secret hits

amber

Redaction surfaced one or more hits.

Source provenance

amber

Source attribution and retrieval lineage need follow-up.

Missing boundaries

What still needs to be enforced

Tenant-scoped retrieval authorizationChunk provenance taggingPoisoned context quarantine

Top 3 tests

Highest-priority harness checks

1Tenant boundary enforcement on retrieval

2Provenance-preserving answer assembly

3PII and secret bleed guardrail

Pipeline map

planRagBoundaries → generateRagTestcases → classifyRagEvidence

Surface inventory

3 RAG paths mapped

done

Boundary planning

72/100 boundary score

done

Testcase generation

3 top tests queued

ready

Evidence classification

3 control paths found

ready

Harness export

5 config files ready

ready

Suggested tests

4 items

Cross-tenant namespace escape regression

Poisoned chunk provenance rejection

Context leak after redaction and rerank

Prompt injection embedded in retrieved documents

Controls found

3 items

packages/governance/policies.ts

docs/ai/trace-runbook.md

apps/web/app/api/assistant/route.ts

Affected paths

3 items

packages/rag/index.ts

packages/rag/vector-store.ts

apps/web/app/api/assistant/stream.ts

seceng-rag / fixture-driven preview / claim-ready

• seceng-rag/seceng-rag.config.json

• seceng-rag/identities.json

• seceng-rag/documents.json

• seceng-rag/tests.json

Comparison snapshot

What changed in this fixture

Security delta

Adoption delta

Risk delta

-1

New prompts

packages/ai/prompt-templates/support.md

New tools

packages/ai/tools/calendar-create.ts

Operating controls

Policy and governance backbone

packages/governance/policies.tsControlled

docs/ai/trace-runbook.mdControlled

apps/web/app/api/assistant/route.tsControlled

packages/workflows/routing.tsControlled