Secure Coding with GenAI

Build, review, test, and ship AI-assisted code safely.

AI-assisted development changes where secure coding work happens.

Security does not begin after code generation. It begins when the developer frames the task, chooses context, grants tool access, accepts dependencies, writes tests, and decides whether generated output is safe enough to merge.

The durable enterprise control is not banning AI coding. The durable control is a paved path where developers can use AI safely, consistently, and measurably.

This course is for developers, AppSec teams, platform engineers, engineering managers, secure SDLC owners, and AI governance teams.

Learners will be able to:

• identify security failure modes introduced by AI-assisted development
• write secure prompts that specify assumptions, constraints, and tests
• review AI-generated code using a security rubric
• define secure coding standards and paved paths
• use shared security components instead of repeated invention
• apply tests, evals, judges, and consensus review safely
• add CI/CD controls for AI-assisted pull requests
• reduce shadow AI with approved tools and model gateways
• produce evidence for security, audit, governance, and enterprise buyers

\pagebreak

# Module 1: Threat Model and Workflow Boundaries

AI-assisted development is not one activity. It is a workflow made of prompts, context, tools, generated code, review decisions, tests, commits, and release gates.

Secure coding with GenAI starts by defining the boundaries of the AI-assisted workflow: what the tool can see, what it can change, what it can run, what it can suggest, and what humans must verify.

• The workflow becomes part of the system.
• Context selection is a security decision.
• Tool permissions define possible impact.
• High-risk tasks need stronger prompts, tighter permissions, review, tests, and evidence.
• A safer workflow frames the task, constrains output, limits context, reviews with a rubric, and preserves evidence.

Classify an AI-assisted password reset change by risk level. List the controls required before merge.

\pagebreak

# Module 2: Secure Prompting and Code Constraints

Prompting is now part of secure software development.

A secure coding prompt should define the task, constraints, assumptions, banned patterns, approved components, test expectations, and review criteria before implementation begins.

• A prompt can act as a specification and security boundary.
• High-risk prompts should ask for assumptions first.
• Approved shared components should be preferred over generated security logic.
• Unsafe patterns should be explicitly banned where relevant.
• Tests should often come before implementation.
• Make it secure is not a requirement. It is a wish.

Rewrite a vague admin delete-user prompt into a secure coding prompt with assumptions, authorization constraints, audit logging, safe errors, tests, and review notes.

\pagebreak

# Module 3: Reviewing AI-Generated Code

AI-generated code can be readable, idiomatic, and wrong.

Readable code is not necessarily safe code. AI-generated output should be reviewed against explicit security expectations, not against how polished it looks.

• Review the diff, the prompt, and the assumptions.
• Require pull request context for AI-assisted portions.
• Apply a consistent rubric.
• Watch for missing authorization, missing tenant filters, superficial validation, sensitive logs, unnecessary dependencies, and happy-path tests.
• Use AI-assisted review as one signal, not as a decision-maker.
• Large generated changes should be split before review.

Write review comments for a generated admin export endpoint that checks login, queries by URL account ID, returns all fields, and logs the full request body on error.

\pagebreak

# Module 4: Secrets, Identity, and Permissions

AI-assisted coding becomes dangerous when generated code touches secrets, identity, authorization, or privileged actions without explicit constraints.

Treat every prompt, tool context, terminal transcript, model trace, and generated log statement as a potential path for secret exposure.

• Secrets are not only passwords.
• Do not paste real secrets into unapproved tools.
• Use placeholders in prompts.
• Generated code must not create secret sprawl.
• Authentication is not authorization.
• Agentic tools can indirectly access local credentials.
• Log decisions and identifiers, not secrets or sensitive content.

Design controls for an AI coding agent that can read a repository, access a terminal, run cloud CLI commands, and inspect environment files.

\pagebreak

# Module 5: Dependencies, Packages, and Provenance

AI coding assistants can introduce packages, examples, APIs, and build steps faster than teams can review them.

Every new dependency expands the trusted computing base. AI-generated package suggestions must be reviewed before they become part of the product.

• Generated packages may be hallucinated, unmaintained, risky, or unnecessary.
• Prefer existing internal components, existing dependencies, and standard libraries before adding packages.
• Lockfile changes are security-relevant.
• Build scripts and CI changes are high-risk.
• Preserve provenance for important generated decisions.

Review a pull request where an AI assistant adds a new validation package even though the project already has a shared schema library.

\pagebreak

# Module 6: Retrieval, Context, and Repo Assistants

AI coding tools become more useful when they can read repositories, tickets, documentation, logs, and internal knowledge.

They also become more dangerous.

Context is not neutral. Every file, ticket, log, document, and trace added to an AI coding session changes the security boundary.

• Context should be classified by sensitivity.
• Retrieval should be narrow.
• Restricted content should be excluded unless explicitly approved.
• Approved standards should rank above random examples.
• Untrusted issue text and external docs should be treated as data, not instruction.
• Evidence should show which context was used and which actions were taken.

Design a policy for a repository-aware assistant including allowed repositories, excluded paths, allowed actions, approvals, context classification, and evidence.

\pagebreak

# Module 7: Agents, Tools, and Autonomy

AI coding tools become more powerful when they can act.

Every tool an agent can use becomes part of the security boundary. Tool access should be scoped, logged, and approved according to risk.

• Use least autonomy by default.
• Start with explain-only or read-only modes where possible.
• Require approval for commands, package installation, CI/CD edits, infrastructure changes, and sensitive code changes.
• Human approval must show the actual command, diff, or permission change.
• Preserve agent evidence including files read, files changed, commands proposed, commands approved, and tests run.
• Model gateways provide policy, visibility, and control points.

Define a policy for AI agents that fix failing tests automatically.

\pagebreak

# Module 8: Tests, Evals, and Failure Modes

AI can generate tests, but tests generated by the same workflow can share the same blind spots.

Generated tests are useful, but they are not proof of safety unless they cover failure modes, abuse cases, and security boundaries.

• Ask for test plans before implementation for high-risk work.
• Failure modes identify how the feature can behave unsafely.
• AI features may need evals for prompt injection, retrieval boundaries, tool behavior, and policy adherence.
• Judge models are useful signals, not authorities.
• Deterministic checks should run before model judgment.
• Consensus is not proof.

Create a security test and eval plan for a support assistant that retrieves account documentation and can create support tickets.

\pagebreak

# Module 9: CI/CD Gates, Telemetry, and Release

Secure AI-assisted development requires controls after the prompt and before production.

The secure path should be visible in CI/CD. If AI-assisted code changes security-sensitive behavior, the pipeline should require tests, scans, review, and evidence.

• Pull request signals should trigger stronger gates.
• Use static checks, secret scanning, dependency review, security tests, AI-specific checks, and human approval.
• Telemetry should improve paved paths, not become surveillance theater.
• Shadow AI response should provide an approved alternative path.
• Release evidence should be understandable outside engineering.

Design a CI/CD gate for AI-assisted pull requests that change authorization logic.

\pagebreak

# Module 10: Capstone Operating Model

The final goal is not one secure prompt or one secure pull request.

The goal is an operating model for AI-assisted development that developers can actually use.

• approved AI coding tools
• allowed and restricted data classes
• model gateway or approved access path
• secure system instructions
• secure prompt templates
• shared security components
• agent tool permission policy
• dependency review policy
• AI-assisted pull request checklist
• test and eval requirements
• CI/CD gates
• telemetry and shadow AI discovery
• exception process
• evidence and reporting workflow

Design the secure AI-assisted development operating model for a SaaS company that uses AI coding assistants, repository-aware chat, and agentic coding tools.

Your answer should include approved use cases, prohibited use cases, data classification rules, secure prompt template, shared components, high-risk review checklist, agent tool policy, CI/CD gates, telemetry plan, and evidence artifacts.

\pagebreak

# Appendix A: Quick Checklists

• State the task clearly.
• Name the context and trust boundary.
• Ask for assumptions before implementation.
• Require approved shared components.
• Specify authentication and authorization expectations.
• Specify tenant and data boundary expectations.
• Specify input validation expectations.
• Specify output handling expectations.
• Specify logging restrictions.
• Ban unsafe patterns relevant to the task.
• Require tests for negative cases.
• Require dependency justification.
• Require residual risk notes.
• Instruct the assistant to stop when context is unclear.

• Is the AI-assisted portion identified?
• Is the task risk level clear?
• Are assumptions listed?
• Are changed trust boundaries identified?
• Is authentication handled correctly?
• Is authorization handled correctly?
• Are tenant boundaries preserved?
• Is untrusted input validated?
• Is output safely handled?
• Are secrets absent from code, logs, tests, and traces?
• Are errors safe?
• Are dependencies justified?
• Are negative tests included?
• Is evidence preserved?

• Which files may the agent read?
• Which files are forbidden?
• Which files may the agent edit?
• Which commands are allowed?
• Does command execution require approval?
• Can it install packages?
• Can it access secrets?
• Can it open pull requests?
• Can it merge or deploy?
• Are actions logged?
• Are stopping conditions defined?

\pagebreak

# Appendix B: Sample Prompt Templates

Task: Describe the feature or change.

Context: List the relevant files, framework, service, and trust boundary.

Security constraints:

• Use existing approved components where possible.
• Enforce authentication and authorization.
• Validate all untrusted input.
• Preserve tenant, account, workspace, or organization boundaries.
• Do not log secrets, tokens, credentials, prompts containing sensitive data, or retrieved private context.
• Use safe error messages.
• Do not introduce new dependencies unless necessary and justified.
• Avoid unsafe patterns such as dynamic code execution, raw query construction, and custom cryptography.

Expected output:

1. 1List assumptions.
2. 2List security-sensitive decisions.
3. 3List failure modes.
4. 4Propose tests before implementation.
5. 5Implement the smallest safe change.
6. 6Explain residual risks and review notes.

Create a release evidence summary for this AI-assisted change.

Include:

• change summary
• AI-assisted portions
• affected trust boundaries
• assumptions
• tests run
• evals run, if relevant
• static checks run
• dependency review outcome
• security review outcome
• residual risks
• release decision
• follow-up items

\pagebreak

# Final Message

Secure Coding with GenAI is not about perfect prompts.

It is about turning AI-assisted development into a governed engineering system:

• developers get speed
• security gets control
• platform teams provide paved paths
• CI/CD enforces repeatable gates
• model gateways provide visibility
• evidence supports trust

Do not make every developer fight AI security from scratch. Give them approved tools, safe defaults, shared components, useful automation, and clear evidence paths.