Outcomes
- Explain the AI and LLM attack surface and the OWASP Top 10 for LLM applications
- Perform prompt injection, jailbreaks, system-prompt leakage and data-exfiltration attacks in a lab
- Run automated AI red teaming with garak and PyRIT and report findings
- Implement defensive controls: hardened metaprompts, grounding, Azure AI Content Safety and Prompt Shields
- Secure Azure AI services with managed identity, private endpoints, RBAC, Key Vault, customer-managed keys and virtual networks
- Apply key rotation, monitoring, abuse monitoring and Responsible AI governance
- Deliver a hardened, tested AI deployment and a residual-risk report
Fee: ₹20,000 per learner. Practice lab: ₹2,000 per month, all material included. No high-end laptop or PC needed. Course details
Part 1: Understand and Attack AI
1Introduction to AI Security and Architecture Layers
Understand what is different about securing AI systems and map where attackers can strike.
You will learn:
- What AI security is and why it matters, with the CIA triad applied to AI and LLMs
- The layers of an AI application: model, application, data, orchestration and infrastructure
- The attack surface and the OWASP Top 10 for LLM Applications
- Key vocabulary: prompt, token, context window, system prompt, RAG and agent
Lab: Open a sample chatbot or RAG app, diagram its layers and mark where an attacker could inject, leak or manipulate. Deliverable: a one-page attack-surface map.
2Prompt Injection and Jailbreaking
See how instructions hidden in user input or in content the model reads can take over its behaviour.
You will learn:
- Direct and indirect prompt injection, including poisoned web pages, documents and emails
- Jailbreak techniques: role-play, instruction override, encoding and obfuscation
- Why guardrails fail
- Real-world impact examples
Lab: Play Lakera's Gandalf challenge and use prompt injection to extract the secret level by level. Record which technique beat each level.
3Model Manipulation and Data Exfiltration
Learn how models leak what they were told to protect and how attackers move data out.
You will learn:
- Training-data poisoning, model inversion and membership inference at concept level
- Sensitive-data disclosure and system-prompt leakage
- Data exfiltration through indirect injection, such as malicious markdown image URLs
- Impact on privacy and compliance
Lab: Against a demo chatbot, craft prompts that reveal its hidden instructions, then show how an injected instruction could exfiltrate data.
4AI Overreliance and Introduction to AI Red Teaming
Understand the risk of trusting model output and how red teaming tests AI systems.
You will learn:
- Overreliance, hallucination and insecure output handling
- What AI red teaming is: security, safety and responsible-AI testing
- How AI red teaming differs from traditional penetration testing
- Classifying and communicating AI risk
Lab: Prompt a chatbot for facts and citations, document a confident but wrong answer, classify the risk and propose a mitigation.
5AI Red Teaming Methodology and Planning
Plan an AI engagement like a professional: scope, threats and test cases.
You will learn:
- Scoping, objectives and rules of engagement
- Threat-modelling an AI system
- MITRE ATLAS for adversarial AI tactics
- Mapping test cases to the OWASP LLM Top 10
Lab: Complete a threat-model and test-plan worksheet for the chatbot from module 1: assets, threats and attack prompts to try.
6Automated Red Teaming with garak
Use an open-source LLM vulnerability scanner to test models at scale.
You will learn:
- What garak is and how probes and detectors work
- Probes for prompt injection, jailbreaks, toxicity and data leakage
- Running scans against local and hosted models
- Reading and triaging the report
Lab: Install garak, scan a model with the prompt-injection probes and review the hit-rate report.
7Microsoft PyRIT: Python Risk Identification Tool
Automate attacks and scoring with the framework Microsoft's AI Red Team uses.
You will learn:
- PyRIT overview and architecture
- Orchestrators, targets, converters and scorers
- Automating multi-turn attacks
- Scoring and measuring responses at scale
Lab: Install PyRIT, run a prompt-sending orchestrator against a target model and auto-score the responses for harmful content.
8Content-Filter Bypass and Reporting
Understand why filters fail and how to report an AI finding so it gets fixed.
You will learn:
- Techniques for bypassing guardrails and content filters, and why they work
- Structuring an AI red-team report: finding, impact, evidence and fix
- Moving from attacker mindset to defender mindset
- Communicating with developers and leadership
Lab: Attempt one guardrail bypass on a demo model, then write a one-page finding with reproduction steps and a recommended mitigation.
Part 2: Defend and Govern AI
9AI Security Controls
Start defending: the controls that reduce the risks you just exploited.
You will learn:
- Reviewing open-source AI libraries securely
- Content filters, metaprompts and system prompts, and grounding with RAG
- AI data security and application-security best practices
- Layering controls instead of relying on one
Lab: Write a system prompt with explicit guardrails and re-run the earlier injection prompts to show the difference.
10Azure AI Content Safety
Use managed safety services to detect harmful content and attacks.
You will learn:
- Categories (hate, sexual, violence, self-harm) and severity levels
- Prompt Shields for jailbreak detection, groundedness detection and protected-material checks
- Risk and safety evaluation of text and image content
- Tuning thresholds for your use case
Lab: In Azure AI Content Safety Studio, test text and image moderation and Prompt Shields, tune severity thresholds and observe the results.
11Secure Azure AI Services: Authentication and Network
Control who and what can reach your AI endpoints.
You will learn:
- API keys versus Microsoft Entra ID and managed identity
- Private endpoints, firewall rules and disabling public access
- End-to-end management of Azure AI Services security
- Common authentication mistakes
Lab: Configure a managed identity and a private endpoint for an Azure AI resource and disable key-based public access.
12Security Recommendations for AI Workloads
Protect AI resources, models, access and execution as one system.
You will learn:
- Securing AI resources, models, access and execution
- Least-privilege RBAC and secrets management
- Guarding the model supply chain and the deployment pipeline
- Reference security baselines for AI workloads
Lab: Apply least-privilege RBAC roles to an Azure OpenAI deployment and store its secrets in Azure Key Vault instead of code.
13Azure AI Security Features, Part 1
Protect data in transit and manage credentials safely.
You will learn:
- Transport Layer Security for AI endpoints
- Authentication options and key rotation
- Using environment variables to keep secrets out of code
- Operational habits that prevent key leaks
Lab: Rotate an Azure AI service key, move credentials to environment variables or Key Vault and confirm the app still works.
14Azure AI Security Features, Part 2
Add stronger data protection and isolation.
You will learn:
- Customer-managed keys and virtual networks
- Data loss prevention and Customer Lockbox
- Bring Your Own Storage
- Choosing the right isolation for your data sensitivity
Lab: Configure customer-managed keys and virtual-network integration on an AI or storage resource and verify encryption and isolation.
15Monitoring, Abuse Monitoring and Responsible AI
See what is happening in production and govern AI responsibly.
You will learn:
- Monitoring Azure OpenAI Service
- Building generative-AI workloads with the Well-Architected Framework
- Abuse monitoring and content-filtering systems
- Responsible AI principles and the Microsoft AI Red Team
Lab: Enable diagnostic logging and metrics for an Azure OpenAI resource and create an alert on throttling or content-filter triggers.
16Trustworthy AI, Data Privacy and Capstone
Bring everything together in an end-to-end hardened and tested deployment.
You will learn:
- Microsoft Trustworthy AI: security, privacy and safety
- What data Azure OpenAI processes, how it is processed and how to verify abuse-monitoring storage settings
- Residual-risk assessment and remediation planning
- Presenting results to technical and business audiences
Lab: Capstone: re-run your earlier attacks (garak and Gandalf-style prompts) against your hardened deployment, then document the residual risk and a remediation plan.