AI Red Teaming: A Beginner's Guide with garak and PyRIT

What AI red teaming is, how it differs from pentesting, the stages of an engagement and how garak and Microsoft PyRIT help.

Last updated 26 October 2026. Written by Zeeshan Aamiri, Security Architect and Director.

AI red teaming is structured adversarial testing of an AI system. The goal is to find failures in security, safety and responsible AI before attackers or users do. It borrows the mindset of traditional red teaming but changes the targets and the techniques.

How It Differs from Traditional Pentesting

  • Targets include model behaviour, not only code and infrastructure
  • Results are probabilistic: an attack may work on one attempt in ten, so success rates matter
  • It tests safety and fairness harms as well as security flaws
  • Fixes often involve prompts, filters, grounding and architecture rather than a single patch

The Three Categories

A useful way to organise a test is by three categories: security (can an attacker steal data or take actions?), safety (can the system be made to produce harmful content?) and responsible AI (does it behave fairly, accurately and transparently?).

Stages of an Engagement

  1. Scope and objectives: what harm matters to the business?
  2. Threat model: use MITRE ATLAS and the OWASP LLM Top 10 to choose tests.
  3. Attack: manual prompts plus automation.
  4. Score: measure how often attacks work and how severe the results are.
  5. Report: findings, evidence and specific fixes.
  6. Re-test after fixes.

garak: An Open-Source LLM Vulnerability Scanner

garak runs probes against a model, covering prompt injection, jailbreaks, toxicity and data leakage, and produces a report of how often each probe succeeded. It is a good way to get a baseline quickly.

PyRIT: Microsoft's Python Risk Identification Tool

PyRIT is the framework Microsoft's AI Red Team uses to automate attacks. It uses orchestrators to drive conversations, converters to transform prompts, targets to represent the system under test and scorers to judge responses. It scales manual ideas into repeatable experiments.

Automation Is Not Enough

Tools find known weaknesses fast. Creative, context-specific attacks still need a human who understands the application. Good engagements use both.

Our AI Security course teaches garak and PyRIT in dedicated modules, and our AI red teaming service applies the method to your systems.

Ready to Start?

Tell us what you want to learn or secure. We reply within one business day.

Send an Enquiry

Leave your details and we will reply within one business day.

Prefer chat? Message us on WhatsApp or call +91 91647 52266.