Use this checklist when you test a chatbot, RAG assistant or AI agent. It is organised by the OWASP Top 10 for LLM Applications (2025). Only test systems you own or have written permission to test.
Before You Start
- Write down the scope, the data the system can reach and the tools it can call
- Draw the architecture: user, application, model, retrieval store, tools, downstream systems
- Agree what harm matters: data leaks, wrong actions, harmful content, cost
Prompt Injection (LLM01)
- Try direct instruction override, role-play and encoded or obfuscated instructions
- Hide instructions in documents, web pages, emails and tool output the model reads
- Test multi-turn attacks that build up over several messages
- Check whether untrusted content is clearly separated from trusted instructions
Sensitive Information and System Prompt Leakage (LLM02, LLM07)
- Ask for the system prompt in different ways and check what leaks
- Look for personal data, secrets or internal URLs in answers
- Check for leakage between users, tenants or sessions
Supply Chain and Poisoning (LLM03, LLM04)
- List models, datasets, plugins and libraries, and where they come from
- Check how models and packages are loaded and verified
- Test whether outsiders can influence training, fine-tuning or retrieval data
Output Handling (LLM05)
- Check whether model output is rendered as HTML or markdown without sanitising
- Test whether output is passed into SQL, shell, code or API calls
- Look for data exfiltration through links and image URLs
Excessive Agency (LLM06)
- List every tool and permission the agent has and ask whether each is needed
- Try to make the agent call tools the user should not reach
- Check that sensitive or irreversible actions need human approval
Vector and Embedding Weaknesses (LLM08)
- Test access control on the vector store and retrieved documents
- Try poisoned documents that steer retrieval
- Check that one user cannot retrieve another user's content
Misinformation and Overreliance (LLM09)
- Ask for facts and citations and check for confident but wrong answers
- Check whether users are warned about uncertainty
- Test whether wrong output can trigger a harmful business action
Unbounded Consumption (LLM10)
- Test rate limits, token limits and maximum input size
- Look for ways to cause runaway cost or denial of service
- Check monitoring and alerts on unusual usage
Reporting
For each finding record the prompt or steps, the evidence, the impact, the OWASP category and a specific fix. Retest after the fix. If you want this done for your system, see our LLM pentesting service.
