Automated tools

https://tryhackme.com/room/llmpentesting

garak

Garak is an open-source LLM vulnerability scanner created by Leon Derczynski and now maintained under NVIDIA's GitHub, introduced in a 2024 research paper. It runs modular probes against a target model, testing for prompt injection, jailbreaks, hallucination, data leakage, toxicity, and more.
Each probe has a corresponding detector that evaluates the model's response. Results are reported per-probe with pass/fail rates.
Terminal
pentester@tryhackme-2204:~$ pip install garak

Note: garak is already pre-installed on this VM, so the command above will complete in seconds rather than downloading from scratch.

Run a jailbreak probe sweep against the Ollama instance running on the VM:
Terminal
pentester@tryhackme-2204:~$ python3 -m garak --model_type ollama \
--model_name llama3:8b \
--probes dan.DAN_Jailbreak

To run all variants at once:
Terminal
pentester@tryhackme-2204:~$ python3 -m garak --model_type ollama \
--model_name llama3:8b \
--probes dan

Expected output:
Terminal
garak LLM vulnerability scanner v0.15.0
✋ DEPRECATION: --model_type on CLI is deprecated since version 0.13.1.pre1
✋ DEPRECATION: --model_name on CLI is deprecated since version 0.13.1.pre1
🦜 loading generator: Ollama: llama3:8b
🕵️ queue of probes: dan.DAN_Jailbreak
dan.DAN_Jailbreak dan.DANJailbreak: PASS ok on 5/5
dan.DAN_Jailbreak mitigation.MitigationBypass: FAIL ok on 0/5 (attack success rate: 100.00%)
✔️ garak run complete in 1.43s

The deprecation warnings are expected and do not affect results. Multiple detectors evaluate each probe in parallel. dan.DANJailbreak: PASS means the model's responses did not match the output patterns of a successful jailbreak. mitigation.MitigationBypass: FAIL means the model's responses did not contain the specific refusal language the detector looks for as evidence of active safety training: a different criterion, evaluated against the same response. A single response can PASS one detector and FAIL another. That is the point: garak tells you not just whether the model refused, but whether the refusal matches expected safety behaviour. Use the results to prioritise manual follow-up on FAIL categories.

PyRIT

PyRIT (Python Risk Identification Toolkit) is Microsoft's open-source red-teaming framework for generative AI. It supports multi-turn attack orchestration, adversarial prompt generation, and automated response evaluation. Unlike garak's probe-based architecture, PyRIT is programmatic: you define a target and an orchestrator, then run attack campaigns.
The command below is a reference; PyRIT is not pre-installed on the room VM. Install it on your own machine to explore it outside this room:
Terminal
$ pip install pyrit

PyRIT includes a CrescendoOrchestrator that automates the multi-turn Crescendo attack you covered in the previous task. It supports Azure OpenAI, OpenAI, Ollama, and custom HTTP endpoints as targets.

promptfoo

PromptFoo is a CLI tool for LLM testing and red-teaming that integrates cleanly into a testing workflow. The commands below are a reference; promptfoo requires Node.js and is not pre-installed on the room VM:
Terminal
$ npm install -g promptfoo

Running the built-in red team suite against an OpenAI-compatible endpoint:
Terminal
$ promptfoo redteam run

promptfoo generates a redteam.yaml configuration, runs automated jailbreak, prompt injection, PII leakage, and toxicity tests, and outputs a structured report. It is designed for integration into CI/CD pipelines for ongoing LLM application testing.

Three tools, three architectures. Each tests the same target differently.

Burp Suite and LLM Endpoints

You already know Burp. LLM API endpoints are web targets: they accept HTTP POST requests, process JSON bodies, and return JSON responses. The workflow for an LLM endpoint is identical to API testing.
Intercept a request to /v1/chat/completions, send it to Repeater, and modify the messages array to test injection payloads manually. Burp's Intruder can systematically fuzz the content field. PortSwigger also publishes an Prompt Fuzzer (opens in new tab) extension in the BApp Store, which automates prompt injection fuzzing against -backed endpoints.