Does garak work with models that require an API key?

Yes. For providers like OpenAI or Replicate, you set the appropriate environment variable (e.g., OPENAI_API_KEY) before running the scan.

Can I use garak to scan a model running on my local machine?

Absolutely. Use `--target_type huggingface` for local Transformers models or `--target_type ggml` for llama.cpp models, pointing to the local path or name.

How does garak's probe system work technically?

Probes are plugins that send crafted prompts. Each probe has associated 'detectors' that analyze the model's output to classify it as pass/fail. Results are logged per probe/detector pair.

Can I add my own custom probes or detectors?

Yes. You can write new Python classes following the base probe/detector interfaces and place them in the appropriate directories. The plugin system is designed for extension.

Where are the results of a garak scan stored?

A detailed JSONL report file is created for each run, and a general log is maintained in `garak.log`. The report includes per-attempt statuses and failure rates.

Directory/Search/AI Safety/garak

garak

garak is a command-line tool for probing Large Language Models (LLMs) to find vulnerabilities like hallucination, data leakage, prompt injection, misinformation, toxicity, and jailbreaks. It uses a plugin-based system of probes, detectors, and buffs to automate red-teaming assessments. The tool supports a wide range of LLM backends including Hugging Face Hub, OpenAI API, Replicate, AWS Bedrock, and local GGUF models. It combines static, dynamic, and adaptive attack methods to stress-test models and generate detailed failure reports. Designed for security researchers and AI developers, it helps identify weaknesses before deployment.

Visit website

Added on

Sep 22, 2026

LLM Vulnerability ScannerMulti-Model SupportPrompt Injection TestingToxicity & Jailbreak ProbesCLI ToolOpen-Source (Apache-2.0)

Product information

Everything worth knowing about garak

The complete picture — from everyday features to the technical detail builders and IT folks dig for.

Core features

What it actually does

Multi-Model Support

Probe any LLM via Hugging Face Hub, OpenAI, AWS Bedrock, Replicate, Cohere, Groq, REST APIs, and local GGUF models.

Comprehensive Vulnerability Probing

Automated checks for hallucination, data leakage, prompt injection, misinformation, toxicity, jailbreaks, and dozens of other weakness types.

Extensible Plugin Architecture

Use built-in probes and detectors or write your own to test custom scenarios and model behaviors.

Command-Line Interface

Simple CLI to scan models, specify probe sets, and generate detailed JSONL reports for analysis.

Technical capabilities

Static, Dynamic & Adaptive Probing

Combines fixed probes with automated attack generation (atkgen) that interacts with the target model in real-time.

Modular Harness System

Executes probes against target models through a standardized harness, managing generation, buffering, and result logging.

Must watch videos

GARAK : An Opensource LLM Vulnerability Scanner to secure your AI web apps

41:24

Tutorial41:24

GARAK : An Opensource LLM Vulnerability Scanner to secure your AI web apps

### This video will walk you through: - Installing and setting up Garak quickly. - Using Garak to run against your API - Customizing the Requests as per your Rest API calls - Using garak to scan against specific vulnerabilties/prompt injections - Running scans on your models and interpreting results. Resources & Links: - Garak : https://github.com/NVIDIA/garak - Documentation : https://docs.garak.ai/garak/ - Rest Generator Reference: https://reference.garak.ai/en/latest/garak.generators.rest.html #AISecurity #LLMVulnerabilities #GarakTutorial #AICybersecurity #MachineLearning #OpenSourceAI #PromptEngineering #AIethics #AI #opensource #SecureAI

LLM Vulnerability Scanning with garak. Tutorial: Test your own chat bots!

19:13

Tutorial19:13

LLM Vulnerability Scanning with garak. Tutorial: Test your own chat bots!

In this video we cover basic usage of garak, an LLM vulnerability scanner. In particular we explore how to test your own REST based LLM applications. This can be done by using the built-in REST generator and configure it to work with the target application. (volume of video after uploading to YoutTube is not super loud, so please adjust accordingly) Garak on Github: https://github.com/leondz/garak/ The code for the demo chat application we are testing is here: wuzzi-chat: https://github.com/wunderwuzzi23/wuzzi-chat Quick Basic Garak Intro 0:00 REST Generators 1:21 Exploring the demo test target Wuzzi Chat (powered by groq and Llama3) 1:53 Garak: Basic command line usage 3:10 Reverse engineering the HTTP requests to create the correct garak configuration 5:13 Running garak using the custom configuration file for the REST generator 14:15 Exploring the results 16:30 Disclaimer: Only test applications and systems that you are authorized to test, to stay fully local you can target endpoints via ollama for instance.

Quick overview of Garak - a free LLM vulnerability scanner

12:14

Tutorial12:14

Quick overview of Garak - a free LLM vulnerability scanner

💡 Ready to master AI security? Spots fill fast—save your seat now! 👉 https://live.haxorplus.com ☕️ Enjoying the content? Support my work with a coffee: 👉 https://buymeacoffee.com/martinvoelk 📚 Want to learn at your own pace? Check out my self-paced pentest courses: 👉 https://penetrationtesting.us/training/

Who uses it

Real use cases, no hype

Pre-deployment LLM Red Teaming

Scan your LLM for vulnerabilities like prompt injection, toxicity, and hallucination before it goes live.

Model Security Auditing

Generate detailed reports on a model's resilience to adversarial attacks for compliance or safety reviews.

API Endpoint Hardening

Test commercial API endpoints (OpenAI, Bedrock) for weaknesses using a suite of automated probes.

AI Safety Research

Benchmark and compare the safety of different open-source models against standard attack frameworks.

Custom Vulnerability Development

Build and integrate custom probes to test for specific, domain-relevant failure modes in your LLM application.

What's great

Open-source and free, ideal for budget-conscious teams.
Supports a wide range of LLMs (HuggingFace, OpenAI, Replicate, REST, etc.).
Comprehensive probe library covers hallucination, prompt injection, toxicity, and more.
Active community and NVIDIA-backed development.

Technical strengths