Five open source tools compared for testing LLM apps against prompt injection

Developers building LLM-powered applications such as support bots, agents, or RAG pipelines face a critical question: whether their systems have been properly tested for prompt injection vulnerabilities. Five open source tools — promptfoo, garak, Giskard, PyRIT, and sentinel-scan-cli — are commonly used for this purpose, but they target different layers and serve distinct use cases. A key distinction separates app-layer tools like promptfoo, Giskard, and sentinel-scan-cli, which test how an application's prompts, guardrails, and logic resist attacks, from model-layer tools like garak, which probe the underlying model's raw vulnerabilities. Promptfoo is highlighted as the most broadly adopted app-layer option, offering 50-plus red-team plugins and compliance mappings to OWASP LLM Top 10, NIST, and MITRE ATLAS, while garak, maintained by NVIDIA with over 8,100 GitHub stars, is better suited for evaluating base models. Notably, Microsoft archived PyRIT on GitHub in March 2026, making it read-only with no further development, so new users are advised to consider alternatives.