Promptfoo
CLI for evaluating and red-teaming prompts, agents and RAG apps
What it is
A command-line tool and library for testing LLM applications. You define prompts, providers and test cases in config, run evals to compare outputs, and can run red-team scans for vulnerabilities; it also installs via Homebrew, pip or npx. For developers who want repeatable tests instead of trial and error.
Good for
- Compare prompts or models on a test set
- Add LLM regression tests to CI
- Red-team an agent for prompt injection
Install or run
npm install -g promptfooCommand as given in the project README. Read the repository before you run anything on your machine.
What you need
Node.js; API keys for the providers under test
Sources
- https://github.com/promptfoo/promptfoo (project README, read 10 Nov 2026)
- https://api.github.com/repos/promptfoo/promptfoo (GitHub API, read 13 Nov 2026)
The summary is ours, written from the README. Spotted an error? See how we work.