Skip to content
  1. Home
  2. AI tools
  3. Promptfoo

Promptfoo

CLI for evaluating and red-teaming prompts, agents and RAG apps

What it is

A command-line tool and library for testing LLM applications. You define prompts, providers and test cases in config, run evals to compare outputs, and can run red-team scans for vulnerabilities; it also installs via Homebrew, pip or npx. For developers who want repeatable tests instead of trial and error.

Good for

  • Compare prompts or models on a test set
  • Add LLM regression tests to CI
  • Red-team an agent for prompt injection

Install or run

npm install -g promptfoo

Command as given in the project README. Read the repository before you run anything on your machine.

What you need

Node.js; API keys for the providers under test

Worth knowingThe company is now part of OpenAI; the README states the project remains open source and MIT licensed.

Sources

The summary is ours, written from the README. Spotted an error? See how we work.

More for security & quality

See the topic →
  • Security Guidance plugin

    NousResearch/hermes-agent

    Warns when written code contains known-dangerous patterns

    ToolGitHub stars: 263k
  • security-guidance

    anthropics/claude-plugins-official

    Security warnings on edits plus LLM diff review

    ToolGitHub stars: 47k
  • Langfuse

    langfuse/langfuse

    Open-source tracing, evals and cost tracking for LLM applications

    ToolGitHub stars: 37k
  • SkillSpector

    NVIDIA/SkillSpector

    Security scanner that checks agent skills before you install them

    ToolGitHub stars: 26k