DeepEval
Pytest-style evaluation framework for LLM apps and agents
What it is
A Python framework for unit-testing LLM systems in a Pytest-like style. It provides metrics such as G-Eval, task completion, answer relevancy and hallucination, mostly using an LLM as judge, and integrates with common agent frameworks. For developers who want eval tests alongside their normal test suite.
Good for
- Write unit tests for an LLM feature
- Score a RAG pipeline on relevancy
- Gate a release on agent task completion
Install or run
pip install -U deepevalCommand as given in the project README. Read the repository before you run anything on your machine.
What you need
Python 3.9+; an LLM API key for judge-based metrics
Sources
- https://github.com/confident-ai/deepeval (project README, read 10 Nov 2026)
- https://api.github.com/repos/confident-ai/deepeval (GitHub API, read 13 Nov 2026)
The summary is ours, written from the README. Spotted an error? See how we work.