llama.cpp
C/C++ LLM inference engine with CLI and OpenAI-compatible server
What it is
A C/C++ inference engine for running language models on CPUs and GPUs. It includes a command-line client and a server with an OpenAI-compatible REST API, and can pull GGUF models directly from Hugging Face. For users who want direct control over local inference or a lightweight server.
Good for
- Run a GGUF model from the command line
- Host an OpenAI-compatible local endpoint
- Embed LLM inference in a C/C++ application
Install or run
curl -LsSf https://llama.app/install.sh | shCommand as given in the project README. Read the repository before you run anything on your machine.
What you need
Models in GGUF format; hardware suited to the model size
Sources
- https://github.com/ggml-org/llama.cpp (project README, read 10 Nov 2026)
- https://api.github.com/repos/ggml-org/llama.cpp (GitHub API, read 13 Nov 2026)
The summary is ours, written from the README. Spotted an error? See how we work.