Introduction
llama.cpp lets you run large language models locally on your laptop, desktop, or server with minimal setup and state-of-the-art performance.
With a single command you can chat with a model in your terminal, or spin up an OpenAI-compatible server with a built-in web interface:
# Chat in your terminal
llama cli -hf ggml-org/gemma-4-e4b-it-GGUF:Q4_0
# Serve an OpenAI-compatible API + web UI
llama serve -hf ggml-org/gemma-4-e4b-it-GGUF:Q4_0llama cli is great for experimentation and terminal use, while llama serve exposes a full-featured web UI).
How it works
llama.cpp runs models stored in the GGUF file format. GGUF is a single file that packages everything you need together: model weights, tokenizer, metadata. Thousands of ready-to-use GGUF models are available on Hugging Face.
On top of this core, the project ships user-facing tools: llama cli for the terminal, llama server for serving an HTTP API and web UI, plus utilities for benchmarking, quantizing, and evaluating models.
Next steps
- Installation — install prebuilt binaries, use a package manager, Docker, or build from source
- Quickstart — download a model and run it in minutes
- Using the CLI — chat and experiment from the terminal with
llama cli - Running a server — serve models over HTTP with
llama server - API server — the OpenAI-compatible REST API reference
- Web UI — the built-in browser chat interface