CLI reference

Everything the freellama binary can do. Model references use the form hf:user/repo:QUANT or hf:user/repo/file.gguf and download straight from Hugging Face.

usage

freellama <command> [args]

  pull <model>            Download a GGUF model from Hugging Face
  run <model> [prompt]    Chat with a model (REPL, or one-shot with a prompt)
  list | ls               List installed models
  rm <model>              Remove an installed model
  serve [--host H] [--port P]
                          Start an OpenAI-compatible server (default 127.0.0.1:11434)

freellama pull <model>

Downloads a GGUF model from Hugging Face and stores it locally. Reference a quantization from a repo, or a specific .gguf file.

# by quantization
freellama pull hf:Qwen/Qwen2.5-0.5B-Instruct-GGUF:Q4_K_M

# by exact file
freellama pull hf:user/repo/file.gguf

Set HF_TOKEN to access gated model repos.

freellama run <model> [prompt]

Starts an interactive streaming chat REPL, or runs a one-shot completion when you pass a prompt. Piping stdin also works: the reply is written to stdout with no prompts.

# interactive REPL
freellama run hf:Qwen/Qwen2.5-0.5B-Instruct-GGUF:Q4_K_M

# one-shot
freellama run hf:Qwen/Qwen2.5-0.5B-Instruct-GGUF:Q4_K_M "Why is the sky blue?"

# piped stdin
echo "Why is the sky blue?" | freellama run hf:Qwen/Qwen2.5-0.5B-Instruct-GGUF:Q4_K_M

In the REPL	Effect
/clear	Reset the conversation
/bye or `CtrlD`	Exit the REPL
`CtrlC`	Interrupt a response without exiting

freellama list | ls

Lists the models installed in your local store.

freellama rm <model>

Removes an installed model from the local store.

freellama serve [--host H] [--port P]

Starts an OpenAI-compatible HTTP server backed by llama.cpp. Models are loaded lazily and swapped on demand.

Flag	Description
--host H	Address to bind (default 127.0.0.1)
--port P	Port to listen on (default 11434)

Endpoint	Description
GET /health	Liveness check
GET /v1/models	List installed models (OpenAI format)
POST /v1/chat/completions	Chat completions; supports `"stream": true` (server-sent events)

freellama serve
curl http://127.0.0.1:11434/v1/models

global flags

Flags handled by the CLI itself, outside any command.

Flag	Description
--version, -v	Print the freellama version
help, --help, -h	Show usage help

environment variables

Configuration is environment-only — no config files.

Variable	Description
FREELLAMA_HOME	Data directory (default ~/.freellama)
FREELLAMA_CTX	Context size passed to llama-server (default 4096)
FREELLAMA_LLAMA_VERSION	Pin a llama.cpp release tag, e.g. `b5900` (default: latest)
FREELLAMA_LLAMA_SERVER	Path to an existing `llama-server` binary (skips downloads)
FREELLAMA_SERVER_ARGS	Extra flags passed through to `llama-server`
FREELLAMA_DEBUG=1	Show llama-server output for troubleshooting
HF_TOKEN	Hugging Face token for gated model repos