freellama_

All the power of llama.cpp with none of the setup — a simple CLI and an OpenAI-compatible API, built with Deno and released into the public domain.

Read the docs GitHub

Install

curl -fsSL https://raw.githubusercontent.com/johnpangalos/freellama/main/install.sh | sh

git clone https://github.com/johnpangalos/freellama
cd freellama
deno task install

openai-compatible api

freellama serve speaks the OpenAI API on localhost:11434 — point any client or SDK at it.

llama.cpp made easy

The official prebuilt llama-server, downloaded and managed for you. No toolchain, no build flags — just pull, run, serve.

straight from hugging face

Models are plain GGUF files pulled directly from Hugging Face. No registry, no account.

public domain

Released under the Unlicense. No telemetry, no tracking, no strings.

Quick start

# download a small model (~400 MB) from Hugging Face
freellama pull hf:Qwen/Qwen2.5-0.5B-Instruct-GGUF:Q4_K_M

# chat with it
freellama run hf:Qwen/Qwen2.5-0.5B-Instruct-GGUF:Q4_K_M