Commands

LlamaNexus ships as a single binary (llamanexus) with four subcommands.

serve

Starts the proxy and the underlying llama-server router. This is the main mode, used in production via Docker Compose alongside Open WebUI.

command: serve --llamaport 8080 --port 11434 -- [llama-server args...]

• --port - port LlamaNexus itself listens on (default 11434, matching Ollama's default).
• --llamaport - port llama-server's router listens on internally (default 8080).
• Any arguments after -- are passed straight through to llama-server (e.g. --n-gpu-layers) - but not --ctx-size, since that conflicts with the per-model preset mechanism (see context size overrides).

run

One-shot CLI inference against a single model, without starting the full proxy/router. Downloads the model automatically if it isn't already cached.

docker exec -it llamanexus llamanexus run -m <repo>:<tag> -- -p "Your prompt here"

• -m / --model - model identifier as repo:tag, e.g. Qwen/Qwen2.5-0.5B-Instruct-GGUF:Q4_K_M. The tag can be a quantization name (Q4_K_M) or an exact filename.
• Everything after -- is passed directly to llama-cli. The -- separator is required.
• LlamaNexus automatically adds -st (--single-turn) so llama-cli exits after one response instead of dropping into an interactive prompt.

pull

Downloads a model from Hugging Face without running inference or starting the server. Useful for pre-warming the cache from a script or cron job.

docker exec -it llamanexus llamanexus pull <repo>:<tag>

Prints a live percentage as the download progresses, then exits.

worker

Starts a ggml-rpc-server instance for distributed/multi-machine inference via llama.cpp's RPC backend. Used in a separate Docker Compose file on machines contributing compute to a primary serve instance - see distributed inference.

command: worker --port 50052

--version

Prints the version and exits immediately - no subcommand needed, and none of the usual startup output.

llamanexus --version