Unload All llama.cpp Router Models Without Restarting

worth using in the first place.

But there is one sharp edge: unloading everything is not a single magic button in the HTTP API.

The router can list models. It can load a model. It can unload a model. It can evict the least recently used model when --models-max is reached. What it does not currently document as a first-class endpoint is a universal unload all models now call.

That is not a real blocker. The correct pattern is simple, explicit, and scriptable:

1. Ask the router which models exist.


2. Filter the models whose status is loaded.


3. Call /models/unload once per loaded model.

This is the approach I recommend for serious can integrate with llama.cpp model unload support. When the provider is configured as llama.cpp, Open WebUI can show loaded-model state and expose an Eject action for admins.

Under the hood, that action calls Open WebUI's own unload API, which then calls llama.cpp's /models/unload endpoint on the configured connection.

That is nice for manual operation, but I would still keep the shell script. A UI button is convenient. A script is auditable, repeatable, and usable on a headless box at 2 AM.

When to use unload all

Unloading every loaded model is useful when you want to:

• Free GPU memory before starting a larger model.


• Reset a development box without restarting llama-server.


• Prepare for a benchmark run with a clean memory state.


• Drain local inference workloads before maintenance.


• Recover from a messy session where too many models were warmed.

It is not the right tool when active users are depending on warm models. In that case, tune --models-max, use deliberate routing, and let LRU eviction do part of the work. If you need smarter timeout-based unloading with per-model lifecycle control, give concrete figures to sanity-check against.

A safer JSON body version

If your model identifiers contain unusual characters, use jq to generate the JSON request body:

curl -s http://localhost:8080/models \

| jq -r '.data[] | select(.status == "loaded") | .id' \

| while IFS= read -r model; do

    echo "Unloading: $model"

    body="$(jq -n --arg model "$model" '{model: $model}')"

    curl -s -X POST http://localhost:8080/models/unload \

      -H "Content-Type: application/json" \

      -d "$body" \

      | jq

  done

This is the version to use if your models are named with repository-style identifiers, custom aliases, or paths.

Final take

llama.cpp router mode is a big step forward for local LLM operations. It gives you dynamic loading, model switching, and memory-aware eviction without giving up the directness of llama-server.

But do not wait for a perfect unload-all endpoint. The clean solution already exists: list loaded models and unload them one by one.

That pattern is explicit. It is scriptable. It works over SSH. It plays nicely with Open WebUI. And most importantly, it frees VRAM without restarting the router.

For local AI infrastructure, that is exactly the kind of boring control surface you want.