Use Local AI Models with DeepSeek Harness: Ollama, LM-Kit

Connecting a local model to DeepSeek Harness requires a running local service and a correct endpoint and model name; the config entry lives under Settings → Models in the Web UI, using the OpenAI-compatible protocol.

Overview

Connecting a local model takes three steps: start the service, fill in the config, switch and verify. Local model services (LM-Kit / llama.cpp / Ollama) basically all expose OpenAI-compatible endpoints, so DSH connects through the generic config without touching source code. Local models suit privacy-sensitive, offline, or cost-controlled scenarios: conversations never leave your machine, and no API key or network round-trip is needed; the trade-off is that you must prepare the model files and hardware yourself. If a configured local model still fails to connect, walk through Fix DeepSeek Harness Model Connection first. DSH is still in developer preview and config items may change — follow the official docs (source).

Before connecting: start the local model service for DeepSeek Harness

Get the local model service running and note the address and model name first. The three common services:

1. Ollama: ollama serve starts the service at http://127.0.0.1:11434/v1; ollama list shows the pulled model names.
2. llama.cpp (llama-server): llama-server -m <model-file> starts at http://127.0.0.1:8080/v1.
3. LM-Kit: a local OpenAI-compatible service; the address follows your launch config.

If you have not downloaded a model yet, pull one with ollama pull <model-name> (for llama.cpp, prepare a GGUF model file first); the model name must match what ollama list reports. Two things to keep in mind: these services mostly run as foreground processes — closing the terminal that started them stops the service, and DSH requests will fail; and both the default port and the model name can be changed on the server side, in which case the DSH config must be updated in sync — mismatched sides always fail to connect.

Verify the service is up with curl before moving on (Ollama example):

curl http://127.0.0.1:11434/v1/models

If it returns the model list, continue; if not, fix the service itself first — do not change DSH yet. This step separates the two variables "service" and "DSH config": if curl works, the problem can only be on the DSH config side; if curl fails, the problem is the service or the port, and editing the Web UI will not help.

When curl fails, check three causes: the port is taken (switch ports or kill the occupying process), the model file is missing or failed to load (look for OOM / out-of-memory lines in the service's startup log), or the address is wrong (127.0.0.1 only, not some other address). Out-of-memory is the most common issue with local models — when the model exceeds VRAM the service either crashes or becomes too slow to use; switch to a smaller model first, then come back to the DSH side. Once the service is stable, the DSH side is basically just the endpoint and model name fields.

Configure the local model in the DeepSeek Harness Web UI

The config entry is Settings → Models; fill in the endpoint and model name with OpenAI-compatible config. Open Settings → Models in the Web UI (source) and add a model:

1. Base URL: the local service's OpenAI-compatible address, e.g. http://127.0.0.1:11434/v1.
2. Model name: must be a model the server actually loaded or pulled — a wrong name returns a direct 404.
3. Save and it works: the model route takes effect immediately without a restart (source).

Give each service its own config entry (e.g. "Local Ollama", "Cloud DeepSeek") and tell them apart by model name so switching is a direct pick; if a wrong model name returns 404, handle it per Fix DeepSeek Harness Model Connection.

Local services usually need no API key — filling the endpoint like http://127.0.0.1:port/v1 is enough; a few local services have an auth switch, and if the server side enables it, fill the key into the corresponding DSH config entry instead of leaving it blank. The model name must match the server side: use the name shown by ollama list, or the name llama.cpp loads with (the --alias or the default).

To confirm the config was written, run dsh --dump-config and check the model rows in the merged config tree (source).

Switching DeepSeek Harness between local and cloud models

Local and cloud models can coexist, and switching takes effect instantly. Add multiple endpoints under Settings → Models: local 127.0.0.1 ports, plus cloud addresses with API keys, then pick the model when you use it. Three things to remember:

1. The endpoint and model name must match — change both when switching models.
2. Cloud models need an API key; local models usually do not.
3. A profile only affects its own environment: web and headless can use different models without interfering.

After switching, send a message to verify it works: a normal reply means the config succeeded; on 401/404/timeouts, go back to Fix DeepSeek Harness Model Connection.

Model-management DSH plugins (model routing, usage stats) can be found in the model category of DSH Plugin Hub — once installed, they show which model is in use and whether calls succeed, making local/cloud switching easy to verify.

Which scenario uses local, which uses cloud, comes down to your constraints: if the data must not leave the machine, you need offline availability, or you want to control API costs, use local models; if you want generation quality and long context, use cloud models. Running both is about switching per task — daily drafts on local, final output on cloud, with instant switching and no restart required.

Notes

1. Start the local model service yourself — DSH will not launch it for you.
2. Use a model name that really exists on the server; a wrong one fails to connect.
3. headless mode uses the same model config; dsh --dump-config confirms it.
4. After changing the server-side port or the model, update the endpoint and model name in DSH in sync.

Source: official Quickstart, dsh CLI README