dsh-llama-model-manager: Managing Local llama.cpp Models for DeepSeek Harness
doctorxpriestess/dsh-llama-model-manager
A DSH plugin that manages local llama.cpp GGUF models behind one OpenAI-compatible gateway, auto-loading, switching and unloading them on demand so DSH always talks to a single fixed URL.
dsh-llama-model-manager is a local model hosting plugin for DeepSeek Harness that wraps llama.cpp GGUF models on Windows behind a single fixed OpenAI-compatible gateway. It removes the old chore of stopping the server, editing provider config, and restarting whenever you want a different model: start, stop, switch, and restart are all owned by the plugin, while DSH keeps talking to one URL. A serialization gate drains in-flight requests before a switch, a real Ctrl+C lets llama.cpp release VRAM itself, and runtime.json plus a pre-flight port check cover orphan processes and port conflicts. Status, logs, the model list, and all controls sit on one settings page.
How to Install
dsh plugin --profile web add github:doctorxpriestess/dsh-llama-model-manager- Category
- Models & Reasoning
- Platform
- DSH Plugin
- Author
- doctorxpriestess
- Distribution
- Plugin
dsh-llama-model-manager Key Features
dsh-llama-model-manager Repo Summary
What Does It Do?
It is a DSH plugin for DeepSeek Harness that manages local llama.cpp GGUF models on Windows and exposes them through a stable, OpenAI-compatible gateway. It solves the pain of switching models, which normally means stopping the server, editing provider config, and restarting while hoping no request was in flight. The plugin owns the whole model lifecycle — start, stop, switch, recover — so DSH only ever talks to one fixed URL. It is maintained by DoctorxPriestess under the MIT license, written in JavaScript with no npm dependencies and no build step.
Core Features
- Model lifecycle management: start, stop, switch, and restart local GGUF models while DSH keeps a single fixed endpoint.
- Serialization gate: inference holds a shared ticket, a model switch needs an exclusive one and waits for in-flight requests to drain, preventing corrupted output.
- Real stop and VRAM release: a genuine Ctrl+C is delivered so llama.cpp frees the model itself, avoiding roughly 12 GB of leaked VRAM.
- Orphan-process safety net: a runtime.json record lets the plugin clean up leftover processes it can positively attribute to itself after DSH dies.
- Pre-flight port check: if the port is already taken, the error names the owning process; everything spawns with a hidden console to avoid flashing windows.
- Settings page: live status, logs, model list, and start/stop/switch/restart controls in one place.
How to Use This Plugin?
Open Settings → 本地模型管理 in the DSH web UI, fill in the full path to llama-server.exe, and add at least one model with an id, a display name, and the full path to a .gguf file. Then point a DSH provider at the gateway in settings.yaml, and you can chat through DSH and switch local models at any time. The plugin is Windows 10/11 only, requires Node.js 20.10 or newer (22+ recommended), and expects your own llama-server build plus .gguf model files.
This page is an independent rewrite of the plugin's official README — for authoritative documentation and the latest changes, refer to the source: doctorxpriestess/dsh-llama-model-manager. The plugin is third-party code that runs on your machine once installed; inclusion does not imply endorsement — please review the source before installing.