dsh-computer-use-win: A Windows Computer Use Plugin for DeepSeek Harness
yu-tao-li/dsh-computer-use-win
Windows computer use for DeepSeek Harness: MCP stdio server + PowerShell UIA backend with 22 desktop tools for agent to see and operate real apps.
dsh-computer-use-win is a Windows computer use plugin for DeepSeek Harness, using an MCP stdio server and a PowerShell/UIA desktop engine to let agents see (UIA accessibility tree, window screenshots, OCR) and act (mouse, keyboard, UIA semantic actions, window management) on real Windows desktop apps. It solves the problem of agents not being able to operate desktop GUI, offering 22 tools with zero runtime dependencies and no modification to the harness. Built-in safety mechanisms like coordinate homing, emergency stop failsafe, and identity guard keep operations controlled. Maintained by Yu-tao-Li under the MIT license, last updated in 2026-08.
How to Install
dsh plugin --profile <your-profile> add dsh-computer-use-winReplace --profile with the profile name you actually run (see ls ~/.dsh/profiles/): usually web in the browser UI, desktop or desktop-NNN in the desktop app.
- Category
- Tools & Capabilities
- Platform
- DSH Plugin
- Author
- yu-tao-li
- Distribution
- Plugin
dsh-computer-use-win Key Features
dsh-computer-use-win Repo Summary
What Does It Do?
dsh-computer-use-win is a DSH plugin for DeepSeek Harness that brings Windows computer use to the harness via an MCP stdio server and a PowerShell/UIA desktop engine. It solves the problem of agents not being able to operate real Windows desktop apps, offering 22 tools to see (UIA accessibility tree, window screenshots, OCR) and act (mouse, keyboard, UIA semantic actions, window management). It requires zero runtime dependencies and does not modify the harness itself. Maintained by Yu-tao-Li under the MIT license, last updated in 2026-08.
Core Features
- Text-first observation: UIA accessibility tree with three views (control/content/raw), locating controls by name/automationId/class/value; element ids use UIA RuntimeId, stable across UI refreshes.
- Three-tier screenshot chain: PrintWindow (captures non-foreground windows) → WGC (Windows.Graphics.Capture) → screen region fallback with occludedPossible flag; PNGs auto-cleaned after 30 minutes.
- Three input paths: semantic (UIA patterns), foreground (SetCursorPos + SendInput + clipboard), and background (PostMessage into the window message queue, honestly reporting verified:false when delivery is unverified).
- OCR fallback for UIA blind spots: Windows.Media.Ocr returns text and per-word bounding boxes; query hits automatically re-query the control under the matched word.
- Built-in safety mechanisms: coordinate homing (auto-compensates if window moves), emergency stop (mouse in corner for 500ms rejects all input), identity guard (rejects if window HWND/PID changes), foreground validation (fail-closed), and Win key blacklist.
- Persistent backend: a resident PowerShell process with JSON line protocol, hot latency 6–100ms; C# P/Invoke helpers compiled into cached DLLs by hash.
How to Use This Plugin?
After enabling it in DSH, restart dsh web and the tools appear as mcp__wincu__windows_computer_use_*. For any window, use snapshot to get the tree and screenshot, find to locate controls, and click/type_text to perform actions. All tools support optional window targeting (windowTitle substring / processId / nativeWindowHandle) plus activate: true; if not specified, they act on the current foreground window. For sensitive operations, take a snapshot first and restrict important windows with windowTitle.
This page is an independent rewrite of the plugin's official README — for authoritative documentation and the latest changes, refer to the source: yu-tao-li/dsh-computer-use-win. The plugin is third-party code that runs on your machine once installed; inclusion does not imply endorsement — please review the source before installing.
DSH Plugin FAQ
