dsh-see-now: A Vision Plugin That Lets DeepSeek Harness See Your Screen in Real Time

What Does It Do?

See Now is a DSH plugin for DeepSeek Harness that turns your screen, camera, or video file into a stream of frames filtered by how much they changed, packed into a single composite image for the model. It solves the dilemma of screenshotting the full screen on a timer: capture every two seconds and you burn tokens on identical frames, capture every thirty seconds and the model misses everything in between. The plugin scores each frame from 0 to 255 and keeps only frames where the picture suddenly cuts or motion accumulates past a threshold, so a static screen costs almost nothing. The kept frames are tiled into one image, spreading the fixed cost of a single image across four to eight frames.

Core Features

• Change-based filtering: each frame gets a 0–255 change score computed as a weighted HSV difference on a 64×36 grid in about 0.2ms, and only cut points or accumulated motion survive.
• Multi-frame tiling: kept frames are placed at their real pixel positions in one image, and four or eight frames cost the same, so keeping one more frame is nearly free.
• Three pacing tiers: calm, normal, and dense set cut thresholds of 20, 12, and 7 points with fallback wake-ups at 30, 15, and 8 seconds respectively.
• Scene mode: each picture change produces its own full-bleed image and wakes the model once, suited to page-by-page narration.
• Multiple sources: screen, camera, and video file playback are all supported, and the capture and pipeline modules can be used standalone as a plain Node library.
• Floating-window speech: only the say parameter reaches the floating window and gets spoken aloud; ordinary replies stay invisible to the user.

How to Use This Plugin?

After enabling it in DSH, have the model call see_now with action: "watch"; it blocks until a full sheet is assembled and hands the image back. The first call returns the current screen in about one second, and later calls return what happened during the interval. Adjust quality across low, standard, high, and ultra, and use pace to control rhythm: saying it is too noisy or too expensive switches to calm, while saying it is not sensitive enough switches to dense. Add the say parameter to have it narrate while watching, since that is the only channel that reaches the floating window and gets spoken. The plugin is maintained by SAOS-3624, free for personal and learning use, with commercial use requiring separate written authorization.

How to Troubleshoot This Plugin?

If the model cannot see the screen after setup, first confirm DSH was restarted, since the capture module only takes effect after a restart. If capture never yields frames, check whether a quality change triggered the roughly 0.6-second capture restart and wait a moment before taking a sheet. If returned sheets contain too few frames, lower the minFrames threshold or switch to dense pacing; if tiles shrink to a quarter or a ninth with gray bars between them, the scenes are too dense and scene mode should be enabled so single frames fill the sheet.