DSH plugin web fails after editing profile package.json: BOM

If you just hand-edited the profile's package.json (say, adjusting dependencies after installing a plugin) and dsh web now dies at boot with SyntaxError: Unexpected token ... is not valid JSON and readProfileManifest in the stack, it is almost certainly saved as UTF-8 with a BOM. The content is valid JSON — the only extra thing is the leading three bytes EF BB BF. readFileSync(path, 'utf8') preserves that BOM, the U+FEFF reaches JSON.parse, and parsing necessarily fails. The fix is to save as UTF-8 without BOM or apply the recommended one-line strip, and dsh-plugin-doctor's manifest-bom check can catch it before boot.

DSH plugin symptom: dsh web crashes, and every edit brings it back

Three traits together pin it down: it dies at boot, the error hides its own cause, and it recurs. Concretely:

1. Immediate failure at startup, with the stack pointing at readProfileManifest: on Windows, running pnpm dsh web from a source checkout (or npx @deepseek-ai/dsh web) fails instantly:

SyntaxError: Unexpected token ...
is not valid JSON
at JSON.parse (<anonymous>)
at readProfileManifest (packages/boot/app-boot/src/profile.ts:272:23)
at loadProfile (packages/boot/app-boot/src/profile.ts:385:55)
...
Node.js v24.19.0

2. The trigger is utterly ordinary: edit the profile's ~/.dsh/profiles/<profile>/package.json (for example hand-tuning dependencies after installing a plugin), save it as UTF-8 with BOM — some Windows editors do this by default — then start dsh web and it immediately reports is not valid JSON (#1842).
3. The error itself offers no clue: the character after Unexpected token may render as a question mark or mojibake in a terminal, because U+FEFF is invisible. The stack only says JSON.parse died; it never says "your file begins with three extra bytes".
4. It is a recurring failure: one user documented the pattern precisely — after removing the BOM, dsh web starts fine; but as soon as the file is edited again (changing plugin dependencies, adjusting profile config), if the save tool writes a UTF-8 BOM again, startup immediately reports the same is not valid JSON. So cleaning the file once is not the finish line; the save habit has to change too (#1842).
5. Typical BOM sources: Windows PowerShell 5.1's Set-Content / Out-File -Encoding utf8 (both write a BOM); PowerShell's > redirection; Notepad's "Save as → UTF-8 with BOM"; and the "save as UTF-8" option in some editors. VS Code is BOM-free by default, so its "Save with Encoding → UTF-8" is usually safe (#1842).

DeepSeek Harness mechanism: a UTF-8 BOM in package.json reaching JSON.parse

The chain is only two steps long, but neither step does anything wrong — which is exactly why the failure point stays hidden. Layer by layer:

1. The BOM is bytes, not content: a UTF-8 BOM is simply the three bytes EF BB BF at the start of the file, decoding to U+FEFF. The JSON spec forbids it before {. So a BOM-bearing file is invalid at the byte level and perfectly normal at the editor-visible level — which is precisely what makes it hard to diagnose (#1842).
2. readFileSync preserves the BOM: readProfileManifest in packages/boot/app-boot/src/profile.ts runs readFileSync(path, 'utf8') at line 267. That call only decodes as UTF-8; it does not strip a BOM — the BOM stays in the string as an ordinary leading character (#1842).
3. JSON.parse rejects U+FEFF: line 272 then calls JSON.parse(raw) directly and throws SyntaxError. A minimal reproduction makes the boundary obvious:

const raw = Buffer.from([0xEF, 0xBB, 0xBF]).toString() + '{"dependencies":{}}'
JSON.parse(raw) // SyntaxError: Unexpected token '?', "?{"dependencies":{}}" is not valid JSON
JSON.parse(raw.replace(/^\uFEFF/, '')) // works

4. It is the same defect at more than one site: the same file has several other JSON.parse(readFileSync(..., 'utf8')) sites (lines 227, 247, 390) of the same class. That is why a shared readJsonFile helper that strips a leading BOM before parsing covers every read point at once, rather than patching each one (#1842).
5. Why "users should just be careful" is not enough: what writes the BOM is the save tool, not user intent — different tools, versions, and default encodings on one machine all affect the outcome, and the file gets rewritten repeatedly. Putting the compatibility in the parse layer (strip \uFEFF before parsing) is what makes the pit stop recurring (#1842).

DSH plugin fix: one-line strip and the manifest-bom preflight

Three tiers of response: strip before parsing in the host, add a self-check you can run, then pin down the save habit. Specifically:

1. The upstream fix: drop the leading BOM before parsing (one line):

- raw = readFileSync(path, 'utf8')
+ raw = readFileSync(path, 'utf8').replace(/^\uFEFF/, '')

Or the more explicit form:

const text = await fs.readFile(path, 'utf8');
const json = text.charCodeAt(0) === 0xFEFF ? text.slice(1) : text;
return JSON.parse(json);

Cover the other manifest read sites in the same file (lines 227, 247, 390) too, or extract a shared readJsonFile. A cherry-pick-ready branch is zoahdev/deepseek-harness @ fix/profile-manifest-bom-strip, built directly on upstream master (47f9438): readProfileManifest strips a leading \uFEFF before JSON.parse, with the same fix applied to sibling manifest reads (#1842). 2. Self-rescue: resave the file as UTF-8 without BOM: in an editor, VS Code is "bottom-right encoding → Save with Encoding → choose UTF-8"; from the command line, this Node snippet drops the first three bytes:

node -e "const fs=require('fs');const p=process.argv[1];const b=fs.readFileSync(p);if(b[0]===0xEF&&b[1]===0xBB&&b[2]===0xBF)fs.writeFileSync(p,b.subarray(3))" "$HOME/.dsh/profiles/web/package.json"

Nothing else needs restarting — just start dsh web again. On Windows the profile path is typically %USERPROFILE%\.dsh\profiles\<profile>\package.json. 3. Avoid save methods that write a BOM: Windows PowerShell 5.1's Set-Content / Out-File -Encoding utf8 write one; use Node or PowerShell 7's utf8NoBOM encoding when writing config files (#1842). 4. Check the three bytes yourself: to see whether a file carries a BOM, look at its first bytes. On macOS / Linux, inspect the hex prefix:

od -An -tx1 -N3 ~/.dsh/profiles/web/package.json
# with a BOM this prints: ef bb bf

5. Preflight before boot: manifest-bom from dsh-plugin-doctor: since v1.6.0, dsh-plugin-doctor adds a manifest-bom check under --profile, turning a post-boot crash into a pre-boot diagnosis:

npx dsh-plugin-doctor --profile ~/.dsh/profiles/web --json

Verified end-to-end on Windows: a profile package.json starting with EF BB BF yields manifest-bom: FAIL, exit code 2, and a message naming Discussion #1842; a clean manifest yields PASS, exit code 0. That release's tests pass 15/15, with a new fixture covering BOM / clean / missing. Note that the upstream one-line strip in readProfileManifest is still the permanent fix; this check merely makes the precondition diagnosable (source). 6. Managing plugins through the front door reduces hand-editing of manifests: the most common trigger for this trap is "hand-edit the profile's package.json dependencies after installing a plugin". Doing plugin install, uninstall, and update in DSH Plugin Hub lets tooling maintain dependencies for you, so you only touch that file when a manual adjustment is genuinely needed — and right after touching it, run the manifest-bom preflight above (source). 7. One lesson for plugin authors: any implementation that reads a JSON config file should strip a leading U+FEFF before parsing, especially in projects where Windows is a primary environment. Treat a BOM as a legal input to be tolerated, rather than leaving users to guess why valid JSON fails to parse (source).

DSH plugin troubleshooting notes

Remember first that the content is fine and you should not go hunting for JSON syntax errors — the only extra thing is the leading three bytes, and because save tools keep writing a BOM this failure recurs, so fixing the file and fixing the save habit have to happen together. Eight points to keep in mind when a DeepSeek Harness plugin install ends in a failed boot:

1. The content is valid — do not go hunting for JSON syntax errors: the only extra thing is the leading three bytes.
2. The whole plugin tree is affected: both DSH plugin and DeepSeek Harness plugin dependencies live in this manifest, and the host dies before it ever reads the dependency list, so every plugin fails to load together.
3. The error never names the BOM: the stack stops at JSON.parse / readProfileManifest, and U+FEFF is invisible. The timeline (you just edited the manifest) is the fastest clue.
4. It recurs: every resave can reintroduce a BOM; fixing the file once is not enough.
5. Treat both halves: save as UTF-8 without BOM, and strip in the host before parsing.
6. PowerShell 5.1 is a common culprit: both Set-Content and Out-File -Encoding utf8 write a BOM.
7. The same defect lives at several sites: other manifest reads in that file lack the strip too; one shared helper beats patching each.
8. A preflight is cheaper than a post-mortem: run the manifest-bom check after changing a profile manifest.

Sources: Discussion #1842, fix/profile-manifest-bom-strip, dsh-plugin-doctor v1.6.0.