dsh-plugin-mlquant-benchmark: Enabling DeepSeek Harness Agents to Reproduce Quant Benchmarks End-to-End
initial-d/dsh-plugin-mlquant-benchmark
A DSH plugin that reproduces the ml-quant-trading protocol v1 benchmark, offering tools to run, validate, read JSON artifacts, and draft GitHub issue reports without trading or market data.
dsh-plugin-mlquant-benchmark is a benchmark reproduction plugin for DeepSeek Harness that lets an agent fully reproduce the ml-quant-trading protocol v1 CPU benchmark. Once installed, the agent can run the fixed-parameter benchmark, read the machine-readable JSON artifact, validate results against the protocol, and draft a GitHub issue report ready for submission. The plugin deliberately stays narrow: it adds no trading agent, calls no market data APIs, and configures no model provider, focusing on engineering reproducibility rather than investment advice. Through four dedicated tools, it turns agent attention into a verifiable benchmark report, not a trading-performance claim.
How to Install
dsh plugin --profile web add github:initial-d/dsh-plugin-mlquant-benchmark- Category
- Tools & Capabilities
- Platform
- DSH Plugin
- Author
- initial-d
- Distribution
- Plugin
dsh-plugin-mlquant-benchmark Key Features
dsh-plugin-mlquant-benchmark Repo Summary
What Does It Do?
mlquant-benchmark is a DSH plugin for DeepSeek Harness that enables an agent to reproduce the ml-quant-trading protocol v1 CPU benchmark end to end. It solves the problem of making benchmark results verifiable and reportable, letting the agent run the fixed-parameter benchmark, read the machine-readable JSON artifact, validate it against the protocol, and draft an issue-ready report. The plugin does not add a trading agent, call market data APIs, or configure any model provider, focusing on engineering reproducibility rather than investment advice.
Core Features
- Runs the fixed protocol v1 CPU benchmark and writes
artifacts/benchmark-v1.json. - Reads the JSON artifact and renders a compact Markdown result table.
- Validates protocol v1 fields, expected cases, fixed parameters, and variance warnings.
- Drafts a DeepSeek Harness benchmark issue body from the JSON artifact, without posting to GitHub.
How to Use This Plugin?
After enabling the plugin in DSH, open a checkout of initial-d/ml-quant-trading and ask the agent to run, validate, summarize, and draft a benchmark report. A suggested prompt is to have the agent read AGENTS.md, docs/benchmarking.md, and docs/reality_check.md, then use the mlquant tools to run the protocol v1 CPU benchmark, validate and read the JSON artifact, and draft a report, keeping the result as an engineering reproducibility benchmark rather than a trading-performance claim. The drafted report can be submitted through the dedicated issue template in the main repository.
This page is an independent rewrite of the plugin's official README — for authoritative documentation and the latest changes, refer to the source: initial-d/dsh-plugin-mlquant-benchmark. The plugin is third-party code that runs on your machine once installed; inclusion does not imply endorsement — please review the source before installing.