Test an idea against overfitting
- Write the rule in words: instrument, timeframe, entry, exit, stop-loss, the one parameter you would like to tune, and the period you will not touch until the end (the out-of-sample window).
- Let the model draft the cBot with the prompt below. Point it at the vendor's command reference at
github.com/spotware/CLI-referencesso it does not invent options; that is the vendor's own advice for AI-drafted commands. - Build and run the in-sample backtest with the broker's costs. The only required inputs are the dates and the data mode; add the commission and the spread yourself, and save the JSON report:
- Run the same cBot on the untouched period. Change nothing but
--startand--end; save a second JSON report. That is a walk-forward step in its simplest form. - Ask the model to write the acceptance report from the two JSON files, not from memory, with the fields listed in the checklist. Every field must quote the report it came from.
- Read the verdict as binary. "Not rejected" means the idea survived the filters you declared in advance. It does not mean "works".
docker pull ghcr.io/spotware/ctrader-console:latest ctrader-cli backtest strategy.algo params.cbotset \ --account=<account-id> --symbol=EURUSD --period=h1 \ --start="01/01/2024" --end="31/12/2024" \ --data-mode=m1 --commission=<per-lot> --spread=<pips> \ --report-json=report-in-sample.json
Syntax as documented by the vendor on 2 September 2026: backtest takes the .algo file, an optional .cbotset parameter file, and the testing context. --report-json writes the machine-readable report the model will read.
You are drafting a cBot for cTrader Automate (C#) that will be backtested with cTrader CLI. Read the command reference at https://github.com/spotware/CLI-references before proposing any command. Do not invent options. Use placeholders for the account id and file paths. Never ask for a password. Rule: <instrument, timeframe, entry, exit, stop-loss, exit rule> Parameter to expose as a cBot parameter: <name, default, sensible range> Deliver: (1) the cBot source; (2) a .cbotset file with the defaults; (3) the exact build and backtest commands for the in-sample window <dates> and the out-of-sample window <dates>, with --commission, --spread and --report-json; (4) your assumptions and anything that is missing. Do not start `run`. Do not place orders. Do not estimate returns.
Attached are two JSON backtest reports from cTrader CLI: in-sample and out-of-sample. This is attempt number N=<n> on this family of ideas (every parameter set and every rule change counts as one attempt). Write an acceptance report with exactly these fields, each quoting the report it comes from: trades (IS / OOS); net result sign (IS / OOS); max balance drawdown (IS / OOS); ProfitFactor (IS / OOS); buy-and-hold of the same instrument over the same period; cost sensitivity (a third report with the spread doubled, if attached); N and the noise threshold for N attempts; bootstrap of trade order (drawdown distribution, if computed). Verdict: "not rejected" or "rejected", with the reason. Do not forecast. Do not recommend a position. Do not use the word "works".
- Number of trades in each window. Too few trades out of sample, and the verdict is "inconclusive", not "not rejected".
- Sign of the net result and the maximum balance drawdown, in-sample and out-of-sample, side by side.
- Buy-and-hold of the same instrument over the same period. An idea that lost to holding the instrument is rejected.
- Cost sensitivity: the same run with the spread doubled. An idea that dies on costs is rejected.
- Declared N: how many attempts this family of ideas has consumed. With N attempts, the best result of pure noise is not zero: for ten thousand attempts over five years the expected best Sharpe ratio of noise is about 1.7 (Bailey and others, 2014). The threshold rises with N.
- Bootstrap: shuffle the order of the trades a few hundred times and look at the distribution of drawdowns, not at the one path history happened to take.
- No equity curve and no return figure. A pass through the filters is not a forecast.
- No "strategy that works". An idea that is not rejected today can be rejected by the next window.
Why the vendor's tools. The backtester is the vendor's own command-line tool; it runs on Linux, macOS and Windows and inside the vendor's Docker image, with no third-party bridge and no paid charting plan. The parameter sweep (optimize) is documented for cTrader CLI 5.10; until it reaches your build, a series of backtest runs with different .cbotset files does the same job, and makes N explicit.
Everything above runs on a demo account. Open a TopFX demo account →