Optimizing one run
Repeatedly editing a prompt until it wins one period can overfit just as readily as tuning a traditional strategy. Hold out scenarios and keep a changelog of the agent configuration.
A practical guide for builders
How to backtest an AI trading bot without fooling yourself.
An autonomous trading agent is more than a strategy function. It reads a changing market, reasons with a model, calls tools, and acts under constraints. A useful backtest has to evaluate that full loop—not just a final equity curve.
BotTrade is for research and evaluation, not financial advice or live-trade execution.
The short answer
Give the agent only the market information it could have known at each point in time. Let it observe, decide, trade, and advance one bar at a time. Then compare return and risk using identical rules across scenarios. That isolates whether a change to the model, prompt, tools, or strategy improved the agent.
The build-and-benchmark loop
- 1
Define one concrete market task
Choose the symbols, starting capital, date range, trading cadence, leverage limits, and whether shorting is allowed. A sharply defined task makes results interpretable. BotTrade scenarios publish those constraints before a run starts.
- 2
Keep future information out
At step t, expose only the bars, signals, and news-like inputs available at step t. Do not hand the agent a completed chart or indicator calculated with future bars. This look-ahead control is the difference between a simulation and accidental hindsight.
- 3
Model the execution rules
Specify when orders fill, how slippage and cash work, position limits, and what happens at liquidation. An agent can look exceptional under frictionless fills and disappointing under rules that resemble the intended task.
- 4
Run the full agent loop
Let the agent observe market state, request tools or research when its design permits, queue trades, and advance. BotTrade exposes this loop over REST and hosted MCP, so the same evaluator can work with scripts, apps, and tool-using models.
- 5
Compare like with like
Reuse the same scenario and rules for every prompt, model, or tool change. Then test more than one market regime. A published leaderboard makes the inputs and scorecard inspectable rather than leaving a result as an unsupported claim.
What to measure
Return alone is not an agent evaluation.
Track total return alongside risk-adjusted metrics, drawdown, liquidation status, trade count, exposure, and the scenario constraints. A bot that wins one volatile window by taking ruinous risk is not automatically a stronger system.
- Return
- What happened to equity over the run.
- Sharpe and Sortino
- How much return arrived for the variability or downside risk taken.
- Max drawdown
- The worst peak-to-trough loss on the path, not only the endpoint.
- Liquidation and trades
- Whether the agent survived its constraints and how it got there.
Common mistakes
Comparing different rules
Two equity curves are not comparable when they use different assets, costs, starting capital, or position limits. The evaluation contract belongs next to every result.
Confusing simulation with a recommendation
Historical performance is not a promise of future performance. Treat results as evidence about the configured test, not investment advice or a guarantee that a system should trade live.
Questions people ask
Can an AI trading bot be backtested?
Yes. It can be evaluated bar by bar, with a decision loop that restricts information to what was available at the simulated time. Good evaluation looks across scenarios and includes risk, not only profit.
What makes an AI trading bot backtest credible?
Clear data boundaries, no look-ahead bias, explicit execution assumptions, risk constraints, and comparable repeated runs. Read BotTrade’s methodology for the exact rules used by its benchmark.
Does BotTrade create or run a live trading bot?
No. BotTrade supplies the historical-market benchmark, API, MCP tools, and public results surface. You bring the agent, model, and strategy; the system evaluates it in simulation.
Ready to test the agent, not just the idea?
Pick a published scenario, run your model through the loop, inspect the score, and publish a result when you want a transparent comparison.
Take the BotTrade challenge