{"id":212275,"date":"2024-09-29T11:09:55","date_gmt":"2024-09-29T15:09:55","guid":{"rendered":"https:\/\/ibkrcampus.com\/campus\/?p=212275"},"modified":"2024-09-29T11:09:10","modified_gmt":"2024-09-29T15:09:10","slug":"youre-probably-backtesting-wrong","status":"publish","type":"post","link":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/youre-probably-backtesting-wrong\/","title":{"rendered":"You’re Probably Backtesting Wrong"},"content":{"rendered":"\n

The article “You’re Probably Backtesting Wrong” was originally posted on Dave Mabe’s blog<\/a><\/em>.<\/p>\n\n\n\n

I\u2019ve worked with a lot of traders over the years, many of whom are quite smart and able to backtest their ideas to generate successful trading strategies. But when I look at their backtesting routine, almost all of them completely ignore the most productive step for creating profitable models with a strong and durable trading edge.<\/p>\n\n\n\n

Backtesting this way is like riding a horse compared to walking while not realizing that airplanes exist: you can travel a long way on horseback (even across the entire country, in theory) and you\u2019ll leave mere walkers in the dust. However, if you\u2019re unaware of air travel then you may as well be a sloth who has no idea that he can travel at least 100 times faster by plane.<\/p>\n\n\n\n

Here\u2019s a standard backtest workflow that I see traders doing over and over:<\/p>\n\n\n\n

    \n
  1. Trading strategy idea generation<\/li>\n\n\n\n
  2. Code the backtest<\/li>\n\n\n\n
  3. Run the backtest<\/li>\n\n\n\n
  4. Analyze aggregate metrics and equity curve<\/li>\n\n\n\n
  5. Decide to trade it or return to step #1<\/li>\n<\/ol>\n\n\n\n

    Let\u2019s look at these steps one at a time in detail.<\/p>\n\n\n\n

    Trading Strategy Idea Generation<\/h2>\n\n\n\n

    Generating ideas is the hardest part of trading for most since it starts off like a chicken and egg problem: ideas come from experience but to have good ideas you first need to have some experience. With enough practice, though, this becomes the easiest aspect of the whole process. I know some readers will find this hard to believe, but the more you\u2019re able to exercise your \u201ctrading idea muscle,\u201d the stronger it will be.<\/strong><\/p>\n\n\n\n

    Because it takes regular practice and hard creative work, this step weeds out a lot of would-be traders.<\/p>\n\n\n\n

    Code the Backtest<\/h2>\n\n\n\n

    For most traders, coding the backtest ranges from difficult to impossible. Even high-level traders who are excellent at coding have a hard time context-switching from trading (or other tasks) back to writing code. This is exacerbated by the fact that running the backtest takes so much time. You can\u2019t stay in a productive coding zone when you get interrupted by a backtest that takes an hour or more to run.<\/p>\n\n\n\n

    Run the Backtest<\/h2>\n\n\n\n

    Depending on the markets and instruments you\u2019re running a backtest on, this part will likely be the most time intensive. If it does NOT take a long time to run, you\u2019re probably not scanning the entire market, thereby missing out on profitable trading opportunities.<\/p>\n\n\n\n

    If your backtesting platform doesn\u2019t have the ability to scan the entire market, then \u2013 back to my analogy \u2013 you\u2019re on horseback. Better than walking? Yes, and certainly faster than a sloth, but you\u2019re not really in the same game as successful systems traders.<\/p>\n\n\n\n

    Analyze Aggregate Metrics and Equity Curve<\/h2>\n\n\n\n

    A lot of traders get tangled up here in deciding which aggregate metrics to focus on. There are plenty to choose from and it\u2019s tempting to try to find the \u201cfanciest\u201d one that supersedes all others. Rather than trying to find the perfect single metric, I look at a handful of metrics in this phase. (While important, how I choose my metrics is beyond the scope of this post. I\u2019ll outline my exact process in a forthcoming message to my newsletter.)<\/p>\n\n\n\n

    Decide to Trade It Or Return To Step 1<\/h2>\n\n\n\n

    Using whatever criteria you\u2019ve settled on, you either move forward and trade the strategy as is, or you start the process over at step 1.<\/p>\n\n\n\n

    Here\u2019s this workflow in visual form \u2013 note that the pink boxes represent particularly time-consuming parts of the process:<\/p>\n\n\n\n

    \"\"<\/figure>\n\n\n\n


    The Step You\u2019re Missing<\/h2>\n\n\n\n

    This workflow is\u2026 fine. You\u2019re on horseback competing against walkers. But there\u2019s a very important phase of my<\/em> routine that is completely missing from this workflow. How important is it?<\/p>\n\n\n\n

    I call this the Filter Phase and I literally spend 95% of my overall backtesting time in this phase. It allows me to reduce the time I spend creating a strategy from days and weeks<\/em> to minutes and hours<\/em>. It\u2019s the reason I\u2019ve been able to create and trade 15+ (and counting) profitable strategies.<\/p>\n\n\n\n

    The Filter Phase<\/h2>\n\n\n\n
    \"\"<\/figure>\n\n\n\n


    In order to add this step to your workflow, you\u2019ll need to be able to make a couple of small adjustments to your overall routine. Your backtesting platform must have the following capabilities<\/strong>, otherwise, you\u2019re stuck on horseback:<\/p>\n\n\n\n