Asset Classes

Free investment financial education

Language

Multilingual content from IBKR

Close Navigation
Learn more about IBKR accounts
Backtesting 180 Million Options Strategies – Insights from ORATS’ Latest Research

Backtesting 180 Million Options Strategies – Insights from ORATS’ Latest Research

Posted July 30, 2024 at 1:58 pm
Tyler Cheves
ORATS
VIX

The article first appeared on ORATS blog.

You read that right – In the largest group of options backtests ever studied, Option Research and Technology Services (ORATS) recently published over 180 million options strategies spanning 100+ stocks and 11 strategies. In this article, we share exclusive insights from this massive study that will help you improve and optimize your trading strategies.

This article is divided into subtopics to help you understand backtesting in detail.

  • Why is backtesting important?
  • Common pitfalls of options backtesting
  • Choosing the right entry and exit criteria
  • Measuring strategy performance
  • The future of backtesting

By the end of this article, you’ll have learned tips and tricks for how to limit failure and maximize success in your own backtesting journey.

Why is backtesting important?

A robust options trading strategy depends on accurate backtesting practices. A backtest shows you how the strategy performed historically, and gives you an idea of what to expect. You can learn a lot about a strategy with a backtest. While historical performance is not an indicator of future performance, a backtest can help you identify things like:

  1. The risk and reward potential of a strategy
  2. The relative performance of different entry criteria like days to expiration and strike deltas
  3. The impact of market conditions on strategy performance
  4. The importance of exit criteria like stop losses and profit targets

These are benefits, but there are also complications with backtesting. Let’s look at some of the common issues traders face when backtesting options.

Common pitfalls of options backtesting

Having been in the options backtesting business for over a decade, we see many traders fall into the same traps. Here, we list methods to help avoid those problems:

  1. Use accurate and realistic trade execution prices: Traders often gravitate to the end-of-day closing price of the option as the correct price to use for a backtest. However, we’ve found that these closing prices are not the best representation of the true value of the end-of-day price. Rather, our data shows that 14 minutes before the close is the closest you can get without experiencing deterioration in the quality of the quote. Our backtester uses quotes from 14 minutes before the close data to simulate trades. Slippage assumptions are key for a realistic backtest and ours are based on years of experience. We use a slippage of 75% of bid ask width for single legs all the way to 56% for four leg spreads. Traveling past the mid-price to trade is a reasonable assumption, however, for multi-legged strategies the percent traveled is not as much.
  2. Avoid overfitting the data to create unrealistic returns: When running multiple backtests on the same data set, there are risks of overfitting. An example would be testing 10 different days to expiration for the same symbol and strategy while varying no other parameters, and then choosing the best performing backtest to trade. Instead, you should look at the other similar backtests to see if they also performed well. Backtests with similar inputs but a wide variety of results is a bad sign.
  3. Avoid following path dependency: Path dependency refers to how the order and timing of trades can significantly affect a strategy's performance. Ignoring this aspect can lead to misleading results and unrealistic expectations. Our backtester tool takes into account path dependency in a unique way, by only putting on one trade per day for every day that it meets the entry criteria. This means you can have 10 trades on at the same time, all entered one day after the previous one. While this isn’t how most people trade in real life, it provides more accurate performance metrics because it eliminates any statistical bias that would occur if you only entered one trade at a time. For example, starting a backtest on the first day of the year might have very different results than starting the backtest a few weeks later.
  4. Understand the difference between notional and marginal return: Notional and margin returns are both crucial elements for assessing the profitability and efficiency of a strategy. By understanding how these returns work, investors and traders can make informed decisions when it comes to selecting the strategies that are right for them. In options trading, the notional value of an option refers to the value of the shares controlled if the option were to either expire or be exercised or assigned. For example, if I buy a $5 call option on a $100 stock, because this option controls 100 shares of the underlying, the notional value of the trade would be 100 * $100 = $10,000. However, I would only need to pay $5 * 100 = $500 to open this trade. Thus, the notional return measure the performance relative to the $10,000, while the margin return measures the performance relative to the $500. It’s easy to see that notional returns are much lower than margin returns. We use notional returns in the backtester because it helps standardize and normalize performance across all different types of strategies and symbols. We show the margin return to highlight how efficiently the strategy used the capital at its disposal. However, you have to be careful with margin returns, as sometimes they are not the most accurate reflection of a real-world trading environment. Generally, if an investor is using options and stocks, or if you want to normalize and compare disparate trading strategies, the notional calculation is best. If you want to see how much you would make on the amount at risk from a brokerage firms perspective the margin returns are best to use. Brokerage firm’s margin do not always present the actual risk, so care needs to be used. For example, a short put may require 20% margin in portfolio margined accounts or 100% margin in cash accounts. The margin returns would be significantly different in both approaches.

Choosing the right entry and exit criteria

There’s nearly an infinite amount of settings you can test when running an options backtest. Through our research, we’ve identified the most important entry criteria, technical indicators, and exit triggers so you can run your own backtest with more confidence.

Entry criteria:

  • Days to Expiration (DTE): Days to Expiration (DTE) indicates the remaining time until an option contract expires. We like to test a diverse range of DTEs, from as little as 2 days to over 300 days. By evaluating the strategy's performance across different time horizons, you can gain insights into the ideal periods for executing specific strategies.
  • Strike Deltas: The strike delta is a measurement of how much an option's price is likely to move with each $1 move in the underlying security. It’s important to test across a variety of absolute deltas, including in-the-money and out-of-the-money strikes.
  • Spread Yield: Spread Yield is a measure of the price paid for the options spread relative to the price of the underlying stock. It's calculated by dividing the price paid for the spread by the stock price. We like to categorize the spread yield target for each backtest as low, moderate, or high, relative to other backtests with comparable DTE and strike deltas. This additional context allows for more informed analysis.

It’s interesting to explore the relationships between different backtest entry criteria and their performance. For example, we found that if you filter down SPY Short Put Spreads by a low VIX entry trigger, and rank them by best overall performance, we see that the spread / stock is almost always low. This showed us that in a low volatility environment, this strategy performed better as we targeted a lower spread yield.

Technical indicators: Technical indicators can provide guidance on when to enter a trade. The following five indicators are a good starting point when assessing what to add to your backtest.

  • VIX Price: The VIX, or volatility index, reflects the market's expectation of 30-day forward-looking volatility. Low VIX levels (<15) suggest a calm market, moderate levels (15-20) indicate normal volatility, while high levels (>20) imply increased uncertainty.
  • Simple Moving Average (SMA): SMA is a commonly used technical indicator that smoothes out price data to capture trends over specific periods. Testing if the price is above or below the 50 or 200-day SMA is a good place to start.
  • 14d RSI: The 14-day Relative Strength Index (RSI) is a momentum oscillator that measures the speed and change of price movements. A reading of less than 40 indicates oversold conditions, 40-60 suggests moderate momentum, and above 60 signals overbought conditions.
  • IV Percentile 1 Year: The IV percentile shows where the current implied volatility of the underlying stands relative to its 1-year range. It's categorized as low (<33), moderate (33-66), or high (>66).
  • Slope Percentile 1 Year: This trigger shows where the current slope of the implied volatility skew stands relative to its 1-year range, categorized as low (<33), moderate (33-66), or high (>66).

Exit Criteria: Exit triggers play a crucial role in risk management and profit protection. It’s important to test stop loss levels of -25%, -50%, and -75% to protect from excessive losses. For locking in profits, we suggest testing profit targets of +25%, +50%, +75%, +100% (if debit strategy), +150% (if debit strategy), and +300% (if debit strategy).

Measuring strategy performance

It’s important to measure the right performance metrics when backtesting, otherwise you’ll end up with a lot of data and no idea what to do with it. Before diving into the specifics, let's outline four categories of metrics we believe are critical to analyzing performance:

Return: Metrics such as annual returns (overall, 1 year, 5 years, bearish and bullish markets), annual margin return, and best/worst monthly and annual returns.

Risk: Quantitative measurements such as Sharpe Ratio, Sortino Ratio, Annual Volatility, Max Drawdown %, Drawdown Days, and Reward to Risk Average.

Profit & Loss: Comprehensive data like average P&L % per day, best and worst trade P&L (both in dollar and percentage), average P&L per trade and per day, and total strategy P&L.

Other: Metrics including % of time in the market, strategy win rate, average days in trade, total strategy trades, credit/debit per trade average, margin per trade average, and margin to stock %.

Let's dive deeper into some of the most important metrics:

Sharpe vs. Sortino ratio: While they might seem similar at first, each ratio offers a distinct perspective on risk and should be understood separately. The Sharpe ratio gauges how much excess return a strategy provides relative to the risk taken, using standard deviation as a proxy for risk. It's excellent for understanding a strategy's overall risk-adjusted return, but it falls short in one critical area – it doesn't distinguish between upside and downside volatility. That's where the Sortino ratio comes in. The Sortino ratio, like the Sharpe ratio, evaluates risk-adjusted return, but it only considers downside volatility. In doing so, it addresses an important asymmetry in trading – traders generally welcome upside volatility while fearing its downside counterpart. A high Sortino ratio signals that a strategy minimizes damaging losses while potentially capitalizing on desirable volatility.

Percent of time in market: The percent of time in market is an important metric in backtesting because it can help reduce overfitting. By setting a minimum filter, you can focus on backtests with a statistically sound number of market days. Imagine two strategies: Strategy A has been in the market for 300 days, and Strategy B for only 30 days. While Strategy B might show an impressive return for its short time, its performance metrics might be unreliable due to the small sample size. By filtering percent of time in market, you can prevent overfitting and enable a more robust evaluation of the strategies' true performance. Like percent of time in market, you should consider filtering other metrics such as drawdown days, total strategy trades, and margin to stock % to fine-tune your risk tolerance and trading preferences.

Profit and loss metrics: It’s important to calculate various measures of profit and loss to provide a comprehensive view of a strategy's performance. The best and worst trade p&l metrics make it easy to quantify your upside potential versus the downside risk of the strategy. Sometimes it’s good to filter strategies based on p&l percent per day, as that can be a good baseline for an effective strategy.

The future of backtesting

I want to end on a brief discussion about the future of backtesting. Gone are the days of tireless, manual backtesting in the form of excel sheets and poor quality data. With the availability of high quality minute-by-minute historical data and powerful cloud computing technology, traders of the future will be answering their backtesting questions in seconds with AI powered research assistants. Going from a backtest to live implementation will take no more than a single click of the mouse – seamlessly integrated into the trading lifecycle. Today, you’ve learned about the key principles of backtesting that are driving this cultural shift. Stay tuned for the next installment from ORATS to keep up with the latest trends in options trading.

You can browse all 180 million of ORATS’ backtests on their website by visiting orats.com/ib?utm_source=ib&utm_campaign=quantblog. This article was originally published on orats.com/blog.

Join The Conversation

If you have a general question, it may already be covered in our FAQs. If you have an account-specific question or concern, please reach out to Client Services.

One thought on “Backtesting 180 Million Options Strategies – Insights from ORATS’ Latest Research”

  • Anonymous

    If you’re backtesting an intraday strategy then you must run the data at multiple times in one day. I understand the idea of overfitting, but I don’t follow the above overfitting statement – doesn’t make sense to me.

Leave a Reply

Disclosure: ORATS

The opinions and ideas presented herein are for informational and educational purposes only and should not be construed to represent trading or investment advice tailored to your investment objectives. You should not rely solely on any content herein and we strongly encourage you to discuss any trades or investments with your broker or investment adviser, prior to execution. None of the information contained herein constitutes a recommendation that any particular security, portfolio, transaction, or investment strategy is suitable for any specific person. Option trading and investing involves risk and is not suitable for all investors.

All opinions are based upon information and systems considered reliable, but we do not warrant the completeness or accuracy, and such information should not be relied upon as such. We are under no obligation to update or correct any information herein. All statements and opinions are subject to change without notice.

Past performance is not indicative of future results. We do not, will not and cannot guarantee any specific outcome or profit. All traders and investors must be aware of the real risk of loss in following any strategy or investment discussed herein.

Owners, employees, directors, shareholders, officers, agents or representatives of ORATS may have interests or positions in securities of any company profiled herein. Specifically, such individuals or entities may buy or sell positions, and may or may not follow the information provided herein. Some or all of the positions may have been acquired prior to the publication of such information, and such positions may increase or decrease at any time. Any opinions expressed and/or information are statements of judgment as of the date of publication only.

Day trading, short term trading, options trading, and futures trading are extremely risky undertakings. They generally are not appropriate for someone with limited capital, little or no trading experience, and/ or a low tolerance for risk. Never execute a trade unless you can afford to and are prepared to lose your entire investment. In addition, certain trades may result in a loss greater than your entire investment. Always perform your own due diligence and, as appropriate, make informed decisions with the help of a licensed financial professional.

Commissions, fees and other costs associated with investing or trading may vary from broker to broker. All investors and traders are advised to speak with their stock broker or investment adviser about these costs. Be aware that certain trades that may be profitable for some may not be profitable for others, after taking into account these costs. In certain markets, investors and traders may not always be able to buy or sell a position at the price discussed, and consequently not be able to take advantage of certain trades discussed herein.

Be sure to read the OCCs Characteristics and Risks of Standardized Options to learn more about options trading.

Disclosure: Interactive Brokers

Information posted on IBKR Campus that is provided by third-parties does NOT constitute a recommendation that you should contract for the services of that third party. Third-party participants who contribute to IBKR Campus are independent of Interactive Brokers and Interactive Brokers does not make any representations or warranties concerning the services offered, their past or future performance, or the accuracy of the information provided by the third party. Past performance is no guarantee of future results.

This material is from ORATS and is being posted with its permission. The views expressed in this material are solely those of the author and/or ORATS and Interactive Brokers is not endorsing or recommending any investment or trading discussed in the material. This material is not and should not be construed as an offer to buy or sell any security. It should not be construed as research or investment advice or a recommendation to buy, sell or hold any security or commodity. This material does not and is not intended to take into account the particular financial conditions, investment objectives or requirements of individual customers. Before acting on this material, you should consider whether it is suitable for your particular circumstances and, as necessary, seek professional advice.

Disclosure: Options Trading

Options involve risk and are not suitable for all investors. Multiple leg strategies, including spreads, will incur multiple commission charges. For more information read the "Characteristics and Risks of Standardized Options" also known as the options disclosure document (ODD) or visit ibkr.com/occ

IBKR Campus Newsletters

This website uses cookies to collect usage information in order to offer a better browsing experience. By browsing this site or by clicking on the "ACCEPT COOKIES" button you accept our Cookie Policy.