When Your Strategy Works, Is It Just Dumb Luck? How to Stack the Odds in Your Favour

The article “When Your Strategy Works, Is It Just Dumb Luck? How to Stack the Odds in Your Favour” first appeared on Robot Wealth blog.

Recently, we had an excellent question on the Trade Like a Quant Discord server:

“How do you know if your strategy is working out of coincidence rather than actual edge? The strategy might work over a long period just because of blind luck.”

Damn good question.

It hits right in the insecurity because the honest answer is: you can never know for sure.

I remember when I first started trading.

I had this trend system that worked beautifully in backtests. It even made money for a short while in live trading. I was convinced I’d found the holy grail.

A few months later, it had all gone horribly wrong, and I was left questioning my life choices.

(It’s actually a good story… You can read about it in my case study if you haven’t already).

Was my backtest capturing a real edge, or was I just lucky in that particular sample? I had no freaking idea.

Since then, I’ve developed a more nuanced approach to this problem. You can’t eliminate uncertainty, but you can stack the deck in your favour.

Here’s how:

Have a strong and credible basis for the effect

This is by far the most important thing. The first thing you should do is ask yourself: “Why should this work?”

If there’s a credible reason for the trade to exist – a risk premium, a structural inefficiency, a behavioural quirk – then that provides solid evidence.

For example, the volatility risk premium exists because people naturally want to buy insurance against market crashes. They’ll pay above expected value for that insurance, which creates an opportunity for those willing to sell it.

That makes sense. There’s a natural imbalance of supply and demand that creates an edge.

Compare that to: “When the 13-day MA crosses the 21-day MA and the RSI is above 70, buy.”

Why would that work?

If you can’t explain it, be suspicious.

The data should support your hypothesis

Once you have a credible hypothesis, check if the data backs it up.

Use simple analysis techniques to look as directly at the effect as possible:

• Is it persistent through time? Or is there one lucky period?
• Does it show up where you expect it to (in markets or conditions that match your hypothesis)?
• Does it NOT show up where you’d expect it not to? (This is often overlooked)

Statistical tests are overrated

Yes, I use them occasionally. We even teach one in Bootcamp. But most of the time, they’re flawed because of p-hacking.

You never know how many permutations were looked at to arrive at the one that looks statistically significant.

Most of the time, the answer is lots.

And there’s a practical issue too: by the time an effect shows up as statistically significant with a high degree of confidence, everyone else has probably noticed it. It’s likely being arbitraged away as we speak.

Markets don’t wait for academic certainty.

Trade multiple strategies

This is your insurance policy. When you trade multiple strategies, you reduce the impact of any single strategy failing.

The reality is that most inefficiencies will eventually disappear, only to be replaced by new ones.

I’ve had strategies that worked beautifully for years stop working. If I’d been all-in on just one approach, I’d have been toast.

But because I was running 5-6 different approaches based on different edges, my portfolio as a whole was on solid ground.

The bottom line

Trading is about making decisions under uncertainty. You’ll never eliminate that uncertainty completely.

But you can:

• Build strategies based on sound logic
• Test them thoroughly using simple, direct methods
• Diversify across multiple approaches
• Remain humble about your edge

Remember, even the best traders in the world have periods where they question whether they actually have an edge or if they’ve just been lucky.

The difference is they’ve stacked enough evidence in their favour that they can weather the doubt and keep going.

Because they’re relying on more than their P&L (which will be highly variable, even for good strategies), they can cope with the anxiety about whether they’re really trading with an edge.

And while you can never be 100% sure, stacking the deck in your favour certainly helps you sleep at night.

If you’d like to learn a practical, simple method for stacking the deck in your favour, join us for Trade Like a Quant Bootcamp. You get lifetime access to the course, and best of all, you get to hang out with me and a bunch of other fun people on Discord.