Why Data Mining Risks Your Trading Career

The article “Why Data Mining Risks Your Trading Career” was originally posted on Robot Wealth blog.

I was recently talking to someone about data mining as an approach to finding edges to trade.

I get the appeal. Feed enough data into a computer, run enough tests, and surely something profitable will emerge, right?

Maybe.

But almost certainly not.

But the worst thing about this approach is that it risks your entire trading career.

Sound extreme?

I’ll explain.

Two types of traders

Let me tell you about two types of systematic-minded traders:

The first group dives into parameter optimisation and backtesting, hunting for that perfect combination of indicators that will print money.

They’re essentially playing a sophisticated game of pin-the-tail-on-the-donkey that’s exceedingly unlikely to work out.

The second group starts by seeking to understand the market.

They observe patterns, study structural inefficiencies, and dig into why certain phenomena exist.

Their edge is based on real market understanding.

Want to guess which group consistently makes money?

It’s not the donkey-tail pinners.

A better approach

Here’s a simplified version of what actually works:

1. Start with observation. Notice something interesting in the markets, or learn about it from other traders.
2. Think about why it exists. Is it driven by risk preferences? Structural constraints? Product mechanics?
3. Look for evidence. Use simple data analysis (not complex backtests) and try to invalidate your hypothesis (not prove yourself right).
4. Trade the edges that actually stack up, learn from the feedback, and refine your approach.
5. Repeat this process until it becomes second nature.

The magic happens

After a few years of this approach, something magical happens:

• You develop what I call “trader smarts” – an intuitive feel for what might work before you even crunch the numbers.
• You become highly efficient at finding and implementing new edges.
• Most importantly, you’re at peace with the inherent uncertainty of the markets.

Now contrast this with the data mining approach:

• Run endless parameter optimisations
• Execute Monte Carlo simulations and statistical tests for data mining bias
• Find something that passes your tests
• Watch it fail miserably in live trading
• Repeat while slowly losing your mind and your money

Occasionally, you’ll stumble onto something that works. But crucially, you’ll never understand why it works. You’re just throwing statistical darts in the dark.

And since you never understand why, you never learn the lessons that develop your trader smarts.

And here’s the kicker: that nagging anxiety from trading something you don’t understand never goes away.

Because deep down, you know you don’t really understand what you’re trading. You’re always a week of negative returns away from questioning everything.

Look, I know the temptation to take shortcuts.

And if I’m completely honest, everyone will dabble with data mining at some point on their journey (I certainly did).

But consider this the next time someone shows you a strategy they “discovered” through data mining:

• Anyone can make a backtest look good if they try enough combinations.
• Anyone can make it pass out-of-sample tests if they try enough combinations.
• Anyone can make it pass statistical tests for robustness if they try enough combinations (and don’t report how many tests they’ve run, which no one ever does).

Occasionally, you might even find something that makes money in live trading.

But you won’t learn anything useful.

You won’t develop those crucial trader smarts. And without them, you’re building your trading career on quicksand.

A contradiction?

Here’s something that might sound contradictory:

Data mining can actually be a useful tool for trading, in some cases.

But it requires that you carefully constrain the parameter space according to what you know about the markets. And that you sanity check everything.

Essentially, it requires trader smarts!

So you might as well focus on acquiring them because you’re going to need them one way or another.

Your time is your most precious resource. Invest it in understanding markets, not mining data. Yes, it’s harder in the short term. But it’s the only path to succeeding in this game in the long term.