Confessions of a Recovering Engineer (or Why Engineers Make Bad Traders)

The article “Confessions of a Recovering Engineer (or Why Engineers Make Bad Traders)” was originally posted on Robot Wealth blog.

Here’s something that might shock you:

Engineers make the worst traders.

I’m speaking from experience here.

As an engineer who transitioned to trading, it wasn’t until I stopped thinking like an engineer that I started to make progress.

You can read all about this in my case study, by the way.

Engineers like me are trained to solve implementation problems. We build bridges based on well-understood physics.

The challenge isn’t understanding gravity; it’s designing an efficient structure given a specific set of constraints.

Engineering solves implementation problems.

And when I first approached trading, I did what engineers do: I assumed I was looking at an implementation problem, and I jumped straight into solving it.

I built elaborate backtesting frameworks, optimized parameters, and created complicated machine learning systems before I even understood what edge I was trying to capture.

This approach is completely backwards for trading.

Here’s the key point:

In trading, you’re not starting with a well-understood phenomenon like you do in engineering.

You’re starting with an observation of some market effect. Something you’ve noticed yourself, read about somewhere, or heard from other traders.

But you don’t know anything about this phenomenon!

• What causes it?
• How consistent has it been?
• How strong is it?
• Is it very noisy?
• Is it slow to converge?
• Under what circumstances might it disappear?

How can you engineer a solution for a problem you don’t understand?

You can’t. At least, not very effectively.

Instead, you should design little data analysis experiments that gradually reveal clues about the thing you observed.

In this respect, trading is more like a science project.

There’s a little art in there as well, just as there is in any good science – creative thinking can help you design good experiments and ask the right questions.

But the only engineering is in the implementation of a trading strategy, and that’s the last piece of the puzzle. You wouldn’t even think about this until you understood the basis of your edge.

• You wouldn’t build a bridge without a deep understanding of structural mechanics.
• And you shouldn’t build a trading strategy without a deep understanding of your edge.

Good trading research looks nothing like what most engineers initially attempt:

1. Think about potential causal effects – don’t assume this is settled and well-understood.
2. Start with simple but maximally data-efficient analysis (think scatter plots and factor plots) – not complicated simulations of trading rules.
3. Make small, testable hypotheses. Design experiments that gradually reveal the nature of your observed phenomenon. Ask “What would I see in the data if this were true?” – not “What trading rules should I design?”
4. Try to disprove your ideas quickly. Be paranoid you’re wrong – not confident you’re right.
5. Focus on understanding the phenomenon – not the machine learning framework you’ll use to trade it.

Another trap we engineers fall into is our attraction to precision and determinism. We fail to appreciate the probabilistic nature of edge exploitation – that short-term results are dominated by randomness, long-term results by skill.

We hear about the markets being “low signal to noise.” And we understand this intellectually, but we don’t appreciate what it means for our trading.

This leads us to think we can build a perfect system for capturing a single edge to deliver that up-and-to-the-right equity curve.

But it turns out this is a waste of time for retail traders, given the variance of the edges we can realistically compete in.

Instead, embrace the mayhem of any single strategy (be OK with it having high variance) and focus on the bigger picture:

• Finding additional things to trade
• Obsessing over your exposures
• Looking at how each component contributes to the whole to produce something extraordinary at the portfolio level.

In fact, the ability to trade multiple edges is your biggest edge as a systematic trader. So embrace it!

Summary

If engineers unlearned some of their engineering training, they’d make much better traders:

• Research before implementation: Unlike engineering, trading requires scientific exploration of your edge before building any strategy. Understand the phenomenon first.
• Simpler analysis is better: Use direct, data-efficient methods to understand market effects rather than complicated backtests.
• Embrace the mayhem: Engineers’ desire for precision and determinism conflicts with markets’ inherently noisy, probabilistic nature.
• Portfolio thinking beats obsessing over a single edge: Your greatest edge as a systematic trader is the ability to trade multiple uncorrelated strategies, not perfecting any single approach.
• Disprove quickly: Be paranoid you’re wrong rather than confident you’re right. Design experiments to challenge your hypotheses, not confirm them.

When you first observe a market effect, don’t be an engineer.

Instead, unleash your inner scientist.

Only when you understand the phenomenon and are ready for implementation should you let an engineer loose on the problem.