{"id":233840,"date":"2025-11-04T12:59:02","date_gmt":"2025-11-04T17:59:02","guid":{"rendered":"https:\/\/ibkrcampus.com\/campus\/?p=233840"},"modified":"2025-11-04T12:59:48","modified_gmt":"2025-11-04T17:59:48","slug":"why-microarchitecture-matters-more-than-algorithms-in-high-frequency-trading","status":"publish","type":"post","link":"https:\/\/www.interactivebrokers.com\/campus\/ibkr-quant-news\/why-microarchitecture-matters-more-than-algorithms-in-high-frequency-trading\/","title":{"rendered":"Why Microarchitecture Matters More Than Algorithms in High-Frequency Trading"},"content":{"rendered":"\n

Abstract<\/h2>\n\n\n\n

In high-frequency trading (HFT), the decisive edge often arises not from a new mathematical model but from the way software exploits hardware. When every nanosecond matters, understanding CPU microarchitecture, cache behavior, branch prediction, speculative execution, memory topology (NUMA), and kernel-bypass networking can produce outsized latency gains compared to incremental improvements in trading logic. This whitepaper presents a formal yet accessible treatment for PMs, traders, and engineers: we model latency, explain pipeline hazards with analogies and equations, demonstrate cache-aware data layouts with C++ code, outline kernel-bypass packet paths, and provide system design guidance, benchmarks, and a one-page summary of actionable takeaways.<\/p>\n\n\n\n

Contents<\/h2>\n\n\n\n
    \n
  1. Preface<\/li>\n\n\n\n
  2. Latency as a Formal Budget<\/li>\n\n\n\n
  3. CPU Pipelines, Branch Prediction, and Speculation<\/li>\n\n\n\n
  4. Cache Hierarchy and Data Locality<\/li>\n\n\n\n
  5. NUMA, Thread Placement, and Contention<\/li>\n\n\n\n
  6. Kernel Bypass and NIC-Level Paths<\/li>\n\n\n\n
  7. Microbenchmarking and Profiling<\/li>\n\n\n\n
  8. Performance Comparison Tables<\/li>\n\n\n\n
  9. System Design Blueprint<\/li>\n\n\n\n
  10. Worked Example: Hot-Path Refactor<\/li>\n\n\n\n
  11. Putting It All Together: A Practical Checklist<\/li>\n\n\n\n
  12. Key Takeaways<\/li>\n\n\n\n
  13. Conclusion<\/li>\n<\/ol>\n\n\n\n

    1. Preface<\/h2>\n\n\n\n

    This paper was written to clarify a recurring misconception in HFT: that algorithmic ingenuity alone dominates performance. In reality, the performance envelope is defined by physics (propagation delay), CPU microarchitecture (pipelines, caches, predictors), memory topology, and operating system boundaries. The goal is to arm mixed audiences\u2014PMs, traders, and engineers\u2014with a shared, rigorous vocabulary and concrete techniques to reduce tick-to-trade latency.<\/p>\n\n\n\n

    2. Latency as a Formal Budget<\/h2>\n\n\n\n

    Let end-to-end latency be decomposed as:<\/p>\n\n\n\n

    L<\/em>total<\/sub> = L<\/em>prop<\/sub> + L<\/em>NIC<\/sub> + L<\/em>kernel<\/sub> + L<\/em>user<\/sub> + L<\/em>tx<\/sub> (1)<\/em><\/p>\n\n\n\n

    where Lprop<\/sub> is physical propagation (fiber\/microwave), LNIC<\/sub> NIC and DMA ingress\/egress, Lkernel<\/sub> OS network stack and scheduling, Luser<\/sub> application processing (parse, decide, risk, build order), and Ltx<\/sub> transmit path.<\/p>\n\n\n\n

    Observation.<\/strong> In colocated HFT, Lprop<\/sub> is bounded by geography; Ltx<\/sub> and LNIC<\/sub> are bounded by hardware. Therefore most controllable variance lies in Lkernel<\/sub> + Luser<\/sub>. Microarchitectural work primarily reduces Luser<\/sub> and avoids Lkernel<\/sub> via bypass.<\/p>\n\n\n\n

    2.1. A simple improvement model<\/h3>\n\n\n\n

    If a fraction p<\/em> of Ltotal<\/sub> is improved by factor S<\/em> (e.g., kernel-bypass improving the stack), then an Amdahl-style bound is:<\/p>\n\n\n\n

    L’total<\/sub> = (1 \u2212 p)Ltotal<\/sub> + (p\/S)Ltotal<\/sub> = (1 \u2212 p(1 \u2212 1\/S))Ltotal<\/sub> (2)<\/p>\n\n\n\n

    Microarchitectural work targets a large p<\/em> (broad code paths) and large S<\/em> (order-of-magnitude wins like bypass or cache hits).<\/p>\n\n\n\n

    3. CPU Pipelines, Branch Prediction, and Speculation<\/h2>\n\n\n\n

    Modern CPUs use deep pipelines and speculative execution to keep functional units busy. A conditional branch that is mispredicted flushes in-flight work.<\/p>\n\n\n\n

    3.1. Penalty model<\/h3>\n\n\n\n

    If the effective misprediction penalty is Cb<\/sub> cycles, at clock f<\/em>,<\/p>\n\n\n\n

    Lb<\/sub> = Cb<\/sub>\/f (3)<\/p>\n\n\n\n

    Typical Cb<\/sub> is on the order of 10\u201320 cycles. Over a hot path with Nb<\/sub> unpredictable branches, the expected stall is Nb<\/sub> \u00b7 Pmiss<\/sub> \u00b7 Lb<\/sub>.<\/p>\n\n\n\n

    3.2. Analogy for non-engineers<\/h3>\n\n\n\n

    Think of an assembly line that guesses which part arrives next. A wrong guess forces the line to eject partially assembled items and restart that stage. Reducing surprises (predictable code paths) cuts waste.<\/p>\n\n\n\n

    Fetch \u2192 Decode \u2192 Issue \u2192 Exec \u2192 Mem \u2192 WB
    \u2193
    Cond. Branch
    \u2193
    Speculative (predicted)
    \u2193
    mispredict: flush pipeline<\/p>\n\n\n\n

    Figure 1: Compact CPU pipeline with speculative branch and misprediction flush.<\/p>\n\n\n\n

    3.3. Branch-aware coding patterns<\/h3>\n\n\n\n

    Prefer predictable control flow. Replace long if\/else chains with table-driven logic or bitwise masks.<\/p>\n\n\n\n

    \/*** Predictable dispatch using lookup tables ***\/\nusing Handler = void(*)(Order&);\nextern Handler STATE_DISPATCH[NUM_STATES];\n\ninline void process(Order& o) {\n    STATE_DISPATCH[o.state](o); \/\/ predictable, branchless index\n}<\/pre>\n\n\n\n
    Mark likely paths. Compilers accept likelihood hints on hot predicates.<\/p>\n\n\n\n
    \/*** GCC\/Clang likelihood hints ***\/\n#define LIKELY(x) __builtin_expect(!!(x), 1)\n#define UNLIKELY(x) __builtin_expect(!!(x), 0)\n\ninline void route(const Quote& q, double th) {\n    if (LIKELY(q.price > th)) {\n        fast_buy_path(q);\n    } else {\n        slow_sell_path(q);\n    }\n}<\/pre>\n\n\n\n
    Use arithmetic\/bit tricks to avoid branches.<\/p>\n\n\n\n
    \/*** Convert boolean to mask and select without a branch ***\/\ninline void execute_order(bool is_buy, const Quote& q) {\n    uint64_t m = -static_cast<uint64_t>(is_buy); \/\/ 0x..00 or 0x..FF\n    \/\/ select() pattern: (a & m) | (b & ~m)\n    auto side = (BUY & m) | (SELL & ~m);\n    place(side, q);\n}<\/pre>\n\n\n\n
    4. Cache Hierarchy and Data Locality<\/h2>\n\n\n\n
    4.1. Hierarchical latencies (indicative)<\/h3>\n\n\n\n
    Level<\/th>Access latency<\/th>Notes<\/th><\/tr><\/thead>
    L1 data cache<\/td>~0.5\u20131 ns<\/td>per-core, tiny, fastest<\/td><\/tr>
    L2 cache<\/td>~3\u20135 ns<\/td>per-core\/cluster<\/td><\/tr>
    L3 (LLC)<\/td>~10\u201315 ns<\/td>shared across cores<\/td><\/tr>
    DRAM<\/td>~100\u2013150 ns<\/td>off-core, orders slower<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
    Implication.<\/strong> A few DRAM misses on a hot path can dominate your entire decision time. Organize data to stream through caches.<\/p>\n\n\n\n
    4.2. From AoS to SoA<\/h3>\n\n\n\n
    \/*** Array-of-Structs (AoS): friendly to objects, unfriendly to caches ***\/\nstruct Order {\n    double px;\n    double qty;\n    char sym[16];\n    uint64_t ts;\n};\nstd::vector<Order> book; \/\/ iterating touches mixed fields -> poor locality<\/pre>\n\n\n\n

    <\/p>\n\n\n\n
    <\/p>\n\n\n\n
    <\/p>\n\n\n\n
    4.2. From AoS to SoA<\/strong><\/p>\n\n\n\n
    \/*** Array-of-Structs (AoS): friendly to objects, unfriendly to caches ***\/\nstruct Order {\n    double px;\n    double qty;\n    char sym[16];\n    uint64_t ts;\n};\nstd::vector<Order> book; \/\/ iterating touches mixed fields -> poor locality<\/pre>\n\n\n\n
    \/*** Structure-of-Arrays (SoA): cache + vectorization friendly ***\/\nstruct Book {\n    std::vector<double> px;\n    std::vector<double> qty;\n    std::vector<uint64_t> ts;\n    \/\/ symbols handled separately (IDs or interned)\n};\ninline double vwap(const Book& b) noexcept {\n    \/\/ contiguous arrays enable SIMD and cache-line efficiency\n    double num=0.0, den=0.0;\n    for (size_t i=0;i<b.px.size();++i){ \n        num += b.px[i]*b.qty[i]; \n        den += b.qty[i];\n    }\n    return num\/den;\n}<\/pre>\n\n\n\n
    4.3. Cache-line alignment and padding (diagram)<\/strong><\/h3>\n\n\n\n
    False sharing: two hot counters share the same cache line 
    [Counter A | Counter B | … … ] \u2192 One 64B cache line 

    Aligned: each counter in its own line 
    [Counter A (64B aligned)] 
    [Counter B (64B aligned)]<\/p>\n\n\n\n
    Figure 2: False sharing vs. aligned counters. Padding\/alignment prevents cache-line contention.<\/p>\n\n\n\n
    4.4. Warm-up & steady state<\/h3>\n\n\n\n
    Pre-touch (“warm”) hot data at startup: parse a few messages, exercise parsers and fast paths so instruction\/data caches and predictors are primed before opening the gate.<\/p>\n\n\n\n
    5. NUMA, Thread Placement, and Contention<\/h2>\n\n\n\n
    5.1. NUMA effects<\/h3>\n\n\n\n
    On multi-socket servers, memory is attached to sockets. Remote-node memory adds tens of ns per access. Pin hot threads and allocate memory from the same NUMA node.<\/p>\n\n\n\n
    \/*** Linux: set thread affinity and memory policy (pseudo) ***\/\n\/\/ Use pthread_setaffinity_np() to bind to CPU(s) on NUMA node 0\n\/\/ Use mbind() or numactl to prefer local memory for hot heaps\/buffers<\/pre>\n\n\n\n
    5.2. Lock avoidance<\/h3>\n\n\n\n
    Contention costs explode under parallel load. Prefer:<\/p>\n\n\n\n