The Photon and the Packet: Inside the Architecture of a High-Frequency Trading System

The opening bell is not a bell anymore. On modern exchanges, the market doesn’t open with a sound; it opens with a photon. A high-frequency trading (HFT) system doesn’t react to the tape—it anticipates the tape. It is a machine built to operate in a temporal dimension that is only partially accessible to humans. We think in milliseconds; an HFT system lives in microseconds and nanoseconds, a realm where the speed of light in a fiber optic cable becomes a limiting factor, not just a number in a physics textbook. The difference between profit and loss in this world is not a good idea or a bad trade; it is the time it takes for a packet of data to travel from a server in New Jersey to a matching engine in Secaucus.

This might sound like the obscure territory of a specialist quant or a hardware hacker, but for any serious software engineer, systems architect, or computer scientist, the internals of an HFT system represent the ultimate expression of applied computer science. It is where all the theoretical constraints you learned in school—memory latency, CPU cache coherency, thread contention, garbage collection—become brutal, existential realities. In a standard web application, a 50-millisecond delay is an annoyance. In a web-scale database, a 5-millisecond lock contention is a crisis. In an HFT system, a single microsecond of unnecessary latency is a catastrophic flaw. The order book is not just a data structure; it is the battlefield. The matching engine is not just a function; it is the arbiter of wealth.

So, why does this matter to you, the architect building a high-throughput microservice or the engineer optimizing a real-time analytics pipeline? Because HFT is where the “best practices” of modern system design are forged under unimaginable pressure. The techniques required to build a fast order book—lock-free programming, cache-line alignment, memory pooling, and kernel bypass—are the same techniques that eventually trickle down to make the rest of our infrastructure faster. Understanding the internals of an HFT system gives you a mental model for thinking about performance at the absolute edge. It forces you to confront the physical limits of computation and communication. And in doing so, it makes you a better engineer, regardless of your domain.

In this deep-dive post, we will peel back the layers of a modern HFT stack—from the network interface card to the trade decision engine. We will examine the data structures that make an order book fast, the concurrency models that handle millions of events per second without locks, the kernel-bypass techniques that let user-space processes talk directly to hardware, and the hardware/software co-design that shaves nanoseconds off critical paths. Along the way, we will include concrete code snippets (in C++), real-world latency numbers, and design trade-offs. By the end, you will not only understand how an HFT system works—you will appreciate why it is the ultimate testbed for computer science principles.

Section 1: The Need for Speed – Latency in HFT

Before we dive into architecture, we must understand the single most important metric in HFT: latency. In a typical distributed system, latency is often measured end-to-end—the time between a client request and a server response. In HFT, the game is won or lost in the tail of that distribution. A trading strategy that is consistently 1 microsecond slower than a competitor will never execute profitable trades, because better prices will always be taken first. Latency is not just important; it is everything.

Types of Latency

In an HFT system, we distinguish between several latency components:

• Propagation latency: The speed-of-light delay in the physical medium. For example, in a standard single-mode fiber, light travels at about 200,000 km/s (refractive index ~1.5). Between New York (NYSE) and Chicago (CME), the straight-line distance is about 1,200 km. The theoretical minimum one-way delay is about 6 milliseconds. In practice, with fiber routing and regeneration, it’s closer to 7–8 ms. HFT firms have built microwave networks that achieve ~4 ms by using line-of-sight transmission in air, which is faster than fiber (air refractive index ~1.0003). Microwave latency between the NY metro area and the Chicago area can be as low as 4.2 ms. That 2–3 ms advantage is worth billions in arbitrage.

• Processing latency: The time taken by hardware and software to process a packet. This includes network interface card (NIC) receive processing, kernel stack traversal, user-space dispatch, protocol parsing, order book update, and strategy evaluation. In a conventional Linux system, this can easily add 10–50 microseconds. In an optimized HFT system, processing latency is driven below 1 microsecond.

• Queuing latency: The delay introduced when packets or events wait in buffers. Even if your software is fast, network congestion, CPU scheduling, or memory bus contention can introduce jitter. HFT systems are designed to minimize buffering, often using busy-polling instead of interrupts, and pinned CPUs to avoid context switches.

• Synchronization latency: The time required to coordinate state between multiple threads or processes. A lock acquisition that takes 100 nanoseconds might be acceptable in a database, but if it occurs on the critical path of a particular trade, it can blow your entire latency budget.

Measuring Microseconds and Nanoseconds

To optimize, you must measure. But measuring at microsecond precision is tricky. The standard Linux gettimeofday() has microsecond resolution but poor accuracy (it relies on the system timer, which can be skewed by NTP adjustments). Modern HFT systems use the TSC (Timestamp Counter) register on x86 CPUs, which increments at the CPU’s clock frequency (e.g., 3 GHz means one tick every 0.33 nanoseconds). By reading the TSC at the start and end of a code path, you can measure elapsed time in CPU cycles, then convert to nanoseconds. Libraries like libpfc or rdtscp instruction provide a low-overhead way to read TSC.

But even TSC has pitfalls: frequency scaling (SpeedStep, Turbo Boost) can change the clock rate, making cycle-based measurements inconsistent. Modern systems mitigate this by using the invariant TSC, which runs at a constant rate. Additionally, you must ensure that your measurement itself does not add overhead—a function call to gettimeofday might take 20 nanoseconds, which is fine, but an interrupt or context switch during measurement can corrupt the reading.

In practice, HFT systems use hardware timestamps from the NIC. Many modern NICs (e.g., Solarflare, Mellanox) can hardware-stamp incoming packets with the TSC value at the moment the packet hits the wire. This provides a precise, low-jitter timestamp that is unaffected by software delays. The trade-off is complexity: you need to synchronize the NIC’s clock with the system’s TSC, often via PTP (Precision Time Protocol).

Example: Measuring a critical code path using TSC

#include <x86intrin.h>  // for __rdtsc()
inline uint64_t rdtsc() {
    return __rdtsc();
}

void process_order() {
    uint64_t start = rdtsc();
    // ... order book update ...
    uint64_t end = rdtsc();
    uint64_t cycles = end - start;
    // Convert to nanoseconds assuming 3 GHz clock
    double ns = cycles / 3.0;
    // Log or compare
}

The above is a minimal example. In production, you would use _mm_lfence() before and after to serialize instruction execution, because modern CPUs can reorder RDTSC around other instructions. A more reliable pattern is:

uint64_t start = _mm_lfence(); // or asm volatile("lfence" ::: "memory")
start = __rdtsc();
// critical section
uint64_t end = __rdtsc();
_mm_lfence();

Why Microsecond Optimization Matters

Let’s do a back-of-the-envelope calculation. Suppose a trading strategy makes a profit of $0.01 per share on a 100-share trade ($1 profit). If the firm trades 10,000 times per day, that’s $10,000/day in profit. Now suppose a competitor is 10 microseconds faster. In a race to execute against a favorable price, they will get their order in first 99.9% of the time. The slower firm gets zero profit. The faster firm captures all the profit. Over a year (250 trading days), that 10-microsecond advantage yields $2.5 million in additional profit. That is why HFT firms spend millions on hardware, colocation, and engineers.

The discipline of latency optimization in HFT has trickled down to other domains. For example, modern web servers like NGINX use event-driven, non-blocking I/O to avoid thread-per-connection overhead. The kernel’s epoll and io_uring systems are inspired by the need to reduce system call overhead, a concept pioneered in HFT. The Linux kernel’s x86 network stack optimizations (RPS, XPS, busy polling) were driven by the HFT community’s demand for low latency.

Section 2: The Order Book – Data Structures Under Fire

The heart of any trading system is the limit order book (LOB). An LOB contains all outstanding buy (bid) and sell (ask) orders for a particular security. Orders are typically price-time priority: the best price gets filled first; within the same price, the earliest order gets filled first. The LOB must support three core operations:

1. Insert a new order (market or limit).
2. Cancel an existing order.
3. Execute a trade (partial or full fill), which removes orders from the book.

And it must do all this in sub-microsecond time for thousands of orders per second. Moreover, the LOB must provide a snapshot of the current best bid and ask (the top of book) for the strategy engine to make decisions.

Data Structure Choices

The naive approach is to use a balanced binary search tree (e.g., std::map in C++). Price levels are keys; each level holds a queue of orders (often a linked list or vector). Insertion and deletion are O(log N) in the number of price levels. But in practice, most trading activity happens at a few near-the-market price levels, so the tree is shallow. However, the tree itself has overhead: each node allocation (new/delete) can be expensive due to memory management. Moreover, the tree is often not cache-friendly; a walk to find the correct price level may touch multiple cache lines.

A more efficient approach is to use an array indexed by price, if the price granularity is fixed (e.g., 0.01 increments for US equities). For a stock trading between $10 and $200, there are about 19,000 possible price points. An array of 19,000 entries, each pointing to a queue, gives O(1) insertion and deletion. Memory is a few kilobytes per price point (mostly null pointers). However, there are two issues:

• Price shifting: If the stock trades at $100 and moves up to $150, the active price range shifts. But the array must cover the full range; it’s wasted memory, but acceptable.
• Cache locality: Access to the array is a direct index, but the order queues themselves are scattered. A better design is to store the entire order book structure in a contiguous block of memory, using custom allocators.

Another popular choice is the skip list, which offers O(log N) expected time with less rebalancing overhead than a tree. Many HFT implementations use a hybrid: a static array for the most active price levels (top of book) and a skip list for deeper levels.

The Order Queue

Within each price level, orders must be maintained in FIFO order (time priority). A common implementation is a doubly linked list of order objects. Each order object contains fields like:

• order_id (unique identifier)
• quantity (remaining shares)
• timestamp (for priority, often the system’s TSC value)
• next and prev pointers (or indices into an array)

When a new order arrives, it is appended to the end of the list at the appropriate price level. Cancellations require searching for the order (O(N) within a price level, but typically a price level may have tens to hundreds of orders). To support fast cancellation, many systems use an order map (e.g., an open-addressing hash table) keyed by order_id that stores a pointer to the order node. Cancellation then becomes: look up the node in the hash table, remove it from the list, and free the node. The hash table also enables fast modification (e.g., changing quantity). In C++, we can implement a custom allocator (see Section 4) to avoid heap fragmentation and speed up allocation.

The Matching Engine

When a market order or a limit order that crosses the spread arrives, the matching engine must execute against available orders at the best price. This involves iterating through the order queue at the best level, reducing quantity, and removing fully filled orders. If multiple price levels are crossed, we move to the next level. This is a straightforward loop, but it must be fast.

To avoid per-order memory allocation during matching, the order objects are pre-allocated from a memory pool (see Section 5). Additionally, the matching engine may use a lock-free design to allow concurrent insertions and cancellations from multiple threads (e.g., a feed handler thread and a strategy thread). This is where it gets interesting.

Side note on price-time priority vs. pro-rata: Some exchanges (e.g., futures markets) use pro-rata allocation, where orders at the same price level are filled proportionally. That requires a different data structure (e.g., maintaining cumulative quantity and distributing fill slices). Pro-rata is more complex but still manageable.

A Simple Order Book Implementation (C++ Example)

Below is a simplified but realistic sketch of a limit order book using a price array and doubly linked list for each level. We use a global memory pool for orders (preallocated array). We omit locking for clarity; assume single-threaded context.

#include <cstdint>
#include <array>
#include <unordered_map>
#include <vector>

constexpr int MAX_PRICE_LEVELS = 20000; // for stock price up to $200 with $0.01 ticks
constexpr int MAX_ORDERS = 1000000;

struct Order {
    uint64_t order_id;
    uint32_t quantity;
    int64_t price_index; // index into price array (negative for bids? but we'll use separate arrays)
    uint64_t timestamp;  // for time priority within level
    Order* prev;
    Order* next;
};

// Memory pool: simple free list
class OrderPool {
    Order* pool;
    Order* free_head;
public:
    OrderPool(size_t size) {
        pool = new Order[size];
        for (size_t i = 0; i < size-1; ++i) {
            pool[i].next = &pool[i+1];
        }
        pool[size-1].next = nullptr;
        free_head = pool;
    }
    Order* allocate() {
        if (!free_head) return nullptr; // out of memory (handle error)
        Order* o = free_head;
        free_head = o->next;
        o->next = nullptr;
        o->prev = nullptr;
        return o;
    }
    void deallocate(Order* o) {
        o->next = free_head;
        free_head = o;
    }
    ~OrderPool() { delete[] pool; }
};

struct PriceLevel {
    Order* head; // oldest order (time priority)
    Order* tail; // newest order
    uint32_t quantity; // total quantity at this level (optional)
    PriceLevel() : head(nullptr), tail(nullptr), quantity(0) {}
};

class OrderBook {
    // For simplicity use separate arrays for bids and asks (or store sign in price)
    std::array<PriceLevel, MAX_PRICE_LEVELS> bids;
    std::array<PriceLevel, MAX_PRICE_LEVELS> asks;
    // map order_id to Order* for fast cancellation
    std::unordered_map<uint64_t, Order*> order_map;
    OrderPool pool{MAX_ORDERS};
public:
    // Insert a limit order
    void insert(uint64_t order_id, bool is_bid, int64_t price_ticks, uint32_t quantity) {
        Order* o = pool.allocate();
        o->order_id = order_id;
        o->quantity = quantity;
        o->price_index = price_ticks;
        o->timestamp = get_timestamp(); // e.g., rdtsc()

        PriceLevel& level = is_bid ? bids[price_ticks] : asks[price_ticks];
        // Append to end (newest) for time priority
        o->prev = level.tail;
        o->next = nullptr;
        if (level.tail) {
            level.tail->next = o;
        } else {
            level.head = o;
        }
        level.tail = o;
        level.quantity += quantity;
        order_map[order_id] = o;
    }

    // Cancel an order
    void cancel(uint64_t order_id) {
        auto it = order_map.find(order_id);
        if (it == order_map.end()) return;
        Order* o = it->second;
        PriceLevel& level = (o->price_index >= 0) ? asks[abs(o->price_index)] : bids[abs(o->price_index)];
        // Remove from doubly linked list
        if (o->prev) o->prev->next = o->next;
        else level.head = o->next;
        if (o->next) o->next->prev = o->prev;
        else level.tail = o->prev;
        level.quantity -= o->quantity;
        order_map.erase(it);
        pool.deallocate(o);
    }

    // Execute a market order of given size
    uint32_t market_order(bool is_bid, uint32_t quantity) {
        uint32_t remaining = quantity;
        // For buy market, we execute against asks; for sell, against bids
        auto& levels = is_bid ? asks : bids;
        // Iterate price levels from best (lowest ask for buys, highest bid for sells)
        // We'll not implement direction here for brevity.
        // Pseudocode: for each price level in ascending (ask) or descending (bid)...
        // ...
        return remaining;
    }
};

Crucial optimizations omitted in this example:

• Cache-line padding: PriceLevel objects are small and often adjacent. To avoid false sharing between threads, each PriceLevel should be padded to a cache line (64 bytes). Similarly, order objects should be aligned.
• Separate bid/ask arrays: In reality, the same price can have both bids and asks (e.g., limit orders at the same price on both sides). But in the above, we split them. A unified array with a flag is also possible.
• Time priority within level: Using TSC timestamps works only if all inserts happen on the same thread. In a multi-threaded scenario, you need a global monotonically increasing sequence number (e.g., via atomic increment) to establish order across threads.
• Memory allocation: The pool is a simple free list, which is fast but can fragment if orders are allocated and freed in random order. A better approach is to use a bump allocator for each thread (thread-local allocation caching) and only return to a global pool rarely.

The order book is only half the battle. Next, we have to protect it from concurrent access.

Section 3: Lock-Free Programming and Concurrency

In classic systems, you would protect the order book with a mutex. But a mutex can cause context switches, sleeping, and priority inversion. Even a spinlock introduces contention on a cache line, causing cache coherence traffic. In HFT, we want to avoid any form of blocking. The solution is lock-free (or less commonly, wait-free) data structures.

The Problem with Locks

Consider a scenario with two threads: a market data feed handler that receives order events and updates the book, and a strategy engine that reads the book to make decisions. If both threads access the same order book, you need synchronization. A mutex would look like:

std::mutex mtx;
void on_order_event(Event e) {
    std::lock_guard<std::mutex> lock(mtx);
    order_book.update(e);
}
void strategy_loop() {
    while (true) {
        auto snapshot = order_book.get_snapshot(); // reads under lock
        // ... decision ...
    }
}

The problem: even if the critical section is 50 ns, the overhead of pthread_mutex_lock can be 100–200 ns (including the atomic compare-and-swap (CAS) to acquire the lock, plus potential cache misses). Also, if the strategy thread holds the lock while computing (which it shouldn’t), the feed handler blocks, increasing latency. Moreover, the lock itself is a shared variable that bounces between caches with every acquisition. This is a disaster for tail latency.

Lock-Free via CAS

Lock-free programming relies on atomic operations like compare-and-swap (CAS) and fetch-and-add, along with careful memory ordering. The basic idea: instead of acquiring a lock, each thread modifies the data structure using atomic operations that fail if another thread interfered. If the operation fails, it retries.

A classic lock-free data structure is the Michael-Scott queue (a lock-free FIFO). For an order book, we often need a lock-free ordered map or skip list. That is significantly harder.

One common HFT approach is to use read-copy-update (RCU). RCU allows readers to access a data structure without any locks, while writers make a copy, modify it, and then atomically swap a pointer. Readers see either the old or the new version, but they never see a partially updated state. In C++, you can implement RCU using std::atomic with memory_order_acquire/release. However, RCU requires a grace period to reclaim memory after the update, which adds complexity.

Another approach is to restrict concurrency design: use one thread for all order book modifications (the “feed handler” thread) and multiple readers (strategy threads) that only read using atomic loads. This is single-writer, multiple-reader (SWMR). It is simpler and very fast because the writer uses no locks—it is the sole mutator. Readers can read without any lock, provided they use atomic loads to ensure they see consistent data (e.g., loading a snapshot that the writer updates periodically). This is often called a versioned snapshot.

Single-Writer, Multiple-Reader with Snapshots

The idea: the order book itself is a mutable structure owned by the writer thread. The writer updates it in real-time as events arrive. For readers, we maintain an atomic shared pointer to a read-only snapshot of the order book. The writer reconstructs a fresh snapshot periodically (e.g., every 100 microseconds) or on demand, and atomically swaps the pointer. The reader loads the pointer (atomic load) and reads the snapshot. The snapshot is immutable, so no locks needed.

But there is a latency issue: the snapshot is always slightly outdated. For some strategies, that is acceptable; for others, it’s not. If the strategy needs the latest top-of-book as fast as possible, it could read directly from the writer’s data structures using atomic loads on specific fields (like the best bid price). This can be done if the writer publishes single atomic variables per price level (e.g., std::atomic<uint64_t> best_bid_price; std::atomic<uint32_t> best_bid_qty;). The writer updates these atomically after each event. The reader reads them with memory_order_relaxed or acquire.

However, there is a risk: the reader might see the best bid price from one update and the best bid quantity from a different update (tearing). For a price and quantity pair, you can use a double-word CAS (std::atomic<__int128>) to update both as a single atomic 16-byte store. Many modern x86 CPUs support 16-byte CAS (cmpxchg16b). This is used in practice to publish a consistent top-of-book snapshot.

Lock-Free Order Book Modification

For a fully lock-free order book that supports concurrent writes from multiple threads (e.g., multiple feed handlers), things become extremely complex. There is a known lock-free double-ended priority queue (used for event scheduling) but not many for LOB. Most production HFT systems avoid this complexity by dedicating a single core to the order book and using bounded queues (ring buffers) to pass events between threads. The ring buffer itself is lock-free (single producer, single consumer variant). This design is simpler and more predictable.

A typical architecture:

![Architecture diagram description]

• Network thread: Receives raw packets from NIC (via kernel bypass, see Section 4). Parses protocol (e.g., binary market data feeds). Pushes parsed events into a lock-free SPSC (single producer, single consumer) queue.
• Order book thread: Consumes events from the queue, updates the order book, and publishes the new top-of-book (atomically) for strategy threads.
• Strategy threads (multiple): Each runs on its own core. They read the current market state via atomic loads. When ready to submit an order, they send a message (again via a lock-free queue) to the order routing thread, which sends it to the exchange.

This architecture separates concerns and avoids concurrency on the order book structure itself. The order book thread has exclusive write access, so it can use efficient mutable structures with no locking. The ring buffers between threads use atomic operations only for head/tail indices, which are fast.

Lock-Free Ring Buffer Example (SPSC)

template<typename T>
class SPSCRingBuffer {
    static constexpr size_t SIZE = 1024;
    T buffer[SIZE];
    std::atomic<size_t> head{0}; // writer index (producer)
    std::atomic<size_t> tail{0}; // reader index (consumer)
    size_t cached_head; // local to reader
    size_t cached_tail; // local to writer
public:
    bool try_push(const T& item) {
        size_t current_tail = cached_tail;
        size_t next_tail = (current_tail + 1) % SIZE;
        if (next_tail == head.load(std::memory_order_acquire)) {
            return false; // full
        }
        buffer[current_tail] = item;
        std::atomic_thread_fence(std::memory_order_release);
        cached_tail = next_tail;
        tail.store(next_tail, std::memory_order_release);
        return true;
    }
    bool try_pop(T& item) {
        size_t current_head = cached_head;
        if (current_head == tail.load(std::memory_order_acquire)) {
            return false; // empty
        }
        item = buffer[current_head];
        std::atomic_thread_fence(std::memory_order_acquire);
        cached_head = (current_head + 1) % SIZE;
        head.store(cached_head, std::memory_order_release);
        return true;
    }
};

This uses local cached copies of the other party’s index to avoid atomic loads on every call (the atomic load is only when the cache might be stale). This is a classic optimization in HFT.

Memory ordering note: The memory_order_release on tail.store ensures that the write to buffer is visible before the consumer sees the updated tail. Similarly, memory_order_acquire on head.load ensures we see all producer writes. The thread fence between store and tail ensures ordering even with compiler reordering.

Memory Ordering Pitfalls

Understanding C++ memory ordering is essential for HFT. memory_order_relaxed is fastest but provides no guarantees; acquire/release is used for producer-consumer patterns; seq_cst is the default but also the most expensive (it generates a full memory barrier mfence on x86). Often, on x86, acquire/release do not generate additional instructions (because x86 is already acquire-release), but seq_cst generates mfence. However, you must still use the ordering arguments to prevent compiler reordering.

In HFT, we want to minimize atomics and barriers. The single-writer design reduces the need for atomics on data itself. The ring buffer needs only two atomic loads/stores per push/pop. That is acceptable.

Section 4: Kernel Bypass – Talking to the Network at Wire Speed

The Linux kernel’s network stack is a marvel of engineering, but it is not designed for ultra-low latency. Every packet that arrives triggers:

1. An interrupt from the NIC.
2. The interrupt handler runs in kernel context.
3. The packet is copied from the NIC’s DMA buffer into a kernel socket buffer (sk_buff).
4. The kernel processes the protocol (e.g., UDP, TCP).
5. Depending on the socket type, the packet is queued in a socket receive buffer.
6. A system call (e.g., recvmsg) is made by the user application to copy the packet into user space.
7. Context switch back to user space.

Each of these steps adds microseconds of latency and, more importantly, jitter. In HFT, we can’t tolerate that. The solution: kernel bypass.

User-Level Networking

Kernel bypass allows the user-space application to talk directly to the NIC hardware without kernel involvement. The application allocates memory that is mapped into the NIC’s DMA region, so the NIC can write packet data directly into user-space buffers. The application polls for new packets by checking a flag in a shared descriptor ring, avoiding system calls entirely.

The two most prominent frameworks are:

• DPDK (Data Plane Development Kit): An open-source project (Intel) that provides a set of libraries and drivers for user-space networking. DPDK takes over the NIC from the kernel, allocates huge pages, and uses polling mode PMD (Poll Mode Driver). It achieves packet throughput of millions of packets per second per core with microsecond latency.

• Solarflare OpenOnload: A proprietary kernel bypass solution from Xilinx (formerly Solarflare). OpenOnload is a TCP/UDP stack that runs in user space, transparently replacing the kernel stack for sockets. It can achieve sub-microsecond latency for TCP.

• Mellanox (now Nvidia) RDMA: Remote Direct Memory Access. For InfiniBand and RoCE (RDMA over Converged Ethernet), applications can read/write directly to remote memory with kernel bypass. This is used for high-speed communication between trading servers.

DPDK Basics

With DPDK, the typical workflow:

1. Bind a NIC to the DPDK driver (e.g., igb_uio or vfio-pci).
2. Initialize the EAL (Environment Abstraction Layer), which sets up huge pages, memory, and cores.
3. Allocate memory pools for packet buffers (mbufs) using rte_pktmbuf_pool_create.
4. Configure port (NIC) with number of RX/TX queues.
5. Main loop: On a dedicated core, repeatedly call rte_eth_rx_burst to get a burst of packets. Process them (parse, update order book, etc.). Optionally transmit using rte_eth_tx_burst.

No system calls, no interrupts. The CPU spins in a tight loop polling the NIC. This is called busy polling and, while it wastes CPU cycles (100% core usage), it provides the lowest and most consistent latency.

Example: DPDK receive loop (simplified)

#include <rte_eal.h>
#include <rte_ethdev.h>
#include <rte_mbuf.h>

#define NUM_MBUFS (8192)
#define BURST_SIZE 64

int main(int argc, char *argv[]) {
    rte_eal_init(argc, argv);
    // ... setup port 0 with one RX queue ...
    struct rte_mempool *mbuf_pool = rte_pktmbuf_pool_create("MBUF_POOL", NUM_MBUFS,
        256, 0, RTE_MBUF_DEFAULT_BUF_SIZE, rte_socket_id());

    for (;;) {
        struct rte_mbuf *bufs[BURST_SIZE];
        uint16_t nb_rx = rte_eth_rx_burst(0, 0, bufs, BURST_SIZE);
        for (int i = 0; i < nb_rx; i++) {
            struct rte_mbuf *m = bufs[i];
            uint8_t *data = rte_pktmbuf_mtod(m, uint8_t*);
            // parse packet, e.g., SIP or binary market data
            parse_packet(data, m->data_len);
            rte_pktmbuf_free(m);
        }
    }
}

DPDK also provides precise hardware timestamping: many NICs can stamp incoming packets with a high-resolution timestamp. DPDK exposes this via the rte_mbuf’s timestamp field (if the NIC is configured for dev_tx_offload_timestamp). This is crucial for measuring latency and for time-based order prioritization.

Challenges of Kernel Bypass

• Complexity: You lose all the kernel’s networking features (firewall, routing, TCP offload). You have to implement or integrate a user-space TCP stack if you need TCP.
• CPU pinning: You must assign dedicated cores to the polling loops. Those cores cannot be shared with other tasks.
• Memory: DPDK uses huge pages (2 MB or 1 GB) for memory pools to reduce TLB misses. This requires system configuration.
• Compatibility: Not all NICs support DPDK. You need specific hardware (Intel, Mellanox, Solarflare). Also, certain features like hardware timestamping may be vendor-specific.

Despite these costs, kernel bypass is standard in HFT. Many proprietary trading firms build their own user-space network stacks using DPDK, often combined with custom protocol parsing that recognizes only the specific market data formats (e.g., Nasdaq’s OUCH, ITCH, or CME’s iLink). This parsing can be done in a few tens of nanoseconds.

Softer Approach: Kernel Pinning and Polling

Not every firm needs full DPDK. Some achieve sub-5 microsecond latency using kernel optimization techniques:

• Set CPU affinity for the network interrupt handler to a dedicated core.
• Use SO_BUSY_POLL on sockets: the kernel busy-waits for data instead of sleeping.
• Use recvmmsg to receive multiple packets per system call.
• Disable nagle, tcp_delay_ack, and use TCP_NODELAY.
• Set socket buffer sizes to avoid drops.
• Use setsockopt(IP_PKTINFO) to get NIC timestamp via SO_TIMESTAMPNS.

These can bring latency down to 1–3 microseconds, which is acceptable for some strategies but not the fastest. The fastest HFT firms all use kernel bypass.

Section 5: Hardware Optimizations – Cache, NUMA, and Beyond

Even with kernel bypass and lock-free data structures, you are still subject to the memory hierarchy. A single DRAM access takes about 100 nanoseconds (on a good day). L1 cache hit is 1 nanosecond (3 cycles at 3 GHz). The difference is huge. In an HFT system, we must keep the hot data in L1 cache.

Cache Line Alignment

False sharing occurs when two threads write to different variables that happen to lie on the same cache line. The cache coherence protocol forces a cache line invalidation, even though the threads are not sharing data. To avoid this, we align frequently accessed, thread-local data to 64-byte boundaries. In C++:

struct alignas(64) PerThreadData {
    uint64_t events_processed;
    // other fields...
};

Similarly, the order book’s price level structure should be padded:

struct PriceLevel {
    Order* head;
    Order* tail;
    uint32_t quantity;
    uint32_t padding; // fill to 16 bytes?
    // But we need 64 bytes: add more padding
    uint8_t _pad[64 - sizeof(Order*)*2 - sizeof(uint32_t)*2];
};
static_assert(sizeof(PriceLevel) == 64);

This ensures that two different price levels are on different cache lines and can be modified independently by different threads without false sharing.

Memory Pooling and Object Reuse

Allocating and deallocating objects (new/delete) in the critical path is unacceptable because:

• It may involve system calls (if the heap manager uses mmap).
• It causes fragmentation.
• It touches global allocator locks.

Instead, HFT systems use object pools or region-based allocation. We saw an example in Section 2. The pool pre-allocates a large contiguous memory block and hands out objects from a free list. The free list operations are just pointer updates (no locks if single-threaded). If you need multiple threads allocating, you can use thread-local caches with occasional global scavenging.

Another technique is bump allocators: for short-lived data (e.g., packet parsing buffers), allocate sequentially from a large block, then reset the pointer at the end of each processing cycle. This avoids per-object overhead.

NUMA Awareness

Modern multi-socket servers have Non-Uniform Memory Access (NUMA). Memory attached to one socket is faster for cores on that socket (20–30 ns faster than remote socket). For an HFT system, you must ensure that the thread modifying the order book and its associated memory all reside on the same NUMA node. Additionally, the NIC should be connected to the same NUMA node via PCIe. You can check this with lstopo and pin threads using libnuma.

In DPDK, the EAL automatically detects NUMA and encourages allocation on the correct socket. When setting up memory pools, pass rte_socket_id().

Use of Fixed-Function Hardware: FPGAs and ASICs

Some HFT firms go beyond software optimization and use FPGAs (Field-Programmable Gate Arrays) to implement trading logic directly in hardware. An FPGA can process packets at line rate (e.g., 10 Gbps or 40 Gbps) with deterministic latency measured in nanoseconds. Common FPGA applications:

• Packet parsing: Decode market data frames (e.g., ITCH) in hardware, with latency under 100 ns.
• Order book construction: Maintain the entire order book in FPGA memory. The FPGA can produce a snapshot of the top of book every few nanoseconds.
• Trade decision: Some simple strategies (e.g., statistical arbitrage) can be run in hardware, bypassing the CPU entirely.

However, FPGA development is expensive (hardware cost, firmware engineers, long debug cycles). Still, the latency advantage is clear: a C++ processing loop that takes 500 ns on a CPU can be done in 10–20 ns on an FPGA.

ASICs (Application-Specific Integrated Circuits) are the next level, typically used by exchanges themselves (e.g., Nasdaq’s FPGA-based matching engine for certain instruments). But ASIC development is even more costly.

Most HFT firms compromise: they use CPU for the general strategy and FPGA for ultra-fast feed processing and order arbitration.

Section 6: Garbage Collection – The Silent Killer

If you are using a garbage-collected language like Java or C#, you will hit a latency wall. A stop-the-world GC pause can last tens to hundreds of milliseconds. Even concurrent GC can cause jitter due to CPU contention. Most HFT systems are written in C++ (or Rust recently). But even in C++, dynamic memory allocation can cause latency spikes if the allocator calls into the OS.

The Java HFT Myth

There are some Java-based trading systems (e.g., using the Chronicle library’s off-heap memory and no GC allocations in the hot path). They can achieve low latency by:

• Pre-allocating all objects off the heap (using direct byte buffers).
• Avoiding object creation in the main loop.
• Using sun.misc.Unsafe for CAS and memory ordering.

But the JVM still has JIT warm-up time, safepoints, and deoptimization. While it’s possible to achieve microseconds, the tail latency is harder to control. For the fastest tier, C++ remains the language of choice.

Memory Pool in Practice

In C++, we implement a memory pool for order objects. For a system processing 1 million orders per day, we allocate 1 million orders at startup. The pool is a simple array plus free list. To make it thread-safe without locks, we can use a per-thread free list and periodically return excess nodes to a global pool using a lock-free stack.

Example skeleton:

class ThreadLocalPool {
    const int CHUNK_SIZE = 4096;
    char* current_chunk;
    int offset;
    // free list of previously freed objects within this thread
    Order* free_list;
public:
    Order* allocate() {
        if (free_list) {
            Order* o = free_list;
            free_list = o->next;
            return o;
        }
        if (offset + sizeof(Order) > CHUNK_SIZE) {
            // allocate new chunk from global pool (with mutex)
            current_chunk = global_allocator::get_chunk();
            offset = 0;
        }
        Order* o = reinterpret_cast<Order*>(current_chunk + offset);
        offset += sizeof(Order);
        return o;
    }
    void deallocate(Order* o) {
        o->next = free_list;
        free_list = o;
    }
};

This minimizes contention and fragmentation.

Zero-Copy Parsing

Another key optimization is zero-copy network parsing. Instead of copying the packet into a separate parse buffer, we parse fields directly from the DPDK mbuf (which is already in user-space memory). We use pointers and offsets. Many market data feeds have a fixed width binary format; we can interpret the packet as a struct cast.

struct MarketDataPacket {
    uint16_t message_type;
    uint64_t timestamp;
    uint32_t order_id;
    char side;
    uint32_t quantity;
    uint64_t price;
    // ... other fields
} __attribute__((packed));

// In the receive loop:
for (int i=0; i<nb_rx; i++) {
    struct rte_mbuf *m = bufs[i];
    const MarketDataPacket* pkt = rte_pktmbuf_mtod_offset(m, const MarketDataPacket*, 0);
    // use pkt directly
}

No copying. This is essential for speed.

Section 7: Clock Synchronization and Time Stamps

Time is everything in HFT. The order in which trades are matched depends on the timestamp assigned by the exchange. But HFT firms also need to measure their own latency precisely. This requires accurate clocks across multiple systems.

Hardware Timestamps from NIC

As mentioned, many NICs can hardware-stamp incoming packets with a TSC value. The NIC has its own free-running counter that feeds into the timestamp register. The counter is driven by a high-quality oscillator (e.g., a 10 MHz reference from a GPS disciplined oscillator). The system software must know the relationship between the NIC’s timestamp counter and the CPU’s TSC to convert to system time.

DPDK provides a function rte_eth_read_clock(port_id, &clock_cycles) to read the NIC’s clock. You can then compute offset.

PTP (Precision Time Protocol)

For coordination across servers (e.g., two colocated servers that both receive market data), you need sub-microsecond clock synchronization. PTP (IEEE 1588) can achieve accuracy of 100 ns or better when implemented with hardware timestamping. The NIC handles clock sync messages in hardware. On Linux, you can use ptp4l and phc2sys to synchronize the system clock to a PTP grandmaster.

In an HFT environment, a common setup is to have a GPS receiver in the data center providing a 1 PPS (pulse per second) signal to a grandmaster clock (e.g., EndRun Technologies). All switches and servers with PTP-capable NICs sync to that.

Timestamping Strategy

Where do we timestamp? There are multiple points:

• Ingress timestamp: When the market data packet hits the NIC. This is the most precise reference for market data latency.
• Egress timestamp: When our order leaves the NIC. This is important to measure round-trip time.
• Software processing timestamp: When our code starts processing a packet (via TSC). This helps us measure internal latency contributions.

By subtracting ingress from egress, we get total round-trip latency including propagation. By subtracting ingress from processing start, we get NIC-to-application latency.

Example of Getting NIC Timestamp in DPDK

// On receiving a packet, get the hardware timestamp if available
struct rte_mbuf *m = bufs[i];
uint64_t nic_timestamp = 0;
if (m->ol_flags & PKT_RX_TIMESTAMP) {
    nic_timestamp = m->timestamp;
}

Then you may convert to nanoseconds using the NIC clock frequency:

uint64_t nic_freq = rte_eth_get_clock_freq(port_id); // e.g., 1000 MHz
uint64_t nic_ns = nic_timestamp * 1e9 / nic_freq;

Section 8: Risk Management – The Safety Net

HFT systems are fast. If there is a bug, it can lose millions in milliseconds. Therefore, any production HFT system has multiple layers of risk checks that operate in less than a microsecond. These checks must be performed before orders are sent to the exchange.

Pre-Trade Risk Checks

Common checks:

• Price collar: The order price must be within a certain percentage of the current market price (e.g., 5%). This prevents fat-finger errors.
• Quantity limit: Max order size per order.
• Rate limit: Maximum number of orders per second.
• Position limit: Cumulative position in a security must not exceed a threshold.
• Duplicate order detection: Same order ID should not be sent twice.

These checks are usually implemented as a separate module that runs on a dedicated core, between the strategy and the order gateway. The strategy creates a “risk ticket” that the risk engine validates. If it fails, the order is rejected and logged.

How Fast Can a Risk Check Be?

A simple price collar check is a few integer comparisons (nano seconds). But to check cumulative position, you need to maintain a map of current positions (on a per-security basis). This map must be updated in real-time as executions (fills) occur. The update must also be atomic or lock-free. One approach: the same order book thread that maintains the order book also maintains risk state (because it sees all fills). Then the risk engine can be a simple function that queries the current state.

Alternatively, a dedicated risk manager thread listens to both market data and fill events, and maintains positions. It receives order requests via a lock-free queue, checks, and forwards to the gateway.

Kill Switch

If a risk breach occurs (e.g., position limit exceeded), the system should not allow any new orders and should attempt to cancel existing ones. This is the kill switch. It can be implemented as a hardware relay that physically disconnects the network cable (managed by a watchdog timer). When the software detects a critical error, it triggers the kill switch, which opens the circuit, preventing any further outgoing packets.

Section 9: Putting It All Together – An End-to-End HFT System Architecture

Let’s now sketch a complete, simplified architecture of an HFT trading engine for a single stock. We will trace the journey of a market data packet.

Hardware:

• Two servers: A “feed handler” server (colocated with the exchange) and an “order gateway” server.
• Each server has a dual-socket CPU, each with 8–16 cores. The NICs are DPDK-compatible (e.g., Mellanox ConnectX-5 or Solarflare SFN8000).
• Both servers are connected to the exchange network via an ultra-low latency switch (e.g., Arista 7130 with cut-through switching).
• GPS clock provides 1 PPS to a PTP grandmaster, which syncs the NIC clocks.

Software Stack:

• Feed Handler Server:

  • Core 0: DPDK polling loop – receives raw packets, hardware-timestamped, parses protocol (e.g., NASDAQ ITCH). Produces a stream of order book events.
  • Core 1: Order book thread – consumes events from a lock-free SPSC ring buffer from Core 0. Updates its internal order book (price array + order map). Publishes atomic top-of-book (bid price, bid size, ask price, ask size) via a 16-byte atomic variable (or via a pointer to a small snapshot struct).
  • Core 2–3: Strategy threads – each reads the atomic top-of-book. Implements a simple strategy (e.g., if bid-ask spread > threshold, place a limit order). When decision made, writes an order request into a lock-free queue destined for order gateway.
  • Core 4: Order routing thread – reads from the outgoing queue, performs pre-trade risk checks (price collar, position limits), then sends the order via a separate DPDK port directly to the exchange’s matching engine.

• Order Gateway Server (optional, may be on same server): Handles order acknowledgments and fill confirmations from the exchange.

• Monitoring:

  • A separate core runs a metrics thread that collects latency histograms (using TSC) and prints them periodically.
  • A health monitor thread checks the kill switch.

Typical Latencies (from market data packet to decision):

• NIC to application (DPDK): 300–500 ns
• Protocol parsing: 50–100 ns
• Order book update (limit order inserted): 100–200 ns
• Atomic publish: 20 ns
• Strategy decision (simple rule): 100 ns
• Total: ~600–900 ns.

Plus propagation to exchange (one-way ~1 ms for microwave Chicago-New York). So the internal part is negligible compared to speed-of-light. That is why microwave networks matter so much.

Section 10: Lessons for Non-HFT Engineers

After this deep dive, you might think HFT is irrelevant to your day job. But many of the techniques described here have been adopted by mainstream software.

Lock-Free Queues are used in modern game engines, high-performance web servers (e.g., Seastar), and database systems (e.g., ScyllaDB). Kernel Bypass inspired technologies like AF_XDP (eXpress Data Path) in Linux, which provides a fast path for packet processing without full DPDK complexity. Memory Pooling is increasingly used in latency-sensitive applications like ad tech, real-time bidding, and video streaming. Cache Line Alignment is a known best practice for any multi-threaded low-latency system.

Understanding HFT also teaches you to think in terms of physical limits. You begin to treat time not as a resource but as a dimension you must measure and minimize. You learn that the difference between a 1 ms latency and a 10 ms latency is not linear—it’s often the difference between existence and non-existence in a market. That mindset can elevate any engineer’s approach to performance.

Conclusion

High-frequency trading is not just about making money faster. It is the extreme sport of computer science—a place where hardware, software, and physics collide. The machines that trade at nanosecond speeds are marvels of optimization, built by engineers who treat every cycle, every byte, and every photon with respect. The order book is a battlefield, but it is also a classroom.

As we have seen, building a modern HFT system requires mastery of data structures, concurrency, memory management, network programming, and even hardware design. The techniques are not theoretical; they are implemented every day in trading floors across the globe. And while the arms race continues (with AI and machine learning adding new dimensions), the fundamentals remain: minimize latency, maximize determinism, and never block.

Next time you send an HTTP request that takes 200 ms, remember that there is a system out there that processes 200,000 such operations in that same interval, and each one of them is a calculated move in a zero-sum game. The photon that opened the market today did so at the speed of light. The engineers who made it possible are the ones who understand that the fastest code is not code at all—it is the absence of code. It is a perfectly tuned machine where every instruction knows its place, every memory access has been pre-warmed, and every nanosecond is accounted for.

That is the art of HFT. And it is, in its own way, beautiful.

If you found this deep dive valuable, share it with a fellow engineer. And if you have ever wondered what it takes to build a system that lives at the edge of physics, consider that you already have the skills—you just need to think in nanoseconds.