How lueasf/hftrading works
Project Overview
• HFTrading is a high-frequency trading tool built using both C++ and Python. It integrates components for real-time data (streams), trading algorithms (algos), backtesting (backtest), communication (via Aeron and NATS), and data exporting (exporter).
• The project uses a mix of lower-level performance-critical modules (C++ source code in streams, communication, and algos) and higher-level business logic and simulation modules in Python (backtest and exporter).
Main Components
a. Streams Layer (C++):
• Contains components for real-time data ingestion and processing.
• Modules include:
• Binance WebSocket client (binance_websocket_client.cpp)
• NATS client for messaging (nats_client.[cpp/h])
• Configuration (config.cpp/h & config.yaml)
• Metrics handling (metrics.cpp/h)
• This layer is responsible for receiving live market data and relaying it via NATS messaging.
b. Communication Layer (C++):
• Implements the Aeron publisher (aeron_publisher.cpp/h) for robust UDP transport.
• Tests exist to ensure data integrity and proper handling.
• This module likely interfaces between low-latency messaging and the trading engine/algorithms.
c. Trading Algorithms (C++):
• Under the “algos” directory, which includes a main entry (main.cpp) along with algorithmic modules such as regression (regression.cpp/h).
• This component implements the statistical or machine learning-based strategies that decide trading actions.
d. Backtesting Module (Python):
• Designed for simulation and testing strategies against historical data.
• Contains sub-components that mirror live trading:
• Core engine for simulation (backtest/core/engine.py)
• Data management (backtest/data with data_source.py and feed.py)
• Exchange emulation (backtest/exchange/exchange.py, orderbook.py)
• Reporters to assess performance and risk (backtest/reporters)
• Serves as a sandboxed environment to measure how algorithms perform on historical data.
e. Exporter Component (Python):
• Handles exporting or interfacing with external systems, possibly for analysis or logging.
• Uses modules like exporter.py and integration with NATS via nats_py.py.
f. External Dependencies & Infrastructure:
• NATS Server: Run via Docker as per the README. Acts as the message broker, especially for the streams and exporter components.
• Aeron (for UDP transport): Built and incorporated in the communication layer.
• Additional Tools: References to Prometheus Client, Boost Beast (WebSocket), and Polars suggest external integrations.
Relationships and Interactions
• Data flows from external market sources into the Streams module (Binance WebSocket client).
• Streams publish data over NATS to downstream components.
• The Communication layer (Aeron) might supplement these data flows ensuring low-latency messaging.
• The Algorithms module consumes data (possibly directly or through intermediary messaging) to execute trading strategies.
• The Backtesting module simulates these interactions in a controlled environment – mimicking data feeds, order/exchange interactions, and reporting performance.
• The Exporter interacts with NATS and maybe other endpoints to output results or analytics.
• All modules are built and managed using CMake (as seen from various CMakeLists.txt files) and have GitHub workflows for CI/CD.
Architectural Patterns & Design Principles
• Modular decomposition separating real-time streams, communication, strategy logic, and simulation.
• A mix of microservice-like separation (each folder is a distinct “service” or component) and layered architecture (data ingestion, processing, simulation, exporting).
• Use of messaging (NATS and Aeron) to decouple producers and consumers, which is key in high-frequency trading systems.
• A hybrid design with performance-critical C++ layers and flexible Python layers for backtesting and exporting.