# ddoqag/binance-trading-system architecture

> Architecture diagram of the GitHub repository ddoqag/binance-trading-system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ddoqag/binance-trading-system
- Repository: https://github.com/ddoqag/binance-trading-system

## Overview

1. **Overall shape**
This repo is not a single app; it is a multi-generation Binance quant-trading platform. The main architectural boundary is between: legacy Python/Node trading and research code, margin/live execution code, and the newer low-latency “new/” stack that adds Go/Rust/Python integration, shared memory, and more advanced agent-based orchestration.

2. **External dependencies and system inputs**
The system primarily integrates with Binance Spot/Margin/Futures APIs and a PostgreSQL database. Market data can also be stored and replayed from CSV/JSON files under `data/`. Redis appears as an optional coordination layer in demos/tests, while monitoring targets are exposed via Prometheus/Grafana in the newer Go engine.

3. **Data ingestion and persistence**
The Node.js side handles much of the data plumbing: Binance REST/WebSocket clients, fetch scripts, DB initialization/migration, and indicator table setup. This feeds PostgreSQL tables such as klines, ticker, order book, and technical indicators. The repository also has Python loaders and generators that can fall back between DB and local files.

4. **Research and feature-generation pipeline**
The core research path is `data_generator/`, `factors/`, `indicators/`, `training_system/`, and `rl/`. Raw market data is cleaned, labeled, and split with leakage/lookahead prevention before being turned into features, factors, and training sets. This is the offline “data → features → labels → models” boundary.

5. **ML / factor / RL modeling**
The modeling layer spans classical alpha factors, supervised ML, and reinforcement learning. `training_system/` defines dataset/model/train/evaluate flows; `rl/` contains DQN/PPO environments and trainers; `new/brain_py/qlib_models/` and `new/brain_py/moe/` add more advanced model ensembles, mixture-of-experts, and regime-aware feature sets. These components produce signals or model artifacts consumed by trading/orchestration code.

6. **Strategy layer**
There are several strategy families: simple rule-based strategies under `strategy/`, higher-level trading logic in `trading_system/`, and plugin-driven strategies under `plugins/` and `plugin_examples/`. This boundary is important: strategies generate intent or signals, but they do not execute orders directly.

7. **Risk and portfolio control**
Risk is enforced in multiple places. The legacy stack uses `risk/` for position sizing, stop-loss, and manager logic; the margin stack adds `margin_trading/*` risk controllers; the newer stack has `new/hedge_fund_os/risk_kernel.py`, `new/core_go/risk_*`, and portfolio allocation modules such as mean-variance, risk parity, and Black-Litterman. The architecture should show risk as a gate between signal generation and execution.

8. **Execution and order management**
Order execution is split into several runtimes. Legacy Python `trading/` supports spot/margin executors and a Rust bridge; `margin_trading/` and `live_trading_*` scripts orchestrate real/paper trading flows. The newer execution path is much more explicit: `new/execution_core/`, `new/core/`, `new/core_go/`, and `rust_execution/` provide order state machines, cancel/reprice logic, queue tracking, slippage/fill models, and low-latency exchange adapters.

9. **Real-time system boundary**
The `new/` tree introduces a hard boundary between a fast execution engine and slower Python decision logic. Shared memory, `protocol.h`, `shared/`, `new/shared/`, and Go/Python bridge code are used to move market state, signals, and order updates across process/language boundaries with minimal latency. This is the architectural center of the “HFT / latency-sensitive” path.

10. **Orchestration and lifecycle management**
There are multiple orchestrators, depending on which generation is in use: `main_trading_system.py`, `live_trading_*`, `margin_trading/orchestrator.py`, `trading_system/trader.py`, `new/hedge_fund_os/orchestrator.py`, and `new/brain_py/live_integrator.py`. These coordinate the flow from market data ingestion, through strategy/risk decisions, into execution, then back through fill and account updates.

11. **Monitoring, alerts, and recovery**
Operational support is a first-class subsystem. `monitoring/` handles structured logs and alerts in the legacy stack; `new/core_go/metrics.go`, dashboards, Prometheus config, alert rules, and recovery/reconciliation code provide production observability for the newer engine. This layer closes the loop after execution by reconciling positions, fills, and failures.

12. **Plugin and extension boundary**
The repo supports extension via plugins rather than hardcoding every strategy or factor. `plugins/` contains event buses, managers, versioning, and specialized plugin packages; `plugin_examples/` demonstrates strategies, factors, and executors. Diagrammatically, this is the “extensible strategy/risk/execution” boundary that sits alongside the core engines.

13. **Repository structure to reflect in the diagram**
A good diagram should show three major lanes:  
- **Legacy research/backtest/live stack**: data, factors, training, strategies, risk, backtest, execution.  
- **Margin/live orchestration stack**: margin managers, account/position/risk controllers, live trading entrypoints.  
- **New low-latency stack**: Go core engine, shared memory bridge, Rust execution, advanced Python agents, hedge-fund-style orchestration, monitoring.  
These lanes share the same external exchange and database, but are otherwise separate implementation paths.

14. **Key runtime mix**
The repo is deliberately polyglot: Python for research, model training, orchestration, and analytics; Node.js for Binance/DB tooling and some integration utilities; Go for the low-latency engine and observability; Rust for execution performance work. That language split is an architectural feature, not just an implementation detail.

Main entry points: [`new/core_go`](https://github.com/ddoqag/binance-trading-system/tree/main/new/core_go) (Go core engine), [`main_trading_system.py`](https://github.com/ddoqag/binance-trading-system/blob/main/main_trading_system.py) (Live orchestrators), [`new/hedge_fund_os/orchestrator.py`](https://github.com/ddoqag/binance-trading-system/blob/main/new/hedge_fund_os/orchestrator.py) (New orchestrator), [`strategy`](https://github.com/ddoqag/binance-trading-system/tree/main/strategy) (Legacy strategies)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_external["External systems"]
  node_binance{{"Binance APIs<br/>exchange"}}
  node_postgres[("PostgreSQL<br/>database")]
  node_market_files["Market files<br/>csv/json replay"]
  node_observability["Prometheus/Grafana<br/>monitoring"]
end

subgraph group_legacy["Legacy stack"]
  node_node_ingestion["Node data layer<br/>node ingestion"]
  node_data_pipeline["Research pipeline<br/>offline feature pipeline"]
  node_factors_models["Factors & models<br/>alpha/ML/RL"]
  node_legacy_strategy["Legacy strategies<br/>rules/ML strategies"]
  node_legacy_risk["Legacy risk<br/>risk controls"]
  node_backtest["Backtest engine<br/>simulation"]
  node_legacy_exec["Legacy execution<br/>spot/margin execution"]
  node_legacy_live["Live orchestrators<br/>runtime orchestration"]
  node_margin_stack["Margin stack<br/>margin orchestration"]
end

subgraph group_newstack["New low-latency stack"]
  node_go_core["Go core engine<br/>low-latency engine"]
  node_shared_mem["Shared memory bridge<br/>ipc bridge"]
  node_rust_exec["Rust execution<br/>rust matcher"]
  node_py_agents["Python agents<br/>agent orchestration"]
  node_new_portfolio_risk["Portfolio & risk<br/>allocation/risk"]
  node_new_exec_core["Execution core<br/>order state machine"]
  node_new_orchestrator["New orchestrator<br/>hedge-fund OS<br/>[orchestrator.py]"]
end

subgraph group_extensible["Plugins"]
  node_plugin_layer["Plugin layer<br/>extension bus"]
end

node_node_ingestion -->|"market/account data"| node_binance
node_node_ingestion -->|"store/reload"| node_postgres
node_market_files -->|"replay input"| node_data_pipeline
node_postgres -->|"historical data"| node_data_pipeline
node_data_pipeline -->|"features/labels"| node_factors_models
node_factors_models -->|"signals"| node_legacy_strategy
node_legacy_strategy -->|"signal gate"| node_legacy_risk
node_legacy_risk -->|"approve orders"| node_legacy_exec
node_legacy_exec -->|"orders"| node_binance
node_legacy_live -->|"feeds"| node_node_ingestion
node_legacy_live -->|"runs strategies"| node_legacy_strategy
node_legacy_live -->|"routes orders"| node_legacy_exec
node_legacy_live -->|"margin mode"| node_margin_stack
node_margin_stack -->|"margin trading"| node_binance
node_plugin_layer -->|"strategy plugins"| node_legacy_strategy
node_plugin_layer -->|"executor plugins"| node_legacy_exec
node_py_agents -->|"allocations"| node_new_portfolio_risk
node_py_agents -->|"signals"| node_shared_mem
node_shared_mem -->|"market/state sync"| node_go_core
node_go_core -->|"execution control"| node_new_exec_core
node_new_exec_core -->|"low-latency orders"| node_binance
node_new_orchestrator -->|"agent loop"| node_py_agents
node_new_orchestrator -->|"risk gate"| node_new_portfolio_risk
node_new_orchestrator -->|"bridge state"| node_shared_mem
node_go_core -->|"metrics"| node_observability
node_go_core -->|"hot path"| node_rust_exec
node_rust_exec -->|"optimized fills"| node_binance
node_new_orchestrator -->|"coordinate"| node_go_core

click node_market_files "https://github.com/ddoqag/binance-trading-system/tree/main/data"
click node_observability "https://github.com/ddoqag/binance-trading-system/tree/main/new/core_go"
click node_node_ingestion "https://github.com/ddoqag/binance-trading-system/tree/main/core"
click node_data_pipeline "https://github.com/ddoqag/binance-trading-system/tree/main/data_generator"
click node_factors_models "https://github.com/ddoqag/binance-trading-system/tree/main/factors"
click node_legacy_strategy "https://github.com/ddoqag/binance-trading-system/tree/main/strategy"
click node_legacy_risk "https://github.com/ddoqag/binance-trading-system/tree/main/risk"
click node_backtest "https://github.com/ddoqag/binance-trading-system/tree/main/backtest"
click node_legacy_exec "https://github.com/ddoqag/binance-trading-system/tree/main/trading"
click node_legacy_live "https://github.com/ddoqag/binance-trading-system/blob/main/main_trading_system.py"
click node_margin_stack "https://github.com/ddoqag/binance-trading-system/tree/main/margin_trading"
click node_plugin_layer "https://github.com/ddoqag/binance-trading-system/tree/main/plugins"
click node_go_core "https://github.com/ddoqag/binance-trading-system/tree/main/new/core_go"
click node_shared_mem "https://github.com/ddoqag/binance-trading-system/tree/main/new/shared"
click node_rust_exec "https://github.com/ddoqag/binance-trading-system/tree/main/rust_execution/src"
click node_py_agents "https://github.com/ddoqag/binance-trading-system/tree/main/new/brain_py"
click node_new_portfolio_risk "https://github.com/ddoqag/binance-trading-system/tree/main/new/hedge_fund_os"
click node_new_exec_core "https://github.com/ddoqag/binance-trading-system/tree/main/new/execution_core"
click node_new_orchestrator "https://github.com/ddoqag/binance-trading-system/blob/main/new/hedge_fund_os/orchestrator.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_binance,node_postgres,node_market_files,node_observability toneBlue
class node_node_ingestion,node_data_pipeline,node_factors_models,node_legacy_strategy,node_legacy_risk,node_backtest,node_legacy_exec,node_legacy_live,node_margin_stack toneAmber
class node_go_core,node_shared_mem,node_rust_exec,node_py_agents,node_new_portfolio_risk,node_new_exec_core,node_new_orchestrator toneMint
class node_plugin_layer toneRose
```

## Components

### External systems

Binance, databases, files, and observability targets used by the trading platform.

- **Binance APIs**: exchange · Spot, margin, futures market and account connectivity.
- **PostgreSQL**: database · Persistent storage for klines, order book, indicators, and account/trading data.
- [Market files](https://github.com/ddoqag/binance-trading-system/tree/main/data): csv/json replay · Local CSV/JSON data used for loading, replay, and fallback workflows. · `data`
- [Prometheus/Grafana](https://github.com/ddoqag/binance-trading-system/tree/main/new/core_go): monitoring · Metrics, dashboards, alerts, and operational visibility for the newer engine. · `new/core_go`

### Legacy stack

Older Python/Node trading, research, and live orchestration paths.

- [Node data layer](https://github.com/ddoqag/binance-trading-system/tree/main/core): node ingestion · REST/WebSocket clients, DB setup, migrations, and market data plumbing. · `core`
- [Research pipeline](https://github.com/ddoqag/binance-trading-system/tree/main/data_generator): offline feature pipeline · Cleaning, loading, labeling, leakage-free splitting, and feature generation. · `data_generator`
- [Factors & models](https://github.com/ddoqag/binance-trading-system/tree/main/factors): alpha/ML/RL · Classical factors, supervised training, and reinforcement learning components. · `factors`
- [Legacy strategies](https://github.com/ddoqag/binance-trading-system/tree/main/strategy): rules/ML strategies · Rule-based and ML strategy logic that emits signals but does not execute orders. · `strategy`
- [Legacy risk](https://github.com/ddoqag/binance-trading-system/tree/main/risk): risk controls · Position sizing, stop-loss, and trade gating for older trading paths. · `risk`
- [Backtest engine](https://github.com/ddoqag/binance-trading-system/tree/main/backtest): simulation · Historical simulation and evaluation for strategy and system behavior. · `backtest`
- [Legacy execution](https://github.com/ddoqag/binance-trading-system/tree/main/trading): spot/margin execution · Python and bridge-based order execution for spot, margin, and leverage flows. · `trading`
- [Live orchestrators](https://github.com/ddoqag/binance-trading-system/blob/main/main_trading_system.py): runtime orchestration · Entry points that coordinate research, strategy, risk, and execution in production or paper flows. · `main_trading_system.py`
- [Margin stack](https://github.com/ddoqag/binance-trading-system/tree/main/margin_trading): margin orchestration · Dedicated margin account, position, signal, and risk management path. · `margin_trading`

### New low-latency stack

The newer multi-runtime execution and agent-based architecture under new/.

- [Go core engine](https://github.com/ddoqag/binance-trading-system/tree/main/new/core_go): low-latency engine · Fast execution, metrics, reconciliation, recovery, and exchange client handling. · `new/core_go`
- [Shared memory bridge](https://github.com/ddoqag/binance-trading-system/tree/main/new/shared): ipc bridge · Protocol headers, shared-memory readers/writers, and cross-language state transfer. · `new/shared`
- [Rust execution](https://github.com/ddoqag/binance-trading-system/tree/main/rust_execution/src): rust matcher · Performance-oriented execution engine and benchmarks for latency-sensitive order handling. · `rust_execution/src`
- [Python agents](https://github.com/ddoqag/binance-trading-system/tree/main/new/brain_py): agent orchestration · Agent registry, expert agents, MoE, regime features, and live integration. · `new/brain_py`
- [Portfolio & risk](https://github.com/ddoqag/binance-trading-system/tree/main/new/hedge_fund_os): allocation/risk · Risk kernel and portfolio construction modules such as mean-variance, risk parity, and Black-Litterman. · `new/hedge_fund_os`
- [Execution core](https://github.com/ddoqag/binance-trading-system/tree/main/new/execution_core): order state machine · Order lifecycle, cancel/reprice, queue tracking, slippage, fill, and position state management. · `new/execution_core`
- [New orchestrator](https://github.com/ddoqag/binance-trading-system/blob/main/new/hedge_fund_os/orchestrator.py): hedge-fund OS · Coordinates agents, risk, execution, monitoring, and recovery in the new stack. · `new/hedge_fund_os/orchestrator.py`

### Plugins

Extension boundary for strategies, factors, and executors.

- [Plugin layer](https://github.com/ddoqag/binance-trading-system/tree/main/plugins): extension bus · Event buses, managers, versioning, and example plugins for extensible behaviors. · `plugins`

## Connections

- Node data layer → Binance APIs: market/account data · Fetches live market and account state from Binance.
- Node data layer → PostgreSQL: store/reload · Initializes and populates trading data tables.
- Market files → Research pipeline: replay input · Supplies file-based market history when DB is absent or for offline workflows.
- PostgreSQL → Research pipeline: historical data · Feeds loaders and generators for cleaning and feature creation.
- Research pipeline → Factors & models: features/labels · Produces model-ready datasets and training splits.
- Factors & models → Legacy strategies: signals · Turns modeled outputs into strategy inputs or predictions.
- Legacy strategies → Legacy risk: signal gate · Strategy intent is filtered by risk controls before orders are created.
- Legacy risk → Legacy execution: approve orders · Allows only risk-compliant orders to reach execution.
- Legacy execution → Binance APIs: orders · Submits spot, margin, and leverage orders to exchange APIs.
- Live orchestrators → Node data layer: feeds · Coordinates live market-data acquisition and system startup.
- Live orchestrators → Legacy strategies: runs strategies · Executes the legacy trading loop.
- Live orchestrators → Legacy execution: routes orders · Hands approved actions to execution.
- Live orchestrators → Margin stack: margin mode · Selects the margin-oriented live path when required.
- Margin stack → Binance APIs: margin trading · Interacts with Binance margin accounts and positions.
- Plugin layer → Legacy strategies: strategy plugins · Extends strategy selection and behavior.
- Plugin layer → Legacy execution: executor plugins · Allows alternative execution implementations.
- Python agents → Portfolio & risk: allocations · Agent outputs are constrained and transformed into portfolio decisions.
- Python agents → Shared memory bridge: signals · Writes decision and state payloads into the IPC bridge.
- Shared memory bridge → Go core engine: market/state sync · Moves fast-changing state between Python and Go processes.
- Go core engine → Execution core: execution control · Coordinates order state, lifecycle, and exchange interaction.
- Execution core → Binance APIs: low-latency orders · Sends orders and receives fills through exchange adapters.
- New orchestrator → Python agents: agent loop · Runs the slower reasoning and synthesis layer.
- New orchestrator → Portfolio & risk: risk gate · Applies portfolio construction and risk checks before execution.
- New orchestrator → Shared memory bridge: bridge state · Publishes market, signal, and order updates to the fast path.
- Go core engine → Prometheus/Grafana: metrics · Exports metrics and dashboards for monitoring and alerting.
- Go core engine → Rust execution: hot path · Connects the Go engine to the Rust performance execution layer where used.
- Rust execution → Binance APIs: optimized fills · Implements a high-performance exchange execution path.
- New orchestrator → Go core engine: coordinate · Orchestrates the production lifecycle around the low-latency engine.
