# sushantgagneja/ultra-low-latency-ml-inference-on-fpga architecture

> Architecture diagram of the GitHub repository sushantgagneja/ultra-low-latency-ml-inference-on-fpga, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga
- Repository: https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga

## Overview

This is a low-latency FPGA market-tick inference system with a separate model-training toolchain. In the live path, a host gateway receives Binance ticks, computes metadata, and sends packed data over UDP; ESP32 firmware forwards it over SPI, and FPGA logic extracts features, quantizes them, and runs the BNN before signaling completion to the firmware. The README also documents model training, RTL generation, historical backtesting, and performance auditing. Several internal RTL connections are documented at subsystem level only, so edges use the README as evidence rather than implying unsampled module calls.

Main entry points: [`esp32_firmware/main/main.c`](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/esp32_firmware/main/main.c) (Firmware orchestration), [`rtl/microstructure/tick_parser.v`](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/microstructure/tick_parser.v) (SPI tick parser), [`rtl/microstructure/hw_quantizer.v`](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/microstructure/hw_quantizer.v) (Feature quantizer), [`rtl/bnn_top.v`](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/bnn_top.v) (Inference top)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_gateway["Market gateway"]
  node_gateway["Tick gateway<br/>[main.go]"]
end

subgraph group_firmware["ESP32 firmware"]
  node_udp["UDP listener<br/>[udp_server.c]"]
  node_firmware_app["Firmware orchestration<br/>[main.c]"]
  node_spi["SPI DMA interface<br/>[spi_interface.c]"]
end

subgraph group_hardware["FPGA inference"]
  node_tick_parser["SPI tick parser<br/>[tick_parser.v]"]
  node_ofi["OFI engine<br/>[ofi_engine.v]"]
  node_vwap["VWAP engine<br/>[vwap_engine.v]"]
  node_lee_ready["Lee-Ready engine<br/>[lee_ready.v]"]
  node_quantizer["Feature quantizer<br/>[hw_quantizer.v]"]
  node_temporal["Temporal memory"]
  node_bnn_top["Inference top<br/>[bnn_top.v]"]
  node_bnn_core["Unrolled BNN"]
  node_xnor["XNOR popcount<br/>[xnor_popcount.v]"]
end

subgraph group_ml["Model pipeline"]
  node_training["Ternary model training"]
  node_retraining["BookTicker retraining"]
  node_rtl_generator["RTL generator"]
  node_legacy_training["Legacy BNN training"]
end

subgraph group_analysis["Backtest and audit"]
  node_backtest["Historical backtest"]
  node_monitor["Performance audit"]
  node_audit_log["Audit records"]
end

node_binance(("Binance feed"))

node_binance -->|"streams ticks"| node_gateway
node_gateway -->|"sends UDP"| node_udp
node_firmware_app -->|"starts listener"| node_udp
node_udp -->|"queues ticks"| node_firmware_app
node_firmware_app -->|"transmits ticks"| node_spi
node_spi -->|"sends SPI frame"| node_tick_parser
node_tick_parser -->|"provides tick data"| node_ofi
node_tick_parser -->|"provides tick data"| node_vwap
node_tick_parser -->|"provides tick data"| node_lee_ready
node_ofi -->|"provides feature"| node_quantizer
node_vwap -->|"provides feature"| node_quantizer
node_lee_ready -->|"provides feature"| node_quantizer
node_quantizer -->|"updates memory"| node_temporal
node_temporal -->|"provides features"| node_bnn_top
node_bnn_top -->|"dispatches inference"| node_bnn_core
node_bnn_core -->|"uses logic"| node_xnor
node_bnn_top -->|"signals completion"| node_firmware_app
node_training -->|"supplies model"| node_rtl_generator
node_retraining -->|"exports weights"| node_rtl_generator
node_legacy_training -->|"provides model"| node_backtest
node_retraining -->|"provides features"| node_backtest
node_firmware_app -.->|"emits audit events"| node_monitor
node_monitor -.->|"records metrics"| node_audit_log

click node_gateway "https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/scripts/gateway/main.go"
click node_udp "https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/esp32_firmware/main/udp_server.c"
click node_firmware_app "https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/esp32_firmware/main/main.c"
click node_spi "https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/esp32_firmware/main/spi_interface.c"
click node_tick_parser "https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/microstructure/tick_parser.v"
click node_ofi "https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/microstructure/ofi_engine.v"
click node_vwap "https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/microstructure/vwap_engine.v"
click node_lee_ready "https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/microstructure/lee_ready.v"
click node_quantizer "https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/microstructure/hw_quantizer.v"
click node_temporal "https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/microstructure/bram_microstructure.v"
click node_bnn_top "https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/bnn_top.v"
click node_bnn_core "https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/bnn_core_unrolled.v"
click node_xnor "https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/xnor_popcount.v"
click node_training "https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/scripts/train_twn_pytorch.py"
click node_retraining "https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/scripts/retrain_bookticker.py"
click node_rtl_generator "https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/scripts/generate_bnn_rtl.py"
click node_legacy_training "https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/train_bnn_standalone.py"
click node_backtest "https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/scripts/historical_backtest.py"
click node_monitor "https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/monitoring/bnn_trading_monitor.py"
click node_audit_log "https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/monitoring/sample_bnn_audit.jsonl"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_gateway toneBlue
class node_udp,node_firmware_app,node_spi toneAmber
class node_tick_parser,node_ofi,node_vwap,node_lee_ready,node_quantizer,node_temporal,node_bnn_top,node_bnn_core,node_xnor toneMint
class node_training,node_retraining,node_rtl_generator,node_legacy_training toneRose
class node_backtest,node_monitor,node_audit_log,node_binance toneIndigo
```

## Components

### Market gateway

- [Tick gateway](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/scripts/gateway/main.go): `scripts/gateway/main.go`

### ESP32 firmware

- [UDP listener](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/esp32_firmware/main/udp_server.c): `esp32_firmware/main/udp_server.c`
- [Firmware orchestration](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/esp32_firmware/main/main.c): `esp32_firmware/main/main.c`
- [SPI DMA interface](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/esp32_firmware/main/spi_interface.c): `esp32_firmware/main/spi_interface.c`

### FPGA inference

- [SPI tick parser](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/microstructure/tick_parser.v): `rtl/microstructure/tick_parser.v`
- [OFI engine](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/microstructure/ofi_engine.v): `rtl/microstructure/ofi_engine.v`
- [VWAP engine](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/microstructure/vwap_engine.v): `rtl/microstructure/vwap_engine.v`
- [Lee-Ready engine](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/microstructure/lee_ready.v): `rtl/microstructure/lee_ready.v`
- [Feature quantizer](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/microstructure/hw_quantizer.v): `rtl/microstructure/hw_quantizer.v`
- [Temporal memory](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/microstructure/bram_microstructure.v): `rtl/microstructure/bram_microstructure.v`
- [Inference top](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/bnn_top.v): `rtl/bnn_top.v`
- [Unrolled BNN](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/bnn_core_unrolled.v): `rtl/bnn_core_unrolled.v`
- [XNOR popcount](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/rtl/xnor_popcount.v): `rtl/xnor_popcount.v`

### Model pipeline

- [Ternary model training](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/scripts/train_twn_pytorch.py): `scripts/train_twn_pytorch.py`
- [BookTicker retraining](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/scripts/retrain_bookticker.py): `scripts/retrain_bookticker.py`
- [RTL generator](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/scripts/generate_bnn_rtl.py): `scripts/generate_bnn_rtl.py`
- [Legacy BNN training](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/train_bnn_standalone.py): `train_bnn_standalone.py`

### Backtest and audit

- [Historical backtest](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/scripts/historical_backtest.py): `scripts/historical_backtest.py`
- [Performance audit](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/monitoring/bnn_trading_monitor.py): `monitoring/bnn_trading_monitor.py`
- [Audit records](https://github.com/sushantgagneja/ultra-low-latency-ml-inference-on-fpga/blob/main/monitoring/sample_bnn_audit.jsonl): `monitoring/sample_bnn_audit.jsonl`

### Other components

- **Binance feed**

## Connections

- Binance feed → Tick gateway: streams ticks (evidence: `README.md`)
- Tick gateway → UDP listener: sends UDP (evidence: `README.md`)
- Firmware orchestration → UDP listener: starts listener (evidence: `esp32_firmware/main/main.c`)
- UDP listener → Firmware orchestration: queues ticks (evidence: `README.md`)
- Firmware orchestration → SPI DMA interface: transmits ticks (evidence: `README.md`)
- SPI DMA interface → SPI tick parser: sends SPI frame (evidence: `README.md`)
- SPI tick parser → OFI engine: provides tick data (evidence: `README.md`)
- SPI tick parser → VWAP engine: provides tick data (evidence: `README.md`)
- SPI tick parser → Lee-Ready engine: provides tick data (evidence: `README.md`)
- OFI engine → Feature quantizer: provides feature (evidence: `README.md`)
- VWAP engine → Feature quantizer: provides feature (evidence: `README.md`)
- Lee-Ready engine → Feature quantizer: provides feature (evidence: `README.md`)
- Feature quantizer → Temporal memory: updates memory (evidence: `README.md`)
- Temporal memory → Inference top: provides features (evidence: `README.md`)
- Inference top → Unrolled BNN: dispatches inference (evidence: `README.md`)
- Unrolled BNN → XNOR popcount: uses logic (evidence: `README.md`)
- Inference top → Firmware orchestration: signals completion (evidence: `README.md`)
- Ternary model training → RTL generator: supplies model (evidence: `README.md`)
- BookTicker retraining → RTL generator: exports weights (evidence: `scripts/retrain_bookticker.py`)
- Legacy BNN training → Historical backtest: provides model (evidence: `scripts/historical_backtest.py`)
- BookTicker retraining → Historical backtest: provides features (evidence: `scripts/historical_backtest.py`)
- Firmware orchestration → Performance audit: emits audit events
- Performance audit → Audit records: records metrics
