# rorysroes/sgx-full-orderbook-tick-data-trading-strategy architecture

> Architecture diagram of the GitHub repository rorysroes/sgx-full-orderbook-tick-data-trading-strategy, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rorysroes/sgx-full-orderbook-tick-data-trading-strategy
- Repository: https://github.com/rorysroes/sgx-full-orderbook-tick-data-trading-strategy

## Overview

##### Main Components
a. Data Transformation Module
- This section is responsible for data ingestion, cleaning, and transformation.
- It contains notebooks in the Data_Transformation directory (with subdirectories for order book data and the Train_Test_Builder) that work on pre-processing trade data from CSVs.
b. Feature Selection Module
- This component extracts key features from the transformed data.
- It includes notebooks dealing with "Rise Ratio" (e.g., SGX_A50_Market_Feature.ipynb) and "Depth Ratio." This module is essential for creating inputs for the predictive models.
c. Model Selection and Training Module
- This area involves training various machine learning classifiers and evaluating them.
- Located in the Model_Selection directory, it encapsulates model evaluation using cross-validation (as reflected in pipeline and best model graphs) and selection strategies.
d. Graphical / Visualization Component
- To document the pipeline, features, cross-validation, predictions, and profit & loss outcomes.
e. Data (Order Book Data)
- The raw order book CSVs are part of the data infrastructure, fed into the transformation module.

##### Relationships and Interactions
- Data Flow: Raw order book CSVs enter the Data Transformation module. After cleaning and pre-processing, the processed data is input into the Feature Selection module.
- Once features (e.g., rise ratio and depth ratio, among others) are generated, they are fed to the Model Selection module where different classifiers are trained and compared.
- The selected model is then used to predict future price changes over the next 10 seconds. The outcomes, including P&L metrics, are visualized in the Graph component.

##### Architectural Patterns
- Pipeline Structure: The project naturally follows a sequential data pipeline (data transformation → feature extraction → model selection → outcome prediction/visualization).
- Modularity: Having distinct directories for transformation, feature selection, and model selection reflects a clean separation of concerns.
- Notebook-Driven Exploration: The project leverages Jupyter notebooks which suggest experimental and iterative development practices, common in data science and machine learning projects.
- Extensibility: The modular separation also hints that new features or models can be added without disrupting the entire structure.

##### Relevant Technologies and Frameworks
- Data Science tools (Jupyter notebooks) for experiments and development.
- Machine learning libraries for classifiers (e.g., implementations of RandomForest, ExtraTrees, etc.) are noted via the README.
- CSV for raw data storage.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Raw Data
    RawData["Raw Order Book CSVs"]:::data

    %% Data Transformation Module
    subgraph "Data Transformation Module"
        DT1["Order Book Data Notebook"]:::transformation
        DT2["Train Test Builder Notebook"]:::transformation
    end
    TData["Transformed Data"]:::transformation

    %% Feature Selection Module
    subgraph "Feature Selection Module"
        FS1["Rise Ratio Feature Notebook"]:::feature
        FS2["Train-Test Data Builder Rise Extractor"]:::feature
        FS3["Train-Test Data Builder Depth Extractor"]:::feature
        FS_Feat["Selected Features"]:::feature
    end

    %% Model Selection & Training Module
    subgraph "Model Selection & Training Module"
        MS1["Model Selection Notebook"]:::model
        ModelOut["Trained Model"]:::model
    end

    %% Visualization Component
    subgraph "Visualization Component"
        VIS1["Visualization Graphs"]:::visual
    end

    %% Data Flow Connections
    RawData --> DT1
    RawData --> DT2
    DT1 --> TData
    DT2 --> TData
    TData --> FS1
    TData --> FS2
    TData --> FS3
    FS1 --> FS_Feat
    FS2 --> FS_Feat
    FS3 --> FS_Feat
    FS_Feat --> MS1
    MS1 --> ModelOut
    ModelOut --> VIS1
    VIS1 -- "feedback" --> FS_Feat

    %% Click Events
    click RawData "https://github.com/rorysroes/sgx-full-orderbook-tick-data-trading-strategy/tree/master/Data_Transformation/order book data/"
    click DT1 "https://github.com/rorysroes/sgx-full-orderbook-tick-data-trading-strategy/blob/master/Data_Transformation/order book data/Data_Transformation_Code.ipynb"
    click DT2 "https://github.com/rorysroes/sgx-full-orderbook-tick-data-trading-strategy/blob/master/Data_Transformation/Train_Test_Builder/Train_Test_Builder.ipynb"
    click FS1 "https://github.com/rorysroes/sgx-full-orderbook-tick-data-trading-strategy/blob/master/Feature_Selection/SGX_A50_Market_Feature.ipynb"
    click FS2 "https://github.com/rorysroes/sgx-full-orderbook-tick-data-trading-strategy/blob/master/Feature_Selection/Train_Test_Data_Builder_Feature_Rise_Extractor.ipynb"
    click FS3 "https://github.com/rorysroes/sgx-full-orderbook-tick-data-trading-strategy/blob/master/Feature_Selection/Train_Test_Data_Builder_Feature_Depth_Extractor.ipynb"
    click MS1 "https://github.com/rorysroes/sgx-full-orderbook-tick-data-trading-strategy/blob/master/Model_Selection/Model_Selection.ipynb"
    click VIS1 "https://github.com/rorysroes/sgx-full-orderbook-tick-data-trading-strategy/tree/master/Graph/"

    %% Styles
    classDef data fill:#f9f,stroke:#333,stroke-width:2px;
    classDef transformation fill:#ffd700,stroke:#333,stroke-width:2px;
    classDef feature fill:#87ceeb,stroke:#333,stroke-width:2px;
    classDef model fill:#90ee90,stroke:#333,stroke-width:2px;
    classDef visual fill:#ffb6c1,stroke:#333,stroke-width:2px;
```
