How rorysroes/sgx-full-orderbook-tick-data-trading-strategy works
Main Components
a. Data Transformation Module
• This section is responsible for data ingestion, cleaning, and transformation.
• It contains notebooks in the Data_Transformation directory (with subdirectories for order book data and the Train_Test_Builder) that work on pre-processing trade data from CSVs.
b. Feature Selection Module
• This component extracts key features from the transformed data.
• It includes notebooks dealing with "Rise Ratio" (e.g., SGX_A50_Market_Feature.ipynb) and "Depth Ratio." This module is essential for creating inputs for the predictive models.
c. Model Selection and Training Module
• This area involves training various machine learning classifiers and evaluating them.
• Located in the Model_Selection directory, it encapsulates model evaluation using cross-validation (as reflected in pipeline and best model graphs) and selection strategies.
d. Graphical / Visualization Component
• To document the pipeline, features, cross-validation, predictions, and profit & loss outcomes.
e. Data (Order Book Data)
• The raw order book CSVs are part of the data infrastructure, fed into the transformation module.
Relationships and Interactions
• Data Flow: Raw order book CSVs enter the Data Transformation module. After cleaning and pre-processing, the processed data is input into the Feature Selection module.
• Once features (e.g., rise ratio and depth ratio, among others) are generated, they are fed to the Model Selection module where different classifiers are trained and compared.
• The selected model is then used to predict future price changes over the next 10 seconds. The outcomes, including P&L metrics, are visualized in the Graph component.
Architectural Patterns
• Pipeline Structure: The project naturally follows a sequential data pipeline (data transformation → feature extraction → model selection → outcome prediction/visualization).
• Modularity: Having distinct directories for transformation, feature selection, and model selection reflects a clean separation of concerns.
• Notebook-Driven Exploration: The project leverages Jupyter notebooks which suggest experimental and iterative development practices, common in data science and machine learning projects.
• Extensibility: The modular separation also hints that new features or models can be added without disrupting the entire structure.
Relevant Technologies and Frameworks
• Data Science tools (Jupyter notebooks) for experiments and development.
• Machine learning libraries for classifiers (e.g., implementations of RandomForest, ExtraTrees, etc.) are noted via the README.
• CSV for raw data storage.