# triquang26/ml-course-01 architecture

> Architecture diagram of the GitHub repository triquang26/ml-course-01, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/triquang26/ml-course-01
- Repository: https://github.com/triquang26/ml-course-01

## Overview

##### Main Components Identification
- Data Storage & Ingestion:
- Data is stored under the “data” directory (raw, processed, external).
- Scripts in src/data (make_dataset.py, preprocess.py and model-specific preprocessing scripts) convert raw data into a usable format.
- Feature Engineering:
- The “src/features” directory handles feature extraction and feature selection with specific scripts/modules for each model type.
- Model Implementations:
- Under “src/models”, each subdirectory (bayesian, decision_tree, genetic_algorithm, graphical_model, neural_network) holds the model-specific code.
- Each model has its own core implementation, training scripts (main.py), prediction scripts (predict.py), testing modules (test.py), and persistence routines.
- Experimentation & Reporting:
- Notebooks in the “notebooks” directory serve for exploratory data analysis, model development, and tutorials.
- Reports and figures are placed for visual analysis (reports, figures directories).
- Orchestration & Integration:
- At a higher level, “src/models/train_model.py” and “src/models/predict_model.py” serve as centralized entry points to train and run the different models.
- Testing & Validation:
- The “tests” directory holds unit tests to ensure data processing scripts and model routines are functioning correctly.

##### Relationships and Data Flow
- Data flow begins at the raw data level, then moves through preprocessing (src/data/preprocess/*) into processed datasets.
- Processed data connects to the feature engineering module (src/features/build_features.py and features_selection submodules) to extract the relevant features.
- The processed and engineered outputs are then fed into the model-specific training pipelines within “src/models/[model_type]”.
- Once trained, models are saved in “trained” subdirectories; predictions and evaluations are computed with the associated scripts.
- Notebooks reflect exploratory experiments, while reports compile the visualizations from the “reports” and “figures” directories.
- Testing modules validate the above processes end to end.

##### Architectural Patterns and Design Principles
- Modularity – each model is implemented in its own module, allowing for independent development and testing.
- Pipeline Architecture – clear sequential stages (data ingestion, preprocessing, feature engineering, modeling, evaluation) which mirror common ML workflows.
- Repository Layout reflects separation of concerns: notebooks for exploration, src for production code, separate directories for each workflow stage (models, data, features, visualization, tests).

##### Relevant Technologies and Frameworks
- Python 3.11+, with key libraries including scikit-learn, numpy, pandas, PyTorch (for neural networks), pgmpy (for graphical models).
- Jupyter Notebooks for experimentation.
- joblib and PyTorch’s native persistence for saving trained models.

##### Detailed Steps
- Break the pipeline into modules:
- Data Acquisition/Storage → Data Preprocessing (src/data) → Feature Engineering (src/features) → Model Training (src/models/{model_type}).
- Bayesian Models block: Show subcomponents like “Core Models”, “Persistence”, and “Evaluation”.
- Decision Trees block: Include “Main Training”, “Prediction”, “Test”.
- Genetic Algorithm Ensemble block: Highlight “Core GA Implementation”, “Ensemble Integration”, and “Trained Weights”.
- Neural Networks block: Indicate “Vision Transformer” with training, testing, and persistence.
- Also create an auxiliary branch for Notebooks/Reports that interact with each modeling component for documentation and visualization.
- Optionally, add a layer for "Testing and Validation" that touches all parts of the pipeline.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Handling Subgraph
    subgraph "Data Handling"
        DS("Data Storage"):::data
        MD("Make Dataset"):::data
        PD("Data Preprocessing"):::data
    end
    DS -->|"flows"| MD
    MD -->|"flows"| PD

    %% Feature Engineering Subgraph
    subgraph "Feature Engineering"
        FE("Feature Extraction"):::feature
        FS("Feature Selection"):::feature
    end
    PD -->|"processed data"| FE
    FE -->|"features"| FS

    %% Orchestration Subgraph
    subgraph "Orchestration & Integration"
        TM("Train Model"):::orchestration
        PM("Predict Model"):::orchestration
    end
    FS -->|"engineered features"| TM
    FS -->|"engineered features"| PM

    %% Model Implementations Subgraph
    subgraph "Model Implementations"
        BAY("Bayesian Model"):::model
        DT("Decision Tree Model"):::model
        GA("Genetic Algorithm Ensemble"):::model
        GM("Graphical Model"):::model
        NN("Neural Network Model"):::model
    end
    TM -->|"triggers"| BAY
    TM -->|"triggers"| DT
    TM -->|"triggers"| GA
    TM -->|"triggers"| GM
    TM -->|"triggers"| NN
    PM -->|"queries"| BAY
    PM -->|"queries"| DT
    PM -->|"queries"| GA
    PM -->|"queries"| GM
    PM -->|"queries"| NN

    %% Experimentation & Testing Subgraph
    subgraph "Experimentation & Testing"
        NB("Notebooks & Tutorials"):::support
        RP("Reports & Figures"):::support
        TS("Testing & Validation"):::support
    end
    TM -->|"documents"| NB
    TM -->|"documents"| RP
    TM -->|"verifies"| TS
    PM -->|"verifies"| TS

    %% Click Events
    click DS "https://github.com/triquang26/ml-course-01/tree/main/data/"
    click MD "https://github.com/triquang26/ml-course-01/blob/main/src/data/make_dataset.py"
    click PD "https://github.com/triquang26/ml-course-01/blob/main/src/data/preprocess.py"
    click FE "https://github.com/triquang26/ml-course-01/blob/main/src/features/build_features.py"
    click FS "https://github.com/triquang26/ml-course-01/tree/main/src/features/features_selection/"
    click TM "https://github.com/triquang26/ml-course-01/blob/main/src/models/train_model.py"
    click PM "https://github.com/triquang26/ml-course-01/blob/main/src/models/predict_model.py"
    click BAY "https://github.com/triquang26/ml-course-01/tree/main/src/models/bayesian/"
    click DT "https://github.com/triquang26/ml-course-01/tree/main/src/models/decision_tree/"
    click GA "https://github.com/triquang26/ml-course-01/tree/main/src/models/genetic_algorithm/"
    click GM "https://github.com/triquang26/ml-course-01/tree/main/src/models/graphical_model/"
    click NN "https://github.com/triquang26/ml-course-01/tree/main/src/models/neural_network/"
    click NB "https://github.com/triquang26/ml-course-01/tree/main/notebooks/"
    click RP "https://github.com/triquang26/ml-course-01/tree/main/reports/"
    click TS "https://github.com/triquang26/ml-course-01/tree/main/tests/"

    %% Styles
    classDef data fill:#AED6F1,stroke:#000,stroke-width:2px;
    classDef feature fill:#85C1E9,stroke:#000,stroke-width:2px;
    classDef orchestration fill:#F7DC6F,stroke:#000,stroke-width:2px;
    classDef model fill:#82E0AA,stroke:#000,stroke-width:2px;
    classDef support fill:#F5B041,stroke:#000,stroke-width:2px;
```
