# seyyidos/cpre architecture

> Architecture diagram of the GitHub repository seyyidos/cpre, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/seyyidos/cpre
- Repository: https://github.com/seyyidos/cpre

## Overview

##### Project Type and Purpose
- This is a data-science/ML pipeline project (“Chess Puzzle Rating Estimation”) whose goal is to predict the difficulty rating of chess puzzles.
- It uses raw chess puzzle data → feature extraction → model training (NN, LightGBM, XGBoost) → hyperparameter tuning → result artifacts (models, images).

##### Key File-Structure Insights
- notebooks/: Jupyter notebooks for EDA, demo, model experiments.
- src/feature/: feature‐engineering modules (board representation, material count, one-hot, graph features).
- src/lib/: helper libraries (data parsing, dataset classes, model-training helpers, hyperparameter tuning via Optuna).
- results/: saved model files (cnn, lgbm variants), images.
- runs/: TensorBoard logs for NN and LGBM experiments.
- requirements.txt & README.md: Python environment and project overview.

##### Main Components
A. Data Layer
- Raw data (data/raw)
- Filtered data (data/filtered)
- Feature‐ready data (data/feature)
- Demo data (data/demo)
B. Processing & Feature Extraction
- EDA notebooks (notebooks/eda*.ipynb)
- src/feature modules (cnn, graph, game_phase, material_advantage, etc.)
C. Core Library & Utilities
- fen_parser, eda_helpers, feature_generator
- graph_helper, move_encoder, nn_dataset, train_helper
D. Modeling & Training
- Learner modules (nn_learner.py, lgbm_learner.py, xgboost_learner.py)
- Hyperparameter tuning (Optuna DBs in results/optuna)
- Experiment tracking (TensorBoard logs in runs/)
E. Artifacts & Outputs
- Saved models under results/models/...
- Evaluation plots under results/images
- Demo notebook for showcasing results

##### Relationships & Data Flows
1. Data Ingestion → EDA & Cleaning
2. Cleaned data ➔ Feature Extraction (src/feature & src/lib/feature_generator)
3. Extracted features ➔ Model Training (three parallel branches: NN, LGBM, XGBoost)
4. Each training branch uses hyperparameter tuning (Optuna) and logs to TensorBoard
5. Best models are serialized to results/models
6. Demo notebook loads serialized models + sample data for inference

##### Architectural Patterns & Principles
- Pipeline pattern: clear sequential stages (ingest → transform → train → evaluate).
- Modular design: feature engineering separated from model code.
- Experiment-tracking integration: TensorBoard + Optuna for reproducibility.
- Separation of concerns: notebooks for exploration, src/ for production-grade code.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Layer"
        direction TB
        RawData["Raw CSV Data"]:::data
        FilteredData["Filtered Data"]:::data
        FeatureData["Feature-ready Data"]:::data
        DemoData["Demo Data"]:::data
    end

    %% Notebooks
    subgraph "Notebooks & EDA"
        direction TB
        EDA1[/EDA Notebook 1/]:::notebooks
        EDA2[/EDA Notebook 2/]:::notebooks
        GraphNB[/Graph EDA Notebook/]:::notebooks
        FlattenedNB[/Flattened Feature NB/]:::notebooks
        CNNGRU[/CNN/GRU Experiment NB/]:::notebooks
        DemoNB[/Demo Notebook/]:::notebooks
    end

    %% Feature Extraction
    subgraph "Feature Extraction" 
        direction TB
        FE["fe.py"]:::process
        Center["center.py"]:::process
        FirstPiece["first_peace.py"]:::process
        GamePhase["game_phase.py"]:::process
        MatAdv["material_advantage.py"]:::process
        MatCount["material_count.py"]:::process
        WhiteToMove["white_to_move.py"]:::process
        subgraph "CNN Sub-modules"
            direction TB
            CNNGray["grayscale.py"]:::process
            CNNRGB["rgb.py"]:::process
            CNNOneHot["one_hot.py"]:::process
            MoveEncoders["move_encoders.py"]:::process
        end
    end

    %% Core Utilities
    subgraph "Utilities & Helpers"
        direction TB
        FenParser(("fen_parser.py")):::util
        GraphHelper(("graph_helper.py")):::util
        FeatureGen(("feature_generator.py")):::util
        GraphFeatureGen(("graph_feature_generator.py")):::util
        EDAHelpers(("eda_helpers.py")):::util
        MoveEncoder(("move_encoder.py")):::util
        NNDataset(("nn_dataset.py")):::util
        TrainHelper(("train_helper.py")):::util
    end

    %% Modeling & Training
    subgraph "Model Training"
        direction TB
        subgraph "Neural Network"
            direction TB
            NNLearner["nn_learner.py"]:::model
            OptunaNN(("Optuna DB")):::model
            TBLogsNN(("TensorBoard Logs")):::model
        end
        subgraph "LightGBM"
            direction TB
            LGBMLearner["lgbm_learner.py"]:::model
            OptunaLGBMFlat(("lgbm_flattened.db")):::model
            OptunaLGBMGraph(("lgbm_graph.db")):::model
            TBLogsLGBM(("TensorBoard Logs")):::model
        end
        subgraph "XGBoost"
            direction TB
            XGBLearner["xgboost_learner.py"]:::model
            TBLogsXGB(("TensorBoard Logs")):::model
        end
    end

    %% Artifacts & Outputs
    subgraph "Artifacts & Outputs"
        direction TB
        SavedCNN(("cnn_best Model")):::artifact
        SavedLGBMFlat(("lgbm_flattened_best Model")):::artifact
        SavedLGBMGraph(("lgbm_graph_best Model")):::artifact
        Images(("Evaluation Images")):::artifact
    end

    %% Flows
    RawData -->|ingestion| EDA1
    RawData -->|ingestion| EDA2
    RawData -->|ingestion| GraphNB
    EDA1 --> FilteredData
    EDA2 --> FilteredData
    GraphNB --> FilteredData

    FilteredData -->|to features| FE
    FilteredData --> GamePhase
    FilteredData --> MatAdv
    FilteredData --> MatCount
    FilteredData --> WhiteToMove
    FilteredData --> Center
    FilteredData --> FirstPiece
    FilteredData --> CNNGray
    FilteredData --> CNNRGB
    FilteredData --> CNNOneHot
    FilteredData --> MoveEncoders

    FE --> FeatureData
    Center --> FeatureData
    FirstPiece --> FeatureData
    GamePhase --> FeatureData
    MatAdv --> FeatureData
    MatCount --> FeatureData
    WhiteToMove --> FeatureData
    CNNGray --> FeatureData
    CNNRGB --> FeatureData
    CNNOneHot --> FeatureData
    MoveEncoders --> FeatureData

    FeatureData --> NNLearner
    FeatureData --> LGBMLearner
    FeatureData --> XGBLearner

    NNLearner --> OptunaNN
    NNLearner --> TBLogsNN
    LGBMLearner --> OptunaLGBMFlat
    LGBMLearner --> OptunaLGBMGraph
    LGBMLearner --> TBLogsLGBM
    XGBLearner --> TBLogsXGB

    OptunaNN --> NNLearner
    OptunaLGBMFlat --> LGBMLearner
    OptunaLGBMGraph --> LGBMLearner

    NNLearner --> SavedCNN
    LGBMLearner --> SavedLGBMFlat
    LGBMLearner --> SavedLGBMGraph

    SavedCNN --> DemoNB
    SavedLGBMFlat --> DemoNB
    SavedLGBMGraph --> DemoNB
    DemoData --> DemoNB

    SavedCNN --> Images
    SavedLGBMFlat --> Images
    SavedLGBMGraph --> Images

    %% Click Events
    click RawData "https://github.com/seyyidos/cpre/tree/main/data/raw"
    click FilteredData "https://github.com/seyyidos/cpre/tree/main/data/filtered"
    click FeatureData "https://github.com/seyyidos/cpre/tree/main/data/feature"
    click DemoData "https://github.com/seyyidos/cpre/tree/main/data/demo"

    click EDA1 "https://github.com/seyyidos/cpre/blob/main/notebooks/eda.ipynb"
    click EDA2 "https://github.com/seyyidos/cpre/blob/main/notebooks/eda2.ipynb"
    click GraphNB "https://github.com/seyyidos/cpre/blob/main/notebooks/graph.ipynb"
    click FlattenedNB "https://github.com/seyyidos/cpre/blob/main/notebooks/flatenned.ipynb"
    click CNNGRU "https://github.com/seyyidos/cpre/blob/main/notebooks/cnn_gru.ipynb"
    click DemoNB "https://github.com/seyyidos/cpre/blob/main/notebooks/demo.ipynb"

    click FE "https://github.com/seyyidos/cpre/blob/main/src/feature/fe.py"
    click Center "https://github.com/seyyidos/cpre/blob/main/src/feature/center.py"
    click FirstPiece "https://github.com/seyyidos/cpre/blob/main/src/feature/first_peace.py"
    click GamePhase "https://github.com/seyyidos/cpre/blob/main/src/feature/game_phase.py"
    click MatAdv "https://github.com/seyyidos/cpre/blob/main/src/feature/material_advantage.py"
    click MatCount "https://github.com/seyyidos/cpre/blob/main/src/feature/material_count.py"
    click WhiteToMove "https://github.com/seyyidos/cpre/blob/main/src/feature/white_to_move.py"

    click CNNGray "https://github.com/seyyidos/cpre/blob/main/src/feature/cnn/grayscale.py"
    click CNNRGB "https://github.com/seyyidos/cpre/blob/main/src/feature/cnn/rgb.py"
    click CNNOneHot "https://github.com/seyyidos/cpre/blob/main/src/feature/cnn/one_hot.py"
    click MoveEncoders "https://github.com/seyyidos/cpre/blob/main/src/feature/cnn/move_encoders.py"

    click FenParser "https://github.com/seyyidos/cpre/blob/main/src/lib/fen_parser.py"
    click GraphHelper "https://github.com/seyyidos/cpre/blob/main/src/lib/graph_helper.py"
    click FeatureGen "https://github.com/seyyidos/cpre/blob/main/src/lib/feature_generator.py"
    click GraphFeatureGen "https://github.com/seyyidos/cpre/blob/main/src/lib/graph_feature_generator.py"
    click EDAHelpers "https://github.com/seyyidos/cpre/blob/main/src/lib/eda_helpers.py"
    click MoveEncoder "https://github.com/seyyidos/cpre/blob/main/src/lib/move_encoder.py"
    click NNDataset "https://github.com/seyyidos/cpre/blob/main/src/lib/nn_dataset.py"
    click TrainHelper "https://github.com/seyyidos/cpre/blob/main/src/lib/train_helper.py"

    click NNLearner "https://github.com/seyyidos/cpre/blob/main/src/lib/models/nn_learner.py"
    click LGBMLearner "https://github.com/seyyidos/cpre/blob/main/src/lib/models/lgbm_learner.py"
    click XGBLearner "https://github.com/seyyidos/cpre/blob/main/src/lib/models/xgboost_learner.py"

    click OptunaLGBMFlat "https://github.com/seyyidos/cpre/blob/main/results/optuna/lgbm_flattened.db"
    click OptunaLGBMGraph "https://github.com/seyyidos/cpre/blob/main/results/optuna/lgbm_graph.db"

    classDef data fill:#ADD8E6,stroke:#000;
    classDef process fill:#90EE90,stroke:#000;
    classDef util fill:#D3D3D3,stroke:#000;
    classDef model fill:#FFA500,stroke:#000;
    classDef artifact fill:#CCCCCC,stroke:#000;
    classDef notebooks fill:#98FB98,stroke:#000;
```
