How seyyidos/cpre works
Project Type and Purpose
• This is a data-science/ML pipeline project (“Chess Puzzle Rating Estimation”) whose goal is to predict the difficulty rating of chess puzzles.
• It uses raw chess puzzle data → feature extraction → model training (NN, LightGBM, XGBoost) → hyperparameter tuning → result artifacts (models, images).
Key File-Structure Insights
• notebooks/: Jupyter notebooks for EDA, demo, model experiments.
• src/feature/: feature‐engineering modules (board representation, material count, one-hot, graph features).
• src/lib/: helper libraries (data parsing, dataset classes, model-training helpers, hyperparameter tuning via Optuna).
• results/: saved model files (cnn, lgbm variants), images.
• runs/: TensorBoard logs for NN and LGBM experiments.
• requirements.txt & README.md: Python environment and project overview.
Main Components
A. Data Layer
• Raw data (data/raw)
• Filtered data (data/filtered)
• Feature‐ready data (data/feature)
• Demo data (data/demo)
B. Processing & Feature Extraction
• EDA notebooks (notebooks/eda*.ipynb)
• src/feature modules (cnn, graph, game_phase, material_advantage, etc.)
C. Core Library & Utilities
• fen_parser, eda_helpers, feature_generator
• graph_helper, move_encoder, nn_dataset, train_helper
D. Modeling & Training
• Learner modules (nn_learner.py, lgbm_learner.py, xgboost_learner.py)
• Hyperparameter tuning (Optuna DBs in results/optuna)
• Experiment tracking (TensorBoard logs in runs/)
E. Artifacts & Outputs
• Saved models under results/models/...
• Evaluation plots under results/images
• Demo notebook for showcasing results
Relationships & Data Flows
1. Data Ingestion → EDA & Cleaning
2. Cleaned data ➔ Feature Extraction (src/feature & src/lib/feature_generator)
3. Extracted features ➔ Model Training (three parallel branches: NN, LGBM, XGBoost)
4. Each training branch uses hyperparameter tuning (Optuna) and logs to TensorBoard
5. Best models are serialized to results/models
6. Demo notebook loads serialized models + sample data for inference
Architectural Patterns & Principles
• Pipeline pattern: clear sequential stages (ingest → transform → train → evaluate).
• Modular design: feature engineering separated from model code.
• Experiment-tracking integration: TensorBoard + Optuna for reproducibility.
• Separation of concerns: notebooks for exploration, src/ for production-grade code.