How jhonnydiaz4753/exoplanet-ml-project works
Project type and purpose
• From the file tree (no README content provided), this is a machine-learning pipeline focused on processing Kepler light-curve data to build and evaluate classification models (CNN and LightGBM). It includes raw data ingestion, preprocessing, feature extraction (tsfresh), model training, evaluation, and reporting.
File structure
• Top-level directories:
• data/raw: original light-curve CSVs, separated into positive/negative classes
• data/processed: cleaned/merged CSVs
• data/features: extracted features and labels CSVs
• notebooks: Jupyter notebooks for each pipeline stage (ingest, feature extract, training, evaluation)
• src:
• src/data/preprocess.py: core preprocessing logic
• src/features/tsfresh_utils.py: feature-extraction helpers
• src/models/{cnn_model.py, lightgbm_model.py}: model definitions
• src/evaluation/metrics.py: evaluation metrics
• scripts/run_pipeline.sh: CLI to execute end-to-end pipeline
• reports/figures: generated plots and descriptions
• Dockerfile & requirements.txt: container definition and dependencies
Key components
a. Data Sources
• Kepler raw light-curve files (data/raw)
b. Data Ingestion & Preprocessing
• preprocess.py (with orchestration via run_pipeline.sh or notebooks)
• intermediate storage: data/processed
c. Feature Extraction
• tsfresh_utils.py producing feature matrix (data/features/features.csv)
d. Model Training
• CNN and LightGBM trainers (src/models), driven from notebooks or scripts
• model artifacts output (e.g., serialized weights or ML artifacts folder)
e. Evaluation
• metrics.py computing accuracy, ROC, etc.
• outputs to reports/figures
f. Reporting
• Jupyter notebooks and static plots (reports/figures)
g. Orchestration & Deployment
• run_pipeline.sh
• Docker container (Dockerfile)
Relationships and data flows
• Raw data → Preprocessing → Processed data
• Processed data → Feature extraction → Feature matrix + labels
• Feature matrix → Model training (two branches: CNN & LightGBM) → Trained models
• Trained models + test data → Evaluation → Metrics & plots
• Metrics & plots → Reports directory
Architectural patterns and principles
• Batch pipeline pattern with clear staging areas (raw, processed, features)
• Modular code separation: data, features, models, evaluation
• Repeatable execution via script and containerization
• Notebooks for exploratory & reproducible steps