# jhonnydiaz4753/exoplanet-ml-project architecture

> Architecture diagram of the GitHub repository jhonnydiaz4753/exoplanet-ml-project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jhonnydiaz4753/exoplanet-ml-project
- Repository: https://github.com/jhonnydiaz4753/exoplanet-ml-project

## Overview

##### Project type and purpose
- From the file tree (no README content provided), this is a machine-learning pipeline focused on processing Kepler light-curve data to build and evaluate classification models (CNN and LightGBM). It includes raw data ingestion, preprocessing, feature extraction (tsfresh), model training, evaluation, and reporting.

##### File structure
- Top-level directories:
- data/raw: original light-curve CSVs, separated into positive/negative classes
- data/processed: cleaned/merged CSVs
- data/features: extracted features and labels CSVs
- notebooks: Jupyter notebooks for each pipeline stage (ingest, feature extract, training, evaluation)
- src:
- src/data/preprocess.py: core preprocessing logic
- src/features/tsfresh_utils.py: feature-extraction helpers
- src/models/{cnn_model.py, lightgbm_model.py}: model definitions
- src/evaluation/metrics.py: evaluation metrics
- scripts/run_pipeline.sh: CLI to execute end-to-end pipeline
- reports/figures: generated plots and descriptions
- Dockerfile & requirements.txt: container definition and dependencies

##### Key components
a. Data Sources
- Kepler raw light-curve files (data/raw)
b. Data Ingestion & Preprocessing
- preprocess.py (with orchestration via run_pipeline.sh or notebooks)
- intermediate storage: data/processed
c. Feature Extraction
- tsfresh_utils.py producing feature matrix (data/features/features.csv)
d. Model Training
- CNN and LightGBM trainers (src/models), driven from notebooks or scripts
- model artifacts output (e.g., serialized weights or ML artifacts folder)
e. Evaluation
- metrics.py computing accuracy, ROC, etc.
- outputs to reports/figures
f. Reporting
- Jupyter notebooks and static plots (reports/figures)
g. Orchestration & Deployment
- run_pipeline.sh
- Docker container (Dockerfile)

##### Relationships and data flows
- Raw data → Preprocessing → Processed data
- Processed data → Feature extraction → Feature matrix + labels
- Feature matrix → Model training (two branches: CNN & LightGBM) → Trained models
- Trained models + test data → Evaluation → Metrics & plots
- Metrics & plots → Reports directory

##### Architectural patterns and principles
- Batch pipeline pattern with clear staging areas (raw, processed, features)
- Modular code separation: data, features, models, evaluation
- Repeatable execution via script and containerization
- Notebooks for exploratory & reproducible steps

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Layer"
        DS1[(Raw Kepler data)]:::datastore
    end

    %% Preprocessing Stage
    subgraph "Data Ingestion & Preprocessing"
        PP["preprocess.py"]:::compute
        Script[/run_pipeline.sh/]:::script
        Processed[(Processed Data)]:::datastore
    end

    %% Feature Extraction Stage
    subgraph "Feature Extraction"
        FE["tsfresh_utils.py"]:::compute
        Features[(features.csv)]:::datastore
        Labels[(labels.csv)]:::datastore
    end

    %% Model Training Stage
    subgraph "Model Training"
        NoteTrain["03_model_training.ipynb"]:::notebook
        CNN["cnn_model.py"]:::model
        LGBM["lightgbm_model.py"]:::model
    end

    %% Evaluation Stage
    subgraph "Evaluation"
        EvalMod["metrics.py"]:::compute
        EvalNote["04_evaluation.ipynb"]:::notebook
    end

    %% Reporting Stage
    subgraph "Reporting"
        Artifacts[(reports/figures)]:::reporting
    end

    %% Environment
    Env["Dockerfile & requirements.txt"]:::infra

    %% Data Flows
    DS1 --> PP
    Script -.-> PP
    PP --> Processed
    Processed --> FE
    FE --> Features
    FE --> Labels
    Features --> NoteTrain
    Labels --> NoteTrain
    NoteTrain --> CNN
    NoteTrain --> LGBM
    CNN --> EvalMod
    LGBM --> EvalMod
    EvalMod --> EvalNote
    EvalNote --> Artifacts

    %% Orchestration
    Env -.-> Script

    %% Click Events
    click DS1 "https://github.com/jhonnydiaz4753/exoplanet-ml-project/tree/main/data/raw"
    click PP "https://github.com/jhonnydiaz4753/exoplanet-ml-project/blob/main/src/data/preprocess.py"
    click Script "https://github.com/jhonnydiaz4753/exoplanet-ml-project/blob/main/scripts/run_pipeline.sh"
    click Processed "https://github.com/jhonnydiaz4753/exoplanet-ml-project/tree/main/data/processed"
    click FE "https://github.com/jhonnydiaz4753/exoplanet-ml-project/blob/main/src/features/tsfresh_utils.py"
    click Features "https://github.com/jhonnydiaz4753/exoplanet-ml-project/blob/main/data/features/features.csv"
    click Labels "https://github.com/jhonnydiaz4753/exoplanet-ml-project/blob/main/data/features/labels.csv"
    click CNN "https://github.com/jhonnydiaz4753/exoplanet-ml-project/blob/main/src/models/cnn_model.py"
    click LGBM "https://github.com/jhonnydiaz4753/exoplanet-ml-project/blob/main/src/models/lightgbm_model.py"
    click NoteTrain "https://github.com/jhonnydiaz4753/exoplanet-ml-project/blob/main/notebooks/03_model_training.ipynb"
    click EvalMod "https://github.com/jhonnydiaz4753/exoplanet-ml-project/blob/main/src/evaluation/metrics.py"
    click EvalNote "https://github.com/jhonnydiaz4753/exoplanet-ml-project/blob/main/notebooks/04_evaluation.ipynb"
    click Artifacts "https://github.com/jhonnydiaz4753/exoplanet-ml-project/tree/main/reports/figures"
    click Env "https://github.com/jhonnydiaz4753/exoplanet-ml-project/tree/main/Dockerfile"
    click Env "https://github.com/jhonnydiaz4753/exoplanet-ml-project/blob/main/requirements.txt"

    %% Styles
    classDef datastore fill:#bbdefb,stroke:#0d47a1,color:#0d47a1;
    classDef compute fill:#c8e6c9,stroke:#1b5e20,color:#1b5e20;
    classDef model fill:#ffe0b2,stroke:#e65100,color:#e65100;
    classDef reporting fill:#e1bee7,stroke:#4a148c,color:#4a148c;
    classDef script fill:#fff9c4,stroke:#f57f17,color:#f57f17;
    classDef notebook fill:#b2ebf2,stroke:#006064,color:#006064;
    classDef infra fill:#cfd8dc,stroke:#37474f,color:#37474f;
```
