# il98/well_log_ml_project architecture

> Architecture diagram of the GitHub repository il98/well_log_ml_project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/il98/well_log_ml_project
- Repository: https://github.com/il98/well_log_ml_project

## Overview

##### Project type and purpose
- This is a data-science / machine-learning pipeline project. Its goal is to predict reservoir porosity from well-log CSV data using feature engineering, classical ML models (RF, XGBoost, KNN, Linear Regression, FNN), and time-series models (RNN/LSTM). It is organized around Jupyter notebooks that implement data ingestion, preprocessing, model training & evaluation, and result reporting.

##### File structure
- Top-level:
- README.md – project overview
- data/ – raw CSV files for wells 169, 170, 171, 183 PL
- notebooks/ – four analysis pipelines, each with a checkpoint folder
- PDF reports – ml_project_well_log.pdf & presentation
- No production code or services; everything runs in notebooks.
- Each notebook covers one stage: raw ML, feature engineering ML, time-series ML, time-series feature engineering.

##### Extract insights from the README
- Four primary workflows:
a. well_log_ml_models.ipynb
b. well_log_feature_engineering.ipynb
c. well_log_time_series.ipynb
d. well_log_time_series_feature_eng.ipynb
- Core libraries and frameworks: Python, pandas, scikit-learn, XGBoost, Keras/TensorFlow (for RNN/LSTM)

##### Components
a. Data Ingestion Layer
- Raw CSV files (data/169.csv, 170.csv, 171.csv, 183 PL.csv)
- Data validation & loading
b. Preprocessing & Feature Engineering
- Notebook 1: basic feature set
- Notebook 2: engineered features (scaling, transformations)
- Notebook 4: time-series feature engine (lag, windowing)
c. Model Training & Selection
- ML models (RF, XGBoost, KNN, LinReg, FNN)
- Time-series models (RNN/LSTM)
- Hyperparameter tuning & cross-validation
d. Evaluation & Comparison
- Metrics (RMSE, MAE, R²)
- Per-well and cross-well performance
e. Reporting & Artifacts
- PDF reports (ml_project_well_log.pdf, ml_project_well_log_presentation.pdf)
f. Notebooks as Orchestration
- Each notebook orchestrates one or more of the above steps
- Checkpoints folder for iterative development

##### Architectural patterns and principles
- Modular pipeline: clear separation of data, features, models, evaluation, and reporting
- Reproducibility via notebooks and checkpoint subdirectories
- Extensibility: easy to add new models or features by dropping a new notebook module

##### Relevant technologies
- Python & Jupyter Notebooks
- pandas, NumPy (data manipulation)
- scikit-learn (classical ML)
- XGBoost (gradient boosting)
- Keras/TensorFlow (RNN/LSTM)
- matplotlib / seaborn (visualization in notebooks)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Layer
    subgraph "Data Ingestion Layer"
        Raw169["169.csv"]:::data
        Raw170["170.csv"]:::data
        Raw171["171.csv"]:::data
        Raw183["183 PL.csv"]:::data
    end

    %% Feature Engineering
    subgraph "Preprocessing & Feature Engineering"
        BasicFE["Basic Feature Eng"]:::feature
        AdvFE["Advanced Feature Eng"]:::feature
        TSFE["Time-Series Feature Eng"]:::feature
    end

    %% Modeling
    subgraph "Model Training & Selection"
        Classical["Classical ML Training"]:::model
        TSModel["Time-Series Model Training"]:::model
    end

    %% Evaluation
    subgraph "Evaluation & Comparison"
        Eval["Evaluation & Comparison"]:::eval
    end

    %% Reporting
    subgraph "Reporting & Artifacts"
        ReportPDF["ml_project_well_log.pdf"]:::report
        ReportPres["ml_project_well_log_presentation.pdf"]:::report
    end

    %% Orchestration
    subgraph "Notebooks Orchestration"
        NotebookDir["notebooks/"]:::orchestration
        Checkpoints[".ipynb_checkpoints/"]:::orchestration
    end

    %% Data Flow
    Raw169 --> BasicFE
    Raw170 --> BasicFE
    Raw171 --> BasicFE
    Raw183 --> BasicFE

    Raw169 --> AdvFE
    Raw170 --> AdvFE
    Raw171 --> AdvFE
    Raw183 --> AdvFE

    Raw169 --> TSFE
    Raw170 --> TSFE
    Raw171 --> TSFE
    Raw183 --> TSFE

    BasicFE --> Classical
    AdvFE --> Classical
    TSFE --> TSModel

    Classical --> Eval
    TSModel --> Eval

    Eval --> ReportPDF
    Eval --> ReportPres

    %% Orchestration Controls
    NotebookDir -.-> BasicFE
    NotebookDir -.-> AdvFE
    NotebookDir -.-> TSFE
    NotebookDir -.-> Classical
    NotebookDir -.-> TSModel
    NotebookDir -.-> Eval
    NotebookDir -.-> ReportPDF

    Checkpoints -.-> BasicFE
    Checkpoints -.-> AdvFE
    Checkpoints -.-> TSFE
    Checkpoints -.-> Classical
    Checkpoints -.-> TSModel
    Checkpoints -.-> Eval

    %% Click Events
    click Raw169 "https://github.com/il98/well_log_ml_project/blob/main/data/169.csv"
    click Raw170 "https://github.com/il98/well_log_ml_project/blob/main/data/170.csv"
    click Raw171 "https://github.com/il98/well_log_ml_project/blob/main/data/171.csv"
    click Raw183 "https://github.com/il98/well_log_ml_project/blob/main/data/183 PL.csv"

    click BasicFE "https://github.com/il98/well_log_ml_project/blob/main/notebooks/well_log_ml_models.ipynb"
    click AdvFE "https://github.com/il98/well_log_ml_project/blob/main/notebooks/well_log_feature_engineering.ipynb"
    click TSFE "https://github.com/il98/well_log_ml_project/blob/main/notebooks/well_log_time_series_feature_eng.ipynb"

    click Classical "https://github.com/il98/well_log_ml_project/blob/main/notebooks/well_log_ml_models.ipynb"
    click TSModel "https://github.com/il98/well_log_ml_project/blob/main/notebooks/well_log_time_series.ipynb"

    click NotebookDir "https://github.com/il98/well_log_ml_project/tree/main/notebooks/"
    click Checkpoints "https://github.com/il98/well_log_ml_project/tree/main/notebooks/.ipynb_checkpoints/"

    click ReportPDF "https://github.com/il98/well_log_ml_project/blob/main/ml_project_well_log.pdf"
    click ReportPres "https://github.com/il98/well_log_ml_project/blob/main/ml_project_well_log_presentation.pdf"

    %% Styles
    classDef data fill:#add8e6,stroke:#333,stroke-width:1px;
    classDef feature fill:#90ee90,stroke:#333,stroke-width:1px;
    classDef model fill:#ffa500,stroke:#333,stroke-width:1px;
    classDef eval fill:#9370db,stroke:#333,stroke-width:1px;
    classDef report fill:#d3d3d3,stroke:#333,stroke-width:1px;
    classDef orchestration fill:#f0f0f0,stroke:#333,stroke-width:1px;
```
