# nshakib30/oil_production_ml architecture

> Architecture diagram of the GitHub repository nshakib30/oil_production_ml, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nshakib30/oil_production_ml
- Repository: https://github.com/nshakib30/oil_production_ml

## Overview

1. **Repository purpose**
   - This is a **single-process ML inference demo** for predicting daily oil production from well operating conditions.
   - It is not a multi-service system: the live app, model loading, preprocessing, and explanations all run in the same Python app.

2. **Runtime entry point**
   - `app.py` is the **serving/UI layer** and is what Streamlit runs.
   - The README indicates the app is started with `streamlit run app.py`, so the architecture is centered on a **Streamlit front end + in-process prediction logic**.

3. **Interactive prediction flow**
   - The user enters well conditions such as pressure, choke size, flow hours, and water volume in the Streamlit UI.
   - Those inputs are converted into the expected feature vector, passed through preprocessing, then sent to the trained model for an oil-rate prediction.
   - The predicted value and explanation are then rendered back in the UI.

4. **Model inference boundary**
   - `model.pkl` is the persisted trained estimator used for inference.
   - `scaler.pkl` is the preprocessing artifact used to normalize/scale inputs before inference.
   - `feature_columns.pkl` defines the exact feature order/schema expected by the model, which is important for keeping training and inference aligned.

5. **Offline training / experimentation boundary**
   - `petroleum_ml_pipeline.py` appears to be the **offline ML pipeline** used to train, compare, and evaluate candidate models.
   - The README says multiple models were tested, with XGBoost selected as best, so this script likely owns dataset loading, feature preparation, model comparison, metrics computation, and artifact serialization.

6. **Source data**
   - `Volve production data-Daily Production Data.csv` is the historical dataset used for training and validation.
   - This is the main data source for the offline pipeline; the live app does not appear to query an external database or API.

7. **Explainability layer**
   - The README calls out SHAP-based explanations, so the app likely computes local feature attributions for each prediction.
   - Architecturally, this is a **post-prediction analysis step**: model output first, explanation second, then both are displayed together.

8. **Data flow split: offline vs online**
   - **Offline path:** CSV dataset → training pipeline → model/scaler/feature artifacts saved to disk.
   - **Online path:** Streamlit user input → load artifacts → preprocess → predict → explain → display.
   - That separation is the key architectural boundary in the repo.

9. **Deployment/runtime**
   - The README references a live Streamlit demo, so the deployment target is likely **Streamlit Community Cloud** or a similar Streamlit-hosted runtime.
   - `config.toml` probably contains Streamlit app configuration, such as theme or runtime settings.

10. **Environment/tooling**
   - `requirements.txt` defines the Python dependencies needed for both training and inference.
   - `.devcontainer/devcontainer.json` indicates a containerized development setup, which helps reproduce the Python environment consistently.

11. **What is not present**
   - There is no evidence of a separate backend API, message queue, database, or scheduled job system.
   - The architecture is intentionally compact: one UI app, one training script, one local model bundle, and one dataset.

12. **How to draw it**
   - A good architecture diagram would show:
     - a **Streamlit UI** at the edge,
     - a **prediction/explanation layer** inside `app.py`,
     - local **serialized artifacts** (`model.pkl`, `scaler.pkl`, `feature_columns.pkl`),
     - an **offline training pipeline** (`petroleum_ml_pipeline.py`) fed by the CSV dataset,
     - and SHAP as an internal explanation component attached to inference.

Main entry points: [`petroleum_ml_pipeline.py`](https://github.com/nshakib30/oil_production_ml/blob/main/petroleum_ml_pipeline.py) (Training pipeline), [`app.py`](https://github.com/nshakib30/oil_production_ml/blob/main/app.py) (Feature prep), [`requirements.txt`](https://github.com/nshakib30/oil_production_ml/blob/main/requirements.txt) (requirements), [`model.pkl`](https://github.com/nshakib30/oil_production_ml/blob/main/model.pkl) (model.pkl)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_online["Online inference"]
  node_app["app.py<br/>Streamlit app<br/>[app.py]"]
  node_ui["Input form<br/>UI layer<br/>[app.py]"]
  node_prep["Feature prep<br/>preprocessing<br/>[app.py]"]
  node_predict["Predictor<br/>inference<br/>[app.py]"]
  node_shap{{"Explain<br/>SHAP attribution<br/>[app.py]"}}
  node_render["Results view<br/>display<br/>[app.py]"]
end

subgraph group_offline["Offline training"]
  node_pipeline["Training pipeline<br/>ML pipeline"]
  node_data[("Production data<br/>dataset")]
  node_model["model.pkl<br/>trained model<br/>[model.pkl]"]
  node_scaler["scaler.pkl<br/>scaler artifact<br/>[scaler.pkl]"]
  node_features["feature_columns.pkl<br/>schema artifact"]
end

subgraph group_runtime["Runtime support"]
  node_readme["README<br/>docs<br/>[README.md]"]
  node_config["config.toml<br/>streamlit config<br/>[config.toml]"]
  node_deps["requirements<br/>python deps<br/>[requirements.txt]"]
  node_devcontainer["Dev container<br/>dev environment<br/>[devcontainer.json]"]
end

node_app -->|"collects input"| node_ui
node_ui -->|"builds features"| node_prep
node_prep -->|"aligns schema"| node_features
node_prep -->|"scales input"| node_scaler
node_prep -->|"passes vector"| node_predict
node_predict -->|"loads estimator"| node_model
node_predict -->|"explains result"| node_shap
node_shap -->|"annotates output"| node_render
node_predict -->|"returns rate"| node_render
node_pipeline -->|"trains on"| node_data
node_pipeline -->|"exports"| node_model
node_pipeline -->|"exports"| node_scaler
node_pipeline -->|"exports"| node_features
node_readme -.->|"starts"| node_app
node_config -.->|"configures"| node_app
node_deps -.->|"supports"| node_app
node_deps -.->|"supports"| node_pipeline
node_devcontainer -.->|"reproduces"| node_deps

click node_app "https://github.com/nshakib30/oil_production_ml/blob/main/app.py"
click node_ui "https://github.com/nshakib30/oil_production_ml/blob/main/app.py"
click node_prep "https://github.com/nshakib30/oil_production_ml/blob/main/app.py"
click node_predict "https://github.com/nshakib30/oil_production_ml/blob/main/app.py"
click node_shap "https://github.com/nshakib30/oil_production_ml/blob/main/app.py"
click node_render "https://github.com/nshakib30/oil_production_ml/blob/main/app.py"
click node_pipeline "https://github.com/nshakib30/oil_production_ml/blob/main/petroleum_ml_pipeline.py"
click node_data "https://github.com/nshakib30/oil_production_ml/blob/main/Volve production data-Daily Production Data.csv"
click node_model "https://github.com/nshakib30/oil_production_ml/blob/main/model.pkl"
click node_scaler "https://github.com/nshakib30/oil_production_ml/blob/main/scaler.pkl"
click node_features "https://github.com/nshakib30/oil_production_ml/blob/main/feature_columns.pkl"
click node_readme "https://github.com/nshakib30/oil_production_ml/blob/main/README.md"
click node_config "https://github.com/nshakib30/oil_production_ml/blob/main/config.toml"
click node_deps "https://github.com/nshakib30/oil_production_ml/blob/main/requirements.txt"
click node_devcontainer "https://github.com/nshakib30/oil_production_ml/blob/main/.devcontainer/devcontainer.json"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_ui,node_prep,node_predict,node_shap,node_render toneBlue
class node_pipeline,node_data,node_model,node_scaler,node_features toneAmber
class node_readme,node_config,node_deps,node_devcontainer toneMint
```

## Components

### Online inference

Streamlit app and in-process prediction flow

- [app.py](https://github.com/nshakib30/oil_production_ml/blob/main/app.py): Streamlit app · Entry point for the interactive UI and in-process inference · `app.py`
- [Input form](https://github.com/nshakib30/oil_production_ml/blob/main/app.py): UI layer · Collects well operating conditions from the user · `app.py`
- [Feature prep](https://github.com/nshakib30/oil_production_ml/blob/main/app.py): preprocessing · Builds the model-ready feature vector and aligns feature order · `app.py`
- [Predictor](https://github.com/nshakib30/oil_production_ml/blob/main/app.py): inference · Loads the trained estimator and produces the oil-rate prediction · `app.py`
- [Explain](https://github.com/nshakib30/oil_production_ml/blob/main/app.py): SHAP attribution · Computes local feature attributions after prediction · `app.py`
- [Results view](https://github.com/nshakib30/oil_production_ml/blob/main/app.py): display · Renders prediction and explanation back to the user · `app.py`

### Offline training

Training and artifact creation from historical data

- [Training pipeline](https://github.com/nshakib30/oil_production_ml/blob/main/petroleum_ml_pipeline.py): ML pipeline · Offline experimentation, model comparison, evaluation, and artifact export · `petroleum_ml_pipeline.py`
- [Production data](https://github.com/nshakib30/oil_production_ml/blob/HEAD/Volve%20production%20data-Daily%20Production%20Data.csv): dataset · Historical daily production data used for training and validation · `Volve production data-Daily Production Data.csv`
- [model.pkl](https://github.com/nshakib30/oil_production_ml/blob/main/model.pkl): trained model · Persisted estimator used for online inference · `model.pkl`
- [scaler.pkl](https://github.com/nshakib30/oil_production_ml/blob/main/scaler.pkl): scaler artifact · Persisted normalization or scaling transform for inference · `scaler.pkl`
- [feature_columns.pkl](https://github.com/nshakib30/oil_production_ml/blob/main/feature_columns.pkl): schema artifact · Defines the exact feature order expected by the model · `feature_columns.pkl`

### Runtime support

Repo-level tooling and configuration

- [README](https://github.com/nshakib30/oil_production_ml/blob/main/README.md): docs · Describes app startup, model choice, and usage · `README.md`
- [config.toml](https://github.com/nshakib30/oil_production_ml/blob/main/config.toml): streamlit config · Streamlit runtime and UI configuration · `config.toml`
- [requirements](https://github.com/nshakib30/oil_production_ml/blob/main/requirements.txt): python deps · Dependency set for training and inference · `requirements.txt`
- [Dev container](https://github.com/nshakib30/oil_production_ml/blob/main/.devcontainer/devcontainer.json): dev environment · Containerized development environment for reproducing the Python stack · `.devcontainer/devcontainer.json`

## Connections

- app.py → Input form: collects input · The Streamlit app gathers well conditions from the user
- Input form → Feature prep: builds features · User inputs are converted into the expected model feature vector
- Feature prep → feature_columns.pkl: aligns schema · Feature preparation uses the persisted column order
- Feature prep → scaler.pkl: scales input · Inputs are normalized with the saved preprocessing artifact
- Feature prep → Predictor: passes vector · Prepared features are sent to the trained estimator
- Predictor → model.pkl: loads estimator · Inference uses the serialized model artifact
- Predictor → Explain: explains result · Local attribution is computed after the prediction step
- Explain → Results view: annotates output · Explanations are rendered alongside the prediction
- Predictor → Results view: returns rate · Predicted oil production is shown in the UI
- Training pipeline → Production data: trains on · The offline pipeline consumes the historical production dataset
- Training pipeline → model.pkl: exports · Trained estimator is serialized for online use
- Training pipeline → scaler.pkl: exports · Fitted preprocessing artifact is serialized for online use
- Training pipeline → feature_columns.pkl: exports · Feature schema is persisted to keep training and inference aligned
- README → app.py: starts · README documents the Streamlit entry point
- config.toml → app.py: configures · Streamlit runtime settings apply to the app
- requirements → app.py: supports · Python dependencies are required by the serving app
- requirements → Training pipeline: supports · Python dependencies are also used by the training pipeline
- Dev container → requirements: reproduces · Containerized development environment mirrors the Python dependency stack