How nshakib30/oil_production_ml works
1. Repository purpose
- This is a single-process ML inference demo for predicting daily oil production from well operating conditions.
- It is not a multi-service system: the live app, model loading, preprocessing, and explanations all run in the same Python app.
2. Runtime entry point
- app.py is the serving/UI layer and is what Streamlit runs.
- The README indicates the app is started with streamlit run app.py, so the architecture is centered on a Streamlit front end + in-process prediction logic.
3. Interactive prediction flow
- The user enters well conditions such as pressure, choke size, flow hours, and water volume in the Streamlit UI.
- Those inputs are converted into the expected feature vector, passed through preprocessing, then sent to the trained model for an oil-rate prediction.
- The predicted value and explanation are then rendered back in the UI.
4. Model inference boundary
- model.pkl is the persisted trained estimator used for inference.
- scaler.pkl is the preprocessing artifact used to normalize/scale inputs before inference.
- feature_columns.pkl defines the exact feature order/schema expected by the model, which is important for keeping training and inference aligned.
5. Offline training / experimentation boundary
- petroleum_ml_pipeline.py appears to be the offline ML pipeline used to train, compare, and evaluate candidate models.
- The README says multiple models were tested, with XGBoost selected as best, so this script likely owns dataset loading, feature preparation, model comparison, metrics computation, and artifact serialization.
6. Source data
- Volve production data-Daily Production Data.csv is the historical dataset used for training and validation.
- This is the main data source for the offline pipeline; the live app does not appear to query an external database or API.
7. Explainability layer
- The README calls out SHAP-based explanations, so the app likely computes local feature attributions for each prediction.
- Architecturally, this is a post-prediction analysis step: model output first, explanation second, then both are displayed together.
8. Data flow split: offline vs online
- Offline path: CSV dataset → training pipeline → model/scaler/feature artifacts saved to disk.
- Online path: Streamlit user input → load artifacts → preprocess → predict → explain → display.
- That separation is the key architectural boundary in the repo.
9. Deployment/runtime
- The README references a live Streamlit demo, so the deployment target is likely Streamlit Community Cloud or a similar Streamlit-hosted runtime.
- config.toml probably contains Streamlit app configuration, such as theme or runtime settings.
10. Environment/tooling
- requirements.txt defines the Python dependencies needed for both training and inference.
- .devcontainer/devcontainer.json indicates a containerized development setup, which helps reproduce the Python environment consistently.
11. What is not present
- There is no evidence of a separate backend API, message queue, database, or scheduled job system.
- The architecture is intentionally compact: one UI app, one training script, one local model bundle, and one dataset.
12. How to draw it
- A good architecture diagram would show:
- a Streamlit UI at the edge,
- a prediction/explanation layer inside app.py,
- local serialized artifacts (model.pkl, scaler.pkl, feature_columns.pkl),
- an offline training pipeline (petroleum_ml_pipeline.py) fed by the CSV dataset,
- and SHAP as an internal explanation component attached to inference.
Main entry points: petroleum_ml_pipeline.py Training pipeline · app.py Feature prep · requirements.txt requirements · model.pkl model.pkl
Components 15
Online inference
Streamlit app and in-process prediction flow
- app.py · Entry point for the interactive UI and in-process inference
app.py - Input form · Collects well operating conditions from the user
app.py - Feature prep · Builds the model-ready feature vector and aligns feature order
app.py - Predictor · Loads the trained estimator and produces the oil-rate prediction
app.py - Explain · Computes local feature attributions after prediction
app.py - Results view · Renders prediction and explanation back to the user
app.py
Offline training
Training and artifact creation from historical data
- Training pipeline · Offline experimentation, model comparison, evaluation, and artifact export
petroleum_ml_pipeline.py - Production data · Historical daily production data used for training and validation
Volve production data-Daily Production Data.csv - model.pkl · Persisted estimator used for online inference
model.pkl - scaler.pkl · Persisted normalization or scaling transform for inference
scaler.pkl - feature_columns.pkl · Defines the exact feature order expected by the model
feature_columns.pkl
Runtime support
Repo-level tooling and configuration
- README · Describes app startup, model choice, and usage
README.md - config.toml · Streamlit runtime and UI configuration
config.toml - requirements · Dependency set for training and inference
requirements.txt - Dev container · Containerized development environment for reproducing the Python stack
.devcontainer/devcontainer.json
Connections 18
- app.py → Input form: collects input · The Streamlit app gathers well conditions from the user
- Input form → Feature prep: builds features · User inputs are converted into the expected model feature vector
- Feature prep → feature_columns.pkl: aligns schema · Feature preparation uses the persisted column order
- Feature prep → scaler.pkl: scales input · Inputs are normalized with the saved preprocessing artifact
- Feature prep → Predictor: passes vector · Prepared features are sent to the trained estimator
- Predictor → model.pkl: loads estimator · Inference uses the serialized model artifact
- Predictor → Explain: explains result · Local attribution is computed after the prediction step
- Explain → Results view: annotates output · Explanations are rendered alongside the prediction
- Predictor → Results view: returns rate · Predicted oil production is shown in the UI
- Training pipeline → Production data: trains on · The offline pipeline consumes the historical production dataset
- Training pipeline → model.pkl: exports · Trained estimator is serialized for online use
- Training pipeline → scaler.pkl: exports · Fitted preprocessing artifact is serialized for online use
- Training pipeline → feature_columns.pkl: exports · Feature schema is persisted to keep training and inference aligned
- README → app.py: starts · README documents the Streamlit entry point
- config.toml → app.py: configures · Streamlit runtime settings apply to the app
- requirements → app.py: supports · Python dependencies are required by the serving app
- requirements → Training pipeline: supports · Python dependencies are also used by the training pipeline
- Dev container → requirements: reproduces · Containerized development environment mirrors the Python dependency stack