How sushantkumar31/feature-engineering-analyzer works
Purpose and Entry Point
This is a single-process Streamlit application for demonstrating the effect of tabular-data feature engineering on ML performance. The executable UI and pipeline implementation are centralized in app.py; users launch it with streamlit run app.py.
Interactive Dataset Intake
The app accepts a user-uploaded CSV, presents dataset preview/statistics, and requires selection/validation of a target column. Uploaded data is in-memory and session-driven rather than backed by a database or persistent dataset service.
• app.py
Baseline Data Path
The original uploaded dataset forms the baseline branch. The selected target is separated from predictors, and the chosen ML algorithm is trained/evaluated on raw-data preparation appropriate for model input. Its metrics establish the comparison reference.
• app.py
Engineering and Preprocessing Path
A separate engineered-data branch applies user-selected transformations: missing-value treatment, outlier handling, numerical scaling, categorical encoding, correlation analysis, and feature selection. This produces the processed feature matrix used for the second training run.
• app.py
Controlled Model Comparison
The architecture’s central invariant is use of the same selected learning algorithm for both raw and engineered datasets. The application trains baseline and engineered models, calculates evaluation metrics, and renders their side-by-side performance comparison so changes are attributable to preprocessing/feature engineering.
• app.py
Visualization and Artifact Export
Streamlit renders interactive tables, metrics, and visual analysis such as correlation heatmaps. Users can download the engineered dataset and serialized trained model; model persistence uses Joblib rather than an external model registry or storage service.
• app.py
• assets/demo.gif
• assets/results.png
Runtime and Dependencies
The runtime is Python, pinned operationally by .python-version and runtime.txt, with Streamlit serving the UI. Core processing uses pandas/NumPy; scikit-learn supplies preprocessing, selection, models, and evaluation; Matplotlib/Seaborn provide plots; Joblib serializes artifacts. Dependency installation is defined by requirements.txt. The included dev container supports reproducible development but is not an application service boundary.
• requirements.txt
• runtime.txt
• .python-version
• .devcontainer/devcontainer.json
Main entry points: app.py Feature engineering · requirements.txt Python dependencies · runtime.txt Python runtime · .python-version Local Python version
Components 16
Streamlit UI
- Streamlit app · Single-process interactive entry point.
app.py - CSV upload · Session-driven uploaded tabular data.
app.py - Target validation
app.py
Comparison Pipeline
Both branches use the same selected algorithm.
- Raw-data preparation
app.py - Feature engineering · Applies missing-value, outlier, scaling, encoding, correlation, and selection options.
app.py - Baseline model
app.py - Engineered model
app.py - Metrics comparison · Calculates and compares baseline and engineered performance.
app.py
Outputs
- Tables and plots · Includes correlation heatmaps and interactive analysis.
app.py - Engineered dataset download
app.py - Serialized model download
app.py
Runtime
- Python dependencies · Streamlit, pandas, NumPy, scikit-learn, Matplotlib, Seaborn, and Joblib.
requirements.txt - Python runtime
runtime.txt - Local Python version
.python-version - Development container · Reproducible development support, not an application service.
.devcontainer/devcontainer.json
Other components
- User
Connections 18
- User → Streamlit app: runs and interacts
- Streamlit app → CSV upload: accepts
- CSV upload → Target validation: previewed and profiled
- Target validation → Raw-data preparation: splits target and predictors
- Target validation → Feature engineering: configures transformations
- Raw-data preparation → Baseline model: raw features
- Feature engineering → Engineered model: processed matrix
- Baseline model → Metrics comparison: baseline metrics
- Engineered model → Metrics comparison: engineered metrics
- Baseline model → Engineered model: same selected algorithm
- Metrics comparison → Tables and plots: side-by-side results
- Feature engineering → Tables and plots: correlation analysis
- Feature engineering → Engineered dataset download: exports processed data
- Engineered model → Serialized model download: serializes with Joblib
- Python dependencies → Streamlit app: provides libraries
- Python runtime → Streamlit app: runs
- Local Python version → Python runtime: local pin
- Development container → Python dependencies: reproducible development