How harishnns/house-price-predictor works
Project Type and Purpose
• This is an end-to-end MLOps pipeline and full-stack application.
• It ingests raw housing data, processes it, engineers features, trains models, tracks experiments via MLflow, exposes a FastAPI inference service, and offers a Streamlit frontend.
File Structure
• Top-level directories
• data/raw & data/processed – storage for raw and cleaned data
• src/data, src/features, src/models – batch pipeline code
• src/api – FastAPI microservice for inference
• streamlit_app – React-like UI for user predictions
• models/trained – serialized model artifacts and preprocessors
• deployment/mlflow & deployment/kubernetes – container/cluster orchestration
• Notebooks – exploratory and prototyping
• Requirements and Docker-Compose snippets scattered for reproducible environments
README Insights
• Three main workflow stages: data processing → feature engineering → model training & MLflow logging
• Two deployment targets: FastAPI service for programmatic prediction and Streamlit app for interactive UI
• MLflow server spun up via Docker-Compose; Kubernetes manifests also included for more advanced deployments
Core Components
a. Data Layer
• Raw Data Store (data/raw/house_data.csv)
• Processed Data Store (data/processed/*.csv)
b. Batch Pipeline
• Data Processing (src/data/run_processing.py)
• Feature Engineering (src/features/engineer.py)
• Model Training & Experimentation (src/models/train_model.py + MLflow Client)
c. Model Registry & Artifact Storage
• MLflow Tracking Server (deployment/mlflow/docker-compose.yaml)
• models/trained/* (.pkl artifacts + config files)
d. Serving Layer
• FastAPI Inference Service (src/api/main.py, inference.py, schemas.py)
• Streamlit User Interface (streamlit_app/app.py)
e. Orchestration & Deployment
• Docker-Compose on local/dev for MLflow, API, and Streamlit
• Kubernetes manifests for production (deployment/kubernetes/)
f. Optional Interactive Layer
• Jupyter Notebooks (notebooks/) for ad-hoc exploration
Architectural Patterns & Design Principles
• Modular pipeline stages (separation of concerns)
• Microservices for serving (FastAPI) and UI (Streamlit)
• Containerization for reproducibility (Docker-Compose, Kubernetes)
• Experiment tracking via external service (MLflow)
• Artifact versioning and configurability (YAML configs, environment variables)
Technologies & Frameworks to Call Out
• Language: Python 3.11
• Web: FastAPI, Uvicorn, Streamlit
• Data Processing: Pandas, custom scripts
• Feature Engineering: sklearn pipelines (preprocessor.pkl)
• Model Training & Tracking: MLflow, scikit-learn (or similar)
• Containers: Docker, Docker-Compose
• Orchestration: Kubernetes (manifests)
• Optional: JupyterLab for notebooks