How ezracalis/singapore-hdb-price-prediction works
Project type and purpose
• This is a data‐science/ML pipeline project whose goal is to predict HDB resale prices using historical tabular data and linear/regression models.
• There is no frontend or web service; instead it consists of a sequence of Jupyter notebooks, raw and processed data files, model artifacts, and a slide deck.
File structure
• Top-level directories:
• code/ – Jupyter notebooks for each stage of the pipeline (Cleaning & EDA, Feature Engineering, Modeling, Feature Selection, Production Model, Kaggle Submission)
• data/ – raw and intermediate CSVs plus model artifact (05_final_lr_model.pkl) and final results
• slidedeck/ – PDF presentation of findings
• The presence of “.ipynb_checkpoints” shows iterative notebook development.
• No REST API or microservice containers; this is a batch pipeline.
README for additional insights
• Problem statement: predict resale value via linear, ridge, lasso regression
• Methodology steps correspond directly to the notebooks in code/
• Final outputs: model pickle file and Kaggle submission CSV
Main components
a. Data Sources & Storage
• Raw data files (train.csv, test.csv)
• Intermediate CSVs (cleaned, feature‐engineered)
• Model artifact store (05_final_lr_model.pkl)
1. Data Cleaning & EDA
2. Feature Engineering
3. Initial Modeling & Evaluation
4. Feature Selection & Model Tuning
5. Production Model Training (Linear, Ridge, Lasso)
6. Model Evaluation & Selection
7. Kaggle Submission Generation
c. External Services
• Kaggle API (for pushing submission.csv)
d. Presentation Layer
• slidedeck/ – summarises results
Architectural patterns & design principles
• Linear, sequential ETL/ML pipeline
• Separation of concerns: each notebook handles one stage
• Reproducibility: explicit intermediate CSVs and versioned notebook checkpoints
• Artifact management: model pickle and CSV results
Technologies & frameworks
• Python, Jupyter Notebook
• pandas, scikit-learn
• pickle for model serialization
• Kaggle API for submission
Elements
• Blue for data files
• Green for processing notebooks
• Orange for model artifacts
• Purple for external services
Detail but avoid over-engineering
• Six notebook stages as distinct components
• Intermediate CSV outputs as separate nodes
• Model outputs (pickle and submission.csv)
• Connection to the presentation (slidedeck) as a “reporting” artifact