# roydontay/sg-property-price-prediction architecture

> Architecture diagram of the GitHub repository roydontay/sg-property-price-prediction, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/roydontay/sg-property-price-prediction
- Repository: https://github.com/roydontay/sg-property-price-prediction

## Overview

##### Main Components
- Data Acquisition Component:
- Represented by the API_data_extraction.py script. This script is responsible for polling the URA REST API and gathering property transaction data.
- Data Storage & Management:
- The Data directory, which stores the raw data (including archive “Raw Data.zip”) and collated datasets (e.g., Data/Collated_Data_170622_modified). This component holds the data files that move through the pipeline.
- Data Processing, Feature Extraction & EDA:
- Represented by Exploratory_Data_Analysis.ipynb. This component encompasses data cleaning, feature extraction (final_features.pkl), and visual exploratory analysis.
- Model Training & Experimentation:
- Represented by Model_selection_experimentation.ipynb. This notebook shows the experiments with different modeling techniques and selects the best-performing model (Random Forest in this case).
- Model Artifacts:
- The best_model.pkl file holds the serialized version of the final model, and the final_features.pkl stores the features used. Think of these as the endpoints from the training pipeline that will be used in production.
- Model Serving / Frontend Application:
- Represented by app.py, which is a Streamlit application. This application uses the trained model to present interactive predictions to the user.

- That data moves from storage into the Data Processing/EDA phase for cleaning, transformation, and feature extraction.
- Finally, illustrate that these stored model artifacts are loaded by the Streamlit app (app.py) which serves predictions to end users.

##### Key Architectural Patterns & Technologies
- Since this is an end-to-end pipeline, emphasize the separation of concerns:
- Data Ingestion, Processing, Modeling, and Serving are distinct components.
- Use of scripting and Jupyter notebooks indicates an iterative and exploratory development style common in data science projects.
- The integration of a Streamlit web app means that the model serving component is built using lightweight, interactive Python dashboards.
- The serialized model and feature artifacts (pkl files) act as interfaces between the training and serving stages.

##### Important Details to Capture
- That initial stages (API extraction and data cleaning) are critical for preparing a robust dataset for modeling.
- The model training component is modular enough to potentially allow swapping or updating the machine learning algorithms.
- The web-based app is the final layer that interacts with the user, making predictions based on input features.
- Optionally, include a note about potential future components (like integrating data from OneMap API) as mentioned in the README for extensibility.

## Diagram (Mermaid)

```mermaid
flowchart TD
    A["API Data Extraction (API_data_extraction.py)"]:::script
    DS["Data Storage (Data/)"]:::storage
    EDA["Data Processing & EDA (Exploratory_Data_Analysis.ipynb)"]:::notebook
    MT["Model Training & Experimentation (Model_selection_experimentation.ipynb)"]:::notebook
    BM["Best Model (best_model.pkl)"]:::artifact
    FF["Final Features (final_features.pkl)"]:::artifact
    SA["Streamlit App (app.py)"]:::frontend

    A -->|"polls_data"| DS
    DS -->|"raw_data_to"| EDA
    EDA -->|"processed_data"| MT
    MT -->|"outputs"| BM
    BM -->|"and"| FF
    FF -->|"serves_predictions_to"| SA

    classDef script fill:#A3E4D7,stroke:#1ABC9C,stroke-width:2px,color:#000;
    classDef storage fill:#F9E79F,stroke:#F39C12,stroke-width:2px,color:#000;
    classDef notebook fill:#D2B4DE,stroke:#8E44AD,stroke-width:2px,color:#000;
    classDef artifact fill:#F5B7B1,stroke:#C0392B,stroke-width:2px,color:#000;
    classDef frontend fill:#AED6F1,stroke:#2471A3,stroke-width:2px,color:#000;

    click A "https://github.com/roydontay/sg-property-price-prediction/blob/main/API_data_extraction.py"
    click DS "https://github.com/roydontay/sg-property-price-prediction/tree/main/Data/"
    click EDA "https://github.com/roydontay/sg-property-price-prediction/blob/main/Exploratory_Data_Analysis.ipynb"
    click MT "https://github.com/roydontay/sg-property-price-prediction/blob/main/Model_selection_experimentation.ipynb"
    click BM "https://github.com/roydontay/sg-property-price-prediction/blob/main/best_model.pkl"
    click FF "https://github.com/roydontay/sg-property-price-prediction/blob/main/final_features.pkl"
    click SA "https://github.com/roydontay/sg-property-price-prediction/blob/main/app.py"
```
