# nolanrobbins/housecanary-ml-project architecture

> Architecture diagram of the GitHub repository nolanrobbins/housecanary-ml-project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nolanrobbins/housecanary-ml-project
- Repository: https://github.com/nolanrobbins/housecanary-ml-project

## Overview

##### Main Components
- Data Ingestion & Storage:
- Raw housing data (e.g., "Housing Data.xlsx" and the housing data files inside the data/ directory)
- Additional data files like the housing data dictionary and processed Excel sheets (e.g., "X_train_scaled.xlsx", "X_train_selected.xlsx", "df_cleaned.xlsx")
- Data Preprocessing:
- Jupyter Notebook “Cleaning the Data.ipynb” for data cleaning and preprocessing
- Model Building & Serialization:
- “Model Building Notebook” that processes the cleaned data, conducts ML training, and then serializes outputs
- Persisted artifacts in the data/ folder such as "HouseCanary_model.pkl" (the trained ML model) and "robust_scaler.pkl" (for feature scaling)
- User Interface (Frontend):
- “st_house.py” indicating a Streamlit application for user interaction, where end users input home characteristics
- External Dependencies & Environment:
- “requirements.txt” listing all necessary libraries and dependencies to recreate the environment

##### Component Relationships and Interactions
- Data flows from raw data sources to preprocessing notebooks:
- Raw data (Excel) is ingested to be cleaned. The cleaning project creates processed datasets (df_cleaned.xlsx) which are then fed into the model-building process.
- The Model Training process:
- The “Model Building Notebook” accesses the cleaned data to train an ML model. It outputs model artifacts (e.g., a serialized .pkl file) that the application will load at runtime.
- User Interaction & Inference:
- The Streamlit app ("st_house.py") acts as the frontend where user inputs are gathered.
- When a user submits home characteristics, the app loads the pre-trained model (and necessary scalers) to compute and return a price estimate.
- Integration with a Sales Funnel:

##### Architectural Patterns and Design Principles
- Separation of Concerns:
- Clearly segregate data ingestion/preprocessing, model training, and serving/user interaction components.
- Serialization for Portability:
- Use of pickle files to store ML model and scaler indicates modularity, allowing the inference component to remain loosely coupled with the training process.
- Deployment Readiness:
- The inclusion of “requirements.txt” emphasizes environmental reproducibility and deployment considerations.

##### Final Checklist
- Component (data sources, notebooks, ML model artifact, web interface) has a clear label.
Raw Data → Data Cleaning → Model Building → ML Artifacts → Streamlit (User Inference).
- A note for external API/integration points (the sales funnel filtering system as described in the README).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Ingestion & Storage"
        A["Raw Housing Data"]
        B["Housing Data Dictionary"]
        C["X_train_scaled.xlsx"]
        D["X_train_selected.xlsx"]
        E["df_cleaned.xlsx"]
    end

    subgraph "Data Preprocessing"
        F["Cleaning the Data.ipynb"]
    end

    subgraph "Model Building & Serialization"
        G["Model Building Notebook"]
        H["HouseCanary_model.pkl"]
        I["robust_scaler.pkl"]
    end

    subgraph "User Interface"
        J["st_house.py"]
    end

    subgraph "External Dependencies & Integration"
        K["requirements.txt"]
        L["Sales Funnel Filtering System"]
    end

    %% Data flow connections
    A -->|"data_cleaning"| F
    B -->|"reference"| F
    F -->|"outputs_X_train_scaled"| C
    F -->|"outputs_X_train_selected"| D
    F -->|"outputs_df_cleaned"| E
    E -->|"model_training"| G
    G -->|"serialize_model"| H
    G -->|"serialize_scaler"| I
    F -->|"provides_data"| G
    J -->|"load_and_infer"| H
    J -->|"load_and_infer"| I
    J -->|"integrates_with"| L
    K -->|"env_setup"| F
    K -->|"env_setup"| G
    K -->|"env_setup"| J

    %% Class definitions for color-coding
    classDef dataLayer fill:#f9e79f,stroke:#333,stroke-width:2px;
    classDef mlLayer fill:#aed6f1,stroke:#333,stroke-width:2px;
    classDef uiLayer fill:#a9dfbf,stroke:#333,stroke-width:2px;
    classDef extLayer fill:#f5b7b1,stroke:#333,stroke-width:2px;

    class A,B,C,D,E,F dataLayer;
    class G,H,I mlLayer;
    class J uiLayer;
    class K,L extLayer;

    %% Click Events
    click A "https://github.com/nolanrobbins/housecanary-ml-project/blob/main/data/Housing Data.xlsx"
    click B "https://github.com/nolanrobbins/housecanary-ml-project/blob/main/data/Housing Data Dictionary.xlsx"
    click C "https://github.com/nolanrobbins/housecanary-ml-project/blob/main/data/X_train_scaled.xlsx"
    click D "https://github.com/nolanrobbins/housecanary-ml-project/blob/main/data/X_train_selected.xlsx"
    click E "https://github.com/nolanrobbins/housecanary-ml-project/blob/main/data/df_cleaned.xlsx"
    click F "https://github.com/nolanrobbins/housecanary-ml-project/blob/main/Cleaning the Data.ipynb"
    click G "https://github.com/nolanrobbins/housecanary-ml-project/tree/main/Model Building Notebook"
    click H "https://github.com/nolanrobbins/housecanary-ml-project/blob/main/data/HouseCanary_model.pkl"
    click I "https://github.com/nolanrobbins/housecanary-ml-project/blob/main/data/robust_scaler.pkl"
    click J "https://github.com/nolanrobbins/housecanary-ml-project/blob/main/st_house.py"
    click K "https://github.com/nolanrobbins/housecanary-ml-project/blob/main/requirements.txt"
```
