# freskoko/cycletrafficweather architecture

> Architecture diagram of the GitHub repository freskoko/cycletrafficweather, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/freskoko/cycletrafficweather
- Repository: https://github.com/freskoko/cycletrafficweather

## Overview

##### Major System Components
- Data Ingestion and Pre-Processing:
– This component is responsible for reading the raw CSV files (found in raw_data directories under both “app” and “src”).
– The “src/utils” modules (such as dataframe_handling.py, file_parsing.py, graphing.py, and models.py) implement the pre-processing, feature engineering, and even parts of the model building (handling outliers, converting wind direction to x/y components, etc.).
- Model Training and Prediction Service:
– The main training and prediction logic resides in “src/project.py”. Here the pre-processed data goes through exploratory data analysis, outlier treatments, and is used to train the RandomForestRegressor (along with experiments on different ML models).
– The processing pipeline includes using KNNImputer for missing values and outputting prediction data in “predictions.csv” and an “out” folder.
- Web Application / Frontend:
– The “app” folder contains application scripts (app.py and appmodels.py) and website assets (templates/home.html). This is the interface that allows users to enter data, view live predictions, and trigger the model (using a pre-built model from a pickle file as noted in the README).
– It has its own “raw_data” folder setup instructions, allowing the same raw data to be used for prediction when a user interacts through a web browser.
- Storage and Outputs:
– File storage components include the raw data directories (under both “src/raw_data” and “app/raw_data”), “figs” for graphs, and “out” directories for processed outputs & prediction results.

##### Data Flow and Interactions Between Components
- Raw Data is ingested from CSV files (weather and traffic data) and passed into the Pre-Processing Pipeline.
- The pre-processing functions (in src/utils) clean the data – for example, replacing malformed values, combining files, engineering features (e.g., hour, day, weekend/holiday flags, summer/winter, wind direction transforms) – and then save cleaned outputs.
- The Model Training component (triggered by src/project.py) reads in the processed data, trains multiple models (with RandomForestRegressor coming out on top), tunes parameters (like “n_estimators”, or KNNImputer’s n_neighbors) and generates predictions. This part also exports trained model artifacts (using pickle) to be loaded later by the web application.
- The Web Application loads the saved model and imputer (via pickle) so that when a user submits new input via the website (hosted by app/app.py and rendered through app/templates/home.html), the input data is pre-processed (again using similar routines) and fed to the model for on-demand predictions.
- Final outputs (e.g., predictions.csv, forecast graphs, feature importance tables) are stored in the “out” and “figs” directories.

##### Architectural Patterns and Important Technologies
- The system follows a separation of concerns:
- Data processing and ML training (in the “src” directory) are separated from the presentation layer (in the “app” folder).
- The code is organized in modules (using subdirectories for utils and raw_data) and distinguishes between training and serving aspects.
- The design leverages popular Python libraries (pandas, scikit-learn for modeling, and likely Flask or a similar framework for the web server — even though the specific framework isn’t named) to implement machine learning and web serving.
- The model persistence (saving with pickle) is central to bridging between a computationally heavy training phase and a lightweight web prediction service.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Backend ML Pipeline
    subgraph "Backend ML Pipeline"
        A["Raw Data (src)"]:::storage
        B["Raw Data (app)"]:::storage
        A --> C
        B --> C
        C["Data Pre-Processing & Feature Engineering"]:::backend
            %% Pre-Processing Modules
            subgraph "Pre-Processing Modules"
                D["dataframe_handling.py"]:::module
                E["file_parsing.py"]:::module
                F["graphing.py"]:::module
                G["models.py"]:::module
            end
        C --> H
        H["Model Training & Prediction (src/project.py)"]:::backend
        H --> I
        I["Model Artifacts (Saved Model & predictions.csv)"]:::backend
        I --> J
        I --> K
        I --> L
        J["Processed Outputs (src/out)"]:::storage
        K["Processed Outputs (out)"]:::storage
        L["Graphs (figs)"]:::storage
    end

    %% Frontend Web Interface
    subgraph "Frontend Web Interface"
        subgraph "Web Server Scripts"
            M1["app/app.py"]:::frontend
            M2["app/appmodels.py"]:::frontend
        end
        I -- "loads_model"--> M1
        I -- "loads_model"--> M2
        M1 --> N
        M2 --> N
        N["Website Assets (app/templates/home.html)"]:::frontend
        N --> O
        O["User Browser"]:::frontend
        O -- "submits_input"--> M1
    end

    %% Click Events
    click A "https://github.com/freskoko/cycletrafficweather/tree/main/src/raw_data"
    click B "https://github.com/freskoko/cycletrafficweather/tree/main/app/raw_data"
    click D "https://github.com/freskoko/cycletrafficweather/blob/main/src/utils/dataframe_handling.py"
    click E "https://github.com/freskoko/cycletrafficweather/blob/main/src/utils/file_parsing.py"
    click F "https://github.com/freskoko/cycletrafficweather/blob/main/src/utils/graphing.py"
    click G "https://github.com/freskoko/cycletrafficweather/blob/main/src/utils/models.py"
    click H "https://github.com/freskoko/cycletrafficweather/blob/main/src/project.py"
    click I "https://github.com/freskoko/cycletrafficweather/blob/main/predictions.csv"
    click J "https://github.com/freskoko/cycletrafficweather/tree/main/src/out"
    click K "https://github.com/freskoko/cycletrafficweather/tree/main/out"
    click L "https://github.com/freskoko/cycletrafficweather/tree/main/figs"
    click M1 "https://github.com/freskoko/cycletrafficweather/blob/main/app/app.py"
    click M2 "https://github.com/freskoko/cycletrafficweather/blob/main/app/appmodels.py"
    click N "https://github.com/freskoko/cycletrafficweather/blob/main/app/templates/home.html"

    %% Styles
    classDef backend fill:#ADD8E6,stroke:#000,stroke-width:1px;
    classDef frontend fill:#90EE90,stroke:#000,stroke-width:1px;
    classDef storage fill:#F0E68C,stroke:#000,stroke-width:1px;
    classDef module fill:#FFD700,stroke:#000,stroke-width:1px;
```
