# smohith2105/crime-analysis-buffalo architecture

> Architecture diagram of the GitHub repository smohith2105/crime-analysis-buffalo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/smohith2105/crime-analysis-buffalo
- Repository: https://github.com/smohith2105/crime-analysis-buffalo

## Overview

##### Project Type & Purpose
- This is a full-stack data analysis and forecasting system centered on crime analysis in Buffalo. It integrates crime risk prediction, forecasting using Facebook Prophet, threat score assessment with XGBoost, and neighborhood trend analysis using SVM.
- The project leverages various data sources (CSV, GeoJSON, pickle files for models and scalers), processes this data, stores it (including in an SQLite database), and then visualizes the outcomes through an interactive Streamlit dashboard.

##### Main Components
a. Data Sources & Storage:
- Data Files: Represent the raw and preprocessed data files located in the "data/" directory. This includes subdirectories for forecast, risk, threat, and trends. Include key file types such as CSVs, GeoJSON, and pickle files.
- Database: Although not directly seen in the file tree, the README mentions a "db" folder with an SQLite database (incidents.db). Show this as a central repository for processed crime data.

b. Data Ingestion & Processing:
- This layer processes raw data inputs (from CSVs, etc.) and prepares them for analysis, including cleaning and transformation.

c. Experimentation & Model Development:
- Notebooks: The "exp/" and "src/" directories host Jupyter notebooks that support experimentation with different models:
- Forecasting notebook using Facebook Prophet (exp/forecast.ipynb)
- Risk prediction notebook using Random Forest (exp/risk.ipynb)
- Threat assessment notebook using XGBoost (exp/threat.ipynb)
- Trend analysis notebook using SVM (exp/trends.ipynb)
- That these notebooks represent the research and model training phase that produces the trained predictive models.

d. Model Artifacts:
- Model Files: Pickle files (stored in data/risk, data/threat, and data/trends) imply storage of pre-trained models (Random Forest, XGBoost, and SVM) and scalers. Represent these as “Model Artifacts” which are loaded for prediction.

e. Application / Presentation Layer (Streamlit Web Interface):
- Main App: The "app.py" acts as the Streamlit dashboard which integrates all functionalities. Show how it ties together the predictive models, visualization (using Plotly, Pydeck, etc.), and user inputs.
- Visualization Components: Highlight the specific visual elements:
- Heatmap for crime risk predictions,
- Line charts for crime forecasting,
- Gauge charts for threat score,
- 3D scatter and map visualizations for neighborhood crime trends.
- User Interaction: Indicate that the user can interact with the parameters (e.g., selecting neighborhoods, days, hours) which feed into the models.

##### Relationships and Interactions
- Data flows from the Data Sources to the Data Preparation layer where scripts process and load data into the database.
- Experimentation notebooks read the prepared data, generate predictive models, and export model artifacts.
- The Streamlit app (app.py) acts as the orchestrator by:
- Fetching data from the SQLite database,
- Loading the model artifacts (using Joblib or pickle),
- Accepting user input,
- Running prediction functions (risk prediction, forecast, threat assessment, crime trends analysis),
- Displaying the results via visual components.

##### Architectural Patterns & Design Principles
- Separation of Concerns: Different modules are separated clearly—data ingestion, processing, model training, and UI.
- Modular Design: The project is structured by function (data, experiments, scripts, application) and includes distinct model components.
- Real-Time Interaction: The Streamlit app demonstrates a real-time prediction and visualization pipeline that brings together various machine learning models.
- Reusability: Model artifacts (pickle files) are created and reused in multiple parts of the system.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Layer"
        DS["Data Sources (CSV, GeoJSON, pickle)"]:::data
        DB["SQLite Database (incidents.db)"]:::database
    end

    %% Data Processing
    subgraph "Data Processing"
        CreateDB["create_database.py"]:::scripts
        DataPrep["data_preperation.py"]:::scripts
    end

    %% Experimentation & Model Training
    subgraph "Experimentation & Model Training"
        EXP["Experiment Notebooks (exp/)"]:::notebooks
        SRC["Additional Notebooks (src/)"]:::notebooks
    end

    %% Model Artifacts
    subgraph "Model Artifacts"
        RF["Random Forest Models"]:::artifacts
        XGB["XGBoost & Encoder"]:::artifacts
        SVM["SVM Models & Scalers"]:::artifacts
    end

    %% Presentation Layer
    subgraph "Presentation Layer"
        APP["Streamlit Dashboard (app.py)"]:::app
        UI["User Interaction"]:::ui
    end

    %% Relationships / Data Flow
    DS -->|"raw_data"| CreateDB
    CreateDB -->|"DB_setup"| DataPrep
    DataPrep -->|"processed_data"| DB
    
    DB -->|"feeds_data"| EXP
    DB -->|"feeds_data"| SRC
    
    EXP -->|"trains_risk_model"| RF
    EXP -->|"trains_threat_model"| XGB
    EXP -->|"trains_trend_model"| SVM

    RF -->|"prediction_input"| APP
    XGB -->|"prediction_input"| APP
    SVM -->|"prediction_input"| APP

    DB -->|"supplies_display_data"| APP
    UI -->|"triggers_prediction"| APP

    %% Click Events
    click DS "https://github.com/smohith2105/crime-analysis-buffalo/tree/main/data/"
    click CreateDB "https://github.com/smohith2105/crime-analysis-buffalo/blob/main/scripts/create_database.py"
    click DataPrep "https://github.com/smohith2105/crime-analysis-buffalo/blob/main/scripts/data_preperation.py"
    click EXP "https://github.com/smohith2105/crime-analysis-buffalo/tree/main/exp/"
    click SRC "https://github.com/smohith2105/crime-analysis-buffalo/tree/main/src/"
    click RF "https://github.com/smohith2105/crime-analysis-buffalo/tree/main/data/risk"
    click XGB "https://github.com/smohith2105/crime-analysis-buffalo/tree/main/data/threat"
    click SVM "https://github.com/smohith2105/crime-analysis-buffalo/tree/main/data/trends"
    click APP "https://github.com/smohith2105/crime-analysis-buffalo/blob/main/app.py"

    %% Styles
    classDef data fill:#e0f7fa,stroke:#006064,stroke-width:2px;
    classDef database fill:#ffe082,stroke:#ff6f00,stroke-width:2px;
    classDef scripts fill:#dcedc8,stroke:#33691e,stroke-width:2px;
    classDef notebooks fill:#f8bbd0,stroke:#880e4f,stroke-width:2px;
    classDef artifacts fill:#d1c4e9,stroke:#4a148c,stroke-width:2px;
    classDef app fill:#fff9c4,stroke:#f57f17,stroke-width:2px;
    classDef ui fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px;
```
