# enriqmancomp/la_crime_project architecture

> Architecture diagram of the GitHub repository enriqmancomp/la_crime_project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/enriqmancomp/la_crime_project
- Repository: https://github.com/enriqmancomp/la_crime_project

## Overview

##### Main Components
- Data Source: Although not explicitly included in the file tree, the raw data would be coming from an external source (CSV files, databases, or other feeds) before being landed into your system.
- Data Ingestion: Represented by the “import_data.sql” script which is responsible for importing the raw data into the system’s database.
- Data Cleaning & Transformation:
- “data_cleaning.sql” & “fix_data_type.sql” scripts – these are used to clean and correct the data.
- “create_partitions.sql” – likely used to partition tables for performance or organization.
- “create_views.sql” – used to define views for easier querying of transformed data.
- Exploratory Data Analysis (EDA): The “eda.sql” script executes the analytical queries which result in summarized statistics and reports (as detailed in the README).
- Data Export & Visualization: The “export_data.sql” script exports processed data, which is then visualized in Tableau (the “Book.twb” Tableau workbook).

##### Relationships and Interactions
- Raw Data flows into the Data Ingestion phase.
- Ingestion creates raw tables in the database.
- The Data Cleaning & Transformation scripts operate on these tables, correcting data types, cleaning records, and creating optimized partitions and views.
- The EDA script then queries these curated data stores for generating statistical summaries.
- Finally, the Export script sends the cleaned, analyzed data into formats consumable by Tableau, which acts as the reporting/visualization layer.

##### Architectural Patterns & Design Principles
- ETL (Extract, Transform, Load) Process: The structure clearly separates the process of ingestion, cleaning/transforming, and then analysis/export.
- Modularity: Each SQL script addresses a specific part of the data pipeline, which allows for clearer separation of concerns.
- Reporting Layer: Using Tableau indicates an emphasis on interactive data visualization and dashboard reporting.

##### Relevant Technologies
- SQL (for data manipulation) and an underlying relational database system (unspecified, but could be PostgreSQL, MySQL, etc.).
- Tableau for visualization and reporting.

##### Instructing the Principal Software Engineer
- Use clear labels to denote each major module: Data Source, Data Ingestion Module (import_data.sql), Data Cleaning & Transformation (data_cleaning.sql, fix_data_type.sql, create_partitions.sql, create_views.sql), Analysis (eda.sql), Data Export (export_data.sql), and Visualization (Tableau Workbook).
- Incorporate any relevant configuration or deployment artifacts if applicable, though in this project the emphasis remains on the SQL-driven ETL and reporting layers.

##### Summing Up
- Pay special attention to how the SQL scripts work together to prepare the data and how Tableau consumes the results for insightful analytics.
- Though each process is discrete, ensure that all dependencies and interactions between components (such as how cleaned data feeds into both partitioning and view creation, and then into analytical queries) are clearly marked.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Main Data Pipeline
    RawData["Raw Data (External)"]:::rawData
    Ingestion["Data Ingestion (import_data.sql)"]:::process
    Cleaning["Data Cleaning & Fixing"]:::process
    Cleaning1["Data Cleaning (data_cleaning.sql)"]:::process
    Cleaning2["Data Fixing (fix_data_type.sql)"]:::process
    Transformation["Data Transformation"]:::process
    Transformation1["Partitioning (create_partitions.sql)"]:::process
    Transformation2["View Creation (create_views.sql)"]:::process
    EDA["Exploratory Data Analysis (eda.sql)"]:::eda
    Export["Data Export (export_data.sql)"]:::export
    Visualization["Visualization (Tableau Workbook)"]:::viz

    %% Documentation Subgraph
    subgraph "Documentation & Diagrams"
        Doc1["Documentation (Description.md)"]:::doc
        Doc2["Architecture Diagram (LAPD Data Analysis.drawio)"]:::doc
    end

    %% Data Pipeline Flow
    RawData --> Ingestion
    Ingestion --> Cleaning
    Cleaning --> Cleaning1
    Cleaning --> Cleaning2
    Cleaning1 --> Transformation
    Cleaning2 --> Transformation
    Transformation --> Transformation1
    Transformation --> Transformation2
    Transformation1 --> EDA
    Transformation2 --> EDA
    EDA --> Export
    Export --> Visualization
    Visualization -.-> Doc1
    Doc1 --> Doc2

    %% Click Events
    click Ingestion "https://github.com/enriqmancomp/la_crime_project/blob/master/src/import_data.sql"
    click Cleaning1 "https://github.com/enriqmancomp/la_crime_project/blob/master/src/data_cleaning.sql"
    click Cleaning2 "https://github.com/enriqmancomp/la_crime_project/blob/master/src/fix_data_type.sql"
    click Transformation1 "https://github.com/enriqmancomp/la_crime_project/blob/master/src/create_partitions.sql"
    click Transformation2 "https://github.com/enriqmancomp/la_crime_project/blob/master/src/create_views.sql"
    click EDA "https://github.com/enriqmancomp/la_crime_project/blob/master/src/eda.sql"
    click Export "https://github.com/enriqmancomp/la_crime_project/blob/master/src/export_data.sql"
    click Visualization "https://github.com/enriqmancomp/la_crime_project/blob/master/tableau viz/Book.twb"
    click Doc1 "https://github.com/enriqmancomp/la_crime_project/blob/master/Description.md"
    click Doc2 "https://github.com/enriqmancomp/la_crime_project/blob/master/LAPD Data Analysis.drawio"

    %% Styles
    classDef rawData fill:#F9E79F,stroke:#F39C12,stroke-width:2px;
    classDef process fill:#AED6F1,stroke:#3498DB,stroke-width:2px;
    classDef eda fill:#F5B7B1,stroke:#E74C3C,stroke-width:2px;
    classDef export fill:#D2B4DE,stroke:#9B59B6,stroke-width:2px;
    classDef viz fill:#E8DAEF,stroke:#8E44AD,stroke-width:2px;
    classDef doc fill:#FAD7A0,stroke:#D35400,stroke-width:2px;
```
