# carobs9/segregation-madrid architecture

> Architecture diagram of the GitHub repository carobs9/segregation-madrid, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/carobs9/segregation-madrid
- Repository: https://github.com/carobs9/segregation-madrid

## Overview

##### Main Components/Modules
- Mobility Data Extraction:
- Folder: mobility_data
- Purpose: Extracts raw trip data (get_viajes.py) and retrieves/cleans geographical information (get_geometrias.py).
- Data Preprocessing & Cleaning for Segregation Analysis:
- Folder: segregation_indices
- Components: preprocessing.py, assortativity_matrix.py, morans_i.py, weight_matrix_analysis.py, and associated utilities.
- Purpose: Cleans income and geographic data, computes segregation measures, and prepares data for further analysis.
- Trip and Mobility Analysis:
- Folder: trip_analysis
- Components: trip_count.py, trip_distances.py, alongside configuration and utility scripts.
- Purpose: Calculates trip counts and distances based on socio-economic stratifications.
- Network Analysis and Graph Modeling:
- Folder: network
- Components: get_graph.py (building the network), regressions.py, visualization.py, and modules for handling adjacency matrices and graphs.
- Purpose: Converts processed mobility data into a graph structure (network) and performs regression analyses to study weight metrics between districts.
- Descriptive and Exploratory Analysis:
- Folder: data_overview
- Components: main_descriptives.py, viajes_stats.py, figures generation, etc.
- Purpose: Provides general summaries and exploratory visualizations of both mobility and income data.
- Logging and Configuration:
- Folders/Files: logs directory and multiple config.py files within modules.
- Purpose: Captures logs for various stages of analysis; centralizes configuration for reproducibility.
- Dependencies/Environment:
- Files: requirements.txt
- Purpose: Ensures that all needed libraries (Python 3.11 and third-party packages) are installed for each module.

##### Relationships and Data Flows
- Raw data is fetched and cleaned in the mobility_data module.
- Outputs from mobility_data feed directly into segregation_indices and trip_analysis.
- The segregation_indices and trip_analysis modules preprocess and analyze the data to generate metrics (e.g., assortativity, Moran’s I, trip counts, trip distances).
- Processed data from these modules becomes the basis for constructing the mobility network in the network module.
- The network module then analyzes relationships (e.g., in-weight, out-weight) and creates visualizations via regression and graph models.
- Parallel to specialized analysis, the data_overview module offers overall descriptive statistics and visual summaries.

##### Architectural Patterns and Principles
- Modular Pipeline: Each folder represents a stage in the analysis workflow. Each component (data extraction, cleaning, network analysis) is self-contained with its own configuration, making the system extendable.
- Separation of Concerns: Clear division between raw data ingestion, preprocessing, analytical computations, and visualization.
- Config-driven Execution: With multiple config.py files, the system expects central configuration parameters (e.g., data paths, study type selections) to be used consistently across modules.
- Logging and Output Generation: Dedicated logs and figures directories ensure reproducibility and traceability of calculation steps.

## Diagram (Mermaid)

```mermaid
flowchart TD

    subgraph "Mobility Data Extraction"
        MD1["get_viajes.py"]:::ingestion
        MD2["get_geometrias.py"]:::ingestion
    end

    subgraph "Data Preprocessing & Segregation Analysis"
        SE1["preprocessing.py"]:::preprocessing
        SE2["assortativity_matrix.py"]:::preprocessing
        SE3["morans_i.py"]:::preprocessing
        SE4["weight_matrix_analysis.py"]:::preprocessing
    end

    subgraph "Trip and Mobility Analysis"
        TA1["trip_count.py"]:::trip
        TA2["trip_distances.py"]:::trip
    end

    subgraph "Network Analysis and Graph Modeling"
        NA1["get_graph.py"]:::network
        NA2["regressions.py"]:::network
        NA3["visualization.py"]:::network
    end

    subgraph "Descriptive and Exploratory Analysis"
        DO1["main_descriptives.py"]:::overview
        DO2["viajes_stats.py"]:::overview
    end

    subgraph "Logging & Configuration"
        LC1["logs"]:::config
        LC2["mobility_data/config.py"]:::config
        LC3["network/config.py"]:::config
        LC4["segregation_indices/config.py"]:::config
        LC5["trip_analysis/config.py"]:::config
    end

    DEP1["requirements.txt"]:::dependency

    %% Data Flow Connections
    MD1 -->|"feeds"| SE1
    MD1 -->|"feeds"| TA1
    MD1 -->|"feeds"| DO1
    SE1 -->|"processed_data"| NA1
    TA1 -->|"processed_data"| NA1

    %% Logging/Configuration links (dotted lines)
    MD1 -.-> LC1
    SE1 -.-> LC1
    TA1 -.-> LC1
    NA1 -.-> LC1
    DO1 -.-> LC1

    %% Dependencies links (dotted lines)
    MD1 -.-> DEP1
    SE1 -.-> DEP1
    TA1 -.-> DEP1
    NA1 -.-> DEP1
    DO1 -.-> DEP1

    %% Click Events for Mobility Data Extraction
    click MD1 "https://github.com/carobs9/segregation-madrid/blob/master/mobility_data/get_viajes.py"
    click MD2 "https://github.com/carobs9/segregation-madrid/blob/master/mobility_data/get_geometrias.py"

    %% Click Events for Data Preprocessing & Segregation Analysis
    click SE1 "https://github.com/carobs9/segregation-madrid/blob/master/segregation_indices/preprocessing.py"
    click SE2 "https://github.com/carobs9/segregation-madrid/blob/master/segregation_indices/assortativity_matrix.py"
    click SE3 "https://github.com/carobs9/segregation-madrid/blob/master/segregation_indices/morans_i.py"
    click SE4 "https://github.com/carobs9/segregation-madrid/blob/master/segregation_indices/weight_matrix_analysis.py"

    %% Click Events for Trip and Mobility Analysis
    click TA1 "https://github.com/carobs9/segregation-madrid/blob/master/trip_analysis/trip_count.py"
    click TA2 "https://github.com/carobs9/segregation-madrid/blob/master/trip_analysis/trip_distances.py"

    %% Click Events for Network Analysis and Graph Modeling
    click NA1 "https://github.com/carobs9/segregation-madrid/blob/master/network/get_graph.py"
    click NA2 "https://github.com/carobs9/segregation-madrid/blob/master/network/regressions.py"
    click NA3 "https://github.com/carobs9/segregation-madrid/blob/master/network/visualization.py"

    %% Click Events for Descriptive and Exploratory Analysis
    click DO1 "https://github.com/carobs9/segregation-madrid/blob/master/data_overview/main_descriptives.py"
    click DO2 "https://github.com/carobs9/segregation-madrid/blob/master/data_overview/viajes_stats.py"

    %% Click Events for Logging & Configuration
    click LC1 "https://github.com/carobs9/segregation-madrid/tree/master/logs"
    click LC2 "https://github.com/carobs9/segregation-madrid/blob/master/mobility_data/config.py"
    click LC3 "https://github.com/carobs9/segregation-madrid/blob/master/network/config.py"
    click LC4 "https://github.com/carobs9/segregation-madrid/blob/master/segregation_indices/config.py"
    click LC5 "https://github.com/carobs9/segregation-madrid/blob/master/trip_analysis/config.py"

    %% Click Event for Dependencies
    click DEP1 "https://github.com/carobs9/segregation-madrid/blob/master/requirements.txt"

    %% Styling
    classDef ingestion fill:#a8dadc,stroke:#457b9d,stroke-width:2px;
    classDef preprocessing fill:#f7d794,stroke:#f4a261,stroke-width:2px;
    classDef trip fill:#ffb4a2,stroke:#e76f51,stroke-width:2px;
    classDef network fill:#bde0fe,stroke:#0077b6,stroke-width:2px;
    classDef overview fill:#c1e1c1,stroke:#2d6a4f,stroke-width:2px;
    classDef config fill:#ffd166,stroke:#fda085,stroke-width:2px;
    classDef dependency fill:#d3d3d3,stroke:#696969,stroke-width:2px;
```
