# fhicar/pulso_social_urbano architecture

> Architecture diagram of the GitHub repository fhicar/pulso_social_urbano, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fhicar/pulso_social_urbano
- Repository: https://github.com/fhicar/pulso_social_urbano

## Overview

##### Project type and purpose
- This is a full-stack, data-driven web application (Flask) that:
- Scrapes social-media data from Twitter, Reddit, and Mastodon via their APIs.
- Processes and analyzes that data with custom pipelines (Pandas, NumPy).
- Runs ML models for sentiment analysis and topic clustering.
- Visualizes results on an interactive geospatial map (Folium) and dashboard (Jinja2 templates).

##### File structure
- Top-level:
- app.py – Flask entry point and HTTP routing
- config.py – central config loader (.env integration)
- data_ingestion/ – modules to scrape and pre-process raw posts
- machine_learning/ – modules implementing sentiment analysis + clustering
- templates/ + static/ – frontend views (Jinja2, CSS)
- Dockerfile, docker-compose.yml – containerized deployment
- Patterns:
- MVC-style: Views (templates), Controllers (routes in app.py), Models (data_ingestion & ML).
- A single monolith rather than microservices, but with clear sub-modules.

##### README for additional insights
- Confirms: Flask backend, Jinja2 frontend, PRAW/Tweepy/Mastodon.py scrapers, Pandas/NumPy data pipelines, scikit-learn/TextBlob/VADER for ML, Folium/Geopy for mapping.

##### Elements
- HTTP flows (user ↔ Flask)
- Data flows (Flask → Data Ingestion → ML → Visualization → templates)
- API calls (Data Ingestion → Twitter/Reddit/Mastodon APIs)
- Blue for user-facing (templates, browser)
- Green for services (scrapers, ML modules)
- Orange for external dependencies (social APIs)
- Gray for infrastructure (Docker, .env)

##### Final recommendations
- Be exhaustive: represent each Python module or package as its own sub-component if clarity demands.
- Config (e.g., “reads TWITTER_API_KEY from .env”).
- Keep layout logical: left = user & frontend, middle = application/back end, right = external services & infra.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Frontend Layer"
        A["Client Browser"]:::frontend
        B["templates/base.html"]:::frontend
        C["templates/index.html"]:::frontend
        D["templates/resultados.html"]:::frontend
        E["static/css/style.css"]:::frontend
    end

    subgraph "Backend Layer"
        F["Flask App (app.py)"]:::service
        G["config.py"]:::service
    end

    subgraph "Data Ingestion Pipeline"
        H["scraper.py"]:::service
        I["data_processor.py"]:::service
    end

    subgraph "Machine Learning Module"
        J["sentiment_analysis.py"]:::service
        K["topic_clustering.py"]:::service
        L["stopwords_es"]:::service
    end

    M["Geospatial Visualization"]:::service

    subgraph "External Services"
        N["Twitter API"]:::external
        O["Reddit API"]:::external
        P["Mastodon API"]:::external
    end

    subgraph "Infrastructure"
        Q["Dockerfile"]:::infra
        R["docker-compose.yml"]:::infra
        S[".env"]:::infra
        T[".dockerignore"]:::infra
    end

    A -->|HTTP GET/POST| F
    F -->|renders| B
    F -->|renders| C
    F -->|renders| D
    F -->|serves| E

    F -->|invokes| H
    H -->|calls API| N
    H -->|calls API| O
    H -->|calls API| P
    H -->|outputs JSON payload| I

    I -->|Pandas DataFrame| J
    I -->|Pandas DataFrame| K
    L -->|used by| K

    J -->|sentiment labels| M
    K -->|clusters| M
    M -->|Folium Map| D

    F -->|reads config| G
    G -->|loads env vars| S

    R -->|orchestrates| F
    R -->|uses| S
    Q -->|containers| F

    click B "https://github.com/fhicar/pulso_social_urbano/blob/main/templates/base.html"
    click C "https://github.com/fhicar/pulso_social_urbano/blob/main/templates/index.html"
    click D "https://github.com/fhicar/pulso_social_urbano/blob/main/templates/resultados.html"
    click E "https://github.com/fhicar/pulso_social_urbano/blob/main/static/css/style.css"
    click F "https://github.com/fhicar/pulso_social_urbano/blob/main/app.py"
    click G "https://github.com/fhicar/pulso_social_urbano/blob/main/config.py"
    click H "https://github.com/fhicar/pulso_social_urbano/blob/main/data_ingestion/scraper.py"
    click I "https://github.com/fhicar/pulso_social_urbano/blob/main/data_ingestion/data_processor.py"
    click J "https://github.com/fhicar/pulso_social_urbano/blob/main/machine_learning/sentiment_analysis.py"
    click K "https://github.com/fhicar/pulso_social_urbano/blob/main/machine_learning/topic_clustering.py"
    click L "https://github.com/fhicar/pulso_social_urbano/tree/main/machine_learning/stopwords_es"
    click Q "https://github.com/fhicar/pulso_social_urbano/tree/main/Dockerfile"
    click R "https://github.com/fhicar/pulso_social_urbano/blob/main/docker-compose.yml"
    click S "https://github.com/fhicar/pulso_social_urbano/blob/main/.env"
    click T "https://github.com/fhicar/pulso_social_urbano/blob/main/.dockerignore"

    classDef frontend fill:#D0E6FF,stroke:#004085,color:#004085;
    classDef service fill:#D4EDDA,stroke:#155724,color:#155724;
    classDef external fill:#FFF3CD,stroke:#856404,color:#856404;
    classDef infra fill:#E2E3E5,stroke:#6C757D,color:#6C757D;
```
