# portfolio-alvaro/proyecto-pulso-social-urbano architecture

> Architecture diagram of the GitHub repository portfolio-alvaro/proyecto-pulso-social-urbano, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/portfolio-alvaro/proyecto-pulso-social-urbano
- Repository: https://github.com/portfolio-alvaro/proyecto-pulso-social-urbano

## Overview

##### Project type and purpose
- This is a data-processing + visualization prototype (full-stack Python service) that scrapes social media posts, performs sentiment and topic analysis, geolocates events, calculates KPIs over time windows, and exposes an interactive map & timeline dashboard via a Flask API. It’s orchestrated locally with Docker Compose.

##### File structure analysis
- Top level:
- urbano.py / version extendida con analisis espacio temporal.py – core scripts for ingestion & processing
- API Flask Dockerfile.py – API server entrypoint or Dockerfile (naming suggests combined file)
- Docker-compose.yml (and typo “Docker compose.yml”) – multi-container orchestration
- requirements.txt – Python dependencies (snscrape, VADER/TextBlob, Folium, TimestampedGeoJson, Postgres driver, etc.)
- README.md – roadmap & feature description
- Estructura de proyecto – likely contains deeper folder structure (not shown)

- Indicators: single-repo, phase-1 MVP, monolithic Flask + processing pipeline, Postgres persistence, Folium/Leaflet frontend.

##### Key insights from README
- Phase-1 MVP pipeline:
- Ingest: snscrape → filter city/hashtags
- Process: sentiment analysis (VADER/TextBlob), NMF topic clustering on TF-IDF, simple geolocation (geopy)
- Store: Postgres (with basic geo-indexes)
- Expose: Flask API, interactive maps (HeatMapWithTime & TimestampedGeoJson)
- Docker Compose orchestrates (Scraper, Processor/API, Postgres, maybe UI server).

##### Main components & interactions
a. Data Ingestion Service
- Inputs: external social-media (snscrape)
- Output: raw posts
b. Processing Pipeline
- Sentiment Analyzer (VADER/TextBlob)
- Topic Clustering (NMF w/ TF-IDF)
- Geolocator (geopy → district aggregates)
- KPI Calculator (velocity, coverage over sliding window)
- Outputs processed records & geojson payloads
c. Persistence Layer
- Postgres DB (tables: raw_posts, analyzed_posts, topics, geodata, kpis)
d. API Layer
- Flask app serving endpoints: /heatmap, /timeline, /kpis
- Reads from Postgres, returns JSON/geojson
e. Dashboard / Frontend
- Folium/Leaflet templates served via Flask or static files
- Components: HeatMapWithTime, TimestampedGeoJson timeline
f. Orchestration
- Docker Compose: defines containers for each service + Postgres

##### Architectural patterns & principles
- Pipeline architecture (linear ETL: ingest → transform → load → serve)
- Separation of concerns: ingestion, processing, storage, API, visualization
- Containerization for reproducibility (Docker Compose)
- Future roadmap: move to microservices for each stage, introduce feature store & model registry

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Dependencies
    SocialMedia("Social Media Platforms"):::external
    Internet("Internet"):::external
    DockerHub("Docker Hub"):::external
    GitRepo("Git Repository"):::external

    %% Orchestration Layer
    subgraph "Orchestration" 
        DockerCompose["Docker Compose"]:::orch
    end

    %% Data Services
    subgraph "Data Services" 
        IngestService["Data Ingestion & Processing Service<br/>(Python + snscrape)"]:::ingestProc
        ProcessingPipeline["Processing Pipeline"]:::ingestProc
    end

    subgraph "Processing Components"
        Sentiment["Sentiment Analyzer<br/>(VADER/TextBlob)"]:::ingestProc
        Topic["Topic Clustering<br/>(NMF & TF-IDF)"]:::ingestProc
        Geolocator["Geolocator<br/>(geopy)"]:::ingestProc
        KPI["KPI Calculator<br/>(sliding windows)"]:::ingestProc
    end

    %% Storage Layer
    DB[(Postgres DB)]:::storage

    %% API & Frontend
    subgraph "Serving Layer"
        API["Flask API & Dockerfile"]:::api
        Dashboard["Dashboard<br/>(Folium/Leaflet)"]:::api
    end

    %% Documentation & Config
    Requirements["requirements.txt"]:::external
    README["README.md"]:::external
    ProjectStructure["Project Source Tree"]:::external

    %% Data Flow
    SocialMedia -->|"scrape posts"| IngestService
    IngestService -->|"raw posts"| ProcessingPipeline
    ProcessingPipeline --> Sentiment
    ProcessingPipeline --> Topic
    ProcessingPipeline --> Geolocator
    ProcessingPipeline --> KPI
    ProcessingPipeline -->|"processed records"| DB
    DB -->|"serve data"| API
    API -->|"JSON/GeoJSON"| Dashboard

    %% Orchestration Connections
    DockerCompose --> IngestService
    DockerCompose --> ProcessingPipeline
    DockerCompose --> DB
    DockerCompose --> API
    DockerCompose --> Dashboard

    %% External Services Usage
    API --> DockerHub
    IngestService --> Internet
    DockerCompose --> GitRepo

    %% Click Events
    click IngestService "https://github.com/portfolio-alvaro/proyecto-pulso-social-urbano/blob/main/urbano.py"
    click IngestService "https://github.com/portfolio-alvaro/proyecto-pulso-social-urbano/blob/main/version extendida con analisis espacio temporal.py"
    click API "https://github.com/portfolio-alvaro/proyecto-pulso-social-urbano/blob/main/API Flask Dockerfile.py"
    click DockerCompose "https://github.com/portfolio-alvaro/proyecto-pulso-social-urbano/blob/main/Docker-compose.yml"
    click DockerCompose "https://github.com/portfolio-alvaro/proyecto-pulso-social-urbano/blob/main/Docker compose.yml"
    click Requirements "https://github.com/portfolio-alvaro/proyecto-pulso-social-urbano/blob/main/requirements.txt"
    click README "https://github.com/portfolio-alvaro/proyecto-pulso-social-urbano/blob/main/README.md"
    click ProjectStructure "https://github.com/portfolio-alvaro/proyecto-pulso-social-urbano/tree/main/Estructura de proyecto/"

    %% Styling
    classDef ingestProc fill:#a8d5e2,stroke:#1f618d,color:#1f618d
    classDef storage fill:#bdd7ee,stroke:#21618c,color:#21618c
    classDef api fill:#f9e79f,stroke:#b7950b,color:#b7950b
    classDef orch fill:#d5dbdb,stroke:#7f8c8d,color:#7f8c8d
    classDef external fill:#f0b27a,stroke:#b9770e,color:#b9770e
```
