How fhicar/pulso_social_urbano works
Project type and purpose
• This is a full-stack, data-driven web application (Flask) that:
• Scrapes social-media data from Twitter, Reddit, and Mastodon via their APIs.
• Processes and analyzes that data with custom pipelines (Pandas, NumPy).
• Runs ML models for sentiment analysis and topic clustering.
• Visualizes results on an interactive geospatial map (Folium) and dashboard (Jinja2 templates).
File structure
• Top-level:
• app.py – Flask entry point and HTTP routing
• config.py – central config loader (.env integration)
• data_ingestion/ – modules to scrape and pre-process raw posts
• machine_learning/ – modules implementing sentiment analysis + clustering
• templates/ + static/ – frontend views (Jinja2, CSS)
• Dockerfile, docker-compose.yml – containerized deployment
• Patterns:
• MVC-style: Views (templates), Controllers (routes in app.py), Models (data_ingestion & ML).
• A single monolith rather than microservices, but with clear sub-modules.
README for additional insights
• Confirms: Flask backend, Jinja2 frontend, PRAW/Tweepy/Mastodon.py scrapers, Pandas/NumPy data pipelines, scikit-learn/TextBlob/VADER for ML, Folium/Geopy for mapping.
Elements
• HTTP flows (user ↔ Flask)
• Data flows (Flask → Data Ingestion → ML → Visualization → templates)
• API calls (Data Ingestion → Twitter/Reddit/Mastodon APIs)
• Blue for user-facing (templates, browser)
• Green for services (scrapers, ML modules)
• Orange for external dependencies (social APIs)
• Gray for infrastructure (Docker, .env)
Final recommendations
• Be exhaustive: represent each Python module or package as its own sub-component if clarity demands.
• Config (e.g., “reads TWITTER_API_KEY from .env”).
• Keep layout logical: left = user & frontend, middle = application/back end, right = external services & infra.