How ignacioe7/proyectodetitulo works
Project type and purpose
• This is a full-stack, academic research tool for sentiment analysis on tourism reviews. It combines a data-scraping pipeline, NLP/ML-based sentiment classification, data storage/export, and a Streamlit-based web UI.
File structure
• Top-level:
• config.json (region settings)
• data/ (raw and consolidated JSON)
• src/ with subdirectories:
– core/ (scraper.py, analyzer.py, data_handler.py, metrics.py, parsers/)
– models/ (Attraction, Review data classes)
– ui/ (Streamlit app + menu modules)
– utils/ (constants, exporters, logger, networking)
• requirements.txt declares Python, Transformers, PyTorch, Pandas, Plotly, Streamlit, etc.
README for architecture clues
• Highlights concurrent processing, export to multiple formats, and interactive visualizations.
Main components & interactions (to represent
a. External Data Source
• TripAdvisor (or other review site)
b. Networking Layer
• utils/networking.py handles HTTP requests, rate-limiting, delays
c. Scraper & Parser
• src/core/scraper.py orchestrates page downloads
• src/core/parsers/review_parser.py parses HTML into Review models
d. Data Handler & Models
• src/core/data_handler.py cleans, normalizes, merges data
• src/models/attraction.py & review.py define in-memory objects
• data/consolidated_data.json persists intermediate results
e. Analyzer & Metrics
• src/core/analyzer.py uses Transformers + PyTorch to classify sentiment
• src/core/metrics.py computes statistics (counts, averages, distributions)
f. UI Layer (Streamlit)
• src/ui/streamlit_app.py + menu modules present configuration screens, run buttons, and results views
• Plotly for charts
g. Exporters & Storage
• src/utils/exporters.py writes JSON, CSV, Excel
• data/ directory holds outputs
h. Logging & Config
• src/utils/logger.py captures pipeline events
• config.json and src/utils/constants.py provide parameters
Architectural patterns & design principles
• Modular pipeline: clearly separated scraping → parsing → data handling → analysis → visualization/export
• MVC-like separation:
• Model: src/models/**
• Controller/Service: src/core/ + src/utils/
• View/UI: src/ui/**
• Concurrency: parallel scraping/analysis threads or async I/O
• Single responsibility modules: each component does one job