How diego-uzc-j/analisis-de-datos-de-repositorio-universitario works
This repository provides a university-repository analytics dashboard backed by a CSV-to-Parquet ETL pipeline. The principal workflow is to prepare article records, load the processed data into Streamlit, apply interactive filters, and explore publication trends, collaboration, communities, keywords, authors, and filtered records. The README documents SQL-based DSpace extraction and scheduled automation; those deployment details are not mapped as runtime components. Some app-to-ETL wiring is not visible in the excerpts, so only documented or excerpt-supported relationships are shown.
Main entry points: app.py Streamlit dashboard · plot.py Plotly generators · etl.py ETL orchestrator · consultas_articulos_dspace.sql Article query
Components 19
Data Preparation
- Article query
sql/consultas_articulos_dspace.sql - Article CSV
data/articulos_comms_fac_y_rev.csv - ETL orchestrator
src/etl.py - CSV extraction
src/etl.py - Date cleaning
src/etl.py - Parquet loading
src/etl.py - Processed articles
data/articulos_procesados.parquet
Dashboard
- Streamlit dashboard
app.py - Interactive filters
app.py - Filtered records
app.py
Analytics
- Publication trends
app.py - Coauthor analysis
app.py - Community analysis
app.py - Production heatmap
app.py - Top keywords
app.py - Top authors
app.py - Plotly generators
src/plot.py
Other components
- Dashboard user
- DSpace repository
Connections 24
- DSpace repository → Article query: queried by
README.md - Article query → Article CSV: produces source
README.md - ETL orchestrator → CSV extraction: calls
src/etl.py - CSV extraction → Article CSV: reads
src/etl.py - ETL orchestrator → Date cleaning: calls
src/etl.py - ETL orchestrator → Parquet loading: calls
src/etl.py - Parquet loading → Processed articles: writes
src/etl.py - Dashboard user → Streamlit dashboard: uses
README.md - Processed articles → Streamlit dashboard: feeds
README.md - Streamlit dashboard → Interactive filters: offers
app.py - Streamlit dashboard → Filtered records: renders
app.py - Streamlit dashboard → Publication trends: shows
app.py - Streamlit dashboard → Coauthor analysis: shows
app.py - Streamlit dashboard → Community analysis: shows
app.py - Streamlit dashboard → Production heatmap: shows
app.py - Streamlit dashboard → Top keywords: shows
app.py - Streamlit dashboard → Top authors: shows
app.py - Streamlit dashboard → Plotly generators: calls
app.py - Publication trends → Plotly generators: renders with
app.py - Coauthor analysis → Plotly generators: renders with
app.py - Community analysis → Plotly generators: renders with
app.py - Production heatmap → Plotly generators: renders with
app.py - Top keywords → Plotly generators: renders with
app.py - Top authors → Plotly generators: renders with
app.py