How datos-fundar/sesgos_llm works
Project Type and Purpose
• This is a reproducible data-analysis pipeline, not a web app or microservice. It combines Jupyter notebooks, Python scripts, R scripts, and raw datasets to:
• Prompt multiple Large Language Models (Bard, ChatGPT, Cohere) with adapted Latinobarómetro questions
• Compute distances between model answers and public-opinion survey results
• Run multivariate regression analyses on those distances
• Generate CSV outputs and visual charts
• The README makes clear that the goal is to expose and quantify algorithmic biases in LLMs, in an Argentinian context, using public-opinion data.
File Structure Analysis
• Top-level files:
• notebooks: bard.ipynb, chatgpt.ipynb, cohere.ipynb, and a variety of distances_*.ipynb under the root and etc/
• data directories: datasets/ (Stata and CSV inputs)
• result directory: outputs/ (CSV and chart outputs)
• processing scripts: scripts/ (.py and .R files)
• utility code: utils.py
• Patterns:
• “Notebooks” fire API calls and exploratory analyses
• “scripts/” run batch, reproducible statistical models (OLS)
• “etc/” holds extra exploratory notebooks
Architecture Insights from README
• Dependencies: Python (pandas, Jupyter, custom utils), R (for .R scripts), external LLM APIs (OpenAI, Google Bard, Cohere).
a. Main Components
1. Data Sources
• Latinobarómetro Stata files (datasets/)
• Categorías and respuestas pickle files
2. Notebook Orchestrators
• LLM-prompt notebooks (bard.ipynb, chatgpt.ipynb, cohere.ipynb)
• Distance-calculation notebooks (distances_*.ipynb)
• Exploratory notebooks (etc/)
3. Scripts and Utils
• Python scripts (scripts/consolidacion_preguntas_respuestas.py, utils.py)
• R scripts (scripts/multivariado.R, scripts/multivariado_2_lapop.R)
4. External Services
• OpenAI API (gpt-3.5 turbo etc.)
• Google Bard API
• Cohere API
5. Outputs
• CSVs and charts (outputs/)
b. Relationships and Interactions
• Data Sources feed into Python notebooks and scripts
• Orchestrator notebooks call external LLM APIs
• LLM responses + survey data → distance notebooks
• Distance results feed into R scripts for regression
• Regression outputs and charts are written to outputs/
c. Architectural Patterns and Principles
• Pipeline pattern: sequential ETL-style flow
• Modular separation: data layer, compute layer (LLM calls & distances), modeling layer, reporting layer
• Polyglot: Python for data and API calls, R for statistical modeling
d. Key Technologies
• Jupyter notebooks
• Python (pandas, requests or openai/cohere SDKs)
• R (stats, OLS)
• External LLM APIs (OpenAI, Bard, Cohere)
• Stata file readers (e.g., pandas read_stata)