# migue-08/research_paper_ms_econ architecture

> Architecture diagram of the GitHub repository migue-08/research_paper_ms_econ, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/migue-08/research_paper_ms_econ
- Repository: https://github.com/migue-08/research_paper_ms_econ

## Overview

##### Project Type and Purpose
- This is an empirical research/data-analysis pipeline—NOT a full-stack web app.
- Purpose: ingest national exam (Saber11) and school census data (DANE), clean and merge them, run regressions, produce visualizations, and publish results as a static site via Quarto.

##### File Structure
- Top-level notebooks (.ipynb) for each processing step (cleaning, merging, visualizations).
- A Quarto munging script (munging.qmd) and a Quarto config (_quarto.yml).
- A STATA .do file (regression_analysis.do) for the final regression step.
- docs/ folder containing a generated static site (HTML, assets).
- requirements.txt lists Python/R dependencies.

##### Main Components
A. External Data Sources
- ICFES Saber11 microdata (2019 & 2022)
- DANE scholar census (2019 & 2022)
B. Data-Ingestion & Cleaning Layer
- Jupyter notebook: Saber_11_Cleaning.ipynb
- Jupyter notebook: Acceso_tecnología_por_estudiante.ipynb
C. Merging & Transformation Layer
- merged.ipynb (combines cleaned Saber11 + census data)
- munging.qmd (Quarto pipeline for further munging & summary stats)
D. Analysis Layer
- regression_analysis.do (STATA do-file executing regression models)
E. Visualization Layer
- Grafico_distribucion_datos.ipynb
- Matricula_ed_* notebooks
- Tasa_deserción.ipynb, etc.
F. Presentation / Documentation Layer
- Quarto (_quarto.yml) generates docs/ site
- docs/index.html + assets (bootstrap, quarto scripts)

##### Relationships & Data Flow
- Cleaning Notebooks → Merged Notebook → munging.qmd
- munging.qmd → Summary tables & cleaned artifacts → regression_analysis.do
- regression_analysis.do → Regression outputs
- All notebooks & munging scripts → Visualization Notebooks → Figures
- Quarto pipeline → Static HTML site in docs/ incorporating figures & narrative

##### Architectural Patterns & Design Principles
- Sequential ETL pipeline: Extract (download) → Transform (clean & merge) → Load (analysis & viz)
- Tool-chain integration: Python Jupyter → Quarto → STATA → Quarto static site
- Clear separation between data processing and documentation/publishing

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Data Sources
    subgraph "External Data Sources"
        style External1 fill:#D0E6FA,stroke:#0366d6
        style External2 fill:#D0E6FA,stroke:#0366d6
        External1["ICFES Saber11 Microdata\n(2019 & 2022)"]:::datasource
        External2["DANE School Census\n(2019 & 2022)"]:::datasource
    end

    %% Data Processing Layer
    subgraph "Data Processing"
        style SaberCleaning fill:#E6F4EA,stroke:#028A0F
        style AccesoTec fill:#E6F4EA,stroke:#028A0F
        style Merged fill:#E6F4EA,stroke:#028A0F
        style Munging fill:#E6F4EA,stroke:#028A0F

        SaberCleaning["Saber_11_Cleaning.ipynb\n(Python/Pandas)"]:::etl
        AccesoTec["Acceso_tecnología_por_estudiante.ipynb\n(Python/Pandas)"]:::etl
        Merged["merged.ipynb\n(Python/Pandas)"]:::etl
        Munging["munging.qmd\n(Quarto)"]:::etl
    end

    %% Analysis Layer
    subgraph "Analysis Engine"
        style RegAnalysis fill:#FFF4DB,stroke:#D9822B
        RegAnalysis["regression_analysis.do\n(STATA)"]:::analysis
    end

    %% Visualization Layer
    subgraph "Visualizations"
        style DistPlot fill:#E6F4EA,stroke:#028A0F
        style MatBasSec fill:#E6F4EA,stroke:#028A0F
        style MatInicPrim fill:#E6F4EA,stroke:#028A0F
        style TasaDesercion fill:#E6F4EA,stroke:#028A0F
        style TasaTecAcc fill:#E6F4EA,stroke:#028A0F

        DistPlot["Grafico_distribucion_datos.ipynb\n(Python/Matplotlib)"]:::etl
        MatBasSec["Matricula_ed_bas_sec_media.ipynb\n(Python/Seaborn)"]:::etl
        MatInicPrim["Matricula_ed_inic_bas_primaria.ipynb\n(Python/Seaborn)"]:::etl
        TasaDesercion["Tasa_deserción.ipynb\n(Python/Seaborn)"]:::etl
        TasaTecAcc["Tasa_tecnología_por_estudiante.ipynb\n(Python/Seaborn)"]:::etl
    end

    %% Presentation Layer
    subgraph "Documentation / Presentation"
        style QuartoConfig fill:#F4E6FA,stroke:#8A2BE2
        style DocsIndex fill:#F4E6FA,stroke:#8A2BE2
        style DocsAssets fill:#F4E6FA,stroke:#8A2BE2

        QuartoConfig["_quarto.yml\n(Quarto Config)"]:::docs
        DocsIndex["docs/index.html\n(Static Site)"]:::docs
        DocsAssets["docs/site_libs/\n(Assets)"]:::docs
    end

    %% Flow Connections
    External1 -->|"clean →"| SaberCleaning
    External2 -->|"clean →"| AccesoTec
    SaberCleaning -->|"merge →"| Merged
    AccesoTec --> Merged
    Merged -->|"quarto munging →"| Munging
    Munging -->|"summary outputs →"| RegAnalysis
    RegAnalysis -->|"regression results →"| DistPlot
    RegAnalysis --> MatBasSec
    RegAnalysis --> MatInicPrim
    RegAnalysis --> TasaDesercion
    RegAnalysis --> TasaTecAcc
    DistPlot -->|"figures→"| DocsIndex
    MatBasSec --> DocsIndex
    MatInicPrim --> DocsIndex
    TasaDesercion --> DocsIndex
    TasaTecAcc --> DocsIndex
    QuartoConfig -->|"build site→"| DocsIndex
    DocsAssets --> DocsIndex

    %% Click Events
    click SaberCleaning "https://github.com/migue-08/research_paper_ms_econ/blob/main/Saber_11_Cleaning.ipynb"
    click AccesoTec "https://github.com/migue-08/research_paper_ms_econ/blob/main/Acceso_tecnología_por_estudiante.ipynb"
    click Merged "https://github.com/migue-08/research_paper_ms_econ/blob/main/merged.ipynb"
    click Munging "https://github.com/migue-08/research_paper_ms_econ/blob/main/munging.qmd"
    click RegAnalysis "https://github.com/migue-08/research_paper_ms_econ/blob/main/regression_analysis.do"
    click DistPlot "https://github.com/migue-08/research_paper_ms_econ/blob/main/Grafico_distribucion_datos.ipynb"
    click MatBasSec "https://github.com/migue-08/research_paper_ms_econ/blob/main/Matricula_ed_bas_sec_media.ipynb"
    click MatInicPrim "https://github.com/migue-08/research_paper_ms_econ/blob/main/Matricula_ed_inic_bas_primaria.ipynb"
    click TasaDesercion "https://github.com/migue-08/research_paper_ms_econ/blob/main/Tasa_deserción.ipynb"
    click TasaTecAcc "https://github.com/migue-08/research_paper_ms_econ/blob/main/Tasa_tecnología_por_estudiante.ipynb"
    click QuartoConfig "https://github.com/migue-08/research_paper_ms_econ/blob/main/_quarto.yml"
    click DocsIndex "https://github.com/migue-08/research_paper_ms_econ/blob/main/docs/index.html"
    click DocsAssets "https://github.com/migue-08/research_paper_ms_econ/tree/main/docs/site_libs/"

    %% Class Definitions
    classDef datasource fill:#D0E6FA,stroke:#0366d6,color:#0366d6,stroke-width:2px,stroke-dasharray: 5 5;
    classDef etl fill:#E6F4EA,stroke:#028A0F,color:#028A0F,stroke-width:2px;
    classDef analysis fill:#FFF4DB,stroke:#D9822B,color:#D9822B,stroke-width:2px;
    classDef docs fill:#F4E6FA,stroke:#8A2BE2,color:#8A2BE2,stroke-width:2px;
```
