# daniel27110/ciencia-de-datos-aplicada-proyecto-final architecture

> Architecture diagram of the GitHub repository daniel27110/ciencia-de-datos-aplicada-proyecto-final, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final
- Repository: https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final

## Overview

##### Project Type and Purpose
- This is a full-stack data analytics and machine learning solution that processes raw student data, trains a predictive model, and exposes it via a web interface to identify at-risk students.
- It comprises three major phases: offline data ingestion & modeling, model artifact management, and a lightweight REST + UI deployment.

##### High-Level Components
a. Data Storage Layer
- Raw Data Directory (“data/raw”) – source CSV files (DIVIPOLA, programas.csv, Demograficos.csv.gz).
- Staging Directory (“data/stage”) – cleaned and partitioned CSVs (clean.csv, train.csv, ultimo_semestre.csv).
b. Analytics Pipeline
- Jupyter Notebooks under “analytics” and “data/analytics”: EDA, data preparation, model selection, training.
- Outputs: clean.csv in data/stage and serialized models (rf_model.joblib, lr_model.joblib, rf_pipeline.joblib) in “deploy”.
c. Model Artifact Repository
- Serialized models and pipelines stored in “deploy” alongside versioned artifacts (clf_model.joblib, rf_model.joblib, etc.).
d. API / Service Layer
- FastAPI application (“deploy/main.py”, run via uvicorn)
- Pydantic schema definitions (“deploy/datamodel.py”)
- Model wrappers (“deploy/model.py”, “deploy/rf_model.py”, “deploy/lr_model.py”)
- Endpoints:
– GET /inicio serves HTML form (index.html)
– POST /predict returns single or batch predictions
e. Frontend Layer
- Static HTML templates (“deploy/index.html”, “deploy/student-details.html”)
- JavaScript client logic (“deploy/charts.js”) for rendering charts and calling API
f. Supporting Files
- Configuration / dependencies (requirements.txt)
- Documentation (docs/, README.md)

##### Component Interactions and Data Flow
1. Offline phase: Raw CSVs → Data preparation notebook → clean.csv (staging) → model selection notebook → trained model artifacts (.joblib)
2. Deployment phase:
- At startup, FastAPI loads model artifacts into memory.
- User opens index.html in browser → fills form or uploads estudiantes.csv → client JS invokes /predict.
- FastAPI endpoint processes JSON or file, invokes model wrapper → returns prediction response.
- Frontend renders results and drill-down chart via charts.js and student-details.html.

##### Architectural Patterns and Design Principles
- Separation of Concerns: distinct offline analytics pipeline vs. online serving components.
- Micro-service pattern: stateless, single-responsibility FastAPI application.
- File-based “database”: CSVs and joblib serializations replace a traditional RDBMS.
- Modular code: Pydantic for input validation, separate model wrappers, static templates for UI.

## Diagram (Mermaid)

```mermaid
flowchart TB
  subgraph "Offline Training"
    direction TB
    Raw["Raw Data\ndata/raw/*.csv"]:::datastore
    PrepNB["Data Prep Notebook\nanalytics/..."]:::process
    DataAnalytics["Parallel Notebooks\ndata/analytics"]:::process
    Stage["Staging Data\ndata/stage/clean.csv"]:::datastore
    ModelNB["Model Selection Notebook\nanalytics/..."]:::process
    subgraph "Model Artifacts"
      direction TB
      Clf["clf_model.joblib"]:::datastore
      RF["rf_model.joblib"]:::datastore
      LR["lr_model.joblib"]:::datastore
      RFPipe["rf_pipeline.joblib"]:::datastore
    end
    Raw --> PrepNB --> DataAnalytics --> Stage --> ModelNB
    ModelNB --> Clf
    ModelNB --> RF
    ModelNB --> LR
    ModelNB --> RFPipe
  end

  subgraph "Online Serving"
    direction TB
    API["FastAPI Server\ndeploy/main.py"]:::service
    Datamodel["Pydantic Schemas\ndeploy/datamodel.py"]:::service
    ModelWrapper["Generic Model Wrapper\ndeploy/model.py"]:::service
    RFWrap["RF Wrapper\ndeploy/rf_model.py"]:::service
    LRWrap["LR Wrapper\ndeploy/lr_model.py"]:::service
    Browser["Browser UI"]:::frontend
    Index["Home Page\ndeploy/index.html"]:::frontend
    Details["Student Details\ndeploy/student-details.html"]:::frontend
    Charts["Charts JS\ndeploy/charts.js"]:::frontend

    Clf --> API
    RF --> API
    LR --> API
    RFPipe --> API

    API --> Datamodel
    API --> ModelWrapper
    ModelWrapper --> RFWrap
    ModelWrapper --> LRWrap

    Browser --> Index
    Browser --> Details
    Browser --> Charts
    Browser -->|calls /predict| API
    API -->|response| Browser
  end

  subgraph "Supporting Files"
    direction TB
    Req["requirements.txt"]:::support
    Readme["README.md"]:::support
    Docs["docs/"]:::support
  end

  classDef datastore fill:#D0E6FF,stroke:#0366D6,color:#0366D6;
  classDef process fill:#E6F4EA,stroke:#2C662D,color:#2C662D;
  classDef service fill:#FFE8D6,stroke:#D9822B,color:#D9822B;
  classDef frontend fill:#FFF5CC,stroke:#D4A017,color:#D4A017;
  classDef support fill:#F0F0F0,stroke:#AAAAAA,color:#555555;

  click Raw "https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final/tree/main/data/raw"
  click Stage "https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final/tree/main/data/stage"
  click PrepNB "https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final/tree/main/analytics/primera entrega"
  click ModelNB "https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final/tree/main/analytics/entrega final"
  click DataAnalytics "https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final/tree/main/data/analytics"
  click Clf "https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final/blob/main/deploy/clf_model.joblib"
  click RF "https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final/blob/main/deploy/rf_model.joblib"
  click LR "https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final/blob/main/deploy/lr_model.joblib"
  click RFPipe "https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final/blob/main/deploy/rf_pipeline.joblib"
  click API "https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final/blob/main/deploy/main.py"
  click Datamodel "https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final/blob/main/deploy/datamodel.py"
  click ModelWrapper "https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final/blob/main/deploy/model.py"
  click RFWrap "https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final/blob/main/deploy/rf_model.py"
  click LRWrap "https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final/blob/main/deploy/lr_model.py"
  click Index "https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final/blob/main/deploy/index.html"
  click Details "https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final/blob/main/deploy/student-details.html"
  click Charts "https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final/blob/main/deploy/charts.js"
  click Req "https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final/blob/main/requirements.txt"
  click Readme "https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final/blob/main/README.md"
  click Docs "https://github.com/daniel27110/ciencia-de-datos-aplicada-proyecto-final/tree/main/docs/"
```
