# puffdapaz/ipeapython architecture

> Architecture diagram of the GitHub repository puffdapaz/ipeapython, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/puffdapaz/ipeapython
- Repository: https://github.com/puffdapaz/ipeapython

## Overview

##### Main Components
a. Data Storage (Bronze / Silver / Gold) – your “data lake” layers, each holding Parquet files.
b. ETL & Processing Layer – Python scripts (e.g., IPEAv1.py, backend.py, Statistical Analysis scripts) that read, clean, aggregate, and analyze data.
c. Visualization / UI Layer – Streamlit application entry points (`_🌐_Início_-_Home.py`, pages/_⚽_pt-BR.py, pages/_👑_en-GB.py).
d. External Dependencies – Python libraries (listed in requirements.txt / packages.txt).
e. Users – end users interacting via a web browser.

##### Relationships & Data Flow
- ETL Layer outputs feed both the Statistical Analysis module (ad hoc reports) and the backend.py data‐serving functions.
- Backend provides processed data to the Streamlit UI.
- Users access the UI; UI calls backend functions; backend reads Gold-layer Parquets.

##### Architectural Patterns
- “Lambda–style” data pipeline: raw → cleaned → aggregated.
- Modular front end: multilingual pages in a Streamlit multipage app.
- Single‐process Python “backend” serving both ETL orchestration and live data queries.
- Directory segmentation by data maturity (Bronze/Silver/Gold) enforces separation of concerns.

##### Key Technologies / Frameworks
- Data formats: Parquet (efficient columnar storage).
- Processing: pandas or PyArrow (implied by Parquet use).
- Visualization/UI: Streamlit.
- Orchestration: plain Python scripts (no heavy scheduler indicated).
- Dependency management: requirements.txt / packages.txt.

##### Final Checklist
- Every component has a clear label.

- Captures the full data lifecycle
- Separates storage, compute, and presentation layers
- Highlights the project’s ETL pattern and multilingual UI
- Remains easy to understand for stakeholders and new engineers alike.

## Diagram (Mermaid)

```mermaid
graph TB
    %% External Dependencies
    Req["requirements.txt"]:::external
    Pkg["packages.txt"]:::external

    %% Storage Layer
    subgraph "Data Storage"
        Bronze["Bronze Data Lake"]:::storage
        Silver["Silver Data Lake"]:::storage
        Gold["Gold Data Lake"]:::storage
    end

    %% ETL & Processing Layer
    subgraph "ETL & Processing"
        IPEA["ETL: IPEAv1.py"]:::processing
        Backend["Backend: backend.py"]:::processing
        Stats["Statistical Analysis"]:::processing
    end

    %% Visualization / UI Layer
    subgraph "Streamlit UI"
        Home["Home Page"]:::ui
        PT["Portuguese Page"]:::ui
        EN["English Page"]:::ui
    end

    %% User
    User[/"User"/] 

    %% Data Flows
    Req --> IPEA
    Req --> Backend
    Req --> Stats
    Pkg --> IPEA
    Pkg --> Backend
    Pkg --> Stats

    Bronze -->|reads raw| IPEA
    IPEA -->|writes cleaned| Silver
    Silver -->|reads cleaned| Backend
    Backend -->|writes aggregated| Gold

    Gold -->|data for| Stats
    Gold -->|serves data to| Home
    Gold -->|serves data to| PT
    Gold -->|serves data to| EN

    User -->|views| Home
    Home -->|calls API| Backend
    User -->|views| PT
    PT -->|calls API| Backend
    User -->|views| EN
    EN -->|calls API| Backend

    %% Click Events
    click Bronze "https://github.com/puffdapaz/ipeapython/tree/main/Bronze/"
    click Silver "https://github.com/puffdapaz/ipeapython/tree/main/Silver/"
    click Gold "https://github.com/puffdapaz/ipeapython/tree/main/Gold/"
    click IPEA "https://github.com/puffdapaz/ipeapython/blob/main/IPEAv1.py"
    click Backend "https://github.com/puffdapaz/ipeapython/blob/main/backend.py"
    click Stats "https://github.com/puffdapaz/ipeapython/tree/main/Statistical Analysis/"
    click Home "https://github.com/puffdapaz/ipeapython/blob/main/_🌐_Início_-_Home.py"
    click PT "https://github.com/puffdapaz/ipeapython/blob/main/pages/_⚽_pt-BR.py"
    click EN "https://github.com/puffdapaz/ipeapython/blob/main/pages/_👑_en-GB.py"
    click Req "https://github.com/puffdapaz/ipeapython/blob/main/requirements.txt"
    click Pkg "https://github.com/puffdapaz/ipeapython/blob/main/packages.txt"

    %% Styles
    classDef storage fill:#add8e6,stroke:#000,stroke-width:1px
    classDef processing fill:#ffa500,stroke:#000,stroke-width:1px
    classDef ui fill:#90ee90,stroke:#000,stroke-width:1px
    classDef external fill:#d3d3d3,stroke:#000,stroke-width:1px
```
