# borges070/projeto_integrador_1 architecture

> Architecture diagram of the GitHub repository borges070/projeto_integrador_1, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/borges070/projeto_integrador_1
- Repository: https://github.com/borges070/projeto_integrador_1

## Overview

1. **What this repository contains**
   - The repo is split into two clear parts: **`ETL/`**, which holds the executable data pipeline, and **`docs/`**, which holds product, requirements, data, wireframe, and delivery artifacts.
   - There is **no front-end or API implementation** in this repo; the runtime code is centered on data ingestion and preparation.

2. **Overall architecture**
   - Civitas is designed as a **legislative transparency platform**: it collects public political/legislative data, cleans and enriches it, then produces curated datasets and insights for dashboards and profiles.
   - The pipeline appears to support the downstream concepts described in the README: law explorer, proposal dashboard, politician profile, party pages, and commission pages.

3. **ETL pipeline orchestrator**
   - `ETL/main.py` is the entry point and likely coordinates the full batch flow.
   - The flow is modular: **extract → validate → transform → load → insight generation**, with logging configured separately.

4. **Extraction boundary**
   - `ETL/extract.py` is the ingestion layer that connects the platform to **external public data sources**.
   - This is the repository’s main inbound boundary: raw legislative or parliamentary data enters here before any normalization.

5. **Data quality and contract enforcement**
   - `ETL/validators.py` enforces input expectations before data is accepted downstream.
   - Architecturally, this is the gate between raw external data and the internal analytical model.

6. **Transformation layer**
   - `ETL/transform.py` is where domain data is normalized and reshaped into analytics-ready structures.
   - This layer is where the repository would align heterogeneous source data into consistent entities such as bills, politicians, votes, parties, commissions, and timelines.

7. **Load/persistence layer**
   - `ETL/load.py` is the outbound boundary of the pipeline and is responsible for storing the processed data into the target system.
   - The exact storage technology is not shown in the tree, so the repo only reveals the existence of a persistence step, not the database/vendor itself.

8. **Insight generation**
   - `ETL/gerar_insights.py` is a post-processing stage that produces derived metrics and summaries.
   - These outputs likely feed the dashboard-style views described in the README, such as KPIs, voting summaries, and other computed indicators.

9. **Operational concerns**
   - `ETL/logger_config.py` centralizes logging, which suggests the pipeline is meant to be run as a repeatable batch job with traceable execution.
   - `ETL/requirements.txt` indicates the pipeline is Python-based and its dependencies are isolated to this subsystem.

10. **Documentation and design artifacts**
   - The `docs/` tree contains the project’s **vision, requirements, data model, ER diagram, wireframes, demo material, and final reports**.
   - These artifacts define the intended domain model and user-facing surfaces, but they are not runtime components.

11. **How to draw the architecture**
   - A good diagram would show **external public legislative sources** feeding the **ETL pipeline**, then a **validated/transformed dataset store**, then **derived insights** consumed by the Civitas product views.
   - The documentation layer should be shown separately as **specification and design inputs**, not as an operational service.

12. **Important boundary to keep in mind**
   - This repo is primarily a **data-processing and product-definition repository**, not a complete application repository.
   - The architecture is therefore centered on **data ingestion, domain normalization, and metric generation** rather than request/response web services.

Main entry points: [`ETL/gerar_insights.py`](https://github.com/borges070/projeto_integrador_1/blob/main/ETL/gerar_insights.py) (Insights), [`ETL/extract.py`](https://github.com/borges070/projeto_integrador_1/blob/main/ETL/extract.py) (Extract), [`ETL/main.py`](https://github.com/borges070/projeto_integrador_1/blob/main/ETL/main.py) (Pipeline), [`ETL/validators.py`](https://github.com/borges070/projeto_integrador_1/blob/main/ETL/validators.py) (Validate)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_external["External Sources"]
  node_sources[("Public Data<br/>external data")]
end

subgraph group_etl["ETL Pipeline"]
  node_main["Pipeline<br/>orchestrator<br/>[main.py]"]
  node_extract["Extract<br/>ingestion<br/>[extract.py]"]
  node_validate["Validate<br/>quality gate<br/>[validators.py]"]
  node_transform["Transform<br/>normalization<br/>[transform.py]"]
  node_load[("Load<br/>persistence<br/>[load.py]")]
  node_insights["Insights<br/>metrics<br/>[gerar_insights.py]"]
  node_logger["Logging<br/>ops<br/>[logger_config.py]"]
  node_dataset[("Curated Data<br/>analytical store")]
end

subgraph group_product["Civitas Views"]
  node_law["Law Explorer<br/>product view"]
  node_proposal["Proposal Dashboard<br/>product view"]
  node_politician["Profile Pages<br/>product view"]
  node_party["Party Pages<br/>product view"]
  node_commission["Commission Pages<br/>product view"]
end

subgraph group_docs["Documentation"]
  node_vision["Vision<br/>product spec"]
  node_requirements["Requirements<br/>product spec"]
  node_data_model["Data Model<br/>data spec"]
  node_wireframe["Wireframes<br/>design spec<br/>[Wireframe.md]"]
  node_insights_doc["Insight Doc<br/>delivery spec"]
  node_delivery["Delivery Pack<br/>delivery docs"]
end

node_sources -->|"raw records"| node_extract
node_extract -->|"checks"| node_validate
node_validate -->|"accepted data"| node_transform
node_transform -->|"curated data"| node_load
node_load -->|"stored"| node_dataset
node_dataset -->|"inputs"| node_insights
node_insights -->|"powers"| node_law
node_insights -->|"powers"| node_proposal
node_dataset -->|"powers"| node_politician
node_dataset -->|"powers"| node_party
node_dataset -->|"powers"| node_commission
node_main -->|"starts"| node_extract
node_main -->|"uses"| node_logger
node_vision -->|"refines"| node_requirements
node_requirements -->|"constrains"| node_data_model
node_data_model -->|"informs"| node_wireframe
node_insights_doc -->|"documents"| node_insights
node_delivery -->|"packages"| node_vision

click node_main "https://github.com/borges070/projeto_integrador_1/blob/main/ETL/main.py"
click node_extract "https://github.com/borges070/projeto_integrador_1/blob/main/ETL/extract.py"
click node_validate "https://github.com/borges070/projeto_integrador_1/blob/main/ETL/validators.py"
click node_transform "https://github.com/borges070/projeto_integrador_1/blob/main/ETL/transform.py"
click node_load "https://github.com/borges070/projeto_integrador_1/blob/main/ETL/load.py"
click node_insights "https://github.com/borges070/projeto_integrador_1/blob/main/ETL/gerar_insights.py"
click node_logger "https://github.com/borges070/projeto_integrador_1/blob/main/ETL/logger_config.py"
click node_vision "https://github.com/borges070/projeto_integrador_1/blob/main/docs/Un1/Documento de Visão do Produto.pdf"
click node_requirements "https://github.com/borges070/projeto_integrador_1/blob/main/docs/Un2/Documento de Requisitos - Civitas.pdf"
click node_data_model "https://github.com/borges070/projeto_integrador_1/blob/main/docs/Un3/Documento de Dados - Civitas.pdf"
click node_wireframe "https://github.com/borges070/projeto_integrador_1/blob/main/docs/Un3/Wireframe.md"
click node_insights_doc "https://github.com/borges070/projeto_integrador_1/blob/main/docs/Un4/civitas_modulo_4_insights.pdf"
click node_delivery "https://github.com/borges070/projeto_integrador_1/blob/main/docs/Un5/Relatorio_Final_CIVITAS.pdf"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_sources toneBlue
class node_main,node_extract,node_validate,node_transform,node_load,node_insights,node_logger,node_dataset toneAmber
class node_law,node_proposal,node_politician,node_party,node_commission toneMint
class node_vision,node_requirements,node_data_model,node_wireframe,node_insights_doc,node_delivery toneRose
```

## Components

### External Sources

Public legislative and political data outside the repo

- **Public Data**: external data · Public legislative and political sources that feed the pipeline

### ETL Pipeline

Executable data processing flow

- [Pipeline](https://github.com/borges070/projeto_integrador_1/blob/main/ETL/main.py): orchestrator · Entry point that coordinates the batch flow · `ETL/main.py`
- [Extract](https://github.com/borges070/projeto_integrador_1/blob/main/ETL/extract.py): ingestion · Connects to external data sources and pulls raw records · `ETL/extract.py`
- [Validate](https://github.com/borges070/projeto_integrador_1/blob/main/ETL/validators.py): quality gate · Checks incoming data before downstream processing · `ETL/validators.py`
- [Transform](https://github.com/borges070/projeto_integrador_1/blob/main/ETL/transform.py): normalization · Reshapes and enriches raw data into analytics-ready entities · `ETL/transform.py`
- [Load](https://github.com/borges070/projeto_integrador_1/blob/main/ETL/load.py): persistence · Writes processed data to the target store · `ETL/load.py`
- [Insights](https://github.com/borges070/projeto_integrador_1/blob/main/ETL/gerar_insights.py): metrics · Generates derived summaries and indicators · `ETL/gerar_insights.py`
- [Logging](https://github.com/borges070/projeto_integrador_1/blob/main/ETL/logger_config.py): ops · Central logging configuration for pipeline runs · `ETL/logger_config.py`
- **Curated Data**: analytical store · Validated and transformed repository data used by later stages

### Civitas Views

Downstream analytical surfaces implied by the project

- **Law Explorer**: product view · Search and browse legislative records
- **Proposal Dashboard**: product view · Dashboard for bills and proposals
- **Profile Pages**: product view · Profiles for politicians and related activity
- **Party Pages**: product view · Pages for parties and their data
- **Commission Pages**: product view · Pages for commissions and their records

### Documentation

Vision, requirements, data model, and delivery artifacts

- [Vision](https://github.com/borges070/projeto_integrador_1/blob/HEAD/docs/Un1/Documento%20de%20Vis%C3%A3o%20do%20Produto.pdf): product spec · Product vision and scope documents · `docs/Un1/Documento de Visão do Produto.pdf`
- [Requirements](https://github.com/borges070/projeto_integrador_1/blob/HEAD/docs/Un2/Documento%20de%20Requisitos%20-%20Civitas.pdf): product spec · Functional and non-functional requirements · `docs/Un2/Documento de Requisitos - Civitas.pdf`
- [Data Model](https://github.com/borges070/projeto_integrador_1/blob/HEAD/docs/Un3/Documento%20de%20Dados%20-%20Civitas.pdf): data spec · Data documentation and ER model · `docs/Un3/Documento de Dados - Civitas.pdf`
- [Wireframes](https://github.com/borges070/projeto_integrador_1/blob/main/docs/Un3/Wireframe.md): design spec · Wireframe and link artifacts for interface design · `docs/Un3/Wireframe.md`
- [Insight Doc](https://github.com/borges070/projeto_integrador_1/blob/main/docs/Un4/civitas_modulo_4_insights.pdf): delivery spec · Insights module documentation · `docs/Un4/civitas_modulo_4_insights.pdf`
- [Delivery Pack](https://github.com/borges070/projeto_integrador_1/blob/main/docs/Un5/Relatorio_Final_CIVITAS.pdf): delivery docs · Demo, final report, and compliance deliverables · `docs/Un5/Relatorio_Final_CIVITAS.pdf`

## Connections

- Public Data → Extract: raw records · Public data enters the extraction boundary
- Extract → Validate: checks · Extracted data is validated before use
- Validate → Transform: accepted data · Validated data is normalized and reshaped
- Transform → Load: curated data · Transformed entities are persisted
- Load → Curated Data: stored · Loaded data becomes the curated analytical store
- Curated Data → Insights: inputs · Curated data feeds derived metrics and summaries
- Insights → Law Explorer: powers · Insights support the law explorer view
- Insights → Proposal Dashboard: powers · Insights support the proposal dashboard
- Curated Data → Profile Pages: powers · Curated data supports politician profiles
- Curated Data → Party Pages: powers · Curated data supports party pages
- Curated Data → Commission Pages: powers · Curated data supports commission pages
- Pipeline → Extract: starts · The orchestrator launches the pipeline stages
- Pipeline → Logging: uses · Execution is traced through central logging
- Vision → Requirements: refines · Vision informs requirements
- Requirements → Data Model: constrains · Requirements shape the data model
- Data Model → Wireframes: informs · Data model and UI concepts inform wireframes
- Insight Doc → Insights: documents · Documentation describes the insights module
- Delivery Pack → Vision: packages · Delivery artifacts include project definition and reporting