# adeniltonr/python-com-banco-de-dados architecture

> Architecture diagram of the GitHub repository adeniltonr/python-com-banco-de-dados, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/adeniltonr/python-com-banco-de-dados
- Repository: https://github.com/adeniltonr/python-com-banco-de-dados

## Overview

##### Main components
- Data Extraction Modules: Represent the several Python scripts designed for web scraping and data extraction. This group includes:
- buscar_arrecadacao_atual.py (fetches current fundraising sums directly from the site)
- extrair_dados_top100_doadores.py (extracts data on the top 100 donors)
- copiar_conteudo_html_top_100_doadores.py (copies HTML content related to donor data)
- extrair_dados_mapa_doacao_por_estados.py (extracts donation data by state)
- Data Persistence Module: Use a dedicated component for the database operations handled by bd.py. This module is responsible for creating tables, inserting data, and connecting with the SQLite database (arenacorinthinas.db).
- Main Control/Interface: The main.py file houses the menu-driven interface that orchestrates user interactions and calls the respective modules.
- Data Files: The “dados” folder, including various text and JSON files, serves as intermediate storage for extracted content and acts as a staging area between web scraping and persistence.
- External Libraries/Utilities: Indicate the use of Selenium (with Firefox’s geckodriver) for web scraping, and other native libraries (e.g., sqlite3, json, re) that underpin the application.

##### Relationships and interactions
- The main.py acts as the central control panel. It directs calls to the extraction modules to gather data from the fundraising website.
- Web scraping components (using Selenium) interact with the external website to collect fundraising and donor information.
- Once data is extracted, it flows either directly to data files in the “dados” folder or is processed and then inserted into the SQLite database via bd.py.
- Some scripts appear to work sequentially: for example, extracting donor data → writing to a file → reading from the file → persisting into the database.
- There is an implicit flow from user-triggered menu options to either update, fetch, or display data.

##### Architectural patterns and design principles
- Modular Design: The implementation is divided among distinct scripts, making it easier to extend or modify parts of the system (e.g., adding new scraping routines or database updates).

##### Incorporate the technologies
- Selenium as part of the “Data Extraction” module to showcase its role in interacting with the external website.
- SQLite database (arenacorinthinas.db) within the “Data Persistence” layer.
- Optionally, indicate external visualization tools – such as DBeaver – as clients that users might use to inspect the database.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Data Source and Libraries
    EWS["Fundraising Website"]:::external
    SEL["Selenium (WebDriver)"]:::external
    EL["requirements.txt"]:::external

    EWS -->|"scraped by"| SEL
    EL -->|"provides"| SEL

    %% Main Control
    MC["Main Menu Controller (main.py)"]:::control

    %% Data Extraction Modules
    subgraph "Data Extraction Modules"
        BA["buscar_arrecadacao_atual.py"]:::extraction
        TD["extrair_dados_top100_doadores.py"]:::extraction
        CP["copiar_conteudo_html_top_100_doadores.py"]:::extraction
        ME["extrair_dados_mapa_doacao_por_estados.py"]:::extraction
    end

    %% Intermediate Data Storage (dados)
    subgraph "Intermediate Data Storage (dados)"
        DF["dados folder"]:::dataFiles
        AET["arrecadacoes_estados.txt"]:::dataFiles
        BT["boletos.txt"]:::dataFiles
        CT["conteudo_para_extracao_top100doadores.txt"]:::dataFiles
        DE["dados_estados.json"]:::dataFiles
        RT["resultado_top_100_doadores.txt"]:::dataFiles
        DF --> AET
        DF --> BT
        DF --> CT
        DF --> DE
        DF --> RT
    end

    %% Data Persistence Modules
    subgraph "Data Persistence"
        BD["Database Manager (bd.py)"]:::persistence
        DB["SQLite Database (arenacorinthinas.db)"]:::persistence
        BD -->|"inserts/updates"| DB
    end

    %% Connections between Modules
    %% Main Control triggers extraction and database operations
    MC -->|"triggers"| BA
    MC -->|"triggers"| TD
    MC -->|"triggers"| CP
    MC -->|"triggers"| ME
    MC -->|"manages"| BD

    %% Selenium triggers data extraction
    SEL --> BA
    SEL --> TD
    SEL --> CP
    SEL --> ME

    %% Data Extraction outputs to Intermediate Storage
    BA -->|"saves data"| DF
    TD -->|"saves data"| DF
    CP -->|"saves data"| DF
    ME -->|"saves data"| DF

    %% Intermediate Storage feeds into the Database Manager
    DF -->|"processed data"| BD

    %% Styles
    classDef external fill:#ffb3b3,stroke:#333,stroke-width:2px;
    classDef extraction fill:#b3e6ff,stroke:#333,stroke-width:2px;
    classDef control fill:#b3ffb3,stroke:#333,stroke-width:2px;
    classDef persistence fill:#ffd9b3,stroke:#333,stroke-width:2px;
    classDef dataFiles fill:#ffffcc,stroke:#333,stroke-width:2px;

    %% Click Events for Data Extraction Modules
    click BA "https://github.com/adeniltonr/python-com-banco-de-dados/blob/main//buscar_arrecadacao_atual.py"
    click TD "https://github.com/adeniltonr/python-com-banco-de-dados/blob/main//extrair_dados_top100_doadores.py"
    click CP "https://github.com/adeniltonr/python-com-banco-de-dados/blob/main//copiar_conteudo_html_top_100_doadores.py"
    click ME "https://github.com/adeniltonr/python-com-banco-de-dados/blob/main//extrair_dados_mapa_doacao_por_estados.py"
    
    %% Click Event for Database Module
    click BD "https://github.com/adeniltonr/python-com-banco-de-dados/blob/main//bd.py"
    
    %% Click Event for Main Controller
    click MC "https://github.com/adeniltonr/python-com-banco-de-dados/blob/main//main.py"
    
    %% Click Event for Data Files folder and its files
    click DF "https://github.com/adeniltonr/python-com-banco-de-dados/tree/main//dados"
    click AET "https://github.com/adeniltonr/python-com-banco-de-dados/blob/main//dados/arrecadacoes_estados.txt"
    click BT "https://github.com/adeniltonr/python-com-banco-de-dados/blob/main//dados/boletos.txt"
    click CT "https://github.com/adeniltonr/python-com-banco-de-dados/blob/main//dados/conteudo_para_extracao_top100doadores.txt"
    click DE "https://github.com/adeniltonr/python-com-banco-de-dados/blob/main//dados/dados_estados.json"
    click RT "https://github.com/adeniltonr/python-com-banco-de-dados/blob/main//dados/resultado_top_100_doadores.txt"
    
    %% Click Event for External Libraries file
    click EL "https://github.com/adeniltonr/python-com-banco-de-dados/blob/main//requirements.txt"
```
