# jerbouma/financedatabase architecture

> Architecture diagram of the GitHub repository jerbouma/financedatabase, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jerbouma/financedatabase
- Repository: https://github.com/jerbouma/financedatabase

## Overview

##### Project Type and Purpose
- This project is a Python package providing a free, community‐driven financial database. It organizes financial product categorizations (e.g., Equities, ETFs, Funds, Indices, Currencies, Cryptocurrencies, Money Markets) into CSV files and compressed data formats. It supplies an API (via modules like Equities.py, ETFs.py, etc.) that makes it easy for users to select, search, and aggregate ticker data. Users can also seamlessly integrate with related projects such as the Finance Toolkit.
- The README emphasizes that while it does not provide live stock data itself (which can be obtained from other sources), it gives organized financial product information to aid deeper financial analytics.

##### Detailed File Structure Analysis
- The top-level directories show good separation of concerns:
- .github/: Contains funding info, issue templates, and CI/CD workflows (e.g., linting, database update). This suggests an emphasis on quality checks and automated maintenance.
- compression/: Holds compressed data files, JSON category files, and even a Jupyter Notebook (compression.ipynb) that likely demonstrates how to inspect or use the compressed datasets.
- database/: Contains CSV files for various asset classes (cryptos.csv, equities.csv, etc.), which are the raw/canonical data sources.
- financedatabase/: Contains the core Python modules. Each asset class has a separate module (Equities.py, ETFs.py, etc.) plus helper routines and an __init__.py to manage package exposure. This modular internal structure is indicative of a design that encapsulates business logic per product category.
- examples/: Provides several Jupyter Notebooks showcasing different usage patterns (from basic queries to advanced analyses such as Dupont Analysis or technical analysis charts).
- The presence of pyproject.toml, requirements.txt, pre-commit, and CONTRIBUTING.md further emphasize good practices and ease for developers to build, contribute, and run tests.

##### Main Components
- Data Source / Storage Layer:
- Database CSV files (in the database/ folder).
- Compressed data files along with category JSON and Excel files (in compression/).
- Business Logic / API Layer:
- The “financedatabase” package (modules such as Equities.py, ETFs.py, Cryptos.py, etc.) that reads the raw CSV/compressed files.
- Shared helper functions found in helpers.py.
- This layer provides functions for “select”, “search” and “options” to query the database.
- Integration Layer:
- Interfaces provided (e.g., to_toolkit functionality) that allow users to pass DataFrames or ticker lists to external systems like FinanceToolkit.
- User/Application Layer:
- Examples and notebooks from the examples/ directory.
- Scripts that utilize the package for financial analysis (including interactions like plotting recommendations, each showing how selections and transformations lead to analytics).
- DevOps/Quality Layer:
- CI/CD workflows (in .github/workflows) that automate linting, testing, and database updates.
- Contribution guidelines and license information.

##### Relationships & Interactions
- Data flows upward from the static CSV/compressed files (data sources) to the “financedatabase” API modules that parse the data.
- The API modules expose methods (such as select, options, search) and convert raw data into Pandas DataFrames.
- These DataFrames can then be passed to other utilities – for example, wrapped with FinanceToolkit functionality (via the to_toolkit method) for further financial analyses.
- External systems (like FinanceToolkit or user-facing notebooks) consume the processed data.
- CI/CD workflows trigger on pushes/PRs and are connected to code quality and automated database update scripts.

##### Architectural Patterns & Design Principles
- Modular Design: Different asset classes are encapsulated in separate modules (e.g., Equities.py, ETFs.py) which simplifies adding new financial products.
- Separation of Concerns: Data storage (CSV/JSON/Compressed files) is clearly separated from the logic that performs queries and data transformations.
- Interoperability: The API is designed so that the output (Pandas DataFrames) can be easily consumed by other tools and libraries.
- Community-Driven Extensibility: The presence of plain CSV files and clear contribution guidelines promote ease of modification and community contributions.
- Automated Maintenance: GitHub workflows ensure that quality checks, linting, and even database refresh/update tasks occur regularly, automating maintenance operations.

b. Above that, group related business logic components (the modules under financedatabase/). Inside this group, partition your modules by functional grouping (e.g., “Asset Classes”).
c. Next, illustrate the Integration Layer: show that outputs (Pandas DataFrames) are consumed by user code (in the examples/) and further processed by external tools like FinanceToolkit.
d. On the side or as an overlay, illustrate the DevOps/CI CD layer from your .github/workflows, linking it to the code repository and automation of tests.
f. Label every component clearly and add annotations where necessary.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Source / Storage Layer"
        DS1["Database CSV Files"]:::data
        DS2["Compressed Data & Categories"]:::data
    end

    subgraph "Business Logic / API Layer"
        subgraph "financedatabase Package"
            BL1["Equities.py"]:::business
            BL2["ETFs.py"]:::business
            BL3["Cryptos.py"]:::business
            BL4["Helpers.py"]:::business
            BL5["__init__.py"]:::business
        end
    end

    subgraph "Integration Layer"
        IN1["User Application Examples & Notebooks"]:::integration
        IN2["FinanceToolkit Integration (to_toolkit)"]:::integration
    end

    subgraph "DevOps / Quality Layer"
        DO1["CI/CD Workflows & Quality Checks"]:::devops
        DO2["Contribution Guidelines & Issue Templates"]:::devops
    end

    DS1 -->|"reads/parses"| BL1
    DS1 -->|"reads/parses"| BL2
    DS1 -->|"reads/parses"| BL3
    DS2 -->|"reads/parses"| BL1
    DS2 -->|"reads/parses"| BL2
    DS2 -->|"reads/parses"| BL3

    BL1 -->|"convertsToDataFrame"| IN1
    BL2 -->|"convertsToDataFrame"| IN1
    BL3 -->|"convertsToDataFrame"| IN1
    BL1 -->|"offersMethods"| IN2
    BL2 -->|"offersMethods"| IN2
    BL3 -->|"offersMethods"| IN2

    DO1 ---|"automatesTests"| BL1
    DO1 ---|"automatesTests"| BL2
    DO1 ---|"automatesTests"| BL3
    DO2 ---|"guidesContributions"| BL1
    DO2 ---|"guidesContributions"| BL2
    DO2 ---|"guidesContributions"| BL3

    %% Click Events
    click DS1 "https://github.com/jerbouma/financedatabase/tree/main/database/"
    click DS2 "https://github.com/jerbouma/financedatabase/tree/main/compression/"
    click BL1 "https://github.com/jerbouma/financedatabase/blob/main/financedatabase/Equities.py"
    click BL2 "https://github.com/jerbouma/financedatabase/blob/main/financedatabase/ETFs.py"
    click BL3 "https://github.com/jerbouma/financedatabase/blob/main/financedatabase/Cryptos.py"
    click BL4 "https://github.com/jerbouma/financedatabase/blob/main/financedatabase/helpers.py"
    click BL5 "https://github.com/jerbouma/financedatabase/blob/main/financedatabase/__init__.py"
    click IN1 "https://github.com/jerbouma/financedatabase/tree/main/examples/"
    click DO1 "https://github.com/jerbouma/financedatabase/tree/main/.github/workflows/"
    click DO2 "https://github.com/jerbouma/financedatabase/tree/main/.github/ISSUE_TEMPLATE/"

    %% Styles
    classDef data fill:#cce5ff,stroke:#333,stroke-width:2px;
    classDef business fill:#d5f5e3,stroke:#333,stroke-width:2px;
    classDef integration fill:#f9e79f,stroke:#333,stroke-width:2px;
    classDef devops fill:#f5b7b1,stroke:#333,stroke-width:2px;
```
