# polydbms/sheetreader-duckdb architecture

> Architecture diagram of the GitHub repository polydbms/sheetreader-duckdb, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/polydbms/sheetreader-duckdb
- Repository: https://github.com/polydbms/sheetreader-duckdb

## Overview

##### Project Type and Purpose
- Project Type: DuckDB extension
- Purpose: To enable DuckDB to read XLSX files efficiently using the SheetReader parser. The extension wraps sheetreader-core functionalities into a DuckDB function, which users can install and load to query Excel files directly.

##### File Structure
- Key Top-Level Components:
- Directories such as src, docs, benchmarks, test, and scripts indicate areas for source code, documentation, performance evaluation, testing, and deployment utilities.
- The “src” folder contains the core C++ code (both header files in src/include and source file(s) in src) that implement the extension interface.
- The “duckdb” directory and files like CMakeLists.txt, Makefile, and vcpkg.json suggest that this extension is built along with DuckDB using CMake and supports external dependency management.
- The .devcontainer, .github, and .vscode directories help ensure consistency across development and CI environments, while benchmarks provide performance insight.

##### Key System Components
a. DuckDB Core:
- DuckDB engine that would load and execute SQL queries.
- Binary shell (./build/release/duckdb) which auto-loads the extension.

b. SheetReader DuckDB Extension:
- Extension library (sheetreader.duckdb_extension) that hooks into DuckDB.
- Integration point where the extension registers the sheetreader() SQL function.

c. SheetReader Core:
- Although not directly embedded in this repo, indicate the dependency on the sheetreader-core (from the external GitHub repository) that handles the actual XLSX parsing.

d. Build and CI Components:
- Brief areas for build scripts (Makefile, CMake instructions) and CI pipelines from the .github/workflows. This is useful to understand automated build and testing flows.

e. Testing & Benchmarking:
- How the tests (in the test directory and benchmark notebooks/data) validate and measure performance of the extension.

##### Relationships and Interactions
- That user SQL commands in DuckDB trigger a call to the sheetreader() function, which is implemented in the extension.
- Optionally, include interactions with build tools (CMake, Make, ninja) that result in the final binaries.
- Present CI/CD components (GitHub workflows) that automate testing and build for the extension.

##### Architectural Patterns and Design Principles
- Modular Separation: Clearly separate concerns by differentiating between the database engine (DuckDB), the extension layer (the C++ components exposed in src), and the XLSX parsing logic (SheetReader core).
- Plugin Architecture: Emphasize how the extension plugs into DuckDB as an add-on—this modular design allows easy replacement or extension of core functionality.
- External Integration: Indicate the integration with external technologies (sheetreader-core library, containerized development via .devcontainer, dependency management via vcpkg).

##### Final Assembly
- At the top, place “User SQL Query” entering the system.
- Branch off from the extension to the SheetReader core, and show the result (data table) returning through the extension back to DuckDB.
- In a side panel or a secondary layer, include the build configuration and CI/CD pipelines, connecting them to the development and build artifacts.
- That all components, relationships, and technology references (e.g., CMake, GitHub workflows, Docker development environment) are clearly annotated.

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "Runtime Data Flow"
        U["User SQL Query"]:::user
        D["DuckDB Engine"]:::db
        E["SheetReader DuckDB Extension"]:::extension
        S["SheetReader Core"]:::core
        R["Query Result"]:::result
        U -->|"submits"| D
        D -->|"loads"| E
        E -->|"calls-parser"| S
        S -->|"returns-data"| E
        E -->|"outputs"| D
        D -->|"returns-table"| R
        R -->|"displayed"| U
    end

    subgraph "Build & CI Pipeline"
        B["Build & CI Pipeline"]:::ci
    end
    B -->|"builds"| D

    subgraph "Test & Benchmark"
        T["Test & Benchmark"]:::testbench
    end
    T -->|"validates"| E

    %% Click Events
    click D "https://github.com/polydbms/sheetreader-duckdb/tree/main/duckdb"
    click E "https://github.com/polydbms/sheetreader-duckdb/blob/main/src/sheetreader_extension.cpp"
    click E "https://github.com/polydbms/sheetreader-duckdb/blob/main/src/include/sheetreader_extension.hpp"
    click S "https://github.com/polydbms/sheetreader-duckdb/tree/main/src/include/sheetreader-core"
    click B "https://github.com/polydbms/sheetreader-duckdb/blob/main/CMakeLists.txt"
    click B "https://github.com/polydbms/sheetreader-duckdb/tree/main/Makefile"
    click B "https://github.com/polydbms/sheetreader-duckdb/blob/main/extension_config.cmake"
    click B "https://github.com/polydbms/sheetreader-duckdb/tree/main/.github/workflows"
    click T "https://github.com/polydbms/sheetreader-duckdb/tree/main/test"
    click T "https://github.com/polydbms/sheetreader-duckdb/tree/main/benchmarks"

    %% Styles
    classDef user fill:#ffddd2,stroke:#333,stroke-width:2px;
    classDef db fill:#a2d2ff,stroke:#333,stroke-width:2px;
    classDef extension fill:#ffb4a2,stroke:#333,stroke-width:2px;
    classDef core fill:#fdffb6,stroke:#333,stroke-width:2px;
    classDef result fill:#caffbf,stroke:#333,stroke-width:2px;
    classDef ci fill:#caffbf,stroke:#333,stroke-width:2px;
    classDef testbench fill:#ffcad4,stroke:#333,stroke-width:2px;
```
