# fireducks-dev/fireducks architecture

> Architecture diagram of the GitHub repository fireducks-dev/fireducks, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fireducks-dev/fireducks
- Repository: https://github.com/fireducks-dev/fireducks

## Overview

##### Main Components
- • Core FireDucks Library: The heart of the project that implements DataFrame operations with compiler acceleration. This is the “engine” similar to pandas but optimized for speed.
- • Import Hook Mechanism: A component that intercepts “import pandas” statements and swaps them for FireDucks. This serves as a seamless integration point for legacy code.
- • Explicit Import Module: An alternative entry point where users directly import FireDucks’ pandas-compatible API.
- • Underlying Compiler/Execution Engine: Although not exposed in detail in the repository, indicate that FireDucks likely leverages compiler acceleration (potentially using native code or another high-performance engine) to optimize DataFrame operations.
- • Dependency Integration: Include external libraries such as pyarrow (noting the version upgrade in the README), and potentially other dependencies that integrate with the accelerated engine.
- • Demo/Notebooks: These components (found in the notebooks directory) are used as interactive examples that demonstrate how to use the library, comparing its performance and API behavior with pandas and other libraries like cudf/polars.

##### Relationships and Interactions
- a) The Import Hook Flow where the Python interpreter, when running “python3 -m fireducks.pandas your_script.py”, automatically replaces pandas with FireDucks. This flow should demonstrate how the hook intercepts module loading.
- b) The Explicit Import Flow where the user explicitly imports “fireducks.pandas” in their code.
- • Integration with Other Libraries: Identify where and how dependencies like pyarrow may interface with the core engine (for example, for efficient columnar data processing).

##### Architectural Patterns and Design Principles
- • API Compatibility and Transparency: Emphasize that the design maintains pandas API compatibility while injecting performance improvements; hence, a “wrapper” or “substitution” design pattern is at work.
- • Modular and Layered Approach: The separation between the import hook mechanism, the user-facing API, and the core acceleration engine is critical. This could be depicted in distinct layers.
- • Extensibility: Although not detailed in the tree, consider a placeholder or annotation for potential extensibility points (for example, if additional optimizations or backends are supported in the future).

## Diagram (Mermaid)

```mermaid
flowchart TD
    U["User Application Code"]:::user

    subgraph "API Layer"
        IH["Import Hook (Adapter Pattern)"]:::api
        EA["Explicit API Module (Facade Pattern)"]:::api
    end

    U --> IH
    U --> EA

    subgraph "Core Engine"
        CFD["Core FireDucks Library"]:::engine
        CE["Underlying Compiler/Execution Engine"]:::engine2
        DI["Dependency Integration (pyarrow)"]:::dep
        CFD --> CE
        CFD --> DI
    end

    IH -->|"calls"| CFD
    EA -->|"calls"| CFD
    U -->|"DataFrame operations"| CFD

    subgraph "Example / Demo Clients"
        DNB["Demo/Notebooks"]:::demo
        NED["NYC Demo - Explicit Import"]:::demo
        NID["NYC Demo - Import Hook"]:::demo
    end

    DNB -->|"triggers"| U
    NED -->|"triggers"| U
    NID -->|"triggers"| U

    click DNB "https://github.com/fireducks-dev/fireducks/tree/main/notebooks"
    click NED "https://github.com/fireducks-dev/fireducks/blob/main/notebooks/nyc_demo/fireducks_pandas_nyc_demo.ipynb"
    click NID "https://github.com/fireducks-dev/fireducks/blob/main/notebooks/nyc_demo/cudf_pandas_nyc_demo.ipynb"

    classDef user fill:#AED6F1,stroke:#1B4F72,stroke-width:2px;
    classDef api fill:#A9DFBF,stroke:#1E8449,stroke-width:2px;
    classDef engine fill:#FAD7A0,stroke:#D35400,stroke-width:2px;
    classDef engine2 fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
    classDef dep fill:#D7BDE2,stroke:#8E44AD,stroke-width:2px;
    classDef demo fill:#F5B7B1,stroke:#C0392B,stroke-width:2px;
```
