# fastfilter/fastfilter_cpp architecture

> Architecture diagram of the GitHub repository fastfilter/fastfilter_cpp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fastfilter/fastfilter_cpp
- Repository: https://github.com/fastfilter/fastfilter_cpp

## Overview

##### Project Type and Purpose
- This is a research library focused on implementing and benchmarking various approximate membership filters.
- Its primary role is to provide different filter implementations (with SIMD, AVX2, etc.) and to compare their performance using benchmarks that run on Linux.
- Additionally, there are unit tests to verify correctness.

##### File Structure
- The top-level directories include:
- .github – Contains CI-related workflows.
- benchmarks – Contains benchmarks (Makefile, scripts, and performance-capturing source files).
- src – Holds the source for various filter implementations organized by filter type (bloom, cuckoo, gcs, gqf, morton, prefix, ribbon, tc-shortcut, vqf, xorfilter).
- tests – Contains unit tests for correctness.
- Notice that many implementations in “src” share common utilities like “src/hashutil.h”, which implies a common dependency across different filter types.

##### Core Components and Their Relationships
a. Filter Implementations:
- Each subdirectory under src (bloom, cuckoo, gqf, etc.) represents a distinct filter algorithm implementation.
b. Benchmarking Suite:
- The benchmarks directory includes various scripts, Makefiles, and source files to run large-scale tests (bulk insert and query, performance tests using Linux performance counters, etc.). Represent this as the benchmarking/testing component that interacts with the filter implementations.
c. Testing Modules:
- The tests directory contains unit tests to verify the correctness of the filters.
d. CI/CD:
- Although not the focus of the functional system, note that the .github directory and associated workflows indicate continuous integration, which builds and tests the project automatically.
e. External Dependencies and Environment:
- The project expects a C++11 environment with a compiler that supports AVX2. Also, Linux-specific performance counters are used.
- An “Environment” or “System Requirements” component (for hardware features and OS configuration) that impacts how benchmarks and filter implementations work.

##### Architectural Patterns and Design Principles
- Modular Design: Each filter type is implemented in its own module within src.
- Shared Libraries/Utilities: There is a common dependency (e.g., hashutil) used by many modules.
- Benchmark-Driven Testing: There is a clear separation between the core functionality (filter algorithms) and the benchmarking framework that tests performance and correctness.
- CI/CD Integration: The existence of GitHub workflows for CI ensures constant validation of code changes.

##### Final Recommendations
- Be very detailed: Ensure every filter type and its dependencies are represented.
- Use clear labels that specify component names, functions (e.g., “Insert”, “Query”), and data flow directions.
- Inter-dependencies such as common configuration files (Makefiles), CI workflows, and testing frameworks.

## Diagram (Mermaid)

```mermaid
graph TD
    subgraph "Filter Implementations"
        bloom("Bloom Filter"):::filter
        cuckoo("Cuckoo Filter"):::filter
        gcs("GCS Filter"):::filter
        gqf("GQF Filter"):::filter
        morton("Morton Filter"):::filter
        prefix("Prefix Filter"):::filter
        ribbon("Ribbon Filter"):::filter
        tcshort("TC-Shortcut Filter"):::filter
        vqf("VQF Filter"):::filter
        xorfilter("XOR Filter"):::filter
    end

    shared("Shared Utilities"):::utility
    benchmark("Benchmarking Suite"):::benchmark
    test("Testing Module"):::testing
    cicd("CI/CD Pipeline"):::cicd
    env("Target Environment"):::env

    %% Benchmarking Suite to Filter Implementations
    benchmark -->|"executes"| bloom
    benchmark -->|"executes"| cuckoo
    benchmark -->|"executes"| gcs
    benchmark -->|"executes"| gqf
    benchmark -->|"executes"| morton
    benchmark -->|"executes"| prefix
    benchmark -->|"executes"| ribbon
    benchmark -->|"executes"| tcshort
    benchmark -->|"executes"| vqf
    benchmark -->|"executes"| xorfilter

    %% Testing Module to Filter Implementations
    test -->|"tests"| bloom
    test -->|"tests"| cuckoo
    test -->|"tests"| gcs
    test -->|"tests"| gqf
    test -->|"tests"| morton
    test -->|"tests"| prefix
    test -->|"tests"| ribbon
    test -->|"tests"| tcshort
    test -->|"tests"| vqf
    test -->|"tests"| xorfilter

    %% Filter Implementations dependency on Shared Utilities
    bloom -->|"uses"| shared
    cuckoo -->|"uses"| shared
    gcs -->|"uses"| shared
    gqf -->|"uses"| shared
    morton -->|"uses"| shared
    prefix -->|"uses"| shared
    ribbon -->|"uses"| shared
    tcshort -->|"uses"| shared
    vqf -->|"uses"| shared
    xorfilter -->|"uses"| shared

    %% CI/CD Pipeline interactions
    cicd -->|"triggers"| benchmark
    cicd -->|"triggers"| test
    cicd -->|"builds"| bloom
    cicd -->|"builds"| cuckoo
    cicd -->|"builds"| gcs
    cicd -->|"builds"| gqf
    cicd -->|"builds"| morton
    cicd -->|"builds"| prefix
    cicd -->|"builds"| ribbon
    cicd -->|"builds"| tcshort
    cicd -->|"builds"| vqf
    cicd -->|"builds"| xorfilter

    %% Target Environment influences
    env -->|"configures"| benchmark
    env -->|"configures"| bloom
    env -->|"configures"| cuckoo
    env -->|"configures"| gcs
    env -->|"configures"| gqf
    env -->|"configures"| morton
    env -->|"configures"| prefix
    env -->|"configures"| ribbon
    env -->|"configures"| tcshort
    env -->|"configures"| vqf
    env -->|"configures"| xorfilter

    %% Click Events
    click bloom "https://github.com/fastfilter/fastfilter_cpp/blob/master/src/bloom/bloom.h"
    click cuckoo "https://github.com/fastfilter/fastfilter_cpp/blob/master/src/cuckoo/cuckoofilter.h"
    click gcs "https://github.com/fastfilter/fastfilter_cpp/blob/master/src/gcs/gcs.h"
    click gqf "https://github.com/fastfilter/fastfilter_cpp/blob/master/src/gqf/gqf.h"
    click morton "https://github.com/fastfilter/fastfilter_cpp/blob/master/src/morton/morton_filter.h"
    click prefix "https://github.com/fastfilter/fastfilter_cpp/blob/master/src/prefix/min_pd256.hpp"
    click ribbon "https://github.com/fastfilter/fastfilter_cpp/blob/master/src/ribbon/ribbon_alg.h"
    click tcshort "https://github.com/fastfilter/fastfilter_cpp/blob/master/src/tc-shortcut/tc-shortcut.hpp"
    click vqf "https://github.com/fastfilter/fastfilter_cpp/blob/master/src/vqf/vqf_filter.h"
    click xorfilter "https://github.com/fastfilter/fastfilter_cpp/blob/master/src/xorfilter/xorfilter.h"
    click shared "https://github.com/fastfilter/fastfilter_cpp/blob/master/src/hashutil.h"
    click benchmark "https://github.com/fastfilter/fastfilter_cpp/tree/master/benchmarks"
    click test "https://github.com/fastfilter/fastfilter_cpp/blob/master/tests/unit.cc"
    click cicd "https://github.com/fastfilter/fastfilter_cpp/blob/master/.github/workflows/ubuntu.yml"

    %% Styles
    classDef filter fill:#AED6F1,stroke:#1F618D,stroke-width:2px;
    classDef utility fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
    classDef benchmark fill:#D5F5E3,stroke:#1E8449,stroke-width:2px;
    classDef testing fill:#FADBD8,stroke:#C0392B,stroke-width:2px;
    classDef cicd fill:#D6DBDF,stroke:#566573,stroke-width:2px;
    classDef env fill:#D2B4DE,stroke:#6C3483,stroke-width:2px;
```
