# dropbox/rust-brotli-decompressor architecture

> Architecture diagram of the GitHub repository dropbox/rust-brotli-decompressor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dropbox/rust-brotli-decompressor
- Repository: https://github.com/dropbox/rust-brotli-decompressor

## Overview

##### Project type and purpose
- This is a Rust library crate (“rust-brotli-decompressor”) whose core functionality is a direct, no-stdlib port of Google’s Brotli decompressor.
- It provides:
- A Rust-native API (std/no-std) for streaming and one-shot decompression.
- A C FFI layer (cdylib) matching the original C Brotli API.
- Command-line/test binaries, example code, fuzz tests, and integration tests.

##### File structure
Top-level:
- Cargo.toml, README.md, LICENSE, CI config (.travis.yml) – crate metadata and build pipeline.
- c/ – C FFI directory containing a Makefile and C example (main.c). Builds a cdylib for C linkage.
- src/ – library code, subdivided into:
– bin/ – multiple Rust binaries (CLI decompressor, tests harness, error tests).
– modules: bit_reader, context, decode, dictionary, ffi, huffman, memory, prefix, reader, state, transform, writer, plus helper files (io_wrappers.rs, brotli_alloc.rs).
– lib.rs – top-level entrypoint; re-exports public API.
- examples/ – standalone Rust example (decompress.rs).
- fuzz/ – Cargo project with fuzz targets that import the library and run fuzzing on decompression routines.
- testdata/ – corpus of sample files for tests and integration.
- c/Makefile – builds the C example and the cdylib.

##### Key architectural notes from README
- Dual build modes: with-std (default) vs. no-stdlib (feature-gated).
- FFI exports are feature-gated (“ffi-api”).
- Custom allocator support for kernel or no-syscall environments.
- Supports streaming (BrotliDecompressStream) and one-shot (BrotliDecoderDecompress).

##### Main components
A. Core Rust Decompressor Library
- Internal modules: bit_reader, huffman, dictionary, prefix, transform, decode, context, state, memory.
- Public API: Decompressor (io::Read), BrotliDecompress (stream copy), low-level BrotliDecompressStream.
- Feature flags: std, no-stdlib, ffi-api.
B. Rust CLI & Test Binaries
- src/bin/brotli-decompressor.rs – CLI entrypoint.
- src/bin/error_handling_tests.rs, integration_tests.rs, tests.rs – test harnesses using testdata.
C. Examples & Fuzzing
- examples/decompress.rs – plain-Rust usage demo.
- fuzz/fuzz_targets/*.rs – cargo-fuzz harness targets.
D. C FFI Layer
- c/ffi (in Rust src/ffi) – binds to core library.
- c/main.c – C example illustrating BrotliDecoderStream API.
- c/Makefile – compiles cdylib and example.
E. Testdata & CI
- testdata/ – decompression inputs for integration tests.
- .travis.yml – CI pipeline runs cargo test, fuzz, and C build.

##### Relationships and data flows
- Rust CLI binaries → call public API in core library → modules decode data → writer outputs decompressed bytes.
- examples/decompress.rs → uses the same public API.
- fuzz targets → consume public API internals to feed random inputs and catch panics.
- C example (main.c) → links against cdylib built from Rust FFI layer → calls BrotliDecoderStream (Rust implementation).
- Tests (src/bin/integration_tests.rs) → read from testdata/ → invoke library → compare with expected outputs.

##### Architectural patterns & design principles
- Layered modular decomposition: low-level bit readers → Huffman & LZ77 decode → transform + dictionary → high-level streaming API.
- Feature gating for optional components (ffi-api, no-stdlib).
- Zero-cost FFI design: cdylib exports match C symbols exactly.
- Separation of concerns: core library vs. CLI vs. FFI vs. test harnesses vs. examples vs. fuzzers.

##### Diagramming conventions
- From CLI/Examples/Fuzz → Core Library (calls).
- From Core Library → Testdata (reads).
- From C example → FFI Layer → Core Library.
- Library core = blue
- CLI/apps = green
- FFI/C = orange
- Tests/fuzz = purple
- Build Tools = gray

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Core Library Layers
    subgraph "Core Decompressor Library"
        direction TB
        API["Public API\n(lib.rs)"]:::core
        subgraph "Reader Layer"
            direction TB
            BR["bit_reader"]:::core
            IO["io_wrappers"]:::core
        end
        subgraph "Decode Layer"
            direction TB
            HU["huffman"]:::core
            DI["dictionary"]:::core
            DE["decode"]:::core
        end
        subgraph "Transform Layer"
            direction TB
            PR["prefix"]:::core
            TR["transform"]:::core
        end
        subgraph "Context & State Layer"
            direction TB
            CO["context"]:::core
            ST["state"]:::core
        end
        subgraph "Memory & Allocator"
            direction TB
            ME["memory"]:::core
            BA["brotli_alloc"]:::core
        end
        subgraph "Output Layer"
            direction TB
            RE["reader"]:::core
            WR["writer"]:::core
        end
    end

    %% Feature‐gated FFI exports
    subgraph "Feature-gated FFI exports"
        direction TB
        FI["ffi module root"]:::ffi
        IF["interface"]:::ffi
        AU["alloc_util"]:::ffi
    end

    %% Rust CLI & Test Binaries
    subgraph "Rust CLI & Test Binaries"
        direction TB
        CLI["brotli-decompressor"]:::cli
        EH["error_handling_tests"]:::cli
        IT["integration_tests"]:::cli
        UT["tests"]:::cli
        IPS1["ipsum.brotli"]:::cli
        IPS2["ipsum.raw"]:::cli
        IPSG["ipsum_gen.py"]:::cli
    end

    %% Rust Example
    subgraph "Rust Example"
        direction TB
        EX["decompress.rs"]:::example
    end

    %% Fuzz Harness
    subgraph "Fuzz Harness"
        direction TB
        FC["fuzz Cargo.toml"]:::fuzz
        FT1["decompress_into_fixed.rs"]:::fuzz
        FT2["decompress_into_vec.rs"]:::fuzz
    end

    %% C FFI Layer
    subgraph "C FFI Layer"
        direction TB
        CARGO["c/Cargo.toml"]:::cffi
        MAKE["c/Makefile"]:::cffi
        MAIN["main.c"]:::cffi
        HDR1["decode.h"]:::cffi
        HDR2["port.h"]:::cffi
        HDR3["types.h"]:::cffi
    end

    %% Testdata & CI
    subgraph "Testdata & CI"
        direction TB
        TD["testdata/"]:::test
        CI[".travis.yml"]:::test
    end

    %% Build Tools
    Cargo["Cargo"]:::tool
    Make["Make"]:::tool

    %% Relationships
    CLI -->|calls| API
    EH -->|uses| API
    IT -->|uses| API
    UT -->|uses| API
    EX -->|calls| API
    FT1 -->|fuzzes| API
    FT2 -->|fuzzes| API

    FI -->|builds into| API
    IF --> FI
    AU --> FI

    CARGO -->|builds cdylib| FI
    MAKE -->|invokes| CARGO
    MAIN -->|links to| FI
    MAIN -->|includes headers| HDR1
    MAIN -->|includes headers| HDR2
    MAIN -->|includes headers| HDR3

    IT -->|reads| TD

    %% Build tools relation
    Cargo -.->|builds| CLI
    Cargo -.->|builds| EX
    Cargo -.->|builds| FC

    Make -.->|builds| MAIN

    %% Click Events
    click API "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/lib.rs"
    click BR "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/bit_reader/mod.rs"
    click IO "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/io_wrappers.rs"
    click HU "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/huffman/mod.rs"
    click DI "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/dictionary/mod.rs"
    click DE "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/decode.rs"
    click PR "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/prefix.rs"
    click TR "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/transform.rs"
    click CO "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/context.rs"
    click ST "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/state.rs"
    click ME "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/memory.rs"
    click BA "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/brotli_alloc.rs"
    click RE "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/reader.rs"
    click WR "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/writer.rs"
    click FI "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/ffi/mod.rs"
    click IF "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/ffi/interface.rs"
    click AU "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/ffi/alloc_util.rs"
    click CLI "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/bin/brotli-decompressor.rs"
    click EH "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/bin/error_handling_tests.rs"
    click IT "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/bin/integration_tests.rs"
    click UT "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/bin/tests.rs"
    click IPS1 "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/bin/ipsum.brotli"
    click IPS2 "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/bin/ipsum.raw"
    click IPSG "https://github.com/dropbox/rust-brotli-decompressor/blob/master/src/bin/ipsum_gen.py"
    click EX "https://github.com/dropbox/rust-brotli-decompressor/blob/master/examples/decompress.rs"
    click FC "https://github.com/dropbox/rust-brotli-decompressor/blob/master/fuzz/Cargo.toml"
    click FT1 "https://github.com/dropbox/rust-brotli-decompressor/blob/master/fuzz/fuzz_targets/decompress_into_fixed.rs"
    click FT2 "https://github.com/dropbox/rust-brotli-decompressor/blob/master/fuzz/fuzz_targets/decompress_into_vec.rs"
    click CARGO "https://github.com/dropbox/rust-brotli-decompressor/blob/master/c/Cargo.toml"
    click MAKE "https://github.com/dropbox/rust-brotli-decompressor/tree/master/c/Makefile"
    click MAIN "https://github.com/dropbox/rust-brotli-decompressor/blob/master/c/main.c"
    click HDR1 "https://github.com/dropbox/rust-brotli-decompressor/blob/master/c/brotli/decode.h"
    click HDR2 "https://github.com/dropbox/rust-brotli-decompressor/blob/master/c/brotli/port.h"
    click HDR3 "https://github.com/dropbox/rust-brotli-decompressor/blob/master/c/brotli/types.h"
    click TD "https://github.com/dropbox/rust-brotli-decompressor/tree/master/testdata/"
    click CI "https://github.com/dropbox/rust-brotli-decompressor/blob/master/.travis.yml"

    %% Styles
    classDef core fill:#cce5ff,stroke:#4d89f9,color:#000
    classDef cli fill:#d4edda,stroke:#28a745,color:#000
    classDef example fill:#e2e3e5,stroke:#6c757d,color:#000
    classDef fuzz fill:#f5e1ff,stroke:#b366ff,color:#000
    classDef cffi fill:#ffe5cc,stroke:#ff8800,color:#000
    classDef test fill:#f8d7da,stroke:#dc3545,color:#000
    classDef tool fill:#e2e3e5,stroke:#6c757d,color:#000
```
