# haskell/text architecture

> Architecture diagram of the GitHub repository haskell/text, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/haskell/text
- Repository: https://github.com/haskell/text

## Overview

##### Project type and purpose
- This is a Haskell library (“text”) that provides efficient, Unicode‐aware text operations. It is *not* a web application but a systems library with both Haskell and C/C++ (via FFI) components, plus test and benchmark harnesses.

##### File structure
- Top‐level
- src/                         — Haskell source organized into public API (Data.Text, Data.Text.Lazy, etc.) and internal modules
- cbits/                       — C and C++ implementations (measure_off.c, validate_utf8.cpp, and the bundled simdutf library)
- scripts/                     — Haskell code generators (e.g., CaseFolding, SpecialCasing) and a regenerate.sh driver
- benchmarks/                  — Benchmark definitions and data harnesses
- tests/                       — Test suites (unit, QuickCheck properties, shell scripts)
- Setup.lhs + text.cabal       — Cabal build instructions
- .github/workflows/…          — CI pipelines for various platforms

##### README for additional insights
- Core feature: space/time‐efficient Unicode text operations
- Benchmarks and tests are first‐class citizens
- Uses stream fusion and UTF‐8 under the hood

##### Capturing essential details
- Don’t stop at “src” vs. “cbits”—break src into API vs. internal subpackages (IO, Builder, Fusion, Encoding).
- That scripts regenerate code automatically as part of the build.
- Bundled simdutf library as an external‐but‐included component.
- How benchmarks rely on external test data (benchmarks/text-test-data).

- What each part of the library does
- How they depend on each other
- Where performance is offloaded to native code
- How testing and benchmarking fit into the life cycle
- How CI/CD orchestrates builds on different targets

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Public API Layer
    subgraph "Public API Layer"
        direction TB
        API1["Data.Text.hs"]:::haskell
        API2["Data.Text.Lazy.hs"]:::haskell
    end

    %% Internal Implementation Layer
    subgraph "Internal Implementation Layer"
        direction TB
        INT["Data.Text.Internal/"]:::haskell
    end

    %% FFI Layer
    subgraph "FFI Layer"
        direction TB
        Cbits["cbits/"]:::native
        Simd["simdutf/"]:::native
    end

    %% Build System
    subgraph "Build System"
        direction TB
        Cabal["text.cabal"]:::build
        Setup["Setup.lhs"]:::build
        Scripts["scripts/"]:::build
    end

    %% Test Suite
    subgraph "Test Suite"
        direction TB
        Tests["tests/"]:::test
    end

    %% Benchmark Suite
    subgraph "Benchmark Suite"
        direction TB
        Benches["benchmarks/haskell/"]:::test
        TestData["benchmarks/text-test-data"]:::test
    end

    %% CI/CD Workflows
    subgraph "CI/CD Workflows"
        direction TB
        CI[".github/workflows/"]:::build
    end

    %% Relationships
    API1 -->|"calls →"| INT
    API2 -->|"calls →"| INT
    INT -->|"FFI →"| Cbits
    INT -->|"FFI →"| Simd
    Cabal -->|"builds →"| API1
    Cabal -->|"builds →"| API2
    Cabal -->|"builds →"| INT
    Cabal -->|"builds →"| Cbits
    Setup -->|"bootstrap →"| Cabal
    Scripts -->|"generates →"| API1
    Scripts -->|"generates →"| INT
    Tests -->|"depends →"| Cabal
    Tests -->|"executes →"| API1
    Tests -->|"executes →"| INT
    Benches -->|"depends →"| Cabal
    Benches -->|"uses →"| TestData
    CI -->|"executes →"| Cabal
    CI -->|"executes →"| Tests
    CI -->|"executes →"| Benches

    %% Click Events
    click API1 "https://github.com/haskell/text/blob/master/src/Data/Text.hs"
    click API2 "https://github.com/haskell/text/blob/master/src/Data/Text/Lazy.hs"
    click INT "https://github.com/haskell/text/tree/master/src/Data/Text/Internal"
    click Cbits "https://github.com/haskell/text/tree/master/cbits"
    click Simd "https://github.com/haskell/text/tree/master/simdutf"
    click Cabal "https://github.com/haskell/text/blob/master/text.cabal"
    click Setup "https://github.com/haskell/text/blob/master/Setup.lhs"
    click Scripts "https://github.com/haskell/text/tree/master/scripts"
    click Tests "https://github.com/haskell/text/blob/master/tests/Tests.hs"
    click Benches "https://github.com/haskell/text/blob/master/benchmarks/haskell/Benchmarks.hs"
    click CI "https://github.com/haskell/text/blob/master/.github/workflows/haskell-ci.yml"

    %% Styles
    classDef haskell fill:#ADD8E6,stroke:#000
    classDef native fill:#C8E6C9,stroke:#000
    classDef build fill:#FFE0B2,stroke:#000
    classDef test fill:#CFD8DC,stroke:#000
```
