# alexpreynolds/sample architecture

> Architecture diagram of the GitHub repository alexpreynolds/sample, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/alexpreynolds/sample
- Repository: https://github.com/alexpreynolds/sample

## Overview

##### Project Type Identification
This is a command-line utility tool focused on performing reservoir sampling on large text files. The architecture should reflect its primary purpose of efficient sampling with minimal memory usage.

##### Main Components

a. Input Processing Layer:
- Regular File Input (emphasize that stdin is not supported)
- mmap-based file access component
- Optional C stdio component (alternative implementation)

b. Core Processing Components:
- Reservoir Sampling Engine
- Mersenne Twister RNG (mt19937)
- Storage Manager (for byte offsets)

c. Output Processing Layer:
- Sample Output Generator
- Order Preservation Component (optional)

**[Input File] → [File Access Layer (mmap/stdio)] →**
- [Position Storage Manager] ↔ [Reservoir Sampling Engine] ↔ [MT19937 RNG] →
- [Sample Output Generator] → [Ordered Output (optional)] → [Final Output]

##### Component Details

a. File Access Layer:
- Two possible paths: mmap (primary) and stdio (secondary)
- Two-pass processing nature
- Byte offset storage instead of full line storage

b. Reservoir Sampling Engine:
- To Position Storage Manager
- Interaction with RNG
- Sampling modes (with/without replacement)

c. Storage Manager:
- 8-byte position storage
- Memory efficiency aspect

##### Important Flows to Highlight

- First Pass: File → mmap → Position Storage → RNG
- Second Pass: Position Storage → mmap → Output Generation

##### Technical Details to Note

- Version control information
- Build/test infrastructure
- File format support (BED, SAM, VCF, etc.)
- Configuration options

##### Implementation Details

- Relationship between:
- src/sample-library/mt19937.c
- src/bin/sample.c
- Files (mt19937.h, sample.h)

##### Performance Characteristics

A sidebar or notes showing:
- 2.25-2.75x faster than shuf for small k
- Memory efficiency metrics
- Trade-offs vs. traditional approaches

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph Input
        IF[Input File]
    end

    subgraph "File Access Layer"
        MMAP["mmap Access"]
        STDIO["stdio Access"]
        IF --> MMAP
        IF --> STDIO
    end

    subgraph "Core Processing"
        PSM[(Position Storage Manager)]
        RSE["Reservoir Sampling Engine"]:::core
        RNG["Mersenne Twister RNG"]:::core
        
        MMAP --> PSM
        STDIO --> PSM
        PSM <--> RSE
        RSE <--> RNG
    end

    subgraph "Output Processing"
        SOG["Sample Output Generator"]
        OPC["Order Preservation"]
        FO["Final Output"]
        
        RSE --> SOG
        SOG --> OPC
        OPC --> FO
    end

    subgraph "Build & Test"
        BC["Build Configuration"]:::build
        TF["Test Framework"]:::test
    end

    %% Click events for component mapping
    click RNG "https://github.com/alexpreynolds/sample/blob/master/src/sample-library/mt19937.c"
    click RSE "https://github.com/alexpreynolds/sample/blob/master/src/bin/sample.c"
    click BC "https://github.com/alexpreynolds/sample/tree/master/makefile"
    click TF "https://github.com/alexpreynolds/sample/blob/master/test/README.md.seed123.txt"

    %% Styling
    classDef core fill:#f9f,stroke:#333,stroke-width:2px
    classDef build fill:#fcf,stroke:#333
    classDef test fill:#cff,stroke:#333

    %% Legend
    subgraph Legend
        L1["Core Components"]:::core
        L2["Build Component"]:::build
        L3["Test Component"]:::test
    end

    %% Notes
    note["Key Features:<br/>- Memory Efficient<br/>- 2.25-2.75x faster than shuf<br/>- No stdin support<br/>- Optional order preservation"]
    style note fill:#fff,stroke:#333,stroke-dasharray: 5 5
```
