# skytwosea/grizzly architecture

> Architecture diagram of the GitHub repository skytwosea/grizzly, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/skytwosea/grizzly
- Repository: https://github.com/skytwosea/grizzly

## Overview

##### Main Components

a) Input Layer:
- File Input Handler (file_tokenizer.zig)
- String Input Handler (string_tokenizer.zig)

b) Parsing Core:
- SIMD-based Parser Components
- Vector Processing Units
- Scalar Parser Fallback
- Escape Character Handlers

c) Processing Pipeline:
- Tokenizer
- Parser Variants
- Sink Interface

**┌─────────────────────────────────────┐**
- Input Sources            │
- (File Stream / String Buffer)     │
- ┘
- ↓
**┌─────────────────────────────────────┐**
- Tokenization Layer          │
- SIMD Search                    │
- Vector Processing              │
- ┘
- ↓
**┌─────────────────────────────────────┐**
- Parser Variants            │
- ┌───────────┐      ┌────────────┐  │
- Vector   │      │  Scalar    │  │
- Parser   │      │  Parser    │  │
- ┘      └────────────┘  │
- ┘
- ↓
**┌─────────────────────────────────────┐**
- Sink Layer               │
- ┘

##### Important Implementation Details to Highlight

a) SIMD Optimization Paths:
- Parallel processing paths
- Vectorized operations
- Scalar fallback paths

b) Parser Variants:
- Vector-based parser
- Scalar parser
- Escaped/Non-escaped handling

##### Performance-Critical Paths
- SIMD search paths
- Memory-efficient processing routes
- Benchmark-critical sections

##### Implementation Notes
- Version/build mode indicators
- Performance metrics at relevant points

##### Special Attention Areas

For this specific CSV parser:
- SIMD optimization paths
- Different parser variants clearly
- Where performance bottlenecks are addressed

Remember to:
- Performance-critical paths
- Both implemented and planned features
- Relevant performance metrics

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Input
        FileInput["File Input Handler"]
        StringInput["String Input Handler"]
    end

    subgraph Tokenization
        SIMD["SIMD Search"]
        VectorProc["Vector Processing"]
    end

    subgraph "Parser Layer"
        direction TB
        ParserVariants["Parser Variants"]
        subgraph "Vector Processing"
            VectorEsc["Vector Parser (with escapes)"]
            VectorNoEsc["Vector Parser (no escapes)"]
        end
        ScalarFallback["Scalar Parser Fallback"]
    end

    subgraph "Output Layer"
        SinkInterface["Sink Interface"]
        ArrowFormat["Future Arrow Format"]:::planned
    end

    FileInput --> SIMD
    StringInput --> SIMD
    SIMD --> VectorProc
    VectorProc --> ParserVariants
    ParserVariants --> VectorEsc
    ParserVariants --> VectorNoEsc
    ParserVariants --> ScalarFallback
    VectorEsc --> SinkInterface
    VectorNoEsc --> SinkInterface
    ScalarFallback --> SinkInterface
    SinkInterface --> ArrowFormat

    %% Component Mappings
    click FileInput "https://github.com/skytwosea/grizzly/blob/main/src/file_tokenizer.zig"
    click StringInput "https://github.com/skytwosea/grizzly/blob/main/src/string_tokenizer.zig"
    click SIMD "https://github.com/skytwosea/grizzly/blob/main/src/simd_search.zig"
    click VectorEsc "https://github.com/skytwosea/grizzly/blob/main/vector_src/vector_esc_parser.zig"
    click VectorNoEsc "https://github.com/skytwosea/grizzly/blob/main/vector_src/vector_noesc_parser.zig"
    click ScalarFallback "https://github.com/skytwosea/grizzly/blob/main/vector_src/scalar_parser.zig"
    click ParserVariants "https://github.com/skytwosea/grizzly/blob/main/src/full_parser.zig"
    click SinkInterface "https://github.com/skytwosea/grizzly/blob/main/src/sink.zig"

    %% Styling
    classDef default fill:#f9f9f9,stroke:#333,stroke-width:2px
    classDef input fill:#b3e0ff,stroke:#333,stroke-width:2px
    classDef processing fill:#98fb98,stroke:#333,stroke-width:2px
    classDef simd fill:#ffeb99,stroke:#333,stroke-width:2px
    classDef planned stroke-dasharray: 5 5

    class FileInput,StringInput input
    class VectorProc,ParserVariants,VectorEsc,VectorNoEsc,ScalarFallback processing
    class SIMD simd
    class ArrowFormat planned

    %% Legend
    subgraph Legend
        L1["Input Components"]:::input
        L2["Processing Components"]:::processing
        L3["SIMD Optimization"]:::simd
        L4["Planned Features"]:::planned
    end
```
