# nietras/sep architecture

> Architecture diagram of the GitHub repository nietras/sep, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nietras/sep
- Repository: https://github.com/nietras/sep

## Overview

This project is a high‐performance .NET library focused solely on parsing and writing separated values (such as CSV, TSV, etc.) – “Sep” positions itself as the world’s fastest .NET CSV parser. Its README immediately reveals that the core purpose is efficiency with low or zero allocations, leveraging modern C# features (e.g. Span<T>, ref structs, generic math, SIMD vectorization, async APIs, and even AOT compatibility) and integrating with performance‐focused libraries like csFastFloat. The file tree shows a multi-project solution that includes:

1. A core library (in the “src/Sep” folder) containing all the parser and writer implementations and public APIs (with types such as Sep, SepReader, and SepWriter, plus supporting options and pooling classes).
2. Multiple test projects (e.g. “src/Sep.Test”, “src/Sep.Tester”, “src/Sep.XyzTest”) that exercise the API and validate correct behavior.
3. Benchmark and comparison suites (e.g. “src/Sep.Benchmarks”, “src/Sep.ComparisonBenchmarks”, and a “benchmarks” directory containing platform‐specific benchmark results) that compare Sep’s performance to established libraries (like CsvHelper and Sylvan).
4. A set of development and automation scripts (.github workflows, PowerShell scripts such as bench.ps1, pack.ps1, publish-tester.ps1, etc.) that set up CI/CD, dependency management, code analysis, and repository policies.

- Core Library Module (“Sep Core”)
– This is the heart of the project. It exposes the primary public API (the “Sep” type, along with extension methods that let clients create a “SepReader” or “SepWriter”). Inside, it implements both reading and writing pipelines for separated values. Include sub-components such as:
▪ Reader API and its configurable options (SepReaderOptions), advanced parsing techniques (e.g. SIMD vectorized routines, pooling strategies, asynchronous and synchronous flows), and internal helper types (ref structs for Row, Col, Cols).
▪ Writer API with its options (SepWriterOptions), the deferred row construction model, and methods to handle escaping.
▪ Integration points with third‐party libraries like csFastFloat for fast numeric parsing.
- Benchmark and Comparison Modules
- Testing Modules
– These include the unit/integration test projects (“Sep.Test”, “Sep.Tester”, “Sep.XyzTest”). They depend on the core library to verify correctness and also serve as sample usage (with the examples shown in the README).
- Automation/DevOps & Configuration

##### Dependencies and interactions
- In the core library module, highlight the flow through the API:
– Client code (or tests/benchmarks) invoke the public “Sep” API that provides “Reader()” and “Writer()” extension methods.
– The Reader then (optionally) auto-detects the separator, manages memory pooling, and returns a stream of “Row” ref structs (with columns – “Col” and “Cols”).
– Similarly, the Writer uses deferred row construction, building rows via new row creation and flushing data only when disposing rows.
– Optionally denote that asynchronous variants exist and that the library leverages .NET 9 and C# 13 features.
- Optionally, illustrate internal layers inside the Core Library:
– A “Parsing Engine” layer handling the low-level work (SIMD routines, optimized allocations, etc.).
– A “Configuration & API” layer that exposes user options and a fluent API for building readers/writers.
– An “External Integration” layer highlighting the use of csFastFloat and potential downstream compatibility with AOT/NativeAOT.

## Diagram (Mermaid)

```mermaid
graph TD
    subgraph "Sep Core Library"
        SEPCore["Sep Core Library"]:::core
        API_R["Reader API & Parsing Engine"]:::core
        API_W["Writer API"]:::core
        TPI["Third-Party Integration"]:::core
        SEPCore --> API_R
        SEPCore --> API_W
        SEPCore --> TPI
    end

    subgraph "Benchmark Modules"
        Bench["Benchmark Module"]:::bench
        CompBench["Comparison Benchmark Module"]:::bench
    end

    subgraph "Testing Modules"
        UnitTest["Unit/Integration Tests"]:::test
        SampleTest["Sample/Test Usage"]:::test
    end

    subgraph "CI/CD & Automation"
        DevOps["CI/CD & Automation"]:::devops
        BuildCfg["Build Configuration"]:::devops
        DevOps --> BuildCfg
    end

    UnitTest -->|"dependsOn"| SEPCore
    SampleTest -->|"uses"| SEPCore
    Bench -->|"benchmarks"| SEPCore
    CompBench -->|"benchmarks"| SEPCore
    DevOps -->|"triggers"| SEPCore

    %% Click Events
    click SEPCore "https://github.com/nietras/sep/tree/main/src/Sep"
    click API_R "https://github.com/nietras/sep/blob/main/src/Sep/SepReader.cs"
    click API_R "https://github.com/nietras/sep/blob/main/src/Sep/SepReader.Col.cs"
    click API_R "https://github.com/nietras/sep/blob/main/src/Sep/SepReader.Cols.cs"
    click API_R "https://github.com/nietras/sep/tree/main/src/Sep/Internals"
    click API_W "https://github.com/nietras/sep/blob/main/src/Sep/SepWriter.cs"
    click API_W "https://github.com/nietras/sep/blob/main/src/Sep/SepWriter.Col.cs"
    click API_W "https://github.com/nietras/sep/blob/main/src/Sep/SepWriter.Cols.cs"
    click TPI "https://github.com/nietras/sep/tree/main/src/Sep/Internals"
    click Bench "https://github.com/nietras/sep/blob/main/src/Sep.Benchmarks"
    click Bench "https://github.com/nietras/sep/tree/main/benchmarks"
    click CompBench "https://github.com/nietras/sep/blob/main/src/Sep.ComparisonBenchmarks"
    click UnitTest "https://github.com/nietras/sep/blob/main/src/Sep.Test"
    click SampleTest "https://github.com/nietras/sep/blob/main/src/Sep.Tester"
    click SampleTest "https://github.com/nietras/sep/blob/main/src/Sep.XyzTest"
    click DevOps "https://github.com/nietras/sep/tree/main/.github/workflows"
    click BuildCfg "https://github.com/nietras/sep/blob/main/src/Directory.Build.props"
    click BuildCfg "https://github.com/nietras/sep/blob/main/src/Directory.Build.targets"

    %% Styles
    classDef core fill:#B3E5FC,stroke:#0288D1,stroke-width:2px;
    classDef bench fill:#C8E6C9,stroke:#388E3C,stroke-width:2px;
    classDef test fill:#FFF9C4,stroke:#FBC02D,stroke-width:2px;
    classDef devops fill:#FFCCBC,stroke:#F57C00,stroke-width:2px;
```
