How nietras/sep works
This project is a high‐performance .NET library focused solely on parsing and writing separated values (such as CSV, TSV, etc.) – “Sep” positions itself as the world’s fastest .NET CSV parser. Its README immediately reveals that the core purpose is efficiency with low or zero allocations, leveraging modern C# features (e.g. Span<T>, ref structs, generic math, SIMD vectorization, async APIs, and even AOT compatibility) and integrating with performance‐focused libraries like csFastFloat. The file tree shows a multi-project solution that includes:
1. A core library (in the “src/Sep” folder) containing all the parser and writer implementations and public APIs (with types such as Sep, SepReader, and SepWriter, plus supporting options and pooling classes).
2. Multiple test projects (e.g. “src/Sep.Test”, “src/Sep.Tester”, “src/Sep.XyzTest”) that exercise the API and validate correct behavior.
3. Benchmark and comparison suites (e.g. “src/Sep.Benchmarks”, “src/Sep.ComparisonBenchmarks”, and a “benchmarks” directory containing platform‐specific benchmark results) that compare Sep’s performance to established libraries (like CsvHelper and Sylvan).
4. A set of development and automation scripts (.github workflows, PowerShell scripts such as bench.ps1, pack.ps1, publish-tester.ps1, etc.) that set up CI/CD, dependency management, code analysis, and repository policies.
• Core Library Module (“Sep Core”)
– This is the heart of the project. It exposes the primary public API (the “Sep” type, along with extension methods that let clients create a “SepReader” or “SepWriter”). Inside, it implements both reading and writing pipelines for separated values. Include sub-components such as:
▪ Reader API and its configurable options (SepReaderOptions), advanced parsing techniques (e.g. SIMD vectorized routines, pooling strategies, asynchronous and synchronous flows), and internal helper types (ref structs for Row, Col, Cols).
▪ Writer API with its options (SepWriterOptions), the deferred row construction model, and methods to handle escaping.
▪ Integration points with third‐party libraries like csFastFloat for fast numeric parsing.
• Benchmark and Comparison Modules
• Testing Modules
– These include the unit/integration test projects (“Sep.Test”, “Sep.Tester”, “Sep.XyzTest”). They depend on the core library to verify correctness and also serve as sample usage (with the examples shown in the README).
• Automation/DevOps & Configuration
Dependencies and interactions
• In the core library module, highlight the flow through the API:
– Client code (or tests/benchmarks) invoke the public “Sep” API that provides “Reader()” and “Writer()” extension methods.
– The Reader then (optionally) auto-detects the separator, manages memory pooling, and returns a stream of “Row” ref structs (with columns – “Col” and “Cols”).
– Similarly, the Writer uses deferred row construction, building rows via new row creation and flushing data only when disposing rows.
– Optionally denote that asynchronous variants exist and that the library leverages .NET 9 and C# 13 features.
• Optionally, illustrate internal layers inside the Core Library:
– A “Parsing Engine” layer handling the low-level work (SIMD routines, optimized allocations, etc.).
– A “Configuration & API” layer that exposes user options and a fluent API for building readers/writers.
– An “External Integration” layer highlighting the use of csFastFloat and potential downstream compatibility with AOT/NativeAOT.