# vicent-dev/exporter architecture

> Architecture diagram of the GitHub repository vicent-dev/exporter, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vicent-dev/exporter
- Repository: https://github.com/vicent-dev/exporter

## Overview

##### Project type and purpose
- This is a single-binary Go CLI tool (“exporter”) that connects to a SQL database, runs an arbitrary query, and writes the full result set as a CSV file. It is neither full-stack nor microservices; it’s a focused, high-performance command-line application.

##### File structure
- Top-level files:
- main.go – entry point, flag parsing, orchestration
- env.go – loads environment variables/configuration
- storage.go – abstracts the database snapshot and query execution
- file.go – handles CSV file creation and writing
- Makefile, .env.dist – build and default config templates
- .github/workflows/go.yml – CI pipeline
- go.mod/go.sum capture dependencies; no sub-directories implies a monolithic structure.
- No external services beyond the SQL database; no web server or UI components.

##### README for additional insights
- CLI flags: -f (filename), -d (directory), -q (SQL query)
- Uses Go’s goroutines to parallelize chunks of the result set.
- Creates a consistent database snapshot, divides the full row count into ranges, and spawns X workers (where X = CPU cores) to fetch and write each chunk concurrently.

a. Main components:
1. CLI Interface (main.go)
2. Configuration Loader (env.go)
3. Database Client & Snapshot Manager (storage.go)
4. Chunk Manager & Worker Pool (storage.go + main.go)
5. CSV Writer (file.go)
6. Local Filesystem (output .csv)
b. Relationships & interactions:
- User invokes CLI → CLI parses flags → Config Loader reads env → initialize DB client
- DB client creates snapshot → Query “SELECT COUNT(*)” to get total rows
- Chunk Manager divides [0…total] into N ranges → Worker Pool of goroutines
- Each Worker pulls its range from DB, streams rows → sends to CSV Writer
- CSV Writer writes rows sequentially or via buffered channel to output file
- After all workers finish → Snapshot is torn down → Tool exits with CSV file on disk
c. Architectural patterns & principles:
- Single-responsibility modules (env loader, storage, file writer)
- Concurrency pattern: worker pool limiting parallel DB queries by CPU count
- Streaming I/O: avoids buffering entire result in memory
- Use of database snapshot to ensure consistent reads across workers
d. Key technologies:
- Go runtime (goroutines, channels)
- Go SQL driver (database/sql)
- CSV encoder (encoding/csv)
- OS file I/O (os package)
- Environment configuration (os.Getenv or a small helper)

## Diagram (Mermaid)

```mermaid
graph TD
    U["User/CLI"]:::actor
    U -->|invokes| CLI["CLI Parser\n(main.go)"]:::process
    CLI -->|"loads config"| CFG["Config Loader\n(env.go)"]:::process
    CFG -->|"init DB client"| DB["DB Client & Snapshot\n(storage.go)"]:::process
    DB -->|"estimate rows"| CHUNK["Chunk Manager"]:::process
    subgraph "Worker Pool"
      direction LR
      W1["Worker 1"]:::concurrency
      Wn["Worker N"]:::concurrency
    end
    CHUNK -->|"spawn ranges"| W1
    CHUNK -->|"spawn ranges"| Wn
    W1 -->|"stream rows"| CSV["CSV Writer\n(file.go)"]:::process
    Wn -->|"stream rows"| CSV
    CSV -->|"writes CSV file"| FS["Filesystem: output.csv"]:::datastore
    DB -.->|"snapshot teardown"| DB

    CI["CI Pipeline\n(.github/workflows/go.yml)"]:::process

    click CLI "https://github.com/vicent-dev/exporter/blob/main/main.go"
    click CFG "https://github.com/vicent-dev/exporter/blob/main/env.go"
    click DB "https://github.com/vicent-dev/exporter/blob/main/storage.go"
    click CHUNK "https://github.com/vicent-dev/exporter/blob/main/storage.go"
    click CSV "https://github.com/vicent-dev/exporter/blob/main/file.go"
    click CI "https://github.com/vicent-dev/exporter/blob/main/.github/workflows/go.yml"

    classDef actor fill:#d3d3d3,stroke:#333,stroke-width:1px
    classDef process fill:#add8e6,stroke:#333,stroke-width:1px
    classDef datastore fill:#90ee90,stroke:#333,stroke-width:1px
    classDef concurrency fill:#fdfd96,stroke:#333,stroke-width:1px
```
