# johanan/mvr architecture

> Architecture diagram of the GitHub repository johanan/mvr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/johanan/mvr
- Repository: https://github.com/johanan/mvr

## Overview

##### Main Components

a) Core Components:
- Command Layer (cmd/)
- Core Configuration (core/)
- Database Abstraction Layer (database/)
- File Output System (file/)
- Data Stream Processing (data/)

b) External Systems:
- Supported Databases:
- PostgreSQL
- MS SQL Server
- Snowflake
- File Output Destinations:
- Local filesystem
- Azure Storage
- Output Formats:
- CSV
- JSONL
- Parquet

**┌─ Command Layer ─────────────────────────────────┐**

- CLI Interface / Configuration Processing        │
- ┬───────────────────────────┘

**┌─────────────────────▼───────────────────────────┐**
- Core Layer                                     │

- Config Management / Task Execution             │
- ┬───────────────────────────┘

- ┌─────────────┴─────────────┐

**┌───────▼────────┐         ┌───────▼────────┐**
- Database Layer │         │   File Layer   │

- Abstractions  │         │ Output Handlers│
- ┬────────┘         └───────┬────────┘

**┌───────▼────────┐         ┌───────▼────────┐**
- Databases    │         │ Storage Systems │

- PostgreSQL   │         │ - Local Files  │
- MS SQL       │         │ - Azure Blob   │
- Snowflake    │         │                │
- ┘         └────────────────┘

##### Key Elements to Highlight

a) Data Flow:

- Blue: Core components
- Green: Database-related components
- Orange: File output components
- Gray: External systems

##### Important Relationships

- Parameter binding system between command layer and database layer
- Type abstraction system (showing how Postgres types are used as the common denominator)
- Timestamp handling logic between different database systems
- Environment variable configuration flow

##### Technical Details

- Supported file formats (CSV, JSONL, Parquet)
- Database connection abstraction
- Type conversion system
- Configuration loading pipeline

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph CLI
        CMD[Command Layer]
        CLI[CLI Interface]
        CMDI[Command Implementations]
    end

    subgraph Core
        CONF[Configuration Management]
        TASK[Task Execution]
    end

    subgraph DBLayer[Database Layer]
        DBA[Database Abstraction]
        PG[PostgreSQL]
        MSSQL[MS SQL Server]
        SF[Snowflake]
    end

    subgraph FileSystem[File Layer]
        FA[File Abstraction]
        CSV[CSV Writer]
        JSONL[JSONL Writer]
        PAR[Parquet Writer]
        AZ[Azure Storage]
    end

    subgraph DataProc[Data Processing]
        DS[Stream Processing]
        TU[Time Utilities]
    end

    subgraph Utils
        TMPL[Template Handler]
    end

    subgraph ExternalSystems[External Systems]
        direction TB
        subgraph Databases
            PostgreSQL[(PostgreSQL)]
            MSSQLS[(MS SQL Server)]
            SFS[(Snowflake)]
        end
        subgraph Storage
            Local[Local Files]
            Azure[Azure Blob]
        end
    end

    CMD --> CLI
    CMD --> CMDI
    CLI --> CONF
    CMDI --> CONF
    CONF --> TASK
    TASK --> DBA
    TASK --> FA
    DBA --> PG
    DBA --> MSSQL
    DBA --> SF
    PG --> PostgreSQL
    MSSQL --> MSSQLS
    SF --> SFS
    FA --> CSV
    FA --> JSONL
    FA --> PAR
    FA --> AZ
    CSV --> Local
    JSONL --> Local
    PAR --> Local
    AZ --> Azure
    DS --> TASK
    TU --> TASK
    TMPL --> TASK

    %% Component Mappings
    click CMD "https://github.com/johanan/mvr/tree/main/cmd"
    click CLI "https://github.com/johanan/mvr/blob/main/cmd/root.go"
    click CMDI "https://github.com/johanan/mvr/blob/main/cmd/mv.go"
    click CONF "https://github.com/johanan/mvr/blob/main/core/config.go"
    click TASK "https://github.com/johanan/mvr/blob/main/core/task.go"
    click DBA "https://github.com/johanan/mvr/blob/main/database/database.go"
    click PG "https://github.com/johanan/mvr/blob/main/database/postgres.go"
    click MSSQL "https://github.com/johanan/mvr/blob/main/database/ms_sql_server.go"
    click SF "https://github.com/johanan/mvr/blob/main/database/snowflake.go"
    click FA "https://github.com/johanan/mvr/blob/main/file/file.go"
    click CSV "https://github.com/johanan/mvr/blob/main/file/csv_writer.go"
    click JSONL "https://github.com/johanan/mvr/blob/main/file/jsonl_writer.go"
    click PAR "https://github.com/johanan/mvr/blob/main/file/parquet_writer.go"
    click AZ "https://github.com/johanan/mvr/blob/main/file/azure.go"
    click DS "https://github.com/johanan/mvr/blob/main/data/datastream.go"
    click TU "https://github.com/johanan/mvr/blob/main/data/time_utils.go"
    click TMPL "https://github.com/johanan/mvr/blob/main/utils/templates.go"

    classDef cli fill:#e1f5fe,stroke:#01579b
    classDef core fill:#e8f5e9,stroke:#1b5e20
    classDef db fill:#fff3e0,stroke:#e65100
    classDef file fill:#fce4ec,stroke:#880e4f
    classDef data fill:#f3e5f5,stroke:#4a148c
    classDef utils fill:#eceff1,stroke:#263238
    classDef external fill:#eeeeee,stroke:#212121

    class CMD,CLI,CMDI cli
    class CONF,TASK core
    class DBA,PG,MSSQL,SF,PostgreSQL,MSSQLS,SFS db
    class FA,CSV,JSONL,PAR,AZ,Local,Azure file
    class DS,TU data
    class TMPL utils
    class ExternalSystems external
```
