How johanan/mvr works
Main Components
a) Core Components:
• Command Layer (cmd/)
• Core Configuration (core/)
• Database Abstraction Layer (database/)
• File Output System (file/)
• Data Stream Processing (data/)
b) External Systems:
• Supported Databases:
• PostgreSQL
• MS SQL Server
• Snowflake
• File Output Destinations:
• Local filesystem
• Azure Storage
• Output Formats:
• CSV
• JSONL
• Parquet
┌─ Command Layer ─────────────────────────────────┐
• CLI Interface / Configuration Processing │
• ┬───────────────────────────┘
┌─────────────────────▼───────────────────────────┐
• Core Layer │
• Config Management / Task Execution │
• ┬───────────────────────────┘
• ┌─────────────┴─────────────┐
┌───────▼────────┐ ┌───────▼────────┐
• Database Layer │ │ File Layer │
• Abstractions │ │ Output Handlers│
• ┬────────┘ └───────┬────────┘
┌───────▼────────┐ ┌───────▼────────┐
• Databases │ │ Storage Systems │
• PostgreSQL │ │ - Local Files │
• MS SQL │ │ - Azure Blob │
• Snowflake │ │ │
• ┘ └────────────────┘
Key Elements to Highlight
a) Data Flow:
• Blue: Core components
• Green: Database-related components
• Orange: File output components
• Gray: External systems
Important Relationships
• Parameter binding system between command layer and database layer
• Type abstraction system (showing how Postgres types are used as the common denominator)
• Timestamp handling logic between different database systems
• Environment variable configuration flow
Technical Details
• Supported file formats (CSV, JSONL, Parquet)
• Database connection abstraction
• Type conversion system
• Configuration loading pipeline