# cstobey/vclods architecture

> Architecture diagram of the GitHub repository cstobey/vclods, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cstobey/vclods
- Repository: https://github.com/cstobey/vclods

## Overview

##### Main Components

a) Core Framework Layer:
- Configuration Management System
- Locking Mechanism
- Logging System
- Extension Processing Engine
- Script Execution Engine

b) Input/Output Components:
- Directory Structure Handler
- File Extension Parser
- Connection Managers (MySQL, MSSQL, Oracle, Postgres)
- Output Destinations (syslog, email, Slack, SQL database)

c) Extension System:
- Operation Extensions (.sh, .sql, .py, etc.)
- Utility Extensions (batch, diff, tee, etc.)
- Custom Extensions Directory

##### Relationships and Data Flow

- Input Flow: Directory → Script Files → Extension Parser → Operations
- Configuration Flow: Global Config → Directory Config → Script Execution
- Execution Flow: Script → Locking → Processing → Output
- Extension Processing: Sequential Extension Handling (.dst.sql → dst processor → sql processor)

##### Architectural Pattern Representation

Use these visual elements:
- Pipeline Architecture: Show how extensions form functional pipes
- Modular Design: Separate components for config, operations, connections
- Plugin System: Extension mechanism
- Multi-destination Output System

**[Directory Input] → [Configuration Layer]**
- ↓
**[Script Processing Engine]**
- ↓
**[Extension Parser] ←→ [Extension Modules]**
- ↓
**[Operation Executor] ←→ [Connection Managers]**
- ↓
**[Output Handlers] → [Multiple Destinations]**

##### Important Details

- Parallel execution capability
- Locking mechanism
- Demonstrate configuration inheritance
- Extension chain processing
- Error handling paths

##### Specific Implementation Notes

- How the ~400 lines of ksh code orchestrates the entire system
- Relationship between file extensions and operations
- Demonstrate how the batching system works
- Post-processing capabilities

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Legend
        L1[Core Component]:::core
        L2[Extension Point]:::extension
        L3[(Database)]:::database
        L4[Configuration]:::config
        L5[Critical Path]:::critical
    end

    subgraph "Core Framework"
        Config[Configuration Management]:::core
        Lock[Locking Mechanism]:::core
        ScriptExec[Script Execution Engine]:::core
        ExtProc[Extension Processing Engine]:::core
    end

    subgraph "Input Layer"
        DirHandler[Directory Structure Handler]:::core
        ExtParser[File Extension Parser]:::core
    end

    subgraph "Connection Layer"
        MySQL[(MySQL)]:::database
        MSSQL[(MSSQL)]:::database
        Oracle[(Oracle)]:::database
        Postgres[(Postgres)]:::database
    end

    subgraph "Extension System"
        OpExt[Operation Extensions]:::extension
        UtilExt[Utility Extensions]:::extension
        CustomExt[Custom Extensions]:::extension
    end

    subgraph "Output System"
        Email[Email Handler]:::output
        Slack[Slack Integration]:::output
        Syslog[Syslog]:::output
        SQLOut[SQL Output]:::output
    end

    DirHandler --> Config
    Config --> ScriptExec
    ScriptExec --> Lock
    Lock --> ExtProc
    ExtProc --> ExtParser
    ExtParser --> OpExt
    ExtParser --> UtilExt
    ExtParser --> CustomExt
    OpExt --> MySQL
    OpExt --> MSSQL
    OpExt --> Oracle
    OpExt --> Postgres
    ExtProc --> Email
    ExtProc --> Slack
    ExtProc --> Syslog
    ExtProc --> SQLOut

    classDef core fill:#blue,stroke:#333,stroke-width:2px
    classDef extension fill:#green,stroke:#333,stroke-width:2px
    classDef database fill:#gray,stroke:#333,stroke-width:2px
    classDef config fill:#orange,stroke:#333,stroke-width:2px
    classDef critical fill:#red,stroke:#333,stroke-width:2px
    classDef output fill:#purple,stroke:#333,stroke-width:2px

    click Config "https://github.com/cstobey/vclods/tree/master/config"
    click Lock "https://github.com/cstobey/vclods/tree/master/includes/locking"
    click ScriptExec "https://github.com/cstobey/vclods/tree/master/includes/script"
    click MySQL "https://github.com/cstobey/vclods/tree/master/connections/mysql"
    click MSSQL "https://github.com/cstobey/vclods/tree/master/connections/mssql"
    click Oracle "https://github.com/cstobey/vclods/tree/master/connections/oracle"
    click Postgres "https://github.com/cstobey/vclods/tree/master/connections/postgres"
    click ExtProc "https://github.com/cstobey/vclods/tree/master/extensions"
    click Email "https://github.com/cstobey/vclods/tree/master/extensions/email"
    click Slack "https://github.com/cstobey/vclods/tree/master/extensions/slack"
    click SQLOut "https://github.com/cstobey/vclods/tree/master/extensions/sql"
    click UtilExt "https://github.com/cstobey/vclods/tree/master/extensions/batch"
    click OpExt "https://github.com/cstobey/vclods/tree/master/extensions/sh"
```
