# thejayesh25/excel-schema-transformer architecture

> Architecture diagram of the GitHub repository thejayesh25/excel-schema-transformer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-02-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thejayesh25/excel-schema-transformer
- Repository: https://github.com/thejayesh25/excel-schema-transformer

## Overview

##### Project type and purpose (what you’re diagramming)
This is a **single-process, config-driven Excel transformation engine** (a data tooling / automation utility), not a distributed system.

**Purpose:** take an input `.xlsx`, apply a deterministic sequence of structural transformations defined in **YAML**, and output a delivery-ready `.xlsx`. The “system” is primarily a **pipeline orchestration + a library of transformation modules**.

##### Read the file tree as architecture hints
Top-level cues:
- `src/main.py`: pipeline entrypoint / orchestrator (reads config, loads workbook, applies steps)
- `src/transformers/*`: the core transformation implementations (each corresponds to a YAML `type`)
- `src/utils.py`: shared helpers (range parsing, column index conversion, sheet utilities, validation helpers, etc.)
- `configs/example_config.yaml`: declarative business rules (the “workflow definition”)
- `data/sample_input.xlsx`, `data/sample_output.xlsx`: example artifacts (I/O boundary of the system)

This suggests a clean separation:
- **Configuration layer** (YAML)
- **Orchestration layer** (main)
- **Transformation execution layer** (transformers)
- **Shared utilities** (utils)
- **External boundary** (Excel files + Excel library dependency)

##### What the README adds (how to reflect it visually)
Key architectural principles explicitly stated:
- **Configuration over hardcoding**
- **Deterministic rule sequencing**
- **Metadata-aware transformations**
- **Validation / safety checks** (e.g., “Validate target emptiness before copy”)

- YAML-driven dispatch (“type” → transformer)
- sequential pipeline ordering (step 1..N)
- validations/guards around destructive actions
- scopes (metadata_only / data_only / both) as a cross-cutting concern

##### Inputs / Outputs (system boundary)
- **Input Excel Workbook** (`.xlsx`)
- **YAML Config** (`configs/*.yaml`)
- **Output Excel Workbook** (`.xlsx`)

- Config + Input Workbook → Engine → Output Workbook

##### Engine / Orchestrator (runtime control)
- `main.py` as **Pipeline Runner**
- load config (YAML parse)
- open workbook
- iterate transformations in order
- dispatch to transformer by `type`
- save workbook

##### Transformer registry / dispatch mechanism
Even if lightweight, represent it explicitly:
- `transformers/__init__.py` (likely exports map/type registry)
- “Transformer Factory / Dispatcher”
- input: `{type: "...", ...params }`
- output: calls `apply(...)` / `transform(...)` on correct module

##### Transformer modules (core business logic)

- `move_column.py`
- `swap_ranges.py`
- `relocate_range.py`
- `replace_values.py`
- `clear_ranges.py`
- `clear_columns.py`
- `mapping_copy_by_labels.py` (note: highlight validation “target emptiness”)
- `split_column_digits.py`

- Dispatcher → each transformer
- Transformers ↔ Workbook object (mutations)
- Transformers → Utils (shared computations)

##### Shared utilities
- `utils.py` as “Range/Column Helpers + Validation Helpers”
- main.py → utils.py (config parsing helpers, workbook helpers)
- transformers → utils.py

##### External library dependency (explicitly show it)
Even though not shown in tree, the engine must use an Excel manipulation library (commonly `openpyxl` in Python).
- main/transformers → excel library → workbook in-memory representation

This helps accuracy: the “Excel workbook object model” is the internal state being mutated.

##### For each transformation rule in order
- validate rule parameters (range specs, columns, scope)
- dispatch by `type` to transformer module
- transformer reads/writes workbook ranges/columns
- optional validation gates (e.g., ensure target range empty)
##### Write output workbook (`data/sample_output.xlsx`)

Workbook as a central node:
- “Workbook (in-memory)” in the middle
- Transformers point into it (mutate)
- Save step outputs file

##### Architectural patterns / principles to call out

- **Pipeline / Chain-of-Responsibility:** ordered list of steps; each step mutates shared state
- **Command pattern (config as commands):** each YAML transformation is a “command” with parameters
- **Plugin-like transformer modules:** new transformation types = add a module + register it
- **Safety/validation layer:** guards against overwrite / schema breakage
- **Separation of concerns:** rules (YAML) vs execution (Python)

##### What “very detailed” means for this repo (don’t overthink; split more)
- workbook in-memory as its own node
- YAML config + input/output files as their own nodes
- excel library dependency as its own node

- shows modules + how data moves (config → engine → workbook → output)
- “Run pipeline” sequence: load → loop steps → dispatch → transform → save
- makes deterministic ordering and validation checkpoints explicit

## Diagram (Mermaid)

```mermaid
flowchart TD
  %% =============== Inputs / Outputs ===============
  subgraph "Inputs & Outputs (Artifacts)"
    CFG["YAML Config\n(workflow definition)"]:::doc
    INXLSX["Input Workbook\n(.xlsx)"]:::doc
    OUTXLSX["Output Workbook\n(.xlsx)"]:::doc
  end

  %% =============== Engine / Runtime ===============
  subgraph "Excel Transformation Engine (Single Process)"
    MAIN["Pipeline Runner / Orchestrator\n- parse YAML\n- load workbook\n- iterate steps (1..N)\n- dispatch by type\n- save workbook"]:::engine

    DISP["Transformer Registry / Dispatcher\n(type → transformer)"]:::dispatcher

    UTIL["Shared Utilities\n- range parsing\n- column index conversion\n- sheet helpers\n- validation helpers"]:::util

    WB[("Workbook (in-memory)\nobject model")]:::state

    EXT["Excel Library\n(e.g., openpyxl)"]:::external
  end

  %% =============== Transformers ===============
  subgraph "Transformer Modules (Catalog / Plugin-like)"
    T_MOVE["move_column\n(reposition single column)"]:::transformer
    T_SWAP["swap_ranges\n(swap equal-width blocks)"]:::transformer
    T_RELOC["relocate_range\n(move block to new location)"]:::transformer
    T_REPL["replace_values\n(replace within range, scoped)"]:::transformer
    T_CLR_R["clear_ranges\n(clear metadata/data/both in ranges)"]:::transformer
    T_CLR_C["clear_columns\n(clear metadata/data/both in columns)"]:::transformer
    T_MAP["mapping_copy_by_labels\n(label-aware copy + target-empty validation)"]:::transformer
    T_SPLIT["split_column_digits\n(derive padded digit columns)"]:::transformer
  end

  %% =============== Primary Dataflow (Happy Path) ===============
  CFG -->|"parse_YAML"| MAIN
  INXLSX -->|"load_xlsx"| EXT
  EXT -->|"create_object_model"| WB

  MAIN -->|"for_each_step(1..N)"| DISP
  DISP -->|"dispatch_by_type"| T_MOVE
  DISP -->|"dispatch_by_type"| T_SWAP
  DISP -->|"dispatch_by_type"| T_RELOC
  DISP -->|"dispatch_by_type"| T_REPL
  DISP -->|"dispatch_by_type"| T_CLR_R
  DISP -->|"dispatch_by_type"| T_CLR_C
  DISP -->|"dispatch_by_type"| T_MAP
  DISP -->|"dispatch_by_type"| T_SPLIT

  %% Transformers mutate workbook + use utils + excel lib
  T_MOVE -->|"mutate_workbook_ranges"| WB
  T_SWAP -->|"mutate_workbook_ranges"| WB
  T_RELOC -->|"mutate_workbook_ranges"| WB
  T_REPL -->|"validate_scope(metadata/data/both)"| UTIL
  T_REPL -->|"mutate_workbook_ranges"| WB
  T_CLR_R -->|"validate_scope(metadata/data/both)"| UTIL
  T_CLR_R -->|"mutate_workbook_ranges"| WB
  T_CLR_C -->|"validate_scope(metadata/data/both)"| UTIL
  T_CLR_C -->|"mutate_workbook_ranges"| WB
  T_MAP -->|"validate_target_empty"| UTIL
  T_MAP -->|"mutate_workbook_ranges"| WB
  T_SPLIT -->|"mutate_workbook_ranges"| WB

  T_MOVE -->|"range_helpers"| UTIL
  T_SWAP -->|"range_helpers"| UTIL
  T_RELOC -->|"range_helpers"| UTIL
  T_MAP -->|"label_lookup_helpers"| UTIL
  T_SPLIT -->|"column_helpers"| UTIL

  MAIN -->|"workbook_helpers"| UTIL

  %% Save output
  WB -->|"serialize_xlsx"| EXT
  EXT -->|"save_xlsx"| OUTXLSX

  %% =============== Principles / Notes ===============
  subgraph "Design Principles (Cross-cutting)"
    P1["Pipeline / Chain-of-Responsibility\n(ordered deterministic steps)"]:::note
    P2["Command Pattern\n(YAML steps = commands + params)"]:::note
    P3["Safety & Validation\n(emptiness checks, scope guards)"]:::note
    P4["Plugin-like Transformers\n(add module + register type)"]:::note
  end

  MAIN -.->|"deterministic_sequencing"| P1
  CFG -.->|"declares_commands"| P2
  UTIL -.->|"validation_layer"| P3
  DISP -.->|"extensible_registry"| P4

  %% =============== Click Events (from component_mapping) ===============
  click MAIN "https://github.com/thejayesh25/excel-schema-transformer/blob/main/src/main.py"
  click CFG "https://github.com/thejayesh25/excel-schema-transformer/blob/main/configs/example_config.yaml"
  click INXLSX "https://github.com/thejayesh25/excel-schema-transformer/blob/main/data/sample_input.xlsx"
  click OUTXLSX "https://github.com/thejayesh25/excel-schema-transformer/blob/main/data/sample_output.xlsx"
  click DISP "https://github.com/thejayesh25/excel-schema-transformer/blob/main/src/transformers/__init__.py"
  click T_MOVE "https://github.com/thejayesh25/excel-schema-transformer/blob/main/src/transformers/move_column.py"
  click T_SWAP "https://github.com/thejayesh25/excel-schema-transformer/blob/main/src/transformers/swap_ranges.py"
  click T_RELOC "https://github.com/thejayesh25/excel-schema-transformer/blob/main/src/transformers/relocate_range.py"
  click T_REPL "https://github.com/thejayesh25/excel-schema-transformer/blob/main/src/transformers/replace_values.py"
  click T_CLR_R "https://github.com/thejayesh25/excel-schema-transformer/blob/main/src/transformers/clear_ranges.py"
  click T_CLR_C "https://github.com/thejayesh25/excel-schema-transformer/blob/main/src/transformers/clear_columns.py"
  click T_MAP "https://github.com/thejayesh25/excel-schema-transformer/blob/main/src/transformers/mapping_copy_by_labels.py"
  click T_SPLIT "https://github.com/thejayesh25/excel-schema-transformer/blob/main/src/transformers/split_column_digits.py"
  click UTIL "https://github.com/thejayesh25/excel-schema-transformer/blob/main/src/utils.py"

  %% =============== Styles ===============
  classDef doc fill:#E3F2FD,stroke:#1565C0,stroke-width:1.5px,color:#0D47A1;
  classDef engine fill:#E8F5E9,stroke:#2E7D32,stroke-width:1.5px,color:#1B5E20;
  classDef dispatcher fill:#FFF3E0,stroke:#EF6C00,stroke-width:1.5px,color:#E65100;
  classDef transformer fill:#F3E5F5,stroke:#6A1B9A,stroke-width:1.5px,color:#4A148C;
  classDef util fill:#E0F7FA,stroke:#00838F,stroke-width:1.5px,color:#004D40;
  classDef state fill:#FCE4EC,stroke:#AD1457,stroke-width:1.5px,color:#880E4F;
  classDef external fill:#FAFAFA,stroke:#616161,stroke-width:1.5px,stroke-dasharray:6 4,color:#212121;
  classDef note fill:#FFFDE7,stroke:#9E9D24,stroke-width:1.2px,color:#827717;
```
