How thejayesh25/excel-schema-transformer works
Project type and purpose (what you’re diagramming)
This is a single-process, config-driven Excel transformation engine (a data tooling / automation utility), not a distributed system.
Purpose: take an input .xlsx, apply a deterministic sequence of structural transformations defined in YAML, and output a delivery-ready .xlsx. The “system” is primarily a pipeline orchestration + a library of transformation modules.
Read the file tree as architecture hints
Top-level cues:
• src/main.py: pipeline entrypoint / orchestrator (reads config, loads workbook, applies steps)
• src/transformers/*: the core transformation implementations (each corresponds to a YAML type)
• src/utils.py: shared helpers (range parsing, column index conversion, sheet utilities, validation helpers, etc.)
• configs/example_config.yaml: declarative business rules (the “workflow definition”)
• data/sample_input.xlsx, data/sample_output.xlsx: example artifacts (I/O boundary of the system)
This suggests a clean separation:
• Configuration layer (YAML)
• Orchestration layer (main)
• Transformation execution layer (transformers)
• Shared utilities (utils)
• External boundary (Excel files + Excel library dependency)
What the README adds (how to reflect it visually)
Key architectural principles explicitly stated:
• Configuration over hardcoding
• Deterministic rule sequencing
• Metadata-aware transformations
• Validation / safety checks (e.g., “Validate target emptiness before copy”)
• YAML-driven dispatch (“type” → transformer)
• sequential pipeline ordering (step 1..N)
• validations/guards around destructive actions
• scopes (metadata_only / data_only / both) as a cross-cutting concern
Inputs / Outputs (system boundary)
• Input Excel Workbook (.xlsx)
• YAML Config (configs/*.yaml)
• Output Excel Workbook (.xlsx)
• Config + Input Workbook → Engine → Output Workbook
Engine / Orchestrator (runtime control)
• main.py as Pipeline Runner
• load config (YAML parse)
• open workbook
• iterate transformations in order
• dispatch to transformer by type
• save workbook
Transformer registry / dispatch mechanism
Even if lightweight, represent it explicitly:
• transformers/__init__.py (likely exports map/type registry)
• “Transformer Factory / Dispatcher”
• input: {type: "...", ...params }
• output: calls apply(...) / transform(...) on correct module
Transformer modules (core business logic)
• move_column.py
• swap_ranges.py
• relocate_range.py
• replace_values.py
• clear_ranges.py
• clear_columns.py
• mapping_copy_by_labels.py (note: highlight validation “target emptiness”)
• split_column_digits.py
• Dispatcher → each transformer
• Transformers ↔ Workbook object (mutations)
• Transformers → Utils (shared computations)
Shared utilities
• utils.py as “Range/Column Helpers + Validation Helpers”
• main.py → utils.py (config parsing helpers, workbook helpers)
• transformers → utils.py
External library dependency (explicitly show it)
Even though not shown in tree, the engine must use an Excel manipulation library (commonly openpyxl in Python).
• main/transformers → excel library → workbook in-memory representation
This helps accuracy: the “Excel workbook object model” is the internal state being mutated.
For each transformation rule in order
• validate rule parameters (range specs, columns, scope)
• dispatch by type to transformer module
• transformer reads/writes workbook ranges/columns
• optional validation gates (e.g., ensure target range empty)
Write output workbook (data/sample_output.xlsx)
Workbook as a central node:
• “Workbook (in-memory)” in the middle
• Transformers point into it (mutate)
• Save step outputs file
Architectural patterns / principles to call out
• Pipeline / Chain-of-Responsibility: ordered list of steps; each step mutates shared state
• Command pattern (config as commands): each YAML transformation is a “command” with parameters
• Plugin-like transformer modules: new transformation types = add a module + register it
• Safety/validation layer: guards against overwrite / schema breakage
• Separation of concerns: rules (YAML) vs execution (Python)
What “very detailed” means for this repo (don’t overthink; split more)
• workbook in-memory as its own node
• YAML config + input/output files as their own nodes
• excel library dependency as its own node
• shows modules + how data moves (config → engine → workbook → output)
• “Run pipeline” sequence: load → loop steps → dispatch → transform → save
• makes deterministic ordering and validation checkpoints explicit