How sdf-labs/dbt-sdf works
Main Components
• CLI Component – Represented by the dbt_sdf/cli directory. This is the entry point for the tool (with main.py and params.py) and will handle user interaction via command-line inputs.
• Model Parser Component – Found under the dbt_sdf/model_parser directory. This module (mainly parser.py) likely takes care of parsing model definitions or files. It will be a key processing engine that interprets input.
• Schema Component – Located in the dbt_sdf/schema directory. This module manages schema logic by providing base functionality (via base.py) and handling generated code (in the schema/generated subdirectory with models.py).
• Support Scripts – The scripts directory contains helper scripts such as cleanup_codegen.py. Though not part of the core package, these support tasks like code generation maintenance.
• Testing Infrastructure – The tests directory contains unit tests (test_cli.py, test_model_parser.py, etc.) and common test setup (conftest.py) to ensure the tool’s functionality.
Relationships and Data Flow
• Within the CLI, the main.py and params.py handle input processing. Once parameters are collected, the CLI component should call the Model Parser.
• Optionally, a feedback loop or error outputs can be shown, flowing from the Schema back to the CLI for logging or result presentation.
• External support where scripts (like cleanup_codegen.py) interact with the generated code in the Schema component, perhaps drawn as a side process that “cleans up” or maintains the code generation output.
• The build/deployment tools (e.g., Makefile and setup.py) can be drawn in a separate “Infrastructure & DevOps” layer that supports packaging and deployment of the entire system.
Architectural Patterns and Design Principles
• Separation of concerns, as the CLI, Parser, and Schema components are clearly delineated. This modular design ensures that each component has a single responsibility.
• That the tool is architected as a command-line application with core business logic abstracted from user interaction.