# dbt-labs/dbt-cll-evaluator architecture

> Architecture diagram of the GitHub repository dbt-labs/dbt-cll-evaluator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dbt-labs/dbt-cll-evaluator
- Repository: https://github.com/dbt-labs/dbt-cll-evaluator

## Overview

##### Project type and purpose
- This is a Python-based command-line application (CLI) whose sole purpose is to fetch, process, and display column-level lineage (CLL) information from dbt Cloud’s internal Discovery API. It pulls data for a single starting node and optionally applies “fix” operations against a local dbt project.

##### File structure
- Top-level:
- pyproject.toml → Python packaging (Poetry)
- README.md, LICENSE.md → documentation
- demo.cast, slides/ → presentation assets
- Source directory `dbt_cll_evaluator/`:
- __main__.py → CLI entry point (console_scripts)
- cll_types/ → domain models & data classes representing the lineage graph
- dbt_operations/ → modules that read/write a local dbt project (the “--fix” commands)
- rules.py → business rules on how to evaluate or transform lineage

##### README for additional insights
- Confirms it’s driven by `poetry run dbt-cll-evaluator …`
- Expects: dbt Cloud API token, environment ID, node_id for lineage extraction
- Two categories of operations:
a) “read” commands that call the Discovery API
b) “fix” commands that modify a target dbt project filesystem

##### Main components & interactions
a. CLI Layer
- Argument parser (built into __main__.py) that routes subcommands (e.g. “fetch‐cll”, “apply‐fix”)
b. API Client Layer
- A thin HTTP client module (likely in dbt_operations or a helper) that authenticates using the provided token, calls the Discovery API, and returns JSON
c. Domain Model Layer
- cll_types package: transforms raw JSON into strongly-typed lineage objects (nodes & edges)
d. Business Logic Layer
- rules.py: applies evaluation rules on the domain model (e.g. flagging lineage issues)
e. I/O & Persistence Layer
- For “read” commands: simply prints or streams to stdout
- For “--fix” commands: dbt_operations modules inspect & patch local dbt project files (parsing .sql/models, dbt_project.yml, etc.)
f. External Dependencies
- dbt Cloud Discovery API (external service)
- Local file system (the target dbt project path)

##### Architectural patterns & design principles
- Modular CLI design: clear separation of subcommands → easier extensibility
- Layered approach:
– Presentation (CLI)
– Service (API client)
– Domain (cll_types)
– Rules/Business logic
– Persistence (file system)
- Single Responsibility: each module has well-defined purpose (parsing, HTTP, domain modeling, rules, fs ops)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Actors
    User[[User]]:::user

    %% Internal Modules
    subgraph "Internal Modules"
        CLI["CLI Parser\n(__main__.py)"]:::internal
        APIClient["API Client"]:::internal
        Domain["Domain Models"]:::internal
        Rules["Rules Engine"]:::internal
        Ops["DBT Project Operator"]:::internal
    end

    %% External Services
    DBTCloud((dbt Cloud API)):::externalService
    LocalProj[/Local DBT Project/]:::externalFS

    %% Documentation
    Doc1["pyproject.toml"]:::doc
    Doc2["README.md"]:::doc
    Doc3["LICENSE.md"]:::doc

    %% Relationships
    User -->|invokes subcommand| CLI
    CLI -->|token, env_id, node_id| APIClient
    APIClient -->|HTTPS request| DBTCloud
    DBTCloud -->|JSON lineage| APIClient
    APIClient -->|JSON lineage| Domain
    Domain -->|Lineage objects| Rules
    Rules -->|evaluation results| CLI
    CLI -->|--fix commands| Ops
    Ops -->|read/modify files| LocalProj

    %% Click Events
    click CLI "https://github.com/dbt-labs/dbt-cll-evaluator/blob/main/dbt_cll_evaluator/__main__.py"
    click APIClient "https://github.com/dbt-labs/dbt-cll-evaluator/tree/main/dbt_cll_evaluator/dbt_operations/"
    click Domain "https://github.com/dbt-labs/dbt-cll-evaluator/tree/main/dbt_cll_evaluator/cll_types/"
    click Rules "https://github.com/dbt-labs/dbt-cll-evaluator/blob/main/dbt_cll_evaluator/rules.py"
    click Ops "https://github.com/dbt-labs/dbt-cll-evaluator/tree/main/dbt_cll_evaluator/dbt_operations/"
    click Doc1 "https://github.com/dbt-labs/dbt-cll-evaluator/blob/main/pyproject.toml"
    click Doc2 "https://github.com/dbt-labs/dbt-cll-evaluator/blob/main/README.md"
    click Doc3 "https://github.com/dbt-labs/dbt-cll-evaluator/blob/main/LICENSE.md"

    %% Styles
    classDef internal fill:#D6EAF8,stroke:#21618C,color:#1B4F72;
    classDef externalService fill:#D5F5E3,stroke:#27AE60,color:#145A32;
    classDef externalFS fill:#FCF3CF,stroke:#F1C40F,color:#7D6608;
    classDef doc fill:#FADBD8,stroke:#C0392B,color:#922B21;
    classDef user fill:#E8DAEF,stroke:#8E44AD,color:#4A235A;
```
