# reata/sqllineage architecture

> Architecture diagram of the GitHub repository reata/sqllineage, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/reata/sqllineage
- Repository: https://github.com/reata/sqllineage

## Overview

##### Main Components

a) Core Processing Components:
- SQL Parser Layer (using SQLFluff and SQLParse)
- Lineage Analyzer
- Graph Storage (using NetworkX)
- Metadata Provider
- Runner/Executor

b) Interface Components:
- CLI Interface
- Web Visualization Interface (React-based)
- API Layer

##### Component Relationships

**[User Interfaces]**

- CLI (sqllineage.cli)
- Web UI (sqllineagejs)

**[Core Processing]**

- Runner (sqllineage.runner)
- Analyzer (core.analyzer)

- Parser Layer
- SQLFluff Parser
- SQLParse Parser

- Metadata Layer
- SQLAlchemy Provider
- Dummy Provider

- Graph Storage
- NetworkX

**[External Dependencies]**

- SQLAlchemy (for metadata)
- SQLFluff (for parsing)
- NetworkX (for graph representation)

##### Key Workflows to Highlight

a) SQL Processing Flow:
**Input SQL → Parser → Analyzer → Graph Storage → Output (CLI/Visual)**

b) Metadata Flow:
**Database → SQLAlchemy Provider → Metadata Layer → Analyzer**

##### Important Details

- Pluggable nature of the parser system (SQLFluff/SQLParse)
- Metadata provider's role in enhancing lineage accuracy
- Demonstrate the separation between core processing and visualization
- Column-level and table-level lineage paths

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph UI["User Interfaces"]
        CLI["CLI Interface"]:::interface
        WebUI["Web UI Interface"]:::interface
        Editor["Editor Interface"]:::interface
        DAG["DAG Visualization"]:::interface
    end

    subgraph Core["Core Processing"]
        Runner["Runner/Executor"]:::core
        Analyzer["Core Analyzer"]:::core
        Config["Configuration"]:::core
        Models["Core Models"]:::core
        
        subgraph Parser["Parser Layer"]
            SQLFluff["SQLFluff Parser"]:::parser
            SQLParse["SQLParse Parser"]:::parser
        end
        
        subgraph Meta["Metadata Layer"]
            SQLAProvider["SQLAlchemy Provider"]:::metadata
            DummyProvider["Dummy Provider"]:::metadata
        end
        
        GraphStorage["Graph Storage"]:::core
    end

    subgraph API["API Layer"]
        APIClient["API Client"]:::api
    end

    subgraph Ext["External Dependencies"]
        SQLAlchemy["SQLAlchemy"]:::external
        SQLFluffLib["SQLFluff"]:::external
        NetworkX["NetworkX"]:::external
    end

    %% Relationships
    CLI --> Runner
    WebUI --> APIClient
    Editor --> APIClient
    DAG --> APIClient
    APIClient --> Runner
    
    Runner --> Analyzer
    Runner --> Config
    Analyzer --> Models
    Analyzer --> Parser
    Analyzer --> Meta
    
    SQLFluff --> SQLFluffLib
    SQLAProvider --> SQLAlchemy
    GraphStorage --> NetworkX

    %% Click Events
    click CLI "https://github.com/reata/sqllineage/blob/master/sqllineage/cli.py"
    click WebUI "https://github.com/reata/sqllineage/tree/master/sqllineagejs/"
    click Runner "https://github.com/reata/sqllineage/blob/master/sqllineage/runner.py"
    click Analyzer "https://github.com/reata/sqllineage/blob/master/sqllineage/core/analyzer.py"
    click SQLFluff "https://github.com/reata/sqllineage/tree/master/sqllineage/core/parser/sqlfluff/"
    click SQLParse "https://github.com/reata/sqllineage/tree/master/sqllineage/core/parser/sqlparse/"
    click SQLAProvider "https://github.com/reata/sqllineage/blob/master/sqllineage/core/metadata/sqlalchemy.py"
    click DummyProvider "https://github.com/reata/sqllineage/blob/master/sqllineage/core/metadata/dummy.py"
    click GraphStorage "https://github.com/reata/sqllineage/blob/master/sqllineage/drawing.py"
    click Config "https://github.com/reata/sqllineage/blob/master/sqllineage/config.py"
    click Models "https://github.com/reata/sqllineage/blob/master/sqllineage/core/models.py"
    click APIClient "https://github.com/reata/sqllineage/blob/master/sqllineagejs/src/api/client.js"
    click DAG "https://github.com/reata/sqllineage/blob/master/sqllineagejs/src/features/editor/DAG.js"
    click Editor "https://github.com/reata/sqllineage/blob/master/sqllineagejs/src/features/editor/Editor.js"

    %% Styles
    classDef core fill:#a8d1ff,stroke:#333
    classDef interface fill:#90EE90,stroke:#333
    classDef external fill:#FFE5B4,stroke:#333
    classDef parser fill:#FFB6C1,stroke:#333
    classDef metadata fill:#DDA0DD,stroke:#333
    classDef api fill:#87CEEB,stroke:#333

    %% Legend
    subgraph Legend
        L1["Core Component"]:::core
        L2["Interface Component"]:::interface
        L3["External Dependency"]:::external
        L4["Parser Component"]:::parser
        L5["Metadata Component"]:::metadata
        L6["API Component"]:::api
    end
```
