# dalgot4d/dbt_arghyam architecture

> Architecture diagram of the GitHub repository dalgot4d/dbt_arghyam, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dalgot4d/dbt_arghyam
- Repository: https://github.com/dalgot4d/dbt_arghyam

## Overview

##### Main Components Identification

The project is a dbt-based data transformation pipeline with the following key components:

a. Source Layer
- Raw data sources (likely external databases/systems)
- CDC (Change Data Capture) feeds indicated by *_cdc.sql files

b. Transformation Layer
- Intermediate models (/models/intermediate/)
- Analytics models (/models/analytics/)
- Custom macros (/macros/)

c. Special Components
- mgramseva module (appears to be a specific application domain)
- One-time data loads

##### Intermediate Models
- encounters_cdc
- location_cdc
- subjects_cdc
- mgramseva specific models (billdetails, demanddetails, etc.)

##### Analytics Models
- Dimension tables (activity_dim, bridge_dim, location_dim)
- Fact tables (various form responses)
- mgramseva analytics (demand_collection, expenditure, etc.)

BOTTOM LAYER - Output/Consumption Layer:
- Final analytics tables
- Reporting views
- Task views

##### Key Relationships to Highlight

- CDC data flow paths
- Dimension to fact table relationships
- Dependencies between intermediate and analytics models
- Special handling for mgramseva module

##### Additional Elements

Supporting components:
- Macros (flatten_json, generate_schema_name)
- One-time load processes
- CI/CD workflow (.github/workflows/dbt-ci.yml)

##### Technical Details

Annotations for:
- SQL transformation types
- Data quality tests
- Schema generation logic
- JSON handling processes

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Style definitions
    classDef source fill:#a8d1f0,stroke:#333
    classDef intermediate fill:#ffe4b5,stroke:#333
    classDef analytics fill:#98fb98,stroke:#333
    classDef mgramseva fill:#dda0dd,stroke:#333
    classDef macro fill:#f0e68c,stroke:#333
    classDef cicd fill:#d3d3d3,stroke:#333
    
    subgraph Sources["Source Layer"]
        Avni[("Avni DB")]:::source
        MGramSeva[("mGramSeva DB")]:::source
        CDC["CDC Feeds"]:::source
    end

    subgraph Intermediate["Transformation Layer"]
        direction TB
        subgraph CDC_Processing["CDC Processing"]
            EncCDC["Encounters CDC"]:::intermediate
            LocCDC["Location CDC"]:::intermediate
            SubCDC["Subjects CDC"]:::intermediate
        end

        subgraph MGramInt["mGramSeva Intermediate"]
            BillDetails["Bill Details"]:::mgramseva
            DemandDetails["Demand Details"]:::mgramseva
        end
    end

    subgraph Analytics["Analytics Layer"]
        direction TB
        subgraph Dimensions["Dimension Tables"]
            ActivityDim["Activity Dim"]:::analytics
            BridgeDim["Bridge Dim"]:::analytics
            LocationDim["Location Dim"]:::analytics
        end

        subgraph Facts["Fact Tables"]
            JalFact["Jal Chaupal Responses"]:::analytics
            LogFact["Log Book Responses"]:::analytics
            TankFact["Tank Cleaning Responses"]:::analytics
            WaterFact["Water Quality Responses"]:::analytics
            WIMCFact["WIMC Meeting Responses"]:::analytics
        end

        subgraph MGramAnalytics["mGramSeva Analytics"]
            DemandColl["Demand Collection"]:::mgramseva
            Expenditure["Expenditure Analysis"]:::mgramseva
        end

        TaskView["Task View"]:::analytics
    end

    subgraph Support["Support Components"]
        direction TB
        Macros["Transformation Macros"]:::macro
        OneTime["One-time Loads"]:::macro
        CICD["CI/CD Pipeline"]:::cicd
    end

    %% Relationships
    Avni --> CDC_Processing
    MGramSeva --> MGramInt
    CDC --> CDC_Processing
    CDC_Processing --> Dimensions
    MGramInt --> MGramAnalytics
    Dimensions --> Facts
    
    %% Click events
    click EncCDC "https://github.com/DalgoT4D/dbt_arghyam/blob/master/models/intermediate/encounters_cdc.sql"
    click LocCDC "https://github.com/DalgoT4D/dbt_arghyam/blob/master/models/intermediate/location_cdc.sql"
    click SubCDC "https://github.com/DalgoT4D/dbt_arghyam/blob/master/models/intermediate/subjects_cdc.sql"
    click ActivityDim "https://github.com/DalgoT4D/dbt_arghyam/blob/master/models/analytics/activity_dim.sql"
    click BridgeDim "https://github.com/DalgoT4D/dbt_arghyam/blob/master/models/analytics/bridge_dim.sql"
    click LocationDim "https://github.com/DalgoT4D/dbt_arghyam/blob/master/models/analytics/location_dim.sql"
    click MGramAnalytics "https://github.com/DalgoT4D/dbt_arghyam/tree/master/models/analytics/mgramseva"
    click MGramInt "https://github.com/DalgoT4D/dbt_arghyam/tree/master/models/intermediate/mgramseva"
    click Macros "https://github.com/DalgoT4D/dbt_arghyam/tree/master/macros"
    click OneTime "https://github.com/DalgoT4D/dbt_arghyam/blob/master/one_time_loads/location_dim_one_time_load.sql"
    click CICD "https://github.com/DalgoT4D/dbt_arghyam/blob/master/.github/workflows/dbt-ci.yml"
    click JalFact "https://github.com/DalgoT4D/dbt_arghyam/blob/master/models/analytics/jal_chaupal_form_responses_fact.sql"
    click LogFact "https://github.com/DalgoT4D/dbt_arghyam/blob/master/models/analytics/log_book_form_responses_fact.sql"
    click TankFact "https://github.com/DalgoT4D/dbt_arghyam/blob/master/models/analytics/tank_cleaning_form_responses_fact.sql"
    click WaterFact "https://github.com/DalgoT4D/dbt_arghyam/blob/master/models/analytics/water_quality_testing_form_responses_fact.sql"
    click WIMCFact "https://github.com/DalgoT4D/dbt_arghyam/blob/master/models/analytics/wimc_meeting_form_responses_fact.sql"
    click TaskView "https://github.com/DalgoT4D/dbt_arghyam/blob/master/models/analytics/task_view.sql"
```
