# feluelle/kind-data-platform architecture

> Architecture diagram of the GitHub repository feluelle/kind-data-platform, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/feluelle/kind-data-platform
- Repository: https://github.com/feluelle/kind-data-platform

## Overview

##### Main Components Identification
The system consists of several key data platform services:
- Airbyte (Data Integration)
- Airflow (Workflow Orchestration)
- dbt (Data Transformation)
- Grafana (Data Visualization)
- Prometheus (Monitoring)
- Superset (Data Analytics)
- LocalStack (AWS Services Emulation)

##### Implementation Details

- Kubernetes namespace boundaries
- Terraform management layer
- LocalStack services (S3, Secrets Manager)
- Airflow DAGs connection to Airbyte and dbt
- Prometheus metrics flow to Grafana
- Clear labeling of all components and their versions

## Diagram (Mermaid)

```mermaid
graph TB
    %% Infrastructure Layer
    subgraph Infrastructure
        KIND[Kind Kubernetes Cluster]
        LOCALSTACK[LocalStack AWS Services]
        S3[(S3 Storage)]
        SECRETS[(Secrets Manager)]
    end

    %% Data Processing Layer
    subgraph DataProcessing
        AIRBYTE[Airbyte]
        DBT[dbt Transformations]
        AIRFLOW[Airflow Orchestration]
    end

    %% Visualization Layer
    subgraph Visualization
        GRAFANA[Grafana Dashboards]
        SUPERSET[Superset Analytics]
        PROMETHEUS[Prometheus Monitoring]
    end

    %% Relationships
    KIND --> LOCALSTACK
    LOCALSTACK --> S3
    LOCALSTACK --> SECRETS
    AIRBYTE --> S3
    S3 --> DBT
    DBT --> S3
    AIRFLOW -.-> AIRBYTE
    AIRFLOW -.-> DBT
    PROMETHEUS --> GRAFANA
    S3 --> SUPERSET

    %% Click Events
    click KIND "https://github.com/feluelle/kind-data-platform/blob/main/infra/kind-with-registry.sh"
    click LOCALSTACK "https://github.com/feluelle/kind-data-platform/tree/main/infra/localstack-services/"
    click AIRBYTE "https://github.com/feluelle/kind-data-platform/blob/main/infra/airbyte.tf"
    click AIRFLOW "https://github.com/feluelle/kind-data-platform/tree/main/airflow/"
    click DBT "https://github.com/feluelle/kind-data-platform/tree/main/dbt/"
    click GRAFANA "https://github.com/feluelle/kind-data-platform/tree/main/grafana/"
    click SUPERSET "https://github.com/feluelle/kind-data-platform/blob/main/infra/superset.tf"
    click PROMETHEUS "https://github.com/feluelle/kind-data-platform/blob/main/infra/prometheus.tf"

    %% Styling
    classDef infrastructure fill:#b3d1ff,stroke:#333,stroke-width:2px
    classDef processing fill:#c2f0c2,stroke:#333,stroke-width:2px
    classDef visualization fill:#e6ccff,stroke:#333,stroke-width:2px
    classDef storage fill:#f2f2f2,stroke:#333,stroke-width:2px

    class KIND,LOCALSTACK infrastructure
    class AIRBYTE,DBT,AIRFLOW processing
    class GRAFANA,SUPERSET,PROMETHEUS visualization
    class S3,SECRETS storage

    %% Legend
    subgraph Legend
        L1[Infrastructure]:::infrastructure
        L2[Processing]:::processing
        L3[Visualization]:::visualization
        L4[(Storage)]:::storage
    end
```
