# databricks/tmm architecture

> Architecture diagram of the GitHub repository databricks/tmm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/databricks/tmm
- Repository: https://github.com/databricks/tmm

## Overview

##### Project type and purpose
- At the top level, this is not a single application but a monorepo of multiple “Tech Marketing” training and demo projects. Each folder (e.g., Cookies-DataEng-DAB, DLT-Kafka-Sink, Genie-Speak, GenAI-Workshop, Lakeflow-DataEng-Workshop, etc.) is a standalone workshop or demo illustrating data-engineering pipelines, ML/GenAI workflows, API services, or ETL patterns on platforms like Databricks, Kafka, Flask, etc.
- The README confirms that these are “examples…for training and tech demos,” with no single cohesive runtime but a catalog of reference implementations.

##### File structure
- Top-level directories ≈ individual demo/workshop modules.
- Common file types per module:
- .py and .ipynb (Python scripts and notebooks) for logic and interactive demos
- .sql, .dbquery, .yml (ETL pipeline definitions, SQL transforms, orchestration configs)
- JSON/YAML dashboards and config templates (e.g., Databricks, Kafka, aibi dashboards)
- resources or misc folders for sample data, schemas, or ancillary scripts
- Cross-cutting files/folders:
- sample_data for seeds
- shared license, .gitignore
- No single build script—each module is self-contained.

##### README for additional insights
- Confirms monorepo of “as-is examples,” no enforced architecture described.

##### Final advice
- Be exhaustive: represent each folder as a component group, then drill into its main scripts, configs, and data flows.
- Medallion pattern in ETL demos and service pipelines in API demos.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Monorepo: Tech Marketing Demos"
        direction TB
        SampleData["Sample Data Assets"]:::util

        subgraph "DataEng DLT Pipelines"
            direction TB
            Cookies["Cookies-DataEng-DAB"]:::dataeng
            KafkaSink["DLT-Kafka-Sink"]:::dataeng
            NYTaxi["NY-Taxi-Fares"]:::dataeng
        end

        subgraph "Lakehouse Workshops"
            direction TB
            Lakeflow["Lakeflow-DataEng-Workshop"]:::dataeng
            HyperPersonal["Lakeflow_Hyper_Personalization"]:::dataeng
        end

        subgraph "Streaming Kafka Demos"
            direction TB
            KafkaSwift["NASA-swift-genie"]:::dataeng
            Circulars["NASA-circulars-rag"]:::dataeng
        end

        subgraph "API / Microservice"
            direction TB
            GenieSpeak["Genie-Speak"]:::api
        end

        subgraph "ML / GenAI Workshops"
            direction TB
            GenAIWorkshop["GenAI-Workshop"]:::ml
            DSMLWorkshop["DSML-Workshop"]:::ml
            Agents["agents-workshop"]:::ml
            EDA["eda-starter-notebook"]:::ml
        end

        subgraph "ETL Workshops"
            direction TB
            Pipelines["Pipelines-Workshop"]:::dataeng
            XMLIngest["XML-ingest-demo"]:::dataeng
            SystemTables["System-Tables-Demo"]:::dataeng
        end
    end

    %% Data flow from sample data
    SampleData --> Cookies
    SampleData --> KafkaSink
    SampleData --> NYTaxi
    SampleData --> Lakeflow
    SampleData --> HyperPersonal
    SampleData --> Pipelines
    SampleData --> XMLIngest
    SampleData --> SystemTables
    SampleData --> KafkaSwift

    classDef dataeng fill:#ADD8E6,stroke:#000,stroke-width:1px;
    classDef ml fill:#90EE90,stroke:#000,stroke-width:1px;
    classDef api fill:#FFD700,stroke:#000,stroke-width:1px;
    classDef util fill:#D3D3D3,stroke:#000,stroke-width:1px;

    %% Click events
    click SampleData "https://github.com/databricks/tmm/tree/main/sample_data/"
    click Cookies "https://github.com/databricks/tmm/tree/main/Cookies-DataEng-DAB/"
    click KafkaSink "https://github.com/databricks/tmm/tree/main/DLT-Kafka-Sink/"
    click NYTaxi "https://github.com/databricks/tmm/tree/main/NY-Taxi-Fares/"
    click Lakeflow "https://github.com/databricks/tmm/tree/main/Lakeflow-DataEng-Workshop/"
    click HyperPersonal "https://github.com/databricks/tmm/tree/main/Lakeflow_Hyper_Personalization/"
    click KafkaSwift "https://github.com/databricks/tmm/tree/main/NASA-swift-genie/"
    click Circulars "https://github.com/databricks/tmm/tree/main/NASA-circulars-rag/"
    click GenieSpeak "https://github.com/databricks/tmm/tree/main/Genie-Speak/"
    click GenAIWorkshop "https://github.com/databricks/tmm/tree/main/GenAI-Workshop/"
    click DSMLWorkshop "https://github.com/databricks/tmm/tree/main/DSML-Workshop/"
    click Agents "https://github.com/databricks/tmm/tree/main/agents-workshop/"
    click EDA "https://github.com/databricks/tmm/tree/main/eda-starter-notebook/"
    click Pipelines "https://github.com/databricks/tmm/tree/main/Pipelines-Workshop/"
    click XMLIngest "https://github.com/databricks/tmm/tree/main/XML-ingest-demo/"
    click SystemTables "https://github.com/databricks/tmm/tree/main/System-Tables-Demo/"
```
