# confluentinc/workshop-data-contracts architecture

> Architecture diagram of the GitHub repository confluentinc/workshop-data-contracts, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/confluentinc/workshop-data-contracts
- Repository: https://github.com/confluentinc/workshop-data-contracts

## Overview

This repository is a hands-on data-contracts workshop, not an application runtime. Its documented workflow prototypes data production, schema governance and quality checks, routes valid and invalid records to separate destinations, and supports review and remediation. The tree identifies schema, generator and infrastructure assets, but no source excerpts were provided; the detailed processing flow below is therefore documented by the README, not verified in implementation. Infrastructure links are limited to those explicit workshop capabilities.

Main entry points: [`terraform/confluent.tf`](https://github.com/confluentinc/workshop-data-contracts/blob/main/terraform/confluent.tf) (Schema Registry), [`data/data_contracts`](https://github.com/confluentinc/workshop-data-contracts/tree/main/data/data_contracts) (Quality rules), [`data/generators`](https://github.com/confluentinc/workshop-data-contracts/tree/main/data/generators) (Data producers), [`data/data_contracts/schemas/clickstream_schema_v1.avsc`](https://github.com/confluentinc/workshop-data-contracts/blob/main/data/data_contracts/schemas/clickstream_schema_v1.avsc) (Clickstream schemas)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_production["Data production"]
  node_producers["Data producers"]
  node_generator_config["Generator configs"]
end

subgraph group_governance["Contract governance"]
  node_contracts["Data contracts"]
  node_schemas["Clickstream schemas"]
  node_schema_evolution["Schema evolution"]
  node_registry["Schema Registry<br/>[confluent.tf]"]
  node_quality["Quality rules"]
end

subgraph group_processing["Stream processing"]
  node_topics["Kafka topics<br/>[confluent.tf]"]
  node_dead_letter["Dead letter queue<br/>[confluent.tf]"]
  node_consumers["Downstream consumers<br/>[databricks.tf]"]
  node_steward(("Data steward"))
  node_remediation["Remediation"]
end

subgraph group_infrastructure["Workshop infrastructure"]
  node_aws_resources["AWS resources<br/>[aws_s3.tf]"]
  node_data_generator_infra["Generator infrastructure<br/>[data_generator.tf]"]
  node_databricks["Databricks workspace<br/>[databricks.tf]"]
end

node_engineer(("Data engineer"))

node_engineer -->|"creates data"| node_producers
node_producers -->|"registers schemas"| node_registry
node_registry -->|"governs validation"| node_quality
node_quality -->|"routes valid"| node_topics
node_quality -->|"routes invalid"| node_dead_letter
node_topics -->|"feeds consumers"| node_consumers
node_dead_letter -->|"flags violations"| node_steward
node_steward -->|"reviews records"| node_remediation
node_remediation -->|"feeds fixes"| node_producers
node_schemas -->|"defines contracts"| node_registry
node_schema_evolution -->|"evolves schemas"| node_registry
node_engineer -->|"provisions environment"| node_aws_resources
node_engineer -->|"uses workspace"| node_databricks

click node_producers "https://github.com/confluentinc/workshop-data-contracts/tree/main/data/generators"
click node_generator_config "https://github.com/confluentinc/workshop-data-contracts/blob/main/data/generators/clickstream_generator.json"
click node_contracts "https://github.com/confluentinc/workshop-data-contracts/tree/main/data/data_contracts"
click node_schemas "https://github.com/confluentinc/workshop-data-contracts/blob/main/data/data_contracts/schemas/clickstream_schema_v1.avsc"
click node_schema_evolution "https://github.com/confluentinc/workshop-data-contracts/blob/main/data/data_contracts/schemas/clickstream_schema_v2.avsc"
click node_registry "https://github.com/confluentinc/workshop-data-contracts/blob/main/terraform/confluent.tf"
click node_quality "https://github.com/confluentinc/workshop-data-contracts/tree/main/data/data_contracts"
click node_topics "https://github.com/confluentinc/workshop-data-contracts/blob/main/terraform/confluent.tf"
click node_dead_letter "https://github.com/confluentinc/workshop-data-contracts/blob/main/terraform/confluent.tf"
click node_consumers "https://github.com/confluentinc/workshop-data-contracts/blob/main/terraform/databricks.tf"
click node_aws_resources "https://github.com/confluentinc/workshop-data-contracts/blob/main/terraform/aws_s3.tf"
click node_data_generator_infra "https://github.com/confluentinc/workshop-data-contracts/blob/main/terraform/data_generator.tf"
click node_databricks "https://github.com/confluentinc/workshop-data-contracts/blob/main/terraform/databricks.tf"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_producers,node_generator_config toneBlue
class node_contracts,node_schemas,node_schema_evolution,node_registry,node_quality toneAmber
class node_topics,node_dead_letter,node_consumers,node_steward,node_remediation toneMint
class node_aws_resources,node_data_generator_infra,node_databricks toneRose
class node_engineer toneIndigo
```

## Components

### Data production

- [Data producers](https://github.com/confluentinc/workshop-data-contracts/tree/main/data/generators): `data/generators`
- [Generator configs](https://github.com/confluentinc/workshop-data-contracts/blob/main/data/generators/clickstream_generator.json): `data/generators/clickstream_generator.json`

### Contract governance

- [Data contracts](https://github.com/confluentinc/workshop-data-contracts/tree/main/data/data_contracts): `data/data_contracts`
- [Clickstream schemas](https://github.com/confluentinc/workshop-data-contracts/blob/main/data/data_contracts/schemas/clickstream_schema_v1.avsc): `data/data_contracts/schemas/clickstream_schema_v1.avsc`
- [Schema evolution](https://github.com/confluentinc/workshop-data-contracts/blob/main/data/data_contracts/schemas/clickstream_schema_v2.avsc): `data/data_contracts/schemas/clickstream_schema_v2.avsc`
- [Schema Registry](https://github.com/confluentinc/workshop-data-contracts/blob/main/terraform/confluent.tf): `terraform/confluent.tf`
- [Quality rules](https://github.com/confluentinc/workshop-data-contracts/tree/main/data/data_contracts): `data/data_contracts`

### Stream processing

- [Kafka topics](https://github.com/confluentinc/workshop-data-contracts/blob/main/terraform/confluent.tf): `terraform/confluent.tf`
- [Dead letter queue](https://github.com/confluentinc/workshop-data-contracts/blob/main/terraform/confluent.tf): `terraform/confluent.tf`
- [Downstream consumers](https://github.com/confluentinc/workshop-data-contracts/blob/main/terraform/databricks.tf): `terraform/databricks.tf`
- **Data steward**
- **Remediation**

### Workshop infrastructure

- [AWS resources](https://github.com/confluentinc/workshop-data-contracts/blob/main/terraform/aws_s3.tf): `terraform/aws_s3.tf`
- [Generator infrastructure](https://github.com/confluentinc/workshop-data-contracts/blob/main/terraform/data_generator.tf): `terraform/data_generator.tf`
- [Databricks workspace](https://github.com/confluentinc/workshop-data-contracts/blob/main/terraform/databricks.tf): `terraform/databricks.tf`

### Other components

- **Data engineer**

## Connections

- Data engineer → Data producers: creates data (evidence: `README.md`)
- Data producers → Schema Registry: registers schemas (evidence: `README.md`)
- Schema Registry → Quality rules: governs validation (evidence: `README.md`)
- Quality rules → Kafka topics: routes valid (evidence: `README.md`)
- Quality rules → Dead letter queue: routes invalid (evidence: `README.md`)
- Kafka topics → Downstream consumers: feeds consumers (evidence: `README.md`)
- Dead letter queue → Data steward: flags violations (evidence: `README.md`)
- Data steward → Remediation: reviews records (evidence: `README.md`)
- Remediation → Data producers: feeds fixes (evidence: `README.md`)
- Clickstream schemas → Schema Registry: defines contracts (evidence: `README.md`)
- Schema evolution → Schema Registry: evolves schemas (evidence: `README.md`)
- Data engineer → AWS resources: provisions environment (evidence: `README.md`)
- Data engineer → Databricks workspace: uses workspace (evidence: `README.md`)
