# sujanks/data-generator architecture

> Architecture diagram of the GitHub repository sujanks/data-generator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sujanks/data-generator
- Repository: https://github.com/sujanks/data-generator

## Overview

Data Generator is a Go command-line tool that turns YAML table manifests into synthetic records and writes them to a configured sink. The main workflow runs from the CLI through manifest parsing, dependency-aware record generation, rule evaluation, and sink insertion. The README also describes validation, rich built-in and Cassandra-specific types, and CSV output. Some of those capabilities are documented but not fully visible in the excerpts; the graph shows them without inferring unshown internal calls.

Main entry points: [`data-gen-app/pkg/generate_data.go`](https://github.com/sujanks/data-generator/blob/main/data-gen-app/pkg/generate_data.go) (Generator), [`data-gen-app/generate.go`](https://github.com/sujanks/data-generator/blob/main/data-gen-app/generate.go) (CLI), [`data-gen-app/pkg/sink/postgres_sink.go`](https://github.com/sujanks/data-generator/blob/main/data-gen-app/pkg/sink/postgres_sink.go) (Postgres sink), [`data-gen-app/pkg/sink/sink.go`](https://github.com/sujanks/data-generator/blob/main/data-gen-app/pkg/sink/sink.go) (Sink interface)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Command entry"]
  node_cli["CLI<br/>[generate.go]"]
  node_manifest["YAML manifest<br/>[application.yaml]"]
end

subgraph group_generation["Generation core"]
  node_generator["Generator<br/>[generate_data.go]"]
  node_schema["Schema model<br/>[schema.go]"]
  node_valuegen["Value generators<br/>[schema.go]"]
  node_rules["Rules engine<br/>[generate_data.go]"]
  node_validation["Constraints<br/>[schema.go]"]
end

subgraph group_outputs["Output sinks"]
  node_csv["CSV sink<br/>[csv_sink.go]"]
  node_sinkapi["Sink interface<br/>[sink.go]"]
  node_pgsink["Postgres sink<br/>[postgres_sink.go]"]
end

subgraph group_database["Database"]
  node_postgres[("Postgres database")]
  node_sqlschema["Application tables<br/>[application.sql]"]
end

node_user(("Operator"))

node_user -->|"runs"| node_cli
node_cli -->|"selects"| node_manifest
node_cli -->|"invokes"| node_generator
node_cli -->|"configures"| node_pgsink
node_generator -->|"parses"| node_schema
node_generator -->|"generates values"| node_valuegen
node_generator -->|"applies rules"| node_rules
node_generator -.->|"validates records"| node_validation
node_generator -->|"inserts records"| node_sinkapi
node_csv -.->|"implements"| node_sinkapi
node_pgsink -->|"implements"| node_sinkapi
node_pgsink -->|"writes records"| node_postgres
node_sqlschema -.->|"defines tables"| node_postgres

click node_cli "https://github.com/sujanks/data-generator/blob/main/data-gen-app/generate.go"
click node_manifest "https://github.com/sujanks/data-generator/blob/main/data-gen-app/manifest/application.yaml"
click node_generator "https://github.com/sujanks/data-generator/blob/main/data-gen-app/pkg/generate_data.go"
click node_schema "https://github.com/sujanks/data-generator/blob/main/data-gen-app/pkg/types/schema.go"
click node_valuegen "https://github.com/sujanks/data-generator/blob/main/data-gen-app/pkg/types/schema.go"
click node_rules "https://github.com/sujanks/data-generator/blob/main/data-gen-app/pkg/generate_data.go"
click node_validation "https://github.com/sujanks/data-generator/blob/main/data-gen-app/pkg/types/schema.go"
click node_csv "https://github.com/sujanks/data-generator/blob/main/data-gen-app/pkg/sink/csv_sink.go"
click node_sinkapi "https://github.com/sujanks/data-generator/blob/main/data-gen-app/pkg/sink/sink.go"
click node_pgsink "https://github.com/sujanks/data-generator/blob/main/data-gen-app/pkg/sink/postgres_sink.go"
click node_sqlschema "https://github.com/sujanks/data-generator/blob/main/db/schemas/application/application.sql"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_manifest toneBlue
class node_generator,node_schema,node_valuegen,node_rules,node_validation toneAmber
class node_csv,node_sinkapi,node_pgsink toneMint
class node_postgres,node_sqlschema toneRose
class node_user toneIndigo
```

## Components

### Command entry

- [CLI](https://github.com/sujanks/data-generator/blob/main/data-gen-app/generate.go): `data-gen-app/generate.go`
- [YAML manifest](https://github.com/sujanks/data-generator/blob/main/data-gen-app/manifest/application.yaml): `data-gen-app/manifest/application.yaml`

### Generation core

- [Generator](https://github.com/sujanks/data-generator/blob/main/data-gen-app/pkg/generate_data.go): `data-gen-app/pkg/generate_data.go`
- [Schema model](https://github.com/sujanks/data-generator/blob/main/data-gen-app/pkg/types/schema.go): `data-gen-app/pkg/types/schema.go`
- [Value generators](https://github.com/sujanks/data-generator/blob/main/data-gen-app/pkg/types/schema.go): `data-gen-app/pkg/types/schema.go`
- [Rules engine](https://github.com/sujanks/data-generator/blob/main/data-gen-app/pkg/generate_data.go): `data-gen-app/pkg/generate_data.go`
- [Constraints](https://github.com/sujanks/data-generator/blob/main/data-gen-app/pkg/types/schema.go): `data-gen-app/pkg/types/schema.go`

### Output sinks

- [CSV sink](https://github.com/sujanks/data-generator/blob/main/data-gen-app/pkg/sink/csv_sink.go): `data-gen-app/pkg/sink/csv_sink.go`
- [Sink interface](https://github.com/sujanks/data-generator/blob/main/data-gen-app/pkg/sink/sink.go): `data-gen-app/pkg/sink/sink.go`
- [Postgres sink](https://github.com/sujanks/data-generator/blob/main/data-gen-app/pkg/sink/postgres_sink.go): `data-gen-app/pkg/sink/postgres_sink.go`

### Database

- **Postgres database**
- [Application tables](https://github.com/sujanks/data-generator/blob/main/db/schemas/application/application.sql): `db/schemas/application/application.sql`

### Other components

- **Operator**

## Connections

- Operator → CLI: runs (evidence: `README.md`)
- CLI → YAML manifest: selects (evidence: `data-gen-app/generate.go`)
- CLI → Generator: invokes (evidence: `data-gen-app/generate.go`)
- CLI → Postgres sink: configures (evidence: `data-gen-app/generate.go`)
- Generator → Schema model: parses (evidence: `data-gen-app/pkg/generate_data.go`)
- Generator → Value generators: generates values (evidence: `data-gen-app/pkg/generate_data.go`)
- Generator → Rules engine: applies rules (evidence: `data-gen-app/pkg/generate_data.go`)
- Generator → Constraints: validates records (evidence: `README.md`)
- Generator → Sink interface: inserts records (evidence: `data-gen-app/pkg/generate_data.go`)
- CSV sink → Sink interface: implements (evidence: `README.md`)
- Postgres sink → Sink interface: implements (evidence: `data-gen-app/pkg/sink/postgres_sink.go`)
- Postgres sink → Postgres database: writes records (evidence: `data-gen-app/pkg/sink/postgres_sink.go`)
- Application tables → Postgres database: defines tables