# scalefreecom/turbovault-engine architecture

> Architecture diagram of the GitHub repository scalefreecom/turbovault-engine, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/scalefreecom/turbovault-engine
- Repository: https://github.com/scalefreecom/turbovault-engine

## Overview

TurboVault Engine turns source metadata into Data Vault models and then generates dbt projects or exports the model in other formats. The principal path runs from CLI or web project setup through metadata import and model management, into export-model construction, validation/planning/rendering, and artifact writing. The README also documents admin-based model editing, configurable templates, and JSON, DBML, and IRiS exports. Some import and generation stage links are documented rather than shown in sampled source; those edges cite the README or are marked unverified.

Main entry points: [`backend/engine/cli/main.py`](https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/cli/main.py) (CLI commands), [`backend/engine/services/generation/pipeline.py`](https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/generation/pipeline.py) (Generation pipeline), [`backend/engine/services/imports/executor.py`](https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/imports/executor.py) (Import executor), [`backend/engine/services/imports`](https://github.com/scalefreecom/turbovault-engine/tree/main/backend/engine/services/imports) (Metadata import)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["User interfaces"]
  node_cli["CLI commands<br/>[main.py]"]
  node_web["Web initializer<br/>[views.py]"]
  node_admin["Django Admin<br/>[admin.py]"]
end

subgraph group_metadata["Metadata and modeling"]
  node_project["Project setup<br/>[project.py]"]
  node_import["Metadata import"]
  node_parsers["Format parsers"]
  node_import_executor["Import executor<br/>[executor.py]"]
  node_source_metadata["Source metadata<br/>[source_metadata.py]"]
  node_vault_model[("Data Vault model")]
  node_staging[("Staging mappings<br/>[staging.py]")]
end

subgraph group_generation["Generation and export"]
  node_export_builder["Export model builder<br/>[builder.py]"]
  node_export_model["Intermediate export model<br/>[models.py]"]
  node_pipeline["Generation pipeline<br/>[pipeline.py]"]
  node_validation["Model validation<br/>[validate.py]"]
  node_planning["Output planning<br/>[plan.py]"]
  node_rendering["Rendering and exports<br/>[render.py]"]
  node_dbt_generator["dbt project generator<br/>[generator.py]"]
  node_artifacts["Generated artifacts<br/>[write.py]"]
  node_other_exports["JSON, DBML, IRiS"]
end

subgraph group_configuration["Configuration"]
  node_runtime_config["Runtime configuration<br/>[runtime_config.py]"]
  node_templates[("Generation templates<br/>[templates.py]")]
end

node_engineer(("Engineer"))

node_engineer -->|"runs commands"| node_cli
node_engineer -->|"uses wizard"| node_web
node_engineer -->|"manages models"| node_admin
node_cli -.->|"creates projects"| node_project
node_web -->|"creates project"| node_project
node_web -->|"imports metadata"| node_import
node_cli -.->|"imports metadata"| node_import
node_import -.->|"selects format parser"| node_parsers
node_import -->|"executes import"| node_import_executor
node_import_executor -->|"writes entities"| node_vault_model
node_import_executor -->|"resolves source data"| node_source_metadata
node_import_executor -->|"creates mappings"| node_staging
node_admin -->|"edits model"| node_vault_model
node_cli -->|"manages entities"| node_vault_model
node_vault_model -->|"builds export from"| node_export_builder
node_runtime_config -->|"configures build"| node_export_builder
node_export_builder -->|"produces"| node_export_model
node_cli -.->|"requests generation"| node_pipeline
node_pipeline -->|"builds export"| node_export_builder
node_pipeline -->|"validates model"| node_validation
node_pipeline -->|"builds plan"| node_planning
node_pipeline -->|"renders output"| node_rendering
node_pipeline -.->|"writes output"| node_artifacts
node_rendering -->|"renders dbt"| node_dbt_generator
node_dbt_generator -->|"resolves templates"| node_templates
node_rendering -->|"renders formats"| node_other_exports
node_cli -->|"exports model"| node_other_exports
node_project -->|"provides defaults"| node_runtime_config

click node_cli "https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/cli/main.py"
click node_web "https://github.com/scalefreecom/turbovault-engine/blob/main/backend/turbovault/views.py"
click node_admin "https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/admin.py"
click node_project "https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/models/project.py"
click node_import "https://github.com/scalefreecom/turbovault-engine/tree/main/backend/engine/services/imports"
click node_parsers "https://github.com/scalefreecom/turbovault-engine/tree/main/backend/engine/services/imports/parsers"
click node_import_executor "https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/imports/executor.py"
click node_source_metadata "https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/models/source_metadata.py"
click node_vault_model "https://github.com/scalefreecom/turbovault-engine/tree/main/backend/engine/models"
click node_staging "https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/models/staging.py"
click node_runtime_config "https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/runtime_config.py"
click node_templates "https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/models/templates.py"
click node_export_builder "https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/export/builder.py"
click node_export_model "https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/export/models.py"
click node_pipeline "https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/generation/pipeline.py"
click node_validation "https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/generation/stages/validate.py"
click node_planning "https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/generation/stages/plan.py"
click node_rendering "https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/generation/stages/render.py"
click node_dbt_generator "https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/generation/generator.py"
click node_artifacts "https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/generation/stages/write.py"
click node_other_exports "https://github.com/scalefreecom/turbovault-engine/tree/main/backend/engine/services/export/exporters"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_web,node_admin toneBlue
class node_project,node_import,node_parsers,node_import_executor,node_source_metadata,node_vault_model,node_staging toneAmber
class node_export_builder,node_export_model,node_pipeline,node_validation,node_planning,node_rendering,node_dbt_generator,node_artifacts,node_other_exports toneMint
class node_runtime_config,node_templates toneRose
class node_engineer toneIndigo
```

## Components

### User interfaces

- [CLI commands](https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/cli/main.py): `backend/engine/cli/main.py`
- [Web initializer](https://github.com/scalefreecom/turbovault-engine/blob/main/backend/turbovault/views.py): `backend/turbovault/views.py`
- [Django Admin](https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/admin.py): `backend/engine/admin.py`

### Metadata and modeling

- [Project setup](https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/models/project.py): `backend/engine/models/project.py`
- [Metadata import](https://github.com/scalefreecom/turbovault-engine/tree/main/backend/engine/services/imports): `backend/engine/services/imports`
- [Format parsers](https://github.com/scalefreecom/turbovault-engine/tree/main/backend/engine/services/imports/parsers): `backend/engine/services/imports/parsers`
- [Import executor](https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/imports/executor.py): `backend/engine/services/imports/executor.py`
- [Source metadata](https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/models/source_metadata.py): `backend/engine/models/source_metadata.py`
- [Data Vault model](https://github.com/scalefreecom/turbovault-engine/tree/main/backend/engine/models): `backend/engine/models`
- [Staging mappings](https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/models/staging.py): `backend/engine/models/staging.py`

### Generation and export

- [Export model builder](https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/export/builder.py): `backend/engine/services/export/builder.py`
- [Intermediate export model](https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/export/models.py): `backend/engine/services/export/models.py`
- [Generation pipeline](https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/generation/pipeline.py): `backend/engine/services/generation/pipeline.py`
- [Model validation](https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/generation/stages/validate.py): `backend/engine/services/generation/stages/validate.py`
- [Output planning](https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/generation/stages/plan.py): `backend/engine/services/generation/stages/plan.py`
- [Rendering and exports](https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/generation/stages/render.py): `backend/engine/services/generation/stages/render.py`
- [dbt project generator](https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/generation/generator.py): `backend/engine/services/generation/generator.py`
- [Generated artifacts](https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/generation/stages/write.py): `backend/engine/services/generation/stages/write.py`
- [JSON, DBML, IRiS](https://github.com/scalefreecom/turbovault-engine/tree/main/backend/engine/services/export/exporters): `backend/engine/services/export/exporters`

### Configuration

- [Runtime configuration](https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/services/runtime_config.py): `backend/engine/services/runtime_config.py`
- [Generation templates](https://github.com/scalefreecom/turbovault-engine/blob/main/backend/engine/models/templates.py): `backend/engine/models/templates.py`

### Other components

- **Engineer**

## Connections

- Engineer → CLI commands: runs commands (evidence: `README.md`)
- Engineer → Web initializer: uses wizard (evidence: `README.md`)
- Engineer → Django Admin: manages models (evidence: `README.md`)
- CLI commands → Project setup: creates projects
- Web initializer → Project setup: creates project (evidence: `backend/turbovault/views.py`)
- Web initializer → Metadata import: imports metadata (evidence: `backend/turbovault/views.py`)
- CLI commands → Metadata import: imports metadata
- Metadata import → Format parsers: selects format parser (evidence: `backend/engine/services/imports/validation/resolver.py`)
- Metadata import → Import executor: executes import (evidence: `backend/engine/services/imports/executor.py`)
- Import executor → Data Vault model: writes entities (evidence: `backend/engine/services/imports/executor.py`)
- Import executor → Source metadata: resolves source data (evidence: `backend/engine/services/imports/executor.py`)
- Import executor → Staging mappings: creates mappings
- Django Admin → Data Vault model: edits model (evidence: `README.md`)
- CLI commands → Data Vault model: manages entities
- Data Vault model → Export model builder: builds export from (evidence: `backend/engine/services/export/builder.py`)
- Runtime configuration → Export model builder: configures build (evidence: `backend/engine/services/export/builder.py`)
- Export model builder → Intermediate export model: produces (evidence: `backend/engine/services/export/builder.py`)
- CLI commands → Generation pipeline: requests generation
- Generation pipeline → Export model builder: builds export (evidence: `backend/engine/services/generation/pipeline.py`)
- Generation pipeline → Model validation: validates model (evidence: `backend/engine/services/generation/pipeline.py`)
- Generation pipeline → Output planning: builds plan (evidence: `backend/engine/services/generation/pipeline.py`)
- Generation pipeline → Rendering and exports: renders output (evidence: `backend/engine/services/generation/pipeline.py`)
- Generation pipeline → Generated artifacts: writes output (evidence: `backend/engine/services/generation/pipeline.py`)
- Rendering and exports → dbt project generator: renders dbt (evidence: `backend/engine/services/generation/stages/render.py`)
- dbt project generator → Generation templates: resolves templates (evidence: `backend/engine/services/generation/generator.py`)
- Rendering and exports → JSON, DBML, IRiS: renders formats (evidence: `backend/engine/services/generation/stages/render.py`)
- CLI commands → JSON, DBML, IRiS: exports model
- Project setup → Runtime configuration: provides defaults (evidence: `backend/engine/services/runtime_config.py`)
