# 5e-bits/5e-database architecture

> Architecture diagram of the GitHub repository 5e-bits/5e-database, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/5e-bits/5e-database
- Repository: https://github.com/5e-bits/5e-database

## Overview

1. **What this repo is**
   - This repository is not the API itself; it is the **canonical data store** for the D&D 5e API.
   - The main product boundary is **source JSON data in the repo** → **MongoDB database content** consumed by the public API at `dnd5eapi.co`.

2. **Primary domain split: editions**
   - The data is separated into two top-level edition tracks:
     - `src/2014/` for the 2014 ruleset
     - `src/2024/` for the 2024 ruleset
   - This is an important architectural boundary: the two editions are managed in parallel, but as distinct datasets.

3. **Data model shape**
   - Most files under `src/2014` and `src/2024` are one-collection JSON documents for core rules entities such as spells, monsters, equipment, feats, classes, backgrounds, traits, and reference tables.
   - These files appear to be the authoritative source of record, likely mapped directly into MongoDB collections with API-friendly structures.

4. **Edition-specific content differences**
   - The 2014 set includes legacy categories such as monsters, races, subclasses, subraces, rule sections, and levels.
   - The 2024 set introduces newer structural concepts like species, subspecies, and weapon mastery properties, while omitting some 2014-only structures.
   - This implies the repository supports two schemas/content models rather than one generic schema.

5. **Ingestion and refresh pipeline**
   - The `scripts/` directory contains the operational layer that turns repo data into a database:
     - `dbRefresh.ts` for rebuilding/loading the database from source data
     - `dbUpdate.ts` for applying updates
     - `dbUtils.ts` for shared database helpers
   - The repo’s local workflow in the README confirms that the database can be loaded into a local MongoDB instance with `npm run db:refresh`.

6. **Update automation**
   - `scripts/update/processor.ts` and `scripts/update/gitUtils.ts` suggest a higher-level update workflow that likely pulls or processes upstream changes before database refresh.
   - This is the repo’s change-application boundary: source data is curated here, then transformed and loaded through scripts.

7. **Database/runtime layer**
   - MongoDB is the only runtime dependency explicitly visible in the architecture.
   - `docker/mongodb.conf` and the Dockerfile indicate the database can be run containerized, with local MongoDB also supported for development.
   - The repository therefore serves both as a data source and as a repeatable database provisioning package.

8. **Validation and correctness**
   - Each edition has its own tests under `src/2014/tests` and `src/2024/tests`, with `tables.test.js` in both.
   - This suggests the tests focus on data integrity/table structure rather than application behavior, which makes sense for a data repository.

9. **Tooling and build concerns**
   - The repo is Node/TypeScript-based, with `vitest`, ESLint, and Prettier used for validation and consistency.
   - `package-lock.json`, `nvmrc`, and `scripts/tsconfig.json` indicate a controlled Node toolchain for reproducible refresh/update jobs.

10. **Operational/CI boundary**
   - GitHub Actions are used for CI, linting, CodeQL analysis, release automation, and a manual DB refresh workflow.
   - The release tooling matters because this repo’s output is effectively a versioned database artifact, not just source code.

11. **How to diagram it**
   - A useful architecture diagram would show:
     - curated JSON data by edition at the top
     - update/refresh scripts in the middle
     - MongoDB as the persistence target
     - the public D&D 5e API as the consumer outside this repo
   - The clearest internal boundary is **data authoring vs. database loading**, with **2014 vs. 2024** as the key domain partition.

Main entry points: [`scripts/dbUpdate.ts`](https://github.com/5e-bits/5e-database/blob/main/scripts/dbUpdate.ts) (DB update), [`scripts/dbRefresh.ts`](https://github.com/5e-bits/5e-database/blob/main/scripts/dbRefresh.ts) (DB refresh), [`.github/workflows`](https://github.com/5e-bits/5e-database/blob/main/.github/workflows) (GitHub Actions), [`docker/mongodb.conf`](https://github.com/5e-bits/5e-database/blob/main/docker/mongodb.conf) (MongoDB)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Source data"]
  node_src2014["2014 data<br/>edition dataset"]
  node_src2024["2024 data<br/>edition dataset"]
  node_d2014core["2014 tables<br/>json collections"]
  node_d2024core["2024 tables<br/>json collections"]
  node_d2014tests["2014 tests<br/>data validation<br/>[tables.test.js]"]
  node_d2024tests["2024 tests<br/>data validation<br/>[tables.test.js]"]
end

subgraph group_pipeline["Refresh pipeline"]
  node_refresh["DB refresh<br/>load job<br/>[dbRefresh.ts]"]
  node_update["DB update<br/>apply job<br/>[dbUpdate.ts]"]
  node_dbutils["DB utils<br/>shared helpers<br/>[dbUtils.ts]"]
  node_processor["Update processor<br/>change processor<br/>[processor.ts]"]
  node_gitutils["Git utils<br/>git helpers<br/>[gitUtils.ts]"]
end

subgraph group_runtime["Database runtime"]
  node_mongodb[("MongoDB<br/>database<br/>[mongodb.conf]")]
  node_dockerfile["Docker build<br/>container setup"]
end

subgraph group_ops["Tooling and CI"]
  node_vitest["Vitest<br/>test runner<br/>[vitest.config.ts]"]
  node_eslint["ESLint<br/>linting<br/>[eslint.config.mjs]"]
  node_github["GitHub Actions<br/>ci automation<br/>[workflows]"]
  node_release["Release flow<br/>release automation<br/>[release.yml]"]
end

node_api(("D&D API<br/>external consumer"))

node_src2014 -->|"contains"| node_d2014core
node_src2024 -->|"contains"| node_d2024core
node_src2014 -->|"validated by"| node_d2014tests
node_src2024 -->|"validated by"| node_d2024tests
node_d2014core -->|"loaded by"| node_refresh
node_d2024core -->|"loaded by"| node_refresh
node_d2014core -->|"updated by"| node_update
node_d2024core -->|"updated by"| node_update
node_refresh -->|"uses"| node_dbutils
node_update -->|"uses"| node_dbutils
node_update -->|"drives"| node_processor
node_processor -->|"uses"| node_gitutils
node_refresh -->|"writes"| node_mongodb
node_update -->|"writes"| node_mongodb
node_dockerfile -->|"packages"| node_mongodb
node_mongodb -->|"serves"| node_api
node_d2014tests -->|"runs on"| node_vitest
node_d2024tests -->|"runs on"| node_vitest
node_github -->|"automates"| node_refresh
node_github -->|"runs"| node_vitest
node_github -->|"runs"| node_eslint
node_github -->|"drives"| node_release

click node_src2014 "https://github.com/5e-bits/5e-database/tree/main/src/2014"
click node_src2024 "https://github.com/5e-bits/5e-database/tree/main/src/2024"
click node_d2014core "https://github.com/5e-bits/5e-database/tree/main/src/2014"
click node_d2024core "https://github.com/5e-bits/5e-database/tree/main/src/2024"
click node_d2014tests "https://github.com/5e-bits/5e-database/blob/main/src/2014/tests/tables.test.js"
click node_d2024tests "https://github.com/5e-bits/5e-database/blob/main/src/2024/tests/tables.test.js"
click node_refresh "https://github.com/5e-bits/5e-database/blob/main/scripts/dbRefresh.ts"
click node_update "https://github.com/5e-bits/5e-database/blob/main/scripts/dbUpdate.ts"
click node_dbutils "https://github.com/5e-bits/5e-database/blob/main/scripts/dbUtils.ts"
click node_processor "https://github.com/5e-bits/5e-database/blob/main/scripts/update/processor.ts"
click node_gitutils "https://github.com/5e-bits/5e-database/blob/main/scripts/update/gitUtils.ts"
click node_mongodb "https://github.com/5e-bits/5e-database/blob/main/docker/mongodb.conf"
click node_dockerfile "https://github.com/5e-bits/5e-database/tree/main/Dockerfile"
click node_vitest "https://github.com/5e-bits/5e-database/blob/main/vitest.config.ts"
click node_eslint "https://github.com/5e-bits/5e-database/blob/main/eslint.config.mjs"
click node_github "https://github.com/5e-bits/5e-database/blob/main/.github/workflows"
click node_release "https://github.com/5e-bits/5e-database/blob/main/.github/workflows/release.yml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_src2014,node_src2024,node_d2014core,node_d2024core,node_d2014tests,node_d2024tests toneBlue
class node_refresh,node_update,node_dbutils,node_processor,node_gitutils toneAmber
class node_mongodb,node_dockerfile toneMint
class node_vitest,node_eslint,node_github,node_release toneRose
class node_api toneNeutral
```

## Components

### Source data

Edition-specific canonical JSON content

- [2014 data](https://github.com/5e-bits/5e-database/tree/main/src/2014): edition dataset · Canonical JSON source data for the 2014 ruleset. · `src/2014`
- [2024 data](https://github.com/5e-bits/5e-database/tree/main/src/2024): edition dataset · Canonical JSON source data for the 2024 ruleset. · `src/2024`
- [2014 tables](https://github.com/5e-bits/5e-database/tree/main/src/2014): json collections · Core rules collections such as spells, monsters, equipment, classes, races, and rules. · `src/2014`
- [2024 tables](https://github.com/5e-bits/5e-database/tree/main/src/2024): json collections · Core rules collections such as spells, equipment, feats, species, subclasses, and traits. · `src/2024`
- [2014 tests](https://github.com/5e-bits/5e-database/blob/main/src/2014/tests/tables.test.js): data validation · Table integrity checks for the 2014 dataset. · `src/2014/tests/tables.test.js`
- [2024 tests](https://github.com/5e-bits/5e-database/blob/main/src/2024/tests/tables.test.js): data validation · Table integrity checks for the 2024 dataset. · `src/2024/tests/tables.test.js`

### Refresh pipeline

Scripts that transform source data into database content

- [DB refresh](https://github.com/5e-bits/5e-database/blob/main/scripts/dbRefresh.ts): load job · Rebuilds or loads MongoDB from the repository source data. · `scripts/dbRefresh.ts`
- [DB update](https://github.com/5e-bits/5e-database/blob/main/scripts/dbUpdate.ts): apply job · Applies incremental updates to the database content. · `scripts/dbUpdate.ts`
- [DB utils](https://github.com/5e-bits/5e-database/blob/main/scripts/dbUtils.ts): shared helpers · Shared database helper functions used by refresh and update jobs. · `scripts/dbUtils.ts`
- [Update processor](https://github.com/5e-bits/5e-database/blob/main/scripts/update/processor.ts): change processor · Coordinates higher-level update handling before database loading. · `scripts/update/processor.ts`
- [Git utils](https://github.com/5e-bits/5e-database/blob/main/scripts/update/gitUtils.ts): git helpers · Git-oriented helpers supporting the update workflow. · `scripts/update/gitUtils.ts`

### Database runtime

MongoDB provisioning and local database environment

- [MongoDB](https://github.com/5e-bits/5e-database/blob/main/docker/mongodb.conf): database · Persistent content store populated from the source JSON. · `docker/mongodb.conf`
- [Docker build](https://github.com/5e-bits/5e-database/tree/main/Dockerfile): container setup · Container image setup for local database/runtime provisioning. · `Dockerfile`

### Tooling and CI

Validation, release, and automation around the data repo

- [Vitest](https://github.com/5e-bits/5e-database/blob/main/vitest.config.ts): test runner · Repository test runner for data validation and script checks. · `vitest.config.ts`
- [ESLint](https://github.com/5e-bits/5e-database/blob/main/eslint.config.mjs): linting · Static code and repository consistency checks. · `eslint.config.mjs`
- [GitHub Actions](https://github.com/5e-bits/5e-database/blob/main/.github/workflows): ci automation · CI, lint, release, CodeQL, and manual refresh workflows. · `.github/workflows`
- [Release flow](https://github.com/5e-bits/5e-database/blob/main/.github/workflows/release.yml): release automation · Versioned release and changelog automation for the data repository. · `.github/workflows/release.yml`

### Other components

- **D&D API**: external consumer · Public API consumer of the MongoDB content store.

## Connections

- 2014 data → 2014 tables: contains · 2014 edition source JSON organized as authoritative collections.
- 2024 data → 2024 tables: contains · 2024 edition source JSON organized as authoritative collections.
- 2014 data → 2014 tests: validated by · 2014 data integrity checks.
- 2024 data → 2024 tests: validated by · 2024 data integrity checks.
- 2014 tables → DB refresh: loaded by · 2014 content is ingested into MongoDB through the refresh job.
- 2024 tables → DB refresh: loaded by · 2024 content is ingested into MongoDB through the refresh job.
- 2014 tables → DB update: updated by · 2014 content can be changed through the update workflow.
- 2024 tables → DB update: updated by · 2024 content can be changed through the update workflow.
- DB refresh → DB utils: uses · Refresh job shares database helpers.
- DB update → DB utils: uses · Update job shares database helpers.
- DB update → Update processor: drives · Update job coordinates higher-level processing.
- Update processor → Git utils: uses · Processor relies on git helpers for update workflow.
- DB refresh → MongoDB: writes · Refresh job rebuilds database content.
- DB update → MongoDB: writes · Update job applies changes to database content.
- Docker build → MongoDB: packages · Containerized runtime for MongoDB development and provisioning.
- MongoDB → D&D API: serves · MongoDB content backs the public API.
- 2014 tests → Vitest: runs on · 2014 validation is executed with Vitest.
- 2024 tests → Vitest: runs on · 2024 validation is executed with Vitest.
- GitHub Actions → DB refresh: automates · Manual and CI refresh workflows invoke the database loader.
- GitHub Actions → Vitest: runs · CI executes validation tests.
- GitHub Actions → ESLint: runs · CI performs lint checks.
- GitHub Actions → Release flow: drives · Release automation publishes versioned dataset updates.
