How 5e-bits/5e-database works
1. What this repo is
- This repository is not the API itself; it is the canonical data store for the D&D 5e API.
- The main product boundary is source JSON data in the repo → MongoDB database content consumed by the public API at dnd5eapi.co.
2. Primary domain split: editions
- The data is separated into two top-level edition tracks:
- src/2014/ for the 2014 ruleset
- src/2024/ for the 2024 ruleset
- This is an important architectural boundary: the two editions are managed in parallel, but as distinct datasets.
3. Data model shape
- Most files under src/2014 and src/2024 are one-collection JSON documents for core rules entities such as spells, monsters, equipment, feats, classes, backgrounds, traits, and reference tables.
- These files appear to be the authoritative source of record, likely mapped directly into MongoDB collections with API-friendly structures.
4. Edition-specific content differences
- The 2014 set includes legacy categories such as monsters, races, subclasses, subraces, rule sections, and levels.
- The 2024 set introduces newer structural concepts like species, subspecies, and weapon mastery properties, while omitting some 2014-only structures.
- This implies the repository supports two schemas/content models rather than one generic schema.
5. Ingestion and refresh pipeline
- The scripts/ directory contains the operational layer that turns repo data into a database:
- dbRefresh.ts for rebuilding/loading the database from source data
- dbUpdate.ts for applying updates
- dbUtils.ts for shared database helpers
- The repo’s local workflow in the README confirms that the database can be loaded into a local MongoDB instance with npm run db:refresh.
6. Update automation
- scripts/update/processor.ts and scripts/update/gitUtils.ts suggest a higher-level update workflow that likely pulls or processes upstream changes before database refresh.
- This is the repo’s change-application boundary: source data is curated here, then transformed and loaded through scripts.
7. Database/runtime layer
- MongoDB is the only runtime dependency explicitly visible in the architecture.
- docker/mongodb.conf and the Dockerfile indicate the database can be run containerized, with local MongoDB also supported for development.
- The repository therefore serves both as a data source and as a repeatable database provisioning package.
8. Validation and correctness
- Each edition has its own tests under src/2014/tests and src/2024/tests, with tables.test.js in both.
- This suggests the tests focus on data integrity/table structure rather than application behavior, which makes sense for a data repository.
9. Tooling and build concerns
- The repo is Node/TypeScript-based, with vitest, ESLint, and Prettier used for validation and consistency.
- package-lock.json, nvmrc, and scripts/tsconfig.json indicate a controlled Node toolchain for reproducible refresh/update jobs.
10. Operational/CI boundary
- GitHub Actions are used for CI, linting, CodeQL analysis, release automation, and a manual DB refresh workflow.
- The release tooling matters because this repo’s output is effectively a versioned database artifact, not just source code.
11. How to diagram it
- A useful architecture diagram would show:
- curated JSON data by edition at the top
- update/refresh scripts in the middle
- MongoDB as the persistence target
- the public D&D 5e API as the consumer outside this repo
- The clearest internal boundary is data authoring vs. database loading, with 2014 vs. 2024 as the key domain partition.
Main entry points: dbUpdate.ts DB update · dbRefresh.ts DB refresh · workflows GitHub Actions · mongodb.conf MongoDB
Components 18
Source data
Edition-specific canonical JSON content
- 2014 data · Canonical JSON source data for the 2014 ruleset.
src/2014 - 2024 data · Canonical JSON source data for the 2024 ruleset.
src/2024 - 2014 tables · Core rules collections such as spells, monsters, equipment, classes, races, and rules.
src/2014 - 2024 tables · Core rules collections such as spells, equipment, feats, species, subclasses, and traits.
src/2024 - 2014 tests · Table integrity checks for the 2014 dataset.
src/2014/tests/tables.test.js - 2024 tests · Table integrity checks for the 2024 dataset.
src/2024/tests/tables.test.js
Refresh pipeline
Scripts that transform source data into database content
- DB refresh · Rebuilds or loads MongoDB from the repository source data.
scripts/dbRefresh.ts - DB update · Applies incremental updates to the database content.
scripts/dbUpdate.ts - DB utils · Shared database helper functions used by refresh and update jobs.
scripts/dbUtils.ts - Update processor · Coordinates higher-level update handling before database loading.
scripts/update/processor.ts - Git utils · Git-oriented helpers supporting the update workflow.
scripts/update/gitUtils.ts
Database runtime
MongoDB provisioning and local database environment
- MongoDB · Persistent content store populated from the source JSON.
docker/mongodb.conf - Docker build · Container image setup for local database/runtime provisioning.
Dockerfile
Tooling and CI
Validation, release, and automation around the data repo
- Vitest · Repository test runner for data validation and script checks.
vitest.config.ts - ESLint · Static code and repository consistency checks.
eslint.config.mjs - GitHub Actions · CI, lint, release, CodeQL, and manual refresh workflows.
.github/workflows - Release flow · Versioned release and changelog automation for the data repository.
.github/workflows/release.yml
Other components
- D&D API · Public API consumer of the MongoDB content store.
Connections 22
- 2014 data → 2014 tables: contains · 2014 edition source JSON organized as authoritative collections.
- 2024 data → 2024 tables: contains · 2024 edition source JSON organized as authoritative collections.
- 2014 data → 2014 tests: validated by · 2014 data integrity checks.
- 2024 data → 2024 tests: validated by · 2024 data integrity checks.
- 2014 tables → DB refresh: loaded by · 2014 content is ingested into MongoDB through the refresh job.
- 2024 tables → DB refresh: loaded by · 2024 content is ingested into MongoDB through the refresh job.
- 2014 tables → DB update: updated by · 2014 content can be changed through the update workflow.
- 2024 tables → DB update: updated by · 2024 content can be changed through the update workflow.
- DB refresh → DB utils: uses · Refresh job shares database helpers.
- DB update → DB utils: uses · Update job shares database helpers.
- DB update → Update processor: drives · Update job coordinates higher-level processing.
- Update processor → Git utils: uses · Processor relies on git helpers for update workflow.
- DB refresh → MongoDB: writes · Refresh job rebuilds database content.
- DB update → MongoDB: writes · Update job applies changes to database content.
- Docker build → MongoDB: packages · Containerized runtime for MongoDB development and provisioning.
- MongoDB → D&D API: serves · MongoDB content backs the public API.
- 2014 tests → Vitest: runs on · 2014 validation is executed with Vitest.
- 2024 tests → Vitest: runs on · 2024 validation is executed with Vitest.
- GitHub Actions → DB refresh: automates · Manual and CI refresh workflows invoke the database loader.
- GitHub Actions → Vitest: runs · CI executes validation tests.
- GitHub Actions → ESLint: runs · CI performs lint checks.
- GitHub Actions → Release flow: drives · Release automation publishes versioned dataset updates.