# wingriders/datum-explorer architecture

> Architecture diagram of the GitHub repository wingriders/datum-explorer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/wingriders/datum-explorer
- Repository: https://github.com/wingriders/datum-explorer

## Overview

##### Main Components
- Frontend Application:
- This is contained in the “app” directory (specifically in app/src/app and its components).
- It provides the interactive UI (pages, layout, and components such as DatumDisplay, local schema forms, etc.) for decoding CBOR data and suggesting schemas.
- Backend API Services:
- Located under the “app/src/api” folder, you have endpoints for GitHub webhooks, healthchecks, CBOR parsing (e.g. parse-cbor-detect) and schema retrieval (including caching logic).
- These endpoints support schema registry updates, cache mechanisms (triggered by events like a push to main), and other backend operations.
- CBOR Parsing & Schema Library:
- The “lib” directory holds the core CBOR parsing functionality, validation, and the CLI tool.
- This includes both standard TypeScript modules (in lib/src) and the Rust-based code (in lib/crate) used for building WebAssembly. These modules encapsulate core logic that both the frontend/backend and CLI leverage.
- Worker Processes:
- Inside “app/src/workers”, workers (parseCborWorker and parseCborDetectWorker) run tasks in separate threads/processes for CBOR parsing, ensuring a non-blocking frontend experience.
- Continuous Integration/Deployment and Docker:
- The .github/workflows directory and the docker directory show that deployment automation and container orchestration are integral parts of the project.
- CI pipelines triggering tests, builds, and deployments as part of the external tooling.

##### Relationships and Data Flow
- User interactions in the frontend trigger API calls (for example, to decode CBOR or access cached schema data) and may also initiate asynchronous tasks that are handled by dedicated workers.
- The backend API layer receives calls from both the frontend and external sources (like GitHub webhook events for automatic schema cache updates).
- The core CBOR parsing logic and CDDL schema validation (located in lib) is used in two contexts:
- Through a CLI tool for command-line parsing and validation.
- As a runtime library for both the frontend/back-end via WebAssembly (built from Rust) and Node/Bun.
- External services (specifically GitHub for the schema registry) are integrated via a dedicated API endpoint that accepts webhook events and triggers caching/update logic.

##### Important Architectural Patterns and Technologies
- Separation of Concerns: Clear separation between the presentation layer (Next.js frontend), business logic (API routes and workers), and core services (CBOR parsing library/CLI).
- Microservice-ish API design: Although part of one codebase, the API endpoints (healthcheck, webhook, parse-cbor-detect, schemas) serve as mini services.
- Use of Workers for heavy parsing tasks: This offloads resource-intensive computations from the main thread.
- Integration with external systems: GitHub’s API for schema registry and automated cache updates.
- Multi-runtime environment: Bun for the JavaScript/TypeScript processes and Rust for performance-critical parts (via wasm-pack) in the CLI.

## Diagram (Mermaid)

```mermaid
graph TD
    UI["Frontend Decoder (Next.js UI)"]:::frontend
    API["Backend API Services"]:::backend
    Lib["CBOR Parsing & Schema Library"]:::lib
    CLI["CLI Tool"]:::cli

    subgraph "Worker Processes"
        PCW["parseCborWorker"]:::worker
        PCDW["parseCborDetectWorker"]:::worker
    end

    GitHub["External GitHub Webhook"]:::external
    CICD["CI/CD & Deployment"]:::cicd

    UI -->|"APIcall"| API
    API -->|"AsyncTask"| PCW
    API -->|"AsyncTask"| PCDW
    API -->|"ParseValidate"| Lib
    CLI -->|"Uses"| Lib
    GitHub -->|"WebhookEvent"| API
    CICD -->|"DeployUpdate"| API

    click UI "https://github.com/wingriders/datum-explorer/tree/main/app/src/app"
    click API "https://github.com/wingriders/datum-explorer/tree/main/app/src/api"
    click API "https://github.com/wingriders/datum-explorer/tree/main/app/src/app/api"
    click Lib "https://github.com/wingriders/datum-explorer/tree/main/lib/src"
    click Lib "https://github.com/wingriders/datum-explorer/tree/main/lib/crate"
    click PCW "https://github.com/wingriders/datum-explorer/tree/main/app/src/workers/parseCborWorker"
    click PCDW "https://github.com/wingriders/datum-explorer/tree/main/app/src/workers/parseCborDetectWorker"
    click CLI "https://github.com/wingriders/datum-explorer/blob/main/lib/src/cli.ts"
    click CLI "https://github.com/wingriders/datum-explorer/tree/main/lib/crate"
    click CICD "https://github.com/wingriders/datum-explorer/tree/main/.github/workflows"
    click CICD "https://github.com/wingriders/datum-explorer/tree/main/docker"

    classDef frontend fill:#bbdefb,stroke:#0d47a1,stroke-width:2px;
    classDef backend fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px;
    classDef lib fill:#fff9c4,stroke:#fdd835,stroke-width:2px;
    classDef worker fill:#d1c4e9,stroke:#7e57c2,stroke-width:2px;
    classDef cli fill:#ffe0b2,stroke:#fb8c00,stroke-width:2px;
    classDef cicd fill:#efebe9,stroke:#6d4c41,stroke-width:2px;
    classDef external fill:#ffcdd2,stroke:#c62828,stroke-width:2px;
```
