How wingriders/datum-explorer works
Main Components
• Frontend Application:
• This is contained in the “app” directory (specifically in app/src/app and its components).
• It provides the interactive UI (pages, layout, and components such as DatumDisplay, local schema forms, etc.) for decoding CBOR data and suggesting schemas.
• Backend API Services:
• Located under the “app/src/api” folder, you have endpoints for GitHub webhooks, healthchecks, CBOR parsing (e.g. parse-cbor-detect) and schema retrieval (including caching logic).
• These endpoints support schema registry updates, cache mechanisms (triggered by events like a push to main), and other backend operations.
• CBOR Parsing & Schema Library:
• The “lib” directory holds the core CBOR parsing functionality, validation, and the CLI tool.
• This includes both standard TypeScript modules (in lib/src) and the Rust-based code (in lib/crate) used for building WebAssembly. These modules encapsulate core logic that both the frontend/backend and CLI leverage.
• Worker Processes:
• Inside “app/src/workers”, workers (parseCborWorker and parseCborDetectWorker) run tasks in separate threads/processes for CBOR parsing, ensuring a non-blocking frontend experience.
• Continuous Integration/Deployment and Docker:
• The .github/workflows directory and the docker directory show that deployment automation and container orchestration are integral parts of the project.
• CI pipelines triggering tests, builds, and deployments as part of the external tooling.
Relationships and Data Flow
• User interactions in the frontend trigger API calls (for example, to decode CBOR or access cached schema data) and may also initiate asynchronous tasks that are handled by dedicated workers.
• The backend API layer receives calls from both the frontend and external sources (like GitHub webhook events for automatic schema cache updates).
• The core CBOR parsing logic and CDDL schema validation (located in lib) is used in two contexts:
• Through a CLI tool for command-line parsing and validation.
• As a runtime library for both the frontend/back-end via WebAssembly (built from Rust) and Node/Bun.
• External services (specifically GitHub for the schema registry) are integrated via a dedicated API endpoint that accepts webhook events and triggers caching/update logic.
Important Architectural Patterns and Technologies
• Separation of Concerns: Clear separation between the presentation layer (Next.js frontend), business logic (API routes and workers), and core services (CBOR parsing library/CLI).
• Microservice-ish API design: Although part of one codebase, the API endpoints (healthcheck, webhook, parse-cbor-detect, schemas) serve as mini services.
• Use of Workers for heavy parsing tasks: This offloads resource-intensive computations from the main thread.
• Integration with external systems: GitHub’s API for schema registry and automated cache updates.
• Multi-runtime environment: Bun for the JavaScript/TypeScript processes and Rust for performance-critical parts (via wasm-pack) in the CLI.