How openpanel-dev/openpanel works
Main Components
• Frontend/Dashboard: Represented in the “apps/dashboard” directory. This is built with Next.js, using Tailwind and Shadcn UI components. It serves as the user interface for interacting with analytics, reports, and configuration.
• Public Website: Found in “apps/public”. This is the landing site and documentation portal (articles, landing pages) for the platform.
• API Server: Located in “apps/api”. It uses Fastify to handle event ingestion, reporting endpoints, and other API requests.
• Worker Process: Found in “apps/worker”. This component processes background jobs (such as event processing, notifications, session management) and enqueues/dequeues tasks using BullMQ.
• Database & Storage Layers:
• Postgres – used for storing basic, transactional information.
• ClickHouse – used for high-volume event storage and fast analytical queries.
• Cache & Queue System: Redis is used for caching, pub/sub, and as a backend for BullMQ queues.
• External & Third-Party Integrations:
• Email: Resend is integrated for transactional emails.
• OAuth & Auth: Arctic and Oslo are used for OAuth and authentication flows.
• Shared Packages & Libraries: The “packages” section organizes modules like auth, common functionality, database interactions, integrations, SDKs, tRPC routers, and more. These are used both by the apps and the worker services.
Out Interactions and Relationships
• The Dashboard (frontend) makes API calls via tRPC and direct HTTP calls to the API server.
• The API server exposes endpoints to record events (analytics ingestion) and retrieve data for dashboards and reports.
• Background tasks (in the worker) interact with the API data and trigger jobs (such as notifications, session handling, and cron jobs) using BullMQ on the Redis queue.
• The API server and worker interact with Postgres (for metadata, user info, configuration) and with ClickHouse (for event storage and fast analytics querying).
• Shared libraries (organized in the “packages” folder) support various aspects like authentication, validation, integrations, and database interactions, ensuring consistency across components.
• External services (e.g., Resend for emails, OAuth providers) are integrated through specific packages and calls from the API or worker components.
Architectural Patterns and Design Principles
• Modular Monorepo Architecture: Different “apps” are separate (API, Dashboard, Public, Worker) but share code from common packages.
• Separation of Concerns: Clear division between presentation (Dashboard/Public), API logic, background processing, and data storage.
• Microservice-like Interactions: Although it’s a monorepo, each component acts as a logically decoupled service communicating over well-defined interfaces.
• Use of API Gateway Pattern (via tRPC and Fastify): Helps in structuring the API endpoints and routing.
Relevant Technologies and Frameworks
• Next.js for the dashboard.
• Fastify for the event API.
• PostgreSQL and ClickHouse as the databases.
• Redis combined with BullMQ for queueing.
• tRPC for API communications.
• Tailwind CSS and Shadcn UI in the frontend.
• Docker (with Dockerfiles and docker-compose setups) for containerization and self-hosting capabilities.
Detailed Steps for the Principal Software Engineer
• Data flow:
• Users interact with the Dashboard which calls the API.
• The API processes events, fetches data from Postgres and ClickHouse.
• The Worker processes background jobs and tasks using BullMQ from Redis.
• Shared libraries (packages) should be represented as foundational components that both API and Worker depend on.
• Adding annotations that point out important technologies (e.g., “Uses Fastify for high performance HTTP routing”, “ClickHouse for event analytics”).