How vemetric/vemetric works
Project Type and Purpose
• Full-stack, open-source Web & Product Analytics platform
• Real-time event ingestion, processing pipelines, dashboard UI, billing & user management
Top-Level Components
A. Frontend (apps/webapp)
– React + Vite app
– Pages & components for dashboard, funnels, events, user detail, settings, onboarding
– trpc client for backend API calls
– Snippet that posts tracking events to the “Hub”
B. Backend API (apps/backend)
– Node.js/TypeScript + tRPC server
– REST/trpc routes: billing, dashboard queries, events, filters, funnels, users, organization, Paddle webhooks
– Uses Prisma client (packages/database) and ClickHouse client (packages/clickhouse)
– Handles authentication (lucia), authorization, rate limiting, logging
C. Ingestion Hub (apps/hub)
– Node.js ingestion service
– Serves the client snippet and receives raw events
– Enriches events (cookies, bots, sessions)
– Pushes jobs/messages into Redis queues
D. Worker Pool (apps/worker)
– Background workers (Bull) subscribed to queues: event-worker, session-worker, user-worker, device-worker, email-worker, etc.
– Processes queues to:
• Write to relational DB (Prisma/Postgres via packages/database)
• Write to analytics DB (ClickHouse via packages/clickhouse)
• Trigger email drip sequences (packages/email + Postmark)
• Send notifications or update usage metrics
E. Queue Infrastructure (packages/queues + apps/bullboard)
– Redis as message broker + Bull for job queues
– apps/bullboard provides a UI to monitor queue statuses
F. Supporting Utilities
– packages/common: shared data types, utility functions, filters
– packages/logger: structured logging across services
– apps/dev-proxy: local dev proxy routing traffic to each app
– apps/health-check: periodic health-check pings
– Docker & docker-compose orchestrate containers: backend, hub, worker, Redis, ClickHouse, Postgres
G. Data Stores
– Postgres (Prisma) for core relational data: users, projects, billing, feature flags
– ClickHouse for analytical time-series & event data
– Redis for queues and session caching
H. External Services
– Paddle (billing, webhooks)
– Postmark (transactional & drip emails)
– Crisp Chat (embedded support chat)
Interactions & Data Flow
1. User visits site → webapp (client) loads tracking snippet from Hub
2. Client records pageviews, events → sends POST to Hub API
3. Hub enriches event → pushes job to Redis queue
4. Worker picks up job → processes event:
• Writes raw analytics to ClickHouse
• Updates session/user info in Postgres
• Emits further jobs (e.g., email drip)
5. Frontend trpc calls → Backend API → reads from Postgres or ClickHouse → returns dashboard/funnel/filter data
6. Billing events handled by backend via Paddle integration → updates billing-info in Postgres
7. Email-worker uses packages/email to send transactional or sequence emails
Architectural Patterns & Principles
– Microservices-style apps under a single monorepo (pnpm + Turborepo)
– Event-driven pipeline: Hub → Redis queues → Workers
– CQRS-like separation:
• Command path: event ingestion & processing
• Query path: ClickHouse + API for dashboards
– Modular packages for cross-cutting concerns (common, queues, logger, clickhouse, database, email)
– Containerized services for isolation & easy deployment