How tom-draper/api-analytics works
Purpose and Topology
API Analytics is a multi-language API-request telemetry product. Framework middleware captures request/response metadata, sends it to a central Go ingestion service, persists it, and exposes it to a SvelteKit dashboard and data API. It also supports scheduled endpoint monitoring and self-hosted deployment.
Instrumentation SDKs
SDKs are framework-specific adapters around common capture-and-submit behavior: measure duration, extract request metadata, apply privacy/user-ID mapping, and emit telemetry keyed by the customer API key. Major implementation families:
• Python shared core and adapters: analytics/python/python/api_analytics/core.py, analytics/python/python/api_analytics/fastapi.py, analytics/python/python/api_analytics/flask.py
• JavaScript framework packages: analytics/javascript/core/index.js, analytics/javascript/express/index.js, analytics/nodejs/analytics.js
• Go core plus router middleware: analytics/go/core/core.go, analytics/go/gin/analytics.go, analytics/go/fiber/analytics.go
• Other runtime bindings: analytics/php/core/src/Client.php, analytics/ruby/api_analytics/lib/api_analytics.rb, analytics/csharp/analytics/analytics.cs, analytics/rust/axum/analytics/src/analytics.rs
Ingestion and Public API
The Go API service is the backend entry point for telemetry, dashboard data access, key-oriented workflows, and monitoring-related operations. Runtime startup and HTTP route registration are centered in server/api/main.go and server/api/lib/routes/routes.go; environment and logging boundaries are server/api/lib/env/env.go and server/api/lib/log/log.go. Clients authenticate data retrieval with an API key in X-AUTH-TOKEN; query parameters constrain raw request data.
Persistence Boundary
Database access is isolated as its own Go module, separating SQL/query construction, sanitization, and retention-oriented storage concerns from HTTP routing:
• server/database/database.go
• server/database/query.go
• server/database/sanitize.go
• server/self-hosting/database/schema.sql
The stored domain is request analytics: method, endpoint, user agent, optional IP/location or custom user ID, timestamp, response status/time, hostname, and framework. Privacy level is enforced at SDK capture boundaries before data reaches ingestion.
Dashboard Application
dashboard/ is a SvelteKit/TypeScript frontend and server-rendered route layer. Its primary entry/routing surface is dashboard/src/routes/+layout.svelte, dashboard/src/routes/+page.svelte, and dashboard/src/hooks.server.ts. Dashboard and explorer screens consume request data, derive aggregates, and render visualizations:
• Analytics fetch/store/aggregation: dashboard/src/lib/fetchRequests.ts, dashboard/src/lib/dataStore.ts, dashboard/src/lib/aggregate.ts
• Dashboard UI: dashboard/src/routes/dashboard/[uuid]/+page.svelte, dashboard/src/lib/components/dashboard/
• Raw-data exploration: dashboard/src/routes/explorer/[uuid]/+page.svelte, dashboard/src/lib/components/explorer/
Server endpoints under dashboard/src/routes/monitoring/ and dashboard/src/routes/frequently-asked-questions/+server.ts form frontend-server boundaries rather than client-only pages.
Active Monitoring and Operations
A separate Go monitor periodically probes customer-configured endpoints, recording uptime and response time. Its executable is server/monitor/main.go, with schedule/container concerns in server/monitor/crontab and server/monitor/Dockerfile. Supporting operational jobs include cleanup, health/checkup, usage calculation, and monitor tooling under server/tools/cleanup/main.go, server/tools/usage/usage.go, and server/tools/monitor/main.go. Email capability is a separate Go module at server/email/email.go.
Deployment and Infrastructure
Primary services are containerized Go API/logger/monitor processes plus the dashboard (server/api/Dockerfile, server/logger/Dockerfile, server/monitor/Dockerfile, dashboard/Dockerfile). Self-hosting composes them with a database and reverse proxy via server/self-hosting/docker-compose.yml; Nginx and optional Traefik configurations are provided in server/self-hosting/nginx/ and server/self-hosting/traefik/. The logger service (server/logger/main.go) is a distinct ingestion-facing runtime with rate limiting at server/logger/lib/ratelimit/ratelimit.go.
Main entry points: docker-compose.yml Self-hosted compose · main.go API service · core.py Capture and privacy · fetchRequests.ts Analytics data layer
Components 20
Instrumentation SDKs
Framework adapters capture and submit API telemetry.
- Python adapters
analytics/python/python/api_analytics/fastapi.py - JavaScript adapters
analytics/javascript/express/index.js - Go middleware
analytics/go/gin/analytics.go - Other runtime bindings · PHP, Ruby, C#, and Rust bindings.
analytics/php/core/src/Client.php - Capture and privacy · Measures requests, maps identities, and applies privacy before submission.
analytics/python/python/api_analytics/core.py
Go Services
Ingestion, query, monitoring, and operational runtimes.
- Logger service · Ingestion-facing runtime with rate limiting.
server/logger/main.go - Rate limiting
server/logger/lib/ratelimit/ratelimit.go - API service · Serves telemetry, dashboard data, keys, and monitoring workflows.
server/api/main.go - API routes
server/api/lib/routes/routes.go - Database access · Owns query construction, sanitization, and retention access.
server/database/database.go - Analytics database · Stores request and monitoring analytics.
server/self-hosting/database/schema.sql - Endpoint monitor · Probes configured endpoints for uptime and response time.
server/monitor/main.go - Operational jobs · Cleanup, usage, health, and monitor support jobs.
server/tools/cleanup/main.go - Email module
server/email/email.go
Dashboard
SvelteKit UI and server-rendered data boundary.
- SvelteKit routes · Dashboard entry, routing, and server hooks.
dashboard/src/routes/+layout.svelte - Analytics data layer · Fetches request data and derives aggregates.
dashboard/src/lib/fetchRequests.ts - Dashboard views · Visualizes request aggregates.
dashboard/src/routes/dashboard/[uuid]/+page.svelte - Raw-data explorer
dashboard/src/routes/explorer/[uuid]/+page.svelte - Monitoring endpoints · Frontend-server boundary for monitoring workflows.
dashboard/src/routes/monitoring/+server.ts
Self Hosting
Containerized deployment topology.
- Self-hosted compose · Composes services, database, and reverse proxy.
server/self-hosting/docker-compose.yml
Connections 23
- Python adapters → Capture and privacy
- JavaScript adapters → Capture and privacy
- Go middleware → Capture and privacy
- Other runtime bindings → Capture and privacy
- Capture and privacy → Logger service: telemetry
- Rate limiting → Logger service: protects
- Logger service → Database access: persist requests
- API service → API routes: registers
- API routes → Database access: query and mutate
- Database access → Analytics database: SQL
- Endpoint monitor → Analytics database: records probe results
- Operational jobs → Analytics database: cleanup and usage
- API service → Email module: notifications
- SvelteKit routes → Analytics data layer: loads
- Analytics data layer → API service: X-AUTH-TOKEN API queries
- Analytics data layer → Dashboard views: aggregates
- Analytics data layer → Raw-data explorer: raw requests
- Monitoring endpoints → API service: monitoring workflows
- Self-hosted compose → API service: runs
- Self-hosted compose → Logger service: runs
- Self-hosted compose → Endpoint monitor: runs
- Self-hosted compose → SvelteKit routes: runs
- Self-hosted compose → Analytics database: provisions