How piyushsachdeva/ai-observability-with-otel-and-honeycomb works
1. This repo is a Kubernetes-based demo application for URL shortening and observability. The main application stack lives under honeycomb-demo/, with Kubernetes manifests, OpenTelemetry config, service entrypoints, and operational scripts.
2. The core runtime is split into three application services: link-service, redirect-service, and stats-service. Their main.go files under patches/services/* suggest each service is independently deployable and wired from a shared codebase or patches layer.
3. link-service is the write path. The presence of patches/internal/adapters/repository/postgres/link.go shows a repository adapter for storing link records in PostgreSQL, so this service is the main persistence boundary for created short links.
4. redirect-service is the read/serve path. It likely resolves a short code into a destination URL and handles the high-volume redirect traffic. This is the most latency-sensitive service in the system.
5. stats-service is the analytics boundary. It aggregates or exposes usage data for links/redirects, likely reading from the database and/or a cache rather than serving the redirect path directly.
6. PostgreSQL is the primary durable datastore, deployed via k8s/02-postgres.yaml. Redis is a separate infrastructure component in k8s/03-redis.yaml, which implies the app uses a fast ephemeral store for caching, counters, or request state.
7. Observability is first-class. k8s/04-otel-collector.yaml and otel/collector-config.yaml define an in-cluster OpenTelemetry Collector, and patches/otel/otel.go indicates shared instrumentation setup inside the services. Telemetry is therefore routed through the collector rather than sent directly from each service to a backend.
8. k8s/05-feature-flags.yaml adds a feature-flag subsystem inside the cluster. That makes runtime behavior configurable without redeploying the services, which is important for demos and controlled experiments.
9. The user-facing edge is split between frontend, nginx, and ingress. frontend is the UI layer, nginx likely serves or reverse-proxies it, and k8s/11-ingress.yaml exposes the stack externally. Requests then fan out from the ingress to the frontend and API services.
10. Deployment is fully Kubernetes-native and ordered by numbered manifests: namespace, secrets, backing services, observability, app services, edge, and ingress. k8s/01-secrets.yaml is the shared configuration boundary for credentials and sensitive settings.
11. The scripts/ folder defines operational workflows around the app: apply.sh deploys the stack, load-test.sh generates traffic, and inject-chaos.sh / remove-chaos.sh support resilience testing. These are not part of the product path, but they are important to the repo’s demo architecture.
12. ubuntu-nodegroup.yaml suggests the cluster runs on a cloud-managed node group, so the deployment model is “Kubernetes on provisioned Linux worker nodes” rather than a local-only demo.
13. For an architecture diagram, the key flows are: external user traffic enters through ingress/nginx to the frontend and API services; link-service writes to Postgres; redirect-service serves redirects and may consult Redis/Postgres; stats-service reads aggregated data; and all services emit telemetry to the OpenTelemetry Collector, which is the central observability hub.
Main entry points: apply.sh Apply Script · redirect-service/main.go Redirect Service · stats-service/main.go Stats Service · link-service/main.go Link Service
Components 18
Platform
Cluster resources and external entrypoints
- Namespace · Shared Kubernetes namespace for the demo stack
honeycomb-demo/k8s/00-namespace.yaml - Secrets · Shared sensitive configuration for services
honeycomb-demo/k8s/01-secrets.yaml - Postgres · Primary durable store for links and analytics
honeycomb-demo/k8s/02-postgres.yaml - Redis · Ephemeral fast store for caching or counters
honeycomb-demo/k8s/03-redis.yaml - Feature Flags · In-cluster feature-flag subsystem for live behavior changes
honeycomb-demo/k8s/05-feature-flags.yaml - Ingress · External entrypoint into the demo stack
honeycomb-demo/k8s/11-ingress.yaml
Application
User-facing services and data path
- Link Service · Creates short links and persists them
honeycomb-demo/patches/services/link-service/main.go - Postgres Adapter · PostgreSQL persistence adapter for link records
honeycomb-demo/patches/internal/adapters/repository/postgres/link.go - Redirect Service · Resolves short codes and serves redirects
honeycomb-demo/patches/services/redirect-service/main.go - Stats Service · Exposes link and redirect usage data
honeycomb-demo/patches/services/stats-service/main.go - Frontend · User interface for creating and viewing links
honeycomb-demo/k8s/10-frontend.yaml - Nginx · Serves or reverse-proxies the frontend and API traffic
honeycomb-demo/k8s/09-nginx.yaml
Observability
Telemetry pipeline and shared instrumentation
- OTel Collector · Central OpenTelemetry aggregation and export point
honeycomb-demo/k8s/04-otel-collector.yaml - Collector Config · Collector pipeline and export configuration
honeycomb-demo/otel/collector-config.yaml - Shared OTEL · Common service-side OpenTelemetry setup
honeycomb-demo/patches/otel/otel.go
Operations
Deployment and test workflows
- Apply Script · Deploys the Kubernetes stack in order
honeycomb-demo/scripts/apply.sh - Load Test · Generates traffic against the running system
honeycomb-demo/scripts/load-test.sh - Chaos Scripts · Injects and removes chaos for fault testing
honeycomb-demo/scripts/inject-chaos.sh
Connections 35
- Apply Script → Namespace: creates · Bootstraps the shared namespace
- Apply Script → Secrets: applies · Installs shared configuration and credentials
- Apply Script → Postgres: deploys · Brings up the durable datastore
- Apply Script → Redis: deploys · Brings up the cache layer
- Apply Script → OTel Collector: deploys · Installs the telemetry collector
- Apply Script → Feature Flags: deploys · Installs runtime feature flags
- Apply Script → Link Service: deploys · Installs the write service
- Apply Script → Redirect Service: deploys · Installs the redirect service
- Apply Script → Stats Service: deploys · Installs the analytics service
- Apply Script → Nginx: deploys · Installs the edge proxy
- Apply Script → Frontend: deploys · Installs the UI layer
- Apply Script → Ingress: exposes · Configures external access
- Ingress → Nginx: routes · Sends public traffic to the edge proxy
- Ingress → Frontend: routes · Exposes the user interface
- Frontend → Link Service: creates links · UI submits new short-link requests
- Frontend → Redirect Service: resolves · UI-facing navigation and redirect interactions
- Frontend → Stats Service: shows stats · UI fetches analytics views
- Nginx → Frontend: serves · Proxies or serves the frontend
- Link Service → Postgres Adapter: uses · Delegates persistence to the repository adapter
- Postgres Adapter → Postgres: writes · Stores link records in PostgreSQL
- Redirect Service → Redis: consults · Uses cache for fast redirect lookups or counters
- Redirect Service → Postgres: falls back · May read durable link data when needed
- Stats Service → Postgres: reads · Aggregates or queries usage data from storage
- Stats Service → Redis: reads · May use cached counters or summaries
- Link Service → Feature Flags: checks · Uses feature flags to alter runtime behavior
- Redirect Service → Feature Flags: checks · Uses feature flags on the hot path
- Stats Service → Feature Flags: checks · Uses feature flags for analytics behavior
- Link Service → Shared OTEL: instruments · Shares telemetry setup with the service runtime
- Redirect Service → Shared OTEL: instruments · Shares telemetry setup with the service runtime
- Stats Service → Shared OTEL: instruments · Shares telemetry setup with the service runtime
- Shared OTEL → OTel Collector: exports · Sends traces and metrics to the collector
- OTel Collector → Collector Config: configured by · Collector behavior is defined by the config file
- Load Test → Ingress: targets · Generates traffic through the public entrypoint
- Chaos Scripts → Link Service: stresses · Used to validate service resilience
- Chaos Scripts → Redirect Service: stresses · Used to validate redirect-path resilience