How openbmp/obmp-docker works
Project Type and Purpose
• This is a Docker-based deployment repo for the OpenBMP system: a BGP Monitoring Protocol (BMP) collector + pipeline.
• It provides Dockerfiles and orchestration (via docker-compose) to stand up collector(s), message broker (Kafka + ZooKeeper), a PostgreSQL database, a consumer/ingestor (psql-app), optional WhoIs enrichment, plus development and visualization services.
File-Structure Analysis
• Top-level:
• docker-compose.yml – defines all services and how they interconnect
• collector/ – BMP collector container
• postgres/ – PostgreSQL DB container + init scripts
• psql-app/ – Kafka→Postgres consumer container + schema-upgrade scripts
• whois/ – optional WhoIs lookup container
• dev-image/ – development/build environment container
• Patterns: each component is its own microservice container; persistent data is mounted via host volumes; schema upgrades are versioned in psql-app/upgrade.
README Insights
• Three primary “product” containers with their own README: collector, PostgreSQL, psql-app.
• Persistent volumes under OBMP_DATA_ROOT for config, Kafka data, zookeeper data/logs, Postgres data, Grafana dashboards.
• BMP Collector
• ZooKeeper
• Kafka Broker
• PSQL Consumer (psql-app)
• PostgreSQL Database
• Grafana UI
• WhoIs Lookup Service (optional)
• Dev-image build environment (optional)
• Collector → publishes BMP messages → Kafka
• PSQL Consumer → subscribes to Kafka → transforms & writes → PostgreSQL
• Grafana → queries → PostgreSQL
• WhoIs Service → REST/API calls → optional enrichment by Collector or PSQL Consumer
• All services communicate over the Docker-compose network
c. Architectural Patterns & Principles:
• Microservices / containerized components
• Pub/Sub messaging via Kafka
• Persistent state via host-mounted volumes
• Versioned schema migrations in psql-app
• Separation of concerns: data ingestion, storage, visualization, enrichment
d. Key Technologies & Frameworks:
• Docker & Docker Compose
• Apache Kafka & ZooKeeper
• PostgreSQL
• Go/Python/C (collector code) as per collector/README
• Grafana for dashboards
• Optional WhoIs lookup daemon