How lasseh/whynoipv6 works
Project Type and Purpose
• This repository is one part of a three-repo ecosystem (backend, frontend, and campaigns) for a service called “WhyNoIPv6,” which tracks and “shames” high-traffic websites for not adopting IPv6.
• Its key functions include scanning domains (using a crawler), aggregating IPv6-related metrics (e.g., DNS and MX record checks), and providing an API for retrieval of this data. There’s also a command-line tool repository (cmd/v6manage) for managing campaigns and related operations.
Analyzing the File Structure
• Two main command directories exist under "cmd":
• cmd/api – holds the main API server (main.go) that likely handles REST requests.
• cmd/v6manage – collections of subcommands (e.g., campaign, crawl, import) for managing domain campaigns and utilities.
• The “internal” directory subdivides into components such as:
• config – configuration handling.
• core – contains business logic for domain, campaign, changelog, metrics, country, and more.
• geoip – for GeoIP functionality, supplemented by the “db/geoip” containing GeoLite2 DB files.
• logger – logging functionality.
• postgres – database-related code and generated SQL queries, including models and integration layer.
• resolver – likely handling domain name resolution tasks.
• rest – API endpoint handlers (campaign, changelog, error handler, etc.).
• toolbox – secondary features like healthchecks and notifications.
• Database migration and query definitions are in the “db” directory.
• Extra directories include deployment resources such as systemd service files, nginx configs, and Grafana dashboards.
• Build/deployment scripts (Makefile, .golangci.yml, go.mod) denote that the project is built and deployed via standard tooling.
Main Components
a. Web API Server (cmd/api):
• Represented as the layer that interfaces with clients using REST endpoints from “internal/rest.”
• It interacts with business logic in “internal/core,” calls on “internal/geoip” and “internal/resolver” as needed, and logs via “internal/logger.”
b. Crawler and Utility Application (cmd/v6manage):
• Handles operational tasks such as domain crawling, campaign imports, campaign listings, changelog generation, etc.
• Acts as the driver for periodic scanning of domains (every 3 days, as per README) and management of campaign data.
c. Database Layer:
• Consists of both physical database assets (migrations in “db/migrations” and actual SQL queries in “db/query”) and the PostgreSQL integration (in “internal/postgres”).
• This component is responsible for storing domain scan results, metrics, and other campaign data.
d. GeoIP Module:
• Incorporate the GeoIP functionalities that use the GeoLite2 databases. It is represented in both the “db/geoip” (data files) and “internal/geoip” (logic).
e. Configuration and Logging Utilities:
• These include “internal/config” for managing configuration settings and “internal/logger” for logging, which cut across the entire system.
f. Deployment/Infrastructure:
• External service configurations are provided via files in the “extra” directory (nginx configurations, systemd service definitions, logrotate, etc.).
• They indicate how the application is deployed and monitored in production.
Relationships and Interactions
• The API layer receives HTTP requests and forwards them to REST handler functions within “internal/rest,” which invoke business logic in “internal/core.”
• The business logic further interacts with the data layer (Postgres operations via “internal/postgres”) and possibly leverages utility functions (healthchecks, notifications) from “internal/toolbox.”
• The crawler tool (cmd/v6manage) operates offline or as scheduled jobs, updating the campaigns and domain scan data in the database.
• External modules such as the GeoIP processor use the provided database files in “db/geoip” along with application logic in “internal/geoip” to resolve additional domain information.
• A resolver component is responsible for handling DNS querying, such as checking DNS and MX records.
• Deployment components (systemd, nginx) tie the API and crawler applications to their hosting environment, ensuring they are properly managed and exposed.
Architectural Patterns & Design Principles
• The project follows a modular monolith with clearly separated concerns: each subdirectory encapsulates a discrete functionality (e.g., core business logic, REST endpoints, database access).
• It follows a layered architecture: the API layer sits atop the business logic/core, which in turn communicates with the database and external services (like DNS, GeoIP).
• Command structure for separate CLI tools (via cmd/api and cmd/v6manage) shows an intent to decouple the management utilities from the API service.
• The separation between deployment (extra) and application logic highlights a design that is deployment-agnostic.