How rahul07bagul/design-load-balancer works
1. System purpose
- This repo is a C++17 load balancer that manages a pool of backend server processes, distributes traffic across them, and self-heals by replacing unhealthy instances.
- It is built around two control planes: a gRPC admin/control API and an HTTP management API via Crow.
2. Top-level runtime shape
- The main application (src/main.cpp) appears to host the load balancer service itself.
- A separate health-checker executable is also expected, based on the README (./health_checker 127.0.0.1:50050), which monitors the balancer and/or server health over gRPC.
3. Core orchestration layer
- include/core/server.hpp, server_manager.hpp, and load_balancer.hpp define the central domain objects.
- src/core/load_balancer.cpp likely owns the active server pool, load-balancing policy, and lifecycle decisions.
- server_manager.cpp is the boundary for adding/removing servers and coordinating server state.
4. Backend server lifecycle and OS abstraction
- Backend servers are managed as OS processes, not just logical endpoints.
- include/core/process/* and src/core/process/* implement a factory-based abstraction for Linux and Windows process handling.
- This is a major architectural boundary: the system can start, stop, and inspect backend instances in a platform-specific way while keeping the rest of the balancer OS-agnostic.
5. Load-balancing policy subsystem
- The include/strategies and src/strategies folders implement the strategy pattern.
- Supported policies are Round Robin, Least Connections, and Resource Based.
- The load balancer delegates request/server selection to these strategy objects, which means policy changes do not require changes to core orchestration.
6. Health monitoring and auto-scaling
- include/monitoring/health_checker.hpp and src/monitoring/health_checker.cpp handle periodic health evaluation.
- The README indicates the system can:
- scale up when CPU usage exceeds a threshold,
- replace unresponsive servers,
- scale down when capacity is no longer needed.
- This monitoring layer feeds decisions back into server management, so health is not purely observational; it drives lifecycle changes.
7. Admin/control APIs
- include/api/admin_service.hpp and src/api/admin_service.cpp implement the gRPC administration interface.
- The protobuf definitions in proto/admin_service.proto and proto/load_balancer.proto define the service contracts and message types.
- This is the primary machine-to-machine control boundary for managing servers and querying state.
8. HTTP management surface
- include/api/crow_service.hpp and src/api/crow_service.cpp expose REST-style endpoints with Crow.
- The documented endpoints are:
- GET /api/status
- POST /api/add_server
- POST /api/remove_server
- This layer is a thin external interface over the same core management functions used by gRPC.
9. Configuration, logging, and shared utilities
- include/utils/config.hpp, logger.hpp, helper.hpp, and exception.hpp provide shared support for parsing runtime settings, logging, and error handling.
- These utilities sit underneath both the core and API layers, so they are cross-cutting concerns rather than independent subsystems.
10. Build and interface generation
- CMake is the build system, and vcpkg is used to manage dependencies such as gRPC, Protobuf, and Crow.
- The generated protobuf sources under proto/*.pb.* are part of the API contract boundary, linking the service definitions to the compiled C++ services.
11. Key data flow
- External admin clients call either the gRPC admin service or the HTTP API.
- Those requests reach server_manager / load_balancer, which update the active backend pool.
- The strategy layer chooses a server for balancing decisions, while the monitoring layer can trigger server replacement or scaling actions.
- Process factories create or terminate backend server processes on Linux or Windows as needed.
12. Architectural boundaries to show in a diagram
- External clients vs. control APIs (gRPC and HTTP).
- API layer vs. core balancer logic.
- Core logic vs. process management.
- Policy selection vs. server lifecycle management.
- Monitoring/health as a feedback loop into the core rather than a passive sidecar.
Main entry points: load_balancer.cpp Balancer core · admin_service.cpp gRPC admin · crow_service.cpp HTTP API · strategy.hpp Strategy base
Components 18
Runtime
Executable entrypoints and external actors
- Admins · Human or machine operators invoking control APIs
- Main app · Starts the load balancer service
src/main.cpp - Backend demo · Example backend server process used in the repo
examples/backend_server.cpp
Control APIs
gRPC and HTTP management surfaces
- gRPC admin · Admin and control interface for server management
src/api/admin_service.cpp - HTTP API · REST management endpoints for status and server changes
src/api/crow_service.cpp
Core balancer
Server state and balancing orchestration
- Balancer core · Owns pool state and balancing decisions
src/core/load_balancer.cpp - Server manager · Adds, removes, and coordinates backend server state
src/core/server_manager.cpp - Server · Represents a backend server instance and its state
src/core/server.cpp
Process mgmt
OS-specific backend process lifecycle
- Process factory · Creates platform-specific backend process objects
src/core/process/process_factory.cpp - Linux process · Linux implementation of backend process control
src/core/process/linux_process.cpp - Windows process · Windows implementation of backend process control
src/core/process/windows_process.cpp
Policies
Load-balancing strategy implementations
- Strategy base · Abstract selection contract used by the balancer
include/strategies/strategy.hpp - Round robin · Cycles through available servers
src/strategies/round_robin.cpp - Least connections · Chooses the least-loaded server
src/strategies/least_connections.cpp - Resource based · Uses resource metrics to choose a server
src/strategies/resource_based.cpp
Monitoring
Health checks and lifecycle feedback
- Health checker · Periodically evaluates health and drives recovery or scaling
src/monitoring/health_checker.cpp
Shared
Common utilities and generated contracts
- Config & logs · Configuration, logging, helpers, and exceptions
include/utils/config.hpp - Proto contracts · gRPC service definitions and generated message/service bindings
proto/admin_service.proto
Connections 24
- Admins → gRPC admin: admin calls · Operators manage servers and query state through gRPC
- Admins → HTTP API: http calls · Operators manage servers and query state through HTTP
- Main app → Balancer core: starts · Application entrypoint boots the balancer core
- gRPC admin → Balancer core: control · gRPC requests update core balancer state
- HTTP API → Balancer core: control · HTTP requests update core balancer state
- gRPC admin → Proto contracts: uses · gRPC service and message contracts
- HTTP API → Proto contracts: shares model · HTTP layer mirrors the same management semantics
- Balancer core → Server manager: delegates lifecycle · Core orchestration manages server membership and recovery through the manager
- Server manager → Server: tracks · Server manager owns server state objects
- Balancer core → Strategy base: selects via · Balancing decisions are delegated to strategy objects
- Strategy base → Round robin: implemented by · Round-robin policy implementation
- Strategy base → Least connections: implemented by · Least-connections policy implementation
- Strategy base → Resource based: implemented by · Resource-based policy implementation
- Health checker → Server manager: feeds back · Health results trigger server replacement and scaling decisions
- Health checker → Balancer core: influences · Monitoring feeds lifecycle decisions back into the balancer
- Server manager → Process factory: spawns · Backend instances are created and terminated as OS processes
- Process factory → Linux process: creates · Linux process backend
- Process factory → Windows process: creates · Windows process backend
- Linux process → Backend demo: runs · Example backend is a managed process target
- Windows process → Backend demo: runs · Example backend is a managed process target
- Balancer core → Config & logs: uses · Core logic depends on shared config, logging, and error handling
- gRPC admin → Config & logs: uses · API layer shares configuration and logging support
- HTTP API → Config & logs: uses · HTTP layer shares configuration and logging support
- Health checker → Config & logs: uses · Monitoring loop uses shared runtime utilities