# shahdhruv777/distributed_assignment1 architecture

> Architecture diagram of the GitHub repository shahdhruv777/distributed_assignment1, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-02-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shahdhruv777/distributed_assignment1
- Repository: https://github.com/shahdhruv777/distributed_assignment1

## Overview

##### Project Type and Purpose
- Type: A distributed, containerized load-balancing system implemented in Python.
- Purpose:
– Accept client requests via a client container.
– Use a multi-threaded load balancer container that maintains shards via consistent hashing, monitors servers via heartbeat, and routes requests.
– Dispatch read/write/update/delete requests to a dynamic pool of server containers.
– Persist metadata about shards and mappings in a database container.

##### File Structure
- Top-level Dockerfiles:
– Dockerfile_lb (load balancer)
– Dockerfile_server
– Dockerfile_client
– (plus dummy & lb variants)
- Python entry points:
– combined_lb.py, load_balancer_rehashed.py, new_lb.py → load-balancer logic
– server.py → server container logic
– client.py, scaled_client.py → client scripts
– Analysis*.py → benchmarking/analysis scripts
- Helpers and data structures:
– Datastructures.py, helper_functions.py, packages.py → core in-memory maps, mutexes, sorted containers
– drs.py, while.py, faltu.py → utility scripts
- Deployment scripts:
– initializer_script_lb, initializer_script_server
– Makefile targets to build/run images and containers
- Logs & tim-ings:
– lb_logs/, server*_logs/, AnalysisTimings

##### README Insights
- Docker orchestration: custom network “my_network”
- Three containers: client, load balancer (with embedded DB), servers
- Metadata class interacts with “ShardT” and “MapT” tables in the database
- Consistent hashing across shards, multi-threading, heartbeat detection, graceful server removal (20s timeout)
- Use of sorted-containers for O(log n) server lookups

##### Main Components & Their Interactions
a. Client Container
– Runs client.py or scaled_client.py
– Issues HTTP requests (e.g. /write, /read, /update, /del) to the load balancer
b. Load Balancer Container
– HTTP API endpoints: /add, /rm, /update, /del, /heartbeat
– ConsistentHashingEngine (uses Shard objects, serv_dict, serv_id_dict, cont_hash)
– MetadataModule (interacts with the metadata database tables)
– HeartbeatMonitor Thread (periodically pings servers; updates server pool)
– Docker initializer script sets up LB on “my_network”
c. Database Container (logical or embedded in LB)
– Holds ShardT and MapT tables
– Serves metadata queries and updates
d. Server Containers (multiple instances)
– server.py exposes endpoints to serve read/write/update/delete on its shard replicas
– Writes are committed across replicas then valid index updated
– Logs operations to server*_logs/
e. Docker Network (“my_network”)
– Virtual LAN connecting client, LB, database, server containers

##### Architectural Patterns and Principles
- Microservices via Docker containers for each role
- Consistent hashing for shard-to-server mapping
- Mutex locks to protect critical sections in Shard and Server class data structures
- Heartbeat pattern for failure detection and dynamic membership
- Graceful removal of servers with a TCP-like FIN timeout
- Use of sorted data structures for efficient nearest-server lookups (O(log n))

2. Load Balancer Container (large rectangle subdivided into:
- HTTP API Layer
- Consistent Hashing Engine (Shard + serv_dict + serv_id_dict)
- Heartbeat Monitor Thread)
3. Server Containers (multiple small rectangles, labeled server-1, server-2, …)
- Client → LB (HTTP requests)
- LB → Servers (routed requests, using consistent hashing)
- LB → DB (metadata reads/writes)
- LB → Servers (heartbeat pings)
– Optionally, show inter-server communication (none in this design) or back-end index updates.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Docker Network: my_network"
        subgraph "Client Container"
            Client["Client Container"]:::external
        end
        subgraph "Load Balancer Container"
            API["HTTP API Layer"]:::lb
            CH["Consistent Hashing Engine\n(Datastructures.py)"]:::lb
            MM["Metadata Module\n(helper_functions.py,\npackages.py)"]:::lb
            HB["Heartbeat Monitor Thread\n(initializer_script_lb)"]:::lb
        end
        subgraph "Database Container"
            DB["Metadata DB\n(ShardT, MapT)"]:::db
        end
        subgraph "Server Containers"
            S1["server-1\n(server.py)"]:::server
            S2["server-2\n(server.py)"]:::server
            S3["server-n\n(server.py)"]:::server
        end
    end

    Client -->|"/write, /read, /update, /del"| API
    API -->|"route request"| CH
    CH -->|"lookup shards"| DB
    MM -->|"db ops"| DB
    API -->|"dispatch"| S1
    API -->|"dispatch"| S2
    API -->|"dispatch"| S3
    HB -->|"heartbeat ping"| S1
    HB -->|"heartbeat ping"| S2
    HB -->|"heartbeat ping"| S3

    click Client "https://github.com/shahdhruv777/distributed_assignment1/tree/main/Dockerfile_client"
    click API "https://github.com/shahdhruv777/distributed_assignment1/blob/main/combined_lb.py"
    click CH "https://github.com/shahdhruv777/distributed_assignment1/blob/main/Datastructures.py"
    click MM "https://github.com/shahdhruv777/distributed_assignment1/blob/main/helper_functions.py"
    click MM "https://github.com/shahdhruv777/distributed_assignment1/blob/main/packages.py"
    click HB "https://github.com/shahdhruv777/distributed_assignment1/tree/main/initializer_script_lb"
    click DB "https://github.com/shahdhruv777/distributed_assignment1/blob/main/readme.md"
    click S1 "https://github.com/shahdhruv777/distributed_assignment1/blob/main/server.py"
    click S1 "https://github.com/shahdhruv777/distributed_assignment1/tree/main/initializer_script_server"
    click S2 "https://github.com/shahdhruv777/distributed_assignment1/blob/main/server.py"
    click S3 "https://github.com/shahdhruv777/distributed_assignment1/blob/main/server.py"

    classDef external fill:#ADD8E6,stroke:#000,stroke-width:1px;
    classDef lb fill:#90EE90,stroke:#000,stroke-width:1px;
    classDef db fill:#FFA500,stroke:#000,stroke-width:1px;
    classDef server fill:#FFFF66,stroke:#000,stroke-width:1px;
```
