How parisasokuti/ds_project works
Project Type and Purpose
• This is a backend-centric, distributed, real-time WebSocket game server with dual persistence (Redis + PostgreSQL), built in Python asyncio.
• Primary goals: horizontal scaling, fault tolerance (circuit breakers, auto-failover), session persistence, and enterprise monitoring.
File Structure for Architectural Clues
• backend/ – core server code: WebSocket handling, game logic, auth, data layer, fault-tolerance modules.
• docker-compose.yml / k8s-deployment.yml – container orchestration, multi-instance scaling.
• scripts/ – deployment, backup, failover automation.
• examples_archive/ & docs/ – sample infra (PostgreSQL HA, Redis clusters), monitoring configs, load-balancer examples.
• tests/ – comprehensive coverage indicating key integration points (DB, Redis, network, auth, fault-tolerance).
README for Additional Insights
• Main components called out: Backend Server, Database Layer, Authentication, Load Balancing, Monitoring.
• Tech stack: Python asyncio, Redis, PostgreSQL, JWT, HAProxy, Docker Compose, Kubernetes.
Main Components and Their Relationships
a. Clients
• Web or CLI clients (e.g. backend/client.py) connect via WebSocket.
• Could be browser JS front-ends or custom Python clients.
b. Load Balancer
• HAProxy (configured for WebSocket sticky sessions).
• Sits in front of multiple game-server instances.
c. Game Server Instances (backend/server.py)
• Stateless WebSocket handlers that implement game logic (game_board.py, game_states.py).
• Fault-tolerance modules: circuit_breaker.py, auto_scaler.py, circuit_breaker_monitor.py.
• Authentication routes (auth_routes.py, auth_service.py).
d. Data Layer
• Redis Cluster: real-time game state and session storage
• Managers & integrations: redis_manager.py, redis_cluster_manager.py, hybrid_data_layer.py.
• PostgreSQL: persistent storage of users, game history
• Handled via SQLAlchemy wrappers (backend/database/*.py, postgresql_persistence.py).
• Circuit Breaker Integration around both Redis and PostgreSQL (see backend/database/circuit_breaker_integration.py).
e. Monitoring & Health
• HTTP /health endpoint in server.
• Prometheus exporters (in examples_archive), Grafana dashboards.
• Health checks & backup monitors (scripts/backup_health_monitor.sh, backend/circuit_breaker_monitor.py).
f. Infrastructure & Orchestration
• Docker Compose (docker-compose.yml, docker-compose.scaling.yml).
• Kubernetes spec (k8s-deployment.yml).
• Auto-scaling logic in backend/auto_scaler.py.
g. Backup & Failover
• Backup server manager (backend/ha_server_manager.py).
• DNS failover scripts (scripts/update_dns.sh, scripts/dns_failover_monitor.sh).
• PostgreSQL and Redis replication examples in examples_archive.
Architectural Patterns & Design Principles
• Micro-instance pattern: multiple identical game-server containers behind a load balancer.
• Circuit Breaker: isolate failures to Redis or PostgreSQL, auto-recovery.
• Hybrid Data Layer: real-time (Redis) + durable (PostgreSQL).
• Auto-scaling: dynamically adjust server count based on load.
• High Availability: HAProxy + Keepalived for LB, Patroni or streaming replication for Postgres.