How sathwikabbaraju/federated-learning-ecommerce-recommendation works
Project Type and Purpose
• This is a distributed federated-learning system for personalized e-commerce recommendations.
• Core goal: train recommendation models locally on user-partitioned data, then aggregate updates centrally to preserve privacy and efficiency.
File Structure
• Top-level directories:
• synthetic_dataset: code and data for generating non-IID user personas and partitioned datasets
• Flower_Main: primary FL implementation (server.py, client*.py, utils.py, notebooks)
• docker_clients & fl_server: containerized versions of clients and server (Dockerfile, docker-compose.yml)
• Supporting files:
• README.md: architecture rationale, tech stack (PyTorch, Flower, Docker, Kubernetes, Kafka)
• .github workflows: CI/CD and CSV validation
• results: logs and metrics outputs
• Tests: basic unit test under tests/
README for Architecture Insights
• Components explicitly mentioned: FL clients, FL server, FedProx, FedAvg+, compression, poisoning defense, datasets, Docker/Kubernetes, future Kafka integration, monitoring via Prometheus/Grafana.
Main Components
a. Data Layer
– Amazon Product Sales Dataset & Synthetic Persona Datasets (CSV files)
– Data partitioner (all_personas_creation.py)
b. Client Layer
– Multiple FL clients (client1.py … client11.py in Flower_Main or docker_clients/client.py)
– Local model training module (PyTorch)
– Preprocessing (utils.py)
– Notebook for metrics (plot_metrics.ipynb)
c. Server Layer
– FL server (Flower flwr server.py in Flower_Main and fl_server/server.py)
– Aggregation strategies: FedAvg+, FedProx, compression, poisoning detection
– Notebook template for simulation (flec_federated_simulation_template.ipynb)
d. Deployment & Orchestration
– Docker containers for clients and server
– docker-compose setups in docker_clients and fl_server
– (Future) Kubernetes cluster
e. External Services
– (Planned) Apache Kafka for streaming updates
– Monitoring: Prometheus, Grafana
Relationships and Interactions
• Aggregator component inside FL Server collects client updates → applies FedAvg+/FedProx → dispatches global model back to clients
• Docker orchestration: group client containers and server container under separate logical clusters
Architectural Patterns & Design Principles
• Decoupling: Clients and server are independently deployable services (microservice style)
• Modularity: Data generation, client logic, server logic, and deployment configs each live in separate directories
• Scalability: Can scale number of clients (each in its own container)
• Security & Privacy: Only model weights exchanged, raw data remains local
• Extensibility: New aggregation strategies or security defenses can plug into server.py