# sathwikabbaraju/federated-learning-ecommerce-recommendation architecture

> Architecture diagram of the GitHub repository sathwikabbaraju/federated-learning-ecommerce-recommendation, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sathwikabbaraju/federated-learning-ecommerce-recommendation
- Repository: https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation

## Overview

##### Project Type and Purpose
- This is a distributed federated-learning system for personalized e-commerce recommendations.
- Core goal: train recommendation models locally on user-partitioned data, then aggregate updates centrally to preserve privacy and efficiency.

##### File Structure
- Top-level directories:
- synthetic_dataset: code and data for generating non-IID user personas and partitioned datasets
- Flower_Main: primary FL implementation (server.py, client*.py, utils.py, notebooks)
- docker_clients & fl_server: containerized versions of clients and server (Dockerfile, docker-compose.yml)
- Supporting files:
- README.md: architecture rationale, tech stack (PyTorch, Flower, Docker, Kubernetes, Kafka)
- .github workflows: CI/CD and CSV validation
- results: logs and metrics outputs
- Tests: basic unit test under tests/

##### README for Architecture Insights
- Components explicitly mentioned: FL clients, FL server, FedProx, FedAvg+, compression, poisoning defense, datasets, Docker/Kubernetes, future Kafka integration, monitoring via Prometheus/Grafana.

##### Main Components
a. Data Layer
– Amazon Product Sales Dataset & Synthetic Persona Datasets (CSV files)
– Data partitioner (all_personas_creation.py)
b. Client Layer
– Multiple FL clients (client1.py … client11.py in Flower_Main or docker_clients/client.py)
– Local model training module (PyTorch)
– Preprocessing (utils.py)
– Notebook for metrics (plot_metrics.ipynb)
c. Server Layer
– FL server (Flower flwr server.py in Flower_Main and fl_server/server.py)
– Aggregation strategies: FedAvg+, FedProx, compression, poisoning detection
– Notebook template for simulation (flec_federated_simulation_template.ipynb)
d. Deployment & Orchestration
– Docker containers for clients and server
– docker-compose setups in docker_clients and fl_server
– (Future) Kubernetes cluster
e. External Services
– (Planned) Apache Kafka for streaming updates
– Monitoring: Prometheus, Grafana

##### Relationships and Interactions
- Aggregator component inside FL Server collects client updates → applies FedAvg+/FedProx → dispatches global model back to clients
- Docker orchestration: group client containers and server container under separate logical clusters

##### Architectural Patterns & Design Principles
- **Decoupling**: Clients and server are independently deployable services (microservice style)
- **Modularity**: Data generation, client logic, server logic, and deployment configs each live in separate directories
- **Scalability**: Can scale number of clients (each in its own container)
- **Security & Privacy**: Only model weights exchanged, raw data remains local
- **Extensibility**: New aggregation strategies or security defenses can plug into server.py

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Layer
    subgraph "Data Layer"
        RawData["Amazon Product Sales Dataset 2023"]:::data
        SyntheticGen["all_personas_creation.py"]:::data
        Persona1["persona_01_ratermax.py"]:::data
        Persona2["persona_02_expertfinder.py"]:::data
        Persona3["persona_03_affinity.py"]:::data
        Persona4["persona_04_trustpilot.py"]:::data
        Persona5["persona_05_reviewer.py"]:::data
        Persona6["persona_06_influencer.py"]:::data
        Persona7["persona_07_bargainhunter.py"]:::data
        Persona8["persona_08_qualityseeker.py"]:::data
        Persona9["persona_09_trendsetter.py"]:::data
        Persona10["persona_10_valueconscious.py"]:::data
        Persona11["persona_11_socialproof.py"]:::data
    end

    %% Client Cluster
    subgraph "Client Cluster (Docker)" 
        direction TB
        C1["client1.py"]:::client
        C2["client2.py"]:::client
        C3["client3.py"]:::client
        C4["client4.py"]:::client
        C5["client5.py"]:::client
        C6["client6.py"]:::client
        C7["client7.py"]:::client
        C8["client8.py"]:::client
        C9["client9.py"]:::client
        C10["client10.py"]:::client
        C11["client11.py"]:::client
        Utils["utils.py"]:::client
        NotebookClient["plot_metrics.ipynb"]:::client
    end

    %% Server Cluster
    subgraph "Server Cluster (Docker)"
        direction TB
        ServerStandalone["Flower Main Server\n(server.py)"]:::server
        ServerContainer["Containerized Server\n(server.py)"]:::server
        Aggregator["Aggregator\n(FedAvg+, FedProx,\nCompression, Defense)"]:::server
        SimulationNB["flec_federated_simulation_template.ipynb"]:::server
    end

    %% Logging & Monitoring
    DBLogs["Logs & Metrics DB"]:::data
    Kafka["Apache Kafka"]:::external
    Prometheus["Prometheus"]:::external
    Grafana["Grafana"]:::external

    %% CI/CD Workflows
    subgraph "CI/CD & Validation" 
        CI1["ci-cd.yml"]:::orchestration
        CI2["validate_csv.yml"]:::orchestration
        CSVScript["validate_csv.py"]:::orchestration
    end

    %% Relationships
    RawData -->|provides data to| SyntheticGen
    SyntheticGen -->|generates personas| Persona1 & Persona2 & Persona3 & Persona4 & Persona5 & Persona6 & Persona7 & Persona8 & Persona9 & Persona10 & Persona11
    Persona1 & Persona2 & Persona3 & Persona4 & Persona5 & Persona6 & Persona7 & Persona8 & Persona9 & Persona10 & Persona11 -->|local data| C1 & C2 & C3 & C4 & C5 & C6 & C7 & C8 & C9 & C10 & C11
    Utils -->|preprocessing| C1 & C2 & C3 & C4 & C5 & C6 & C7 & C8 & C9 & C10 & C11
    C1 & C2 & C3 & C4 & C5 & C6 & C7 & C8 & C9 & C10 & C11 -->|"Flower RPC over gRPC/TCP"| ServerStandalone
    ServerStandalone -->|"Flower RPC over gRPC/TCP"| C1 & C2 & C3 & C4 & C5 & C6 & C7 & C8 & C9 & C10 & C11
    ServerContainer -->|"Flower RPC"| C1 & C2 & C3 & C4 & C5 & C6 & C7 & C8 & C9 & C10 & C11
    C1 & C2 & C3 & C4 & C5 & C6 & C7 & C8 & C9 & C10 & C11 -->|push metrics| DBLogs
    ServerStandalone & ServerContainer -->|push metrics| DBLogs
    DBLogs -->|scraped by| Prometheus
    Prometheus -->|visualized in| Grafana
    C1 & C2 & C3 & C4 -->|future stream| Kafka
    CI1 & CI2 & CSVScript -->|CI/CD pipelines| C1

    %% Click Events
    click RawData "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/tree/main/Amazon Product Sales Dataset 2023/"
    click SyntheticGen "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/synthetic_dataset/Persona Files/all_personas_creation.py"
    click Persona1 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/synthetic_dataset/Persona Files/persona_01_ratermax.py"
    click Persona2 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/synthetic_dataset/Persona Files/persona_02_expertfinder.py"
    click Persona3 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/synthetic_dataset/Persona Files/persona_03_affinity.py"
    click Persona4 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/synthetic_dataset/Persona Files/persona_04_trustpilot.py"
    click Persona5 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/synthetic_dataset/Persona Files/persona_05_reviewer.py"
    click Persona6 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/synthetic_dataset/Persona Files/persona_06_influencer.py"
    click Persona7 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/synthetic_dataset/Persona Files/persona_07_bargainhunter.py"
    click Persona8 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/synthetic_dataset/Persona Files/persona_08_qualityseeker.py"
    click Persona9 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/synthetic_dataset/Persona Files/persona_09_trendsetter.py"
    click Persona10 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/synthetic_dataset/Persona Files/persona_10_valueconscious.py"
    click Persona11 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/synthetic_dataset/Persona Files/persona_11_socialproof.py"
    click C1 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/Flower_Main/client1.py"
    click C2 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/Flower_Main/client2.py"
    click C3 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/Flower_Main/client3.py"
    click C4 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/Flower_Main/client4.py"
    click C5 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/Flower_Main/client5.py"
    click C6 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/Flower_Main/client6.py"
    click C7 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/Flower_Main/client7.py"
    click C8 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/Flower_Main/client8.py"
    click C9 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/Flower_Main/client9.py"
    click C10 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/Flower_Main/client10.py"
    click C11 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/Flower_Main/client11.py"
    click Utils "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/Flower_Main/utils.py"
    click NotebookClient "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/Flower_Main/plot_metrics.ipynb"
    click ServerStandalone "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/Flower_Main/server.py"
    click ServerContainer "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/fl_server/server.py"
    click SimulationNB "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/flec_federated_simulation_template.ipynb"
    click CI1 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/.github/workflows/ci-cd.yml"
    click CI2 "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/.github/workflows/validate_csv.yml"
    click CSVScript "https://github.com/sathwikabbaraju/federated-learning-ecommerce-recommendation/blob/main/.github/scripts/validate_csv.py"

    %% Styles
    classDef data fill:#cce5ff,stroke:#004085,color:#004085;
    classDef client fill:#d4edda,stroke:#155724,color:#155724;
    classDef server fill:#ffe5b4,stroke:#cc8400,color:#663c00;
    classDef orchestration fill:#e2e3e5,stroke:#6c757d,color:#6c757d;
    classDef external fill:#e8d4f8,stroke:#6f42c1,color:#4e0f7e;
```
