# tcknickerbocker/toy_and_game_recommender architecture

> Architecture diagram of the GitHub repository tcknickerbocker/toy_and_game_recommender, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tcknickerbocker/toy_and_game_recommender
- Repository: https://github.com/tcknickerbocker/toy_and_game_recommender

## Overview

##### System’s primary components
- Frontend: Located in the “src/frontend” directory, this is the interface through which users interact with the recommender.
- API Backend: Found under “src/api”, this acts as the main service handling HTTP requests from the frontend.
- Model Services: There are multiple model microservices (e.g., “src/models/generate_new_products”, “src/models/model_1”, “src/models/model_2”, “src/models/model_3”, “src/models/model_4”). Each has its own Dockerfile and .env to allow independent scaling/deployment.
- ETL Pipeline: Under “src/etl”, these scripts handle data processing tasks (e.g., computing scores, embeddings, sentiment analysis, and filtering inappropriate content).
- Deployment & Orchestration: Docker Compose, Kubernetes manifests (found in k8s/ and the kubernetes.yaml file), and Airflow DAGs (“src/deployment/airflow_dag.py”) show that the services are containerized and orchestrated.
- Monitoring: Services for Prometheus and Grafana in the “docker/monitoring” folders provide observability of system metrics.
- External Integration: The README mentions integration with AWS, Snowflake, HuggingFace, OpenAI, and external login/authentication.

##### Defining Relationships and Interactions
- Users interact with the frontend (browser) which makes REST API calls to the backend.
- The API service (e.g., “index.py” in “src/api”) acts as a gateway to query the model services based on recommendation requests received from the frontend.
- Each model service is called (e.g., via “call_model_n.py”) independently as microservices that run recommendation algorithms.
- The ETL system processes raw data from the original Amazon dataset and prepares the inputs for the models and API.
- Monitoring components gather metrics from API, model services, and possibly ETL jobs to track system health.
- Docker and Kubernetes are used for containerization and deployment, with orchestration files in both docker-compose.yaml and k8s/ directories.

##### Architectural Patterns & Design Principles
- Microservices architecture: Separate model services and API backend allow individual deployment and scaling.
- Containerization: Dockerfiles for each service enable reproducible environments. Ensure each service uses its own .env file.
- Separation of concerns: The division between frontend, backend, business logic (models), and data processing (ETL) is clearly maintained.
- Monitoring and orchestration: Using Kubernetes, Prometheus, and Grafana highlights a production-ready architecture with observability and scalability in mind.

##### Detailed Steps
- Below that, add the API Backend service (“Python API in src/api”).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interaction
    subgraph "User Interaction"
        U["User"]:::external
    end

    %% Frontend Layer
    subgraph "Frontend Layer"
        FE["Frontend [React/TypeScript]"]:::frontend
    end

    %% API Layer
    subgraph "API Layer"
        API["API Backend (Python)"]:::backend
    end

    %% Model Services
    subgraph "Model Services"
        subgraph "Generate New Products"
            GNP_CODE["Service Code"]:::models
            GNP_DOCKER["Docker Container"]:::models
            GNP_CODE --> GNP_DOCKER
        end
        subgraph "Model 1"
            M1_CODE["Service Code"]:::models
            M1_DOCKER["Docker Container"]:::models
            M1_CODE --> M1_DOCKER
        end
        subgraph "Model 2"
            M2_CODE["Service Code"]:::models
            M2_DOCKER["Docker Container"]:::models
            M2_CODE --> M2_DOCKER
        end
        subgraph "Model 3"
            M3_CODE["Service Code"]:::models
            M3_DOCKER["Docker Container"]:::models
            M3_CODE --> M3_DOCKER
        end
        subgraph "Model 4"
            M4_CODE["Service Code"]:::models
            M4_DOCKER["Docker Container"]:::models
            M4_CODE --> M4_DOCKER
        end
    end

    %% ETL Pipeline
    subgraph "ETL Pipeline"
        ETL["ETL Processes"]:::etl
    end

    %% Deployment & Orchestration
    subgraph "Deployment & Orchestration"
        DC["Docker Compose"]:::deployment
        K8S["Kubernetes Manifests"]:::deployment
        DAG["Airflow DAG"]:::deployment
        DEP_DOCKER["Deployment Dockerfile"]:::deployment
    end

    %% Monitoring
    subgraph "Monitoring"
        PROM["Prometheus"]:::monitoring
        GRAF["Grafana"]:::monitoring
    end

    %% External Integration
    subgraph "External Integration"
        AWS["AWS"]:::external
        SNOW["Snowflake"]:::external
        HF["HuggingFace"]:::external
        OAI["OpenAI"]:::external
    end

    %% Connections
    U -->|"uses"| FE
    FE -->|"calls"| API

    API -->|"queries"| GNP_CODE
    API -->|"queries"| M1_CODE
    API -->|"queries"| M2_CODE
    API -->|"queries"| M3_CODE
    API -->|"queries"| M4_CODE

    ETL -->|"processes"| API
    ETL -->|"feeds"| GNP_CODE
    ETL -->|"feeds"| M1_CODE
    ETL -->|"feeds"| M2_CODE
    ETL -->|"feeds"| M3_CODE
    ETL -->|"feeds"| M4_CODE

    API ---|"metrics"| PROM
    GNP_CODE ---|"metrics"| PROM
    M1_CODE ---|"metrics"| PROM
    M2_CODE ---|"metrics"| PROM
    M3_CODE ---|"metrics"| PROM
    M4_CODE ---|"metrics"| PROM
    ETL ---|"metrics"| PROM

    PROM -->|"dashboards"| GRAF

    FE ---|"deployed_via"| DC
    API ---|"deployed_via"| DC
    GNP_CODE ---|"deployed_via"| DC
    M1_CODE ---|"deployed_via"| DC
    M2_CODE ---|"deployed_via"| DC
    M3_CODE ---|"deployed_via"| DC
    M4_CODE ---|"deployed_via"| DC
    ETL ---|"deployed_via"| DC
    PROM ---|"deployed_via"| DC
    GRAF ---|"deployed_via"| DC

    DC ---|"orchestrated_by"| K8S
    DAG ---|"schedules"| ETL

    API ---|"integrates_with"| AWS
    API ---|"integrates_with"| SNOW
    GNP_CODE ---|"integrates_with"| HF
    M1_CODE ---|"integrates_with"| OAI
    M2_CODE ---|"integrates_with"| OAI
    M3_CODE ---|"integrates_with"| OAI
    M4_CODE ---|"integrates_with"| OAI

    %% Click Events
    click FE "https://github.com/tcknickerbocker/toy_and_game_recommender/tree/main/src/frontend"
    click API "https://github.com/tcknickerbocker/toy_and_game_recommender/blob/main/src/api/index.py"
    click GNP_CODE "https://github.com/tcknickerbocker/toy_and_game_recommender/tree/main/src/models/generate_new_products"
    click GNP_DOCKER "https://github.com/tcknickerbocker/toy_and_game_recommender/tree/main/docker/generate_new_products/Dockerfile"
    click M1_CODE "https://github.com/tcknickerbocker/toy_and_game_recommender/tree/main/src/models/model_1"
    click M1_DOCKER "https://github.com/tcknickerbocker/toy_and_game_recommender/tree/main/docker/model_1/Dockerfile"
    click M2_CODE "https://github.com/tcknickerbocker/toy_and_game_recommender/tree/main/src/models/model_2"
    click M2_DOCKER "https://github.com/tcknickerbocker/toy_and_game_recommender/tree/main/docker/model_2/Dockerfile"
    click M3_CODE "https://github.com/tcknickerbocker/toy_and_game_recommender/tree/main/src/models/model_3"
    click M3_DOCKER "https://github.com/tcknickerbocker/toy_and_game_recommender/tree/main/docker/model_3/Dockerfile"
    click M4_CODE "https://github.com/tcknickerbocker/toy_and_game_recommender/tree/main/src/models/model_4"
    click M4_DOCKER "https://github.com/tcknickerbocker/toy_and_game_recommender/tree/main/docker/model_4/Dockerfile"
    click ETL "https://github.com/tcknickerbocker/toy_and_game_recommender/tree/main/src/etl"
    click DC "https://github.com/tcknickerbocker/toy_and_game_recommender/blob/main/docker-compose.yaml"
    click K8S "https://github.com/tcknickerbocker/toy_and_game_recommender/tree/main/k8s/"
    click K8S "https://github.com/tcknickerbocker/toy_and_game_recommender/blob/main/kubernetes.yaml"
    click DAG "https://github.com/tcknickerbocker/toy_and_game_recommender/blob/main/src/deployment/airflow_dag.py"
    click DEP_DOCKER "https://github.com/tcknickerbocker/toy_and_game_recommender/tree/main/src/deployment/docker/Dockerfile"
    click PROM "https://github.com/tcknickerbocker/toy_and_game_recommender/tree/main/docker/monitoring/prometheus"
    click GRAF "https://github.com/tcknickerbocker/toy_and_game_recommender/tree/main/docker/monitoring/grafana-provisioning"

    %% Styling
    classDef frontend fill:#FFCC00,stroke:#333,stroke-width:2px;
    classDef backend fill:#66CCFF,stroke:#333,stroke-width:2px;
    classDef models fill:#AAFFAA,stroke:#333,stroke-width:2px;
    classDef etl fill:#FFAAAA,stroke:#333,stroke-width:2px;
    classDef deployment fill:#CCCCFF,stroke:#333,stroke-width:2px;
    classDef monitoring fill:#FF99CC,stroke:#333,stroke-width:2px;
    classDef external fill:#DADADA,stroke:#333,stroke-width:2px;
```
