How tcknickerbocker/toy_and_game_recommender works
System’s primary components
• Frontend: Located in the “src/frontend” directory, this is the interface through which users interact with the recommender.
• API Backend: Found under “src/api”, this acts as the main service handling HTTP requests from the frontend.
• Model Services: There are multiple model microservices (e.g., “src/models/generate_new_products”, “src/models/model_1”, “src/models/model_2”, “src/models/model_3”, “src/models/model_4”). Each has its own Dockerfile and .env to allow independent scaling/deployment.
• ETL Pipeline: Under “src/etl”, these scripts handle data processing tasks (e.g., computing scores, embeddings, sentiment analysis, and filtering inappropriate content).
• Deployment & Orchestration: Docker Compose, Kubernetes manifests (found in k8s/ and the kubernetes.yaml file), and Airflow DAGs (“src/deployment/airflow_dag.py”) show that the services are containerized and orchestrated.
• Monitoring: Services for Prometheus and Grafana in the “docker/monitoring” folders provide observability of system metrics.
• External Integration: The README mentions integration with AWS, Snowflake, HuggingFace, OpenAI, and external login/authentication.
Defining Relationships and Interactions
• Users interact with the frontend (browser) which makes REST API calls to the backend.
• The API service (e.g., “index.py” in “src/api”) acts as a gateway to query the model services based on recommendation requests received from the frontend.
• Each model service is called (e.g., via “call_model_n.py”) independently as microservices that run recommendation algorithms.
• The ETL system processes raw data from the original Amazon dataset and prepares the inputs for the models and API.
• Monitoring components gather metrics from API, model services, and possibly ETL jobs to track system health.
• Docker and Kubernetes are used for containerization and deployment, with orchestration files in both docker-compose.yaml and k8s/ directories.
Architectural Patterns & Design Principles
• Microservices architecture: Separate model services and API backend allow individual deployment and scaling.
• Containerization: Dockerfiles for each service enable reproducible environments. Ensure each service uses its own .env file.
• Separation of concerns: The division between frontend, backend, business logic (models), and data processing (ETL) is clearly maintained.
• Monitoring and orchestration: Using Kubernetes, Prometheus, and Grafana highlights a production-ready architecture with observability and scalability in mind.
Detailed Steps
• Below that, add the API Backend service (“Python API in src/api”).