How databytes-organisation/discountmate_new works
1. Overall shape
- DiscountMate is a multi-repo style monorepo with a production app plus supporting data/ML work.
- The live user-facing path is: browser/custom domain → Firebase Hosting → Cloud Run frontend → Cloud Run backend → internal ML sidecars.
2. Frontend subsystem
- Frontend/ is an Expo + React Native Web application built for web delivery.
- It contains the public shopping experience and the authenticated user hub: product browse/search, categories, product detail, compare, lists, basket, wishlist, profile, notifications, subscription, support, and dashboard pages.
- The frontend talks to the backend through EXPO_PUBLIC_API_URL; it does not connect directly to MongoDB or ML services.
3. Backend API subsystem
- Backend/ is a Node.js/Express API that acts as the main application boundary.
- It exposes routes for products, categories, users, baskets, lists, notifications, blogs/news, dashboard, analytics, contact/support, and ML-related features.
- Swagger is included for API documentation, and middleware covers auth, admin access, throttling, scraping protection, and request validation.
4. Backend data access boundary
- The backend is the only public server-side entrypoint for application data.
- It connects to MongoDB for app data, user state, and content initialization scripts.
- This keeps frontend code and external clients isolated from the database and centralizes auth, validation, and business rules.
5. ML service sidecar
- Backend/ml-service/ is a Python service used for model-backed features such as weekly specials, recommendations, price prediction, OCR receipt processing, and recipe assistance/RAG.
- In deployment, it runs as a private sidecar container in the same Cloud Run service as the backend and is reached over localhost, not the public internet.
- The backend proxies user requests to this service, which reduces the public attack surface.
6. Reverse image search sidecar
- Backend/ml-service/reverse_image_search/ is a separate FastAPI service for product image lookup.
- It is also deployed as an internal sidecar and loads FAISS assets from Google Cloud Storage.
- The backend exposes a stable public API route and forwards requests to the local reverse-image-search container.
7. Data engineering pipeline
- DE/ contains the ingestion and ETL layer that prepares supermarket product data.
- The ingestion-pipeline/ scrapes retailer data into local/GCS storage, and etl-pipeline/ transforms it into structured silver-layer tables and product-history outputs.
- This pipeline is the source of the application’s curated catalog, pricing, promotions, and product identity normalization.
8. Scraping and enrichment sources
- Several folders under Scrapping/, Catalogue_Scraping_2025/, and older archived areas contain retailer-specific scrapers, OCR, and catalogue-processing experiments.
- These are upstream data acquisition tools feeding the DE pipeline and ML experiments rather than the live app directly.
- They form the ingestion boundary from external retailer websites/catalogues into internal datasets.
9. ML/analytics experiments vs runtime services
- The ML/, Data Analysis/, and notebook-heavy folders are research and model-development areas.
- They produce concepts, trained artifacts, and analysis for price prediction, recommendations, reverse image search, OCR, and basket optimisation.
- Only selected capabilities are operationalized into the runtime services; most notebooks are offline experimentation support.
10. Infrastructure and deployment
- discount-mate-infra/ defines Terraform for Cloud Run, Artifact Registry, GCS buckets, PostgreSQL, and GitHub Actions identity.
- The active deployment pattern is containerized services on Google Cloud Run, with images pushed to Artifact Registry.
- Firebase Hosting provides the custom domain and rewrites all traffic to the frontend Cloud Run service.
11. CI/CD and release flow
- .github/workflows/ contains the automation for checks and deployment prep.
- The main workflow validates frontend, backend, and ML code, then builds/pushes images and can deploy to Cloud Run when enabled.
- This is the release boundary between source control and production cloud infrastructure.
12. Documentation and operational knowledge
- docs-site/ is the Docusaurus documentation site for architecture, engineering practices, and project handover.
- The repository also includes many PDFs, notebooks, and archived folders that preserve prior implementations, reports, and alternative approaches.
- For architecture diagrams, those archived areas should be treated as historical context, not the current production path.
Main entry points: Backend Backend API · workflows CI/CD · docs-site Docs Site · discount-mate-infra Infra
Components 19
Production App
Live customer-facing product and API path.
- Browser · Customer entry point for the shopping experience.
- Frontend · Expo + React Native Web app with shopping and user hub pages.
Frontend - Backend API · Main application boundary for product, user, list, dashboard, and ML routes.
Backend - API Middleware · Auth, admin, throttling, anti-scraping, and request validation.
Backend/src/middleware - Routes · Public route surface for products, categories, baskets, lists, notifications, blog, news, dashboard, analytics, contact, and ML.
Backend/src/routers - Controllers · Business logic for API endpoints and orchestration to services and data.
Backend/src/controllers - MongoDB · Primary application datastore for user state and content.
- ML Proxy · Backend-side proxy for private ML sidecars and local model services.
Backend/src/services/reverseImageSearchProcess.js
Data Pipeline
Ingestion and ETL that curate catalog and pricing data.
ML Runtime
Operational Python services behind application features.
- ML Service · Private service for specials, recommendations, price prediction, OCR, and recipe assistance.
Backend/ml-service - Reverse Image · Internal reverse-image-search service backed by FAISS assets in cloud storage.
Backend/ml-service/reverse_image_search/api.py
Research & Experiments
Offline notebooks, prototypes, and historical implementations.
- Research ML · Offline model development for price prediction, recommendations, substitutions, OCR, and search.
ML - Analytics · Data analysis, dashboards, and reporting artifacts supporting product decisions.
Data Analysis
Delivery & Docs
Infrastructure, CI/CD, hosting, and documentation.
- Firebase Hosting · Custom domain and rewrite layer in front of the web app.
firebase.json - Infra · Terraform for Cloud Run, Artifact Registry, storage, databases, and identity.
discount-mate-infra - CI/CD · Workflows for checks, release prep, documentation deploy, and service delivery.
.github/workflows - Docs Site · Project documentation and handover site for engineering and architecture knowledge.
docs-site
Connections 27
- Browser → Firebase Hosting: opens · Public traffic enters through the custom domain and hosting layer.
- Firebase Hosting → Frontend: rewrites · Hosting forwards web requests to the frontend service.
- Frontend → Backend API: API calls · Frontend consumes backend APIs via the configured public API URL.
- Backend API → API Middleware: guards · Requests pass through auth, throttling, validation, and anti-scraping middleware.
- API Middleware → Routes: routes · Middleware protects the route surface.
- Routes → Controllers: handles · Routes dispatch to controllers.
- Controllers → MongoDB: reads/writes · Controllers persist and query application state in MongoDB.
- Controllers → ML Proxy: proxies to · Backend forwards ML-related requests to the local service bridge.
- ML Proxy → ML Service: invokes · Proxy reaches the internal ML runtime over localhost/private networking.
- ML Proxy → Reverse Image: forwards · Reverse-image requests are routed to the dedicated sidecar.
- Backend API → MongoDB: initializes · Backend bootstraps content and user data from MongoDB.
- Ingestion → ETL: feeds · Raw ingested data becomes ETL input.
- Scrapers → Ingestion: sources · Retailer scrapers and catalogue tools provide upstream acquisition.
- ETL → MongoDB: curates · Processed product and pricing outputs populate the application datastore boundary.
- Research ML → ML Service: informs · Offline experiments shape runtime ML features and models.
- Research ML → Reverse Image: informs · Reverse-image research informs the sidecar implementation.
- Analytics → ETL: consumes · Analysis and reporting use curated pipeline outputs.
- Infra → Firebase Hosting: supports · Infrastructure defines the hosting and cloud deployment targets.
- Infra → Backend API: deploys · Terraform provisions backend Cloud Run and supporting cloud resources.
- Infra → Frontend: deploys · Infrastructure also supports the frontend deployment path.
- CI/CD → Frontend: builds · CI validates and prepares the frontend application.
- CI/CD → Backend API: builds · CI validates and prepares the backend API.
- CI/CD → ML Service: builds · CI validates and prepares ML service images and checks.
- CI/CD → Docs Site: publishes · Workflow can deploy the documentation site.
- Docs Site → Infra: documents · Docs capture infrastructure and operational knowledge.
- Docs Site → Ingestion: documents · Docs explain the ingestion pipeline architecture.
- Docs Site → Research ML: documents · Docs cover ML and experimentation areas.