How gchq/bailo works
Based on our analysis of the file tree and README, Bailo is a full‐stack web application built to help manage the machine learning lifecycle – enabling collaboration, compliance, and sharing in an ML environment. The project is divided into several clearly defined parts with dedicated directories for the frontend, backend, infrastructure, and libraries (such as Python and model scanning APIs), plus deployment artifacts (Dockerfiles, Compose, and Helm charts).
Main Components
• Frontend: Built with Next.js and React. All UI-related code, including routing (files under frontend/pages) and fetching data (frontend/actions with SWR), should be another major component.
• Backend: A Node.js/Express server (in backend/src) that exposes RESTful APIs through a set of routes. This layer handles business logic, employs middleware (for logging, authentication, error handling), and interacts with the database.
• Database: The backend uses MongoDB (with connection logic probably in backend/src/utils/mongo.ts) via Mongoose models (see backend/src/models).
• External Services and Infrastructure:
○ Registry Service: A container registry on port 5000 (Docker registry) for handling docker images.
○ Object Storage: Minio is used (ports 9000 and 9001) to store files for image storage and file uploads.
○ Email Service: MailCrab (port 1080) is integrated as a fake email server for development.
○ JWKS/Authentication: Certificates and JWKS files produced by scripts (npm run certs and generateJWKS.ts) assist in token authentication, and there are various clients (e.g., cognito.ts in backend/src/clients) that interface with cloud authentication/KMS.
• Additional Components:
○ Infrastructure: Contains Kubernetes (EKS), Helm charts, and standalone deployment configurations.
○ Libs: Both “lib/modelscan_api” and “lib/python” provide additional toolsets/APIs, which may be used to extend functionality or act as SDKs.
○ Testing and CI/CD: With many configuration files (GitHub workflows, GitLab CI, Husky, Prettier, Cypress) to enforce coding standards, automated tests, and container builds.
Relationships and Data Flow
• A user initiates a request via the Frontend (Next.js pages, routed using dynamic TSX files), and the frontend fetches data using SWR from API endpoints.
• The backend’s Express server (located in backend/src/routes) handles API calls, processing middleware before invoking business logic in controllers and services.
• The backend interacts with MongoDB using Mongoose to fetch or update models and also calls out to external services (e.g., file scanning, registry, and authentication) by using dedicated client libraries in backend/src/clients.
• The deployed environment (managed by Docker Compose, Helm charts under infrastructure, and standalone configuration files) acts as the glue to interconnect these containers and external services.
• The Libraries (in lib/) enrich functionality (for instance, providing model scanning APIs or python helpers for client interaction) and are meant to be extensible.
Architectural Patterns and Design Principles
• MVC/Microservice Separation: The backend code is modularized into routes, services, models, and connectors – a pattern that enforces separation of concerns.
• Containerization: With numerous Dockerfiles (one for backend and one for frontend, plus additional ones under lib/ and infrastructure), the project stresses reproducibility and ease of deployment.
• CI/CD Integration: Automated testing and deployment pipelines are integrated (GitHub workflows, GitLab CI) ensuring continuous integration.
• Extensibility and Service Isolation: The use of connectors (for different functionalities like file scanning, authentication, authorisation, etc.) allows for easier extension and maintenance.
Important Technical Details to be Explicitly Shown
• That the Frontend and Backend are separate containers communicating over HTTP/REST.
• How the backend routes (with middleware layers) interface with MongoDB and call out to additional micro-services or connectors.
• Existence of auxiliary libraries in “lib/” that provide extensibility points (for model scanning or Python SDKs).
• That authentication is handled using token mechanisms with JWKS and custom connectors.