How mlcommons/dynabench works
Main Components of the System
• Frontend Interfaces:
• There are two main frontend sections: one in “frontends/web” (the main user-facing web client) and documentation in “frontends/docs”. Emphasize that the web client interacts with the API and user services.
• Backend/API Servers:
• The “api” directory contains controllers, common utilities, cron tasks, migrations, and even specific modules for evaluation and building. This indicates that the backend not only serves API endpoints to support the frontends, but also runs asynchronous processes (e.g., via cron jobs).
• The “backend” directory also holds another layer of API endpoints (e.g., REST endpoints with middleware, domain services, and repositories) as well as separate modules for legacy “dynalab” applications supporting model building and evaluation.
• Evaluation and Building Modules:
• Within “api/evaluation” and “api/builder”, there are components for handling model evaluation, model deployment, and building tasks. They should be drawn as distinct processing services that also communicate with the persistent storage.
• Data Persistence and Migrations:
• The project contains a “migrations” folder (inside “api/migrations”) and model definitions (inside “api/models”). Indicate that a database (likely relational) is used, and that both the “api” and “backend” layers interact with it.
• External and Legacy Services:
• The “legacy” directories (annotators, mlcube, torchserve, etc.) represent older or supplementary components that may be integrated for handling specific machine learning models or tasks.
• Infrastructure and DevOps:
• There are docker-related files (Dockerfile, docker-compose.yaml, ECS task definitions) and configuration scripts (e.g., bootstrap.sh, various .env examples) that indicate containerization and deployment processes.
Relationships and Interactions
• Frontend Interaction:
• That the frontend may also serve static content (from “frontends” static assets) and documentation content.
• API Gateway / RESTful Services:
• The “api” and “backend” components both surface REST endpoints. The “api/controllers” handle various resources (auth, datasets, models, tasks, etc.), while “backend/app/api” has endpoints for builder and evaluation as well as other core services.
• Asynchronous and Task Processing:
• Model Building & Evaluation:
• The specialized modules in “api/builder”, “api/evaluation”, and the corresponding builder and evaluation services in “backend/app/domain/services/builder_and_evaluation” should be depicted as separate yet inter-related processing flows. These modules may also interact with external storage (e.g., S3) and messaging or email services (as implied by “api/infrastructure/email” and “backend/app/domain/helpers/email”).
• Data Persistence:
Architectural Patterns and Technologies
• Clear Separation of Concerns:
• The project separates concerns among frontend, API (controllers), domain logic (services), and data persistence (models/repositories).
• The existence of Dockerfiles, docker-compose setups, and ECS definitions speaks to containerized microservices or at least modular deployment.
• Modular and Layered Architecture:
• The system is modular—look at separate folders for controllers, domain logic, and infrastructure. This layered separation should be reflected by grouping related components.
• Asynchronous Processing:
• Cron jobs and background workers indicate an asynchronous processing pattern.
• Legacy Integration:
• The “legacy” folder components (annotators, mlcube, torchserve) may be integrated via adapter patterns or serve as optional processing pipelines.
Elements
• Clear Component Labels:
• From Frontend to API (user request flows).
• From API to Database (CRUD operations), email services, and external storage.
• Between asynchronous processors (cron jobs, worker tasks) and the main API services.
• Visual Distinctions:
• Additional Metadata:
• Brief labels indicating key technologies or frameworks (for example, Docker, Celery for asynchronous tasks, REST for API communications).