How opendatalab/labelllm works
Project Type and Purpose
• LabelLLM is a full-stack, web-based data annotation platform.
• The backend is a single FastAPI service (monolith) with RESTful endpoints, background scheduler, and worker processes.
• The frontend consists of three single-page React/TypeScript apps (Login, Operator, Supplier) built with Vite and served via Nginx or a static host.
• Core services include a relational database (PostgreSQL/MySQL), object storage (MinIO), and optional SSO.
Main Components
A. Frontend Layer
• Login App (src/apps/login)
• Operator App (src/apps/operator)
• Supplier App (src/apps/supplier)
• Shared UI components (src/components) and utility modules (src/hooks, src/utils)
• Build & Serve: Vite, Rollup, Nginx configuration, Dockerfile
B. API Layer (Backend)
• FastAPI application (app/main.py, app/api/v1/router.py)
• Dependency injection (app/api/deps.py), middleware, security (JWT, CORS)
• CRUD services (app/crud), schemas (app/schemas), Pydantic models
C. Data Layer
• Relational Database (via SQLAlchemy in app/db/session.py, migrations)
• Object Storage: MinIO client (app/client/minio.py) for file uploads/downloads
D. Background Tasks
• Scheduler (app/scheduler/init_scheduler.py) for periodic jobs (stats aggregation, cleanup)
• Worker (backend/worker.py) for asynchronous tasks (file processing, AI pre-annotation)
E. Infrastructure
• Docker Compose orchestrates:
– Backend container (FastAPI + Gunicorn)
– Worker/scheduler container
– MinIO container
– Database container
– Frontend container(s) or static host + Nginx container
• External SSO endpoint (optional)
Relationships and Data Flow
1. User (Browser) → Nginx (or Vite dev server) → React SPA (Login/Operator/Supplier)
2. React SPA → REST API calls → FastAPI backend (HTTP port)
3. FastAPI →
• Reads/writes from/to the relational database
• Stores/retrieves files from MinIO
4. Scheduler triggers internal backend tasks (via APScheduler or Celery beat)
5. Worker process picks up jobs (message queue or direct DB polling) for heavy or AI-assisted annotation tasks
6. Backend emits task results to DB → Frontend polling or WebSocket (if implemented)
Architectural Patterns & Design Principles
• Layered architecture: Presentation (React), API, Service/CRUD, Data
• Clear separation of concerns:
• Routers define endpoints →
• CRUD modules implement business logic →
• Schemas/Pydantic enforce validation →
• Models/ORM map to tables
• Dependency injection in FastAPI for config, DB sessions, security
• Micro-app structure for frontend (three distinct apps sharing common components)
• Infrastructure as code: Docker Compose, Dockerfiles, build scripts