How vmware/data-annotator-for-machine-learning works
• annotation-app – An Angular-based frontend providing the UI for users to interact with annotation projects.
• annotation-service – A Node.js/Express backend service that handles core backend APIs, authentication, user and project management, file handling, Slack integration, and more.
• active-learning-service – A Django application that implements an active learning API using uncertainty sampling (via the modAL library) and integrates machine learning-specific functionalities (e.g., model training, query ranking).
Main Components
a. Frontend Layer: Represent the Angular-based annotation-app.
– Include its modules, routing, shared components, services, and assets.
– Note that it’s responsible for rendering UI, handling user interactions, and making API calls.
b. Backend Layer: Represent the annotation-service built with Node.js/Express.
– Include its routers (for auth, projects, emails, integrations, etc.), middleware (JWT auth, etc.), services (S3, Slack, database operations), and database connection modules.
c. Active Learning Service: Represent the Django-based active-learning-service.
– Highlight its configuration modules, ML service scripts (active learning, model training/inference), and utility packages.
Establish Relationships and Interactions
– Include communication between the annotation-service and active-learning-service, if applicable, for triggering active learning workflows or retrieving ML outcomes.
– Represent external connections, such as:
• Database (MongoDB, as indicated by the “mongo_connect” in active-learning-service and the db folders in annotation-service).
• External integrations like AWS services (S3, STS), Slack services (several slack-related service files), and SQS.
– Show Docker Compose orchestration managing these containers/services (as evidenced by the docker-compose files).
Architectural Patterns and Design Principles
– Emphasize the separation of concerns (frontend vs. backend vs. ML service).
– Call out the use of microservices / service separation (since each component runs in its own service/container).
– Note RESTful API design for the backend with clearly identified controllers/routers.
– Show integration points (e.g., file I/O, active learning logic, and third-party integrations like Slack and AWS).
Mention Relevant Technologies and Frameworks
– Angular (annotation-app)
– Node.js with Express (annotation-service)
– Django with modAL (active-learning-service)
– Docker (Dockerfile and docker-compose configurations)
– AWS services integrations (S3, STS, SQS)
– Slack API integrations (multiple service files and resources)
– MongoDB (database connection modules)
Capturing Essential Details
– Capture not only the high-level components but also emphasize internal modules if necessary (like the various Angular modules in the annotation-app or the multiple router modules and middleware in the annotation-service).
– If desired, add notes on deployment orchestration with Docker Compose and how CI/CD pipelines (as evidenced by GitHub workflows) fit into the overall architecture.