# vmware/data-annotator-for-machine-learning architecture

> Architecture diagram of the GitHub repository vmware/data-annotator-for-machine-learning, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vmware/data-annotator-for-machine-learning
- Repository: https://github.com/vmware/data-annotator-for-machine-learning

## Overview

- annotation-app – An Angular-based frontend providing the UI for users to interact with annotation projects.
- annotation-service – A Node.js/Express backend service that handles core backend APIs, authentication, user and project management, file handling, Slack integration, and more.
- active-learning-service – A Django application that implements an active learning API using uncertainty sampling (via the modAL library) and integrates machine learning-specific functionalities (e.g., model training, query ranking).

##### Main Components
a. Frontend Layer: Represent the Angular-based annotation-app.
– Include its modules, routing, shared components, services, and assets.
– Note that it’s responsible for rendering UI, handling user interactions, and making API calls.
b. Backend Layer: Represent the annotation-service built with Node.js/Express.
– Include its routers (for auth, projects, emails, integrations, etc.), middleware (JWT auth, etc.), services (S3, Slack, database operations), and database connection modules.
c. Active Learning Service: Represent the Django-based active-learning-service.
– Highlight its configuration modules, ML service scripts (active learning, model training/inference), and utility packages.
##### Establish Relationships and Interactions
– Include communication between the annotation-service and active-learning-service, if applicable, for triggering active learning workflows or retrieving ML outcomes.
– Represent external connections, such as:
- Database (MongoDB, as indicated by the “mongo_connect” in active-learning-service and the db folders in annotation-service).
- External integrations like AWS services (S3, STS), Slack services (several slack-related service files), and SQS.
– Show Docker Compose orchestration managing these containers/services (as evidenced by the docker-compose files).
##### Architectural Patterns and Design Principles
– Emphasize the separation of concerns (frontend vs. backend vs. ML service).
– Call out the use of microservices / service separation (since each component runs in its own service/container).
– Note RESTful API design for the backend with clearly identified controllers/routers.
– Show integration points (e.g., file I/O, active learning logic, and third-party integrations like Slack and AWS).
##### Mention Relevant Technologies and Frameworks
– Angular (annotation-app)
– Node.js with Express (annotation-service)
– Django with modAL (active-learning-service)
– Docker (Dockerfile and docker-compose configurations)
– AWS services integrations (S3, STS, SQS)
– Slack API integrations (multiple service files and resources)
– MongoDB (database connection modules)
##### Capturing Essential Details
– Capture not only the high-level components but also emphasize internal modules if necessary (like the various Angular modules in the annotation-app or the multiple router modules and middleware in the annotation-service).
– If desired, add notes on deployment orchestration with Docker Compose and how CI/CD pipelines (as evidenced by GitHub workflows) fit into the overall architecture.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend Layer: Angular annotation-app
    subgraph "Angular Frontend (annotation-app)"
        A1["Annotation UI (Angular)"]:::frontend
        A2["App Modules & Components (annotation-app/src/app)"]:::frontend
        A3["Configurations (angular.json, package.json, proxy.conf.json)"]:::frontend
        A1 -.-> A2
        A1 -.-> A3
    end

    %% Backend Layer: Node.js/Express annotation-service
    subgraph "Backend Layer (annotation-service)"
        B1["API Server (Node/Express)"]:::backend
        B2["Routers (annotation-service/routers)"]:::backend
        B3["Middleware (annotation-service/middlewares)"]:::backend
        B4["Services (annotation-service/services)"]:::backend
        B5["DB Connectors (annotation-service/db)"]:::backend
        B1 -.-> B2
        B1 -.-> B3
        B1 -.-> B4
        B1 -.-> B5
    end

    %% Active Learning Service: Django active-learning-service
    subgraph "Active Learning Service (Django)"
        C1["ML Service (Django Active Learning)"]:::ml
        C2["Configuration (active-learning-service/mysite)"]:::ml
        C3["ML Modules (active-learning-service/src/al)"]:::ml
        C4a["AWS Utils (active-learning-service/src/aws)"]:::ml
        C4b["DB Utils (active-learning-service/src/db)"]:::ml
        C5["Models & Utilities (active-learning-service/models)"]:::ml
        C1 -.-> C2
        C1 -.-> C3
        C1 -.-> C4a
        C1 -.-> C4b
        C1 -.-> C5
    end

    %% External and Orchestration Components
    D["Database (MongoDB)"]:::external
    E["External Services (AWS)"]:::external
    F["External Services (Slack,SQS)"]:::external
    G["Docker Compose Orchestration"]:::orchestration

    %% Interactions Between Layers
    A1 -->|"RESTAPICall"| B1
    B1 -->|"ActiveLearningTrigger"| C1
    B1 -->|"DBOperations"| D
    C1 -->|"DBAccess"| D
    B1 -->|"AWSIntegration"| E
    B1 -->|"Slack/SQS"| F
    G -->|"Orchestrates"| A1
    G -->|"Orchestrates"| B1
    G -->|"Orchestrates"| C1

    %% Click Events for Angular Frontend
    click A1 "https://github.com/vmware/data-annotator-for-machine-learning/tree/master/annotation-app"
    click A2 "https://github.com/vmware/data-annotator-for-machine-learning/tree/master/annotation-app/src/app"
    click A3 "https://github.com/vmware/data-annotator-for-machine-learning/blob/master/angular.json"

    %% Click Events for Node.js/Express Backend
    click B1 "https://github.com/vmware/data-annotator-for-machine-learning/blob/master/annotation-service/server.js"
    click B2 "https://github.com/vmware/data-annotator-for-machine-learning/tree/master/annotation-service/routers"
    click B3 "https://github.com/vmware/data-annotator-for-machine-learning/tree/master/annotation-service/middlewares"
    click B4 "https://github.com/vmware/data-annotator-for-machine-learning/tree/master/annotation-service/services"
    click B5 "https://github.com/vmware/data-annotator-for-machine-learning/tree/master/annotation-service/db"

    %% Click Events for Django Active Learning Service
    click C1 "https://github.com/vmware/data-annotator-for-machine-learning/blob/master/active-learning-service/manage.py"
    click C2 "https://github.com/vmware/data-annotator-for-machine-learning/tree/master/active-learning-service/mysite"
    click C3 "https://github.com/vmware/data-annotator-for-machine-learning/tree/master/active-learning-service/src/al"
    click C4a "https://github.com/vmware/data-annotator-for-machine-learning/tree/master/active-learning-service/src/aws"
    click C4b "https://github.com/vmware/data-annotator-for-machine-learning/tree/master/active-learning-service/src/db"
    click C5 "https://github.com/vmware/data-annotator-for-machine-learning/tree/master/active-learning-service/models"

    %% Styles
    classDef frontend fill:#f9c74f,stroke:#333,stroke-width:2px;
    classDef backend fill:#90be6d,stroke:#333,stroke-width:2px;
    classDef ml fill:#f94144,stroke:#333,stroke-width:2px;
    classDef external fill:#577590,stroke:#333,stroke-width:2px;
    classDef orchestration fill:#f3722c,stroke:#333,stroke-width:2px;
```
