# sign/translate architecture

> Architecture diagram of the GitHub repository sign/translate, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sign/translate
- Repository: https://github.com/sign/translate

## Overview

##### Project type and purpose
- This is a full-stack, real-time sign-language translation application (“Sign Translate”) with:
- A rich Angular/Ionic front-end for desktop and mobile (wrapped via Capacitor)
- A serverless back-end on Firebase Functions exposing HTTP APIs
- TFJS and Mediapipe models loaded in the client for on-device inference
- GitHub Actions CI/CD, Fastlane pipelines for mobile publishing, and VitePress docs

##### File-structure analysis
- Top-level “src/” folder holds the Angular app with:
- “app/” – components, modules, pages (Translation pages, Landing, Settings, Playground)
- “environments/” – environment.ts (prod/test/dev)
- “main.ts”, “server.ts”, “main.server.ts” – hybrid client/server entrypoints
- “functions/” – Firebase Functions code with:
- “gateway/” – HTTP controllers (avatars, me, spoken-to-signed, etc.)
- “text-normalization/”, “text-to-text/”, “prerender/”, plus shared middlewares
- Mobile wrappers: “android/” and “ios/” with Capacitor/Gradle and Xcode projects, plus Fastlane metadata
- “docs/” – VitePress documentation site
- CI/CD: “.github/workflows” for client, server, Lighthouse audits

##### README insights
- It highlights two core flows:
a. Spoken-to-Signed (Audio → Text → Normalization → SignWriting → Pose → Skeleton/3D Avatar → Human GAN)
b. Signed-to-Spoken (Video Upload/Camera → Segmentation → SignWriting → Text → Audio)
- The front-end uses modules (pose, detector, pix2pix, sign-writing, translate) with NGXS state management
- TFJS models and web workers are used for on-device inference

##### Key components & interactions
a. Frontend (Angular/Ionic)
- UI pages/components for capture (video, audio), model loading (TFJS, Mediapipe), translation display
- Services: HTTP client → Firebase Functions endpoints; TFJS loader/service; Google Analytics; AppCheck
- State: NGXS stores for video frames, translation state, pose, settings
- Workers: pix2pix.worker.ts for heavy ML tasks off the main thread
b. Back-end (Firebase Functions)
- Gateway routers expose REST endpoints: /avatars, /spoken-to-signed, /me, /prerender, /text-normalization, /text-to-text
- Middlewares: AppCheck validation, rate-limiting, error handling
- Controllers orchestrate model invocation or proxy to model servers / TFJS on Cloud Functions
- (Optionally) Cloud Storage holds large model files, accessed by client and functions
c. External services
- Google AppCheck for bot protection
- Google Analytics for usage tracking
- Firebase Hosting / Cloud Functions for deployment
- Fastlane for mobile app distribution
d. Local assets
- /assets/models – TFJS and Mediapipe model files loaded by the client
- /assets/promotional – media for marketing and docs site

##### Architectural patterns & design principles
- Client-heavy: offload inference to browser via TFJS, Mediapipe, and Web Workers
- Reactive state management (NGXS) guiding UI flows
- Serverless micro-services: each function folder is an independent HTTP endpoint
- Clear separation: front-end vs. back-end vs. mobile wrapper vs. docs

##### Don’t overthink—be exhaustive
- Break the front-end into UI, state, ML services, HTTP client
- Break the back-end into gateway, controllers, model logic, middlewares
- CI/CD pipelines and documentation site as side components

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Devices
    subgraph "User Devices"
        Desktop["Desktop Browser"]
        AndroidApp["Android App"]
        iOSApp["iOS App"]
    end

    %% Frontend Application
    subgraph "Frontend Application"
        direction TB
        Frontend["Angular/Ionic App"]:::frontendApp
        subgraph "UI Pages"
            direction TB
            UIpages["Translation, Landing, Settings, Playground"]:::ui
        end
        subgraph "ML Modules"
            direction TB
            TFJSLoader["TFJS Loader"]:::ml
            TFJSService["TFJS Service"]:::ml
            Mediapipe["Mediapipe Detector"]:::ml
            Pix2PixWorker["pix2pix Web Worker"]:::ml
            PoseNorm["Pose Normalization"]:::ml
        end
        subgraph "State Management"
            direction TB
            NGXS["NGXS Stores"]:::state
            SignWriting["Sign-Writing Module & State"]:::state
            TranslateMod["Translation Module & Services"]:::state
        end
        HTTPClient["HTTP Client & Token Interceptor"]:::service
        AppCheckHelper["AppCheck Validation"]:::helper
        Analytics["Google Analytics"]:::service
        Assets["Local Assets (models & i18n)"]:::assets

        Frontend --> UIpages
        Frontend --> TFJSLoader
        Frontend --> TFJSService
        Frontend --> Mediapipe
        Frontend --> Pix2PixWorker
        Frontend --> PoseNorm
        Frontend --> NGXS
        Frontend --> SignWriting
        Frontend --> TranslateMod
        Frontend --> HTTPClient
        Frontend --> AppCheckHelper
        Frontend --> Analytics
        Frontend --> Assets
    end

    %% Backend Functions
    subgraph "Serverless Backend (Firebase Functions)"
        direction TB
        index["functions/src/index.ts"]:::backend
        subgraph "Gateway Controllers"
            direction TB
            Avatars["avatars.ts"]:::backend
            Me["me.ts"]:::backend
            SpokenToSigned["spoken-to-signed.ts"]:::backend
            PrerenderGW["prerender/controller.ts"]:::backend
        end
        subgraph "Function Services"
            direction TB
            TextNormCtrl["text-normalization/controller.ts"]:::backend
            TextNormModel["text-normalization/model.ts"]:::backend
            TextToTextCtrl["text-to-text/controller.ts"]:::backend
            TextToTextModel["text-to-text/model.ts"]:::backend
            PrerenderCtrl["prerender/controller.ts"]:::backend
        end
        Middlewares["middlewares"]:::middleware
        Utils["utils/memory.ts"]:::backend
        Storage["Cloud Storage"]:::storage

        index --> Avatars
        index --> Me
        index --> SpokenToSigned
        index --> PrerenderGW
        SpokenToSigned --> TextNormCtrl
        TextNormCtrl --> TextNormModel
        SpokenToSigned --> TextToTextCtrl
        TextToTextCtrl --> TextToTextModel
        PrerenderGW --> PrerenderCtrl
        Avatars --> Utils
        Me --> Utils
        Avatars --> Middlewares
        Me --> Middlewares
        SpokenToSigned --> Middlewares
        TextNormCtrl --> Middlewares
        TextToTextCtrl --> Middlewares
        PrerenderCtrl --> Middlewares
        TextNormModel --> Storage
        TextToTextModel --> Storage
    end

    %% CI/CD & Docs
    subgraph "CI/CD & Documentation"
        direction TB
        GH["GitHub Actions"]:::ci
        FastlaneA["Android Fastlane"]:::ci
        FastlaneI["iOS Fastlane"]:::ci
        Docs["VitePress Docs Site"]:::docs
        Hosting["Firebase Hosting"]:::infra

        GH -->|deploy client| Hosting
        GH -->|deploy functions| index
        FastlaneA --> AndroidApp
        FastlaneI --> iOSApp
        GH --> Docs
    end

    %% Data Flows
    Desktop -->|open app| Frontend
    AndroidApp -->|native bridge| Frontend
    iOSApp -->|native bridge| Frontend
    HTTPClient -->|POST /spoken-to-signed| SpokenToSigned
    HTTPClient -->|POST /text-normalization| TextNormCtrl
    HTTPClient -->|POST /text-to-text| TextToTextCtrl
    HTTPClient -->|GET /prerender| PrerenderCtrl
    AppCheckHelper -.->|validate| Middlewares
    Analytics -.->|track| GH

    %% Click Events
    click Frontend "https://github.com/sign/translate/tree/master/src/"
    click UIpages "https://github.com/sign/translate/tree/master/src/app/pages/"
    click TFJSLoader "https://github.com/sign/translate/blob/master/src/app/core/services/tfjs/tfjs.loader.ts"
    click TFJSService "https://github.com/sign/translate/blob/master/src/app/core/services/tfjs/tfjs.service.ts"
    click Mediapipe "https://github.com/sign/translate/blob/master/src/app/modules/detector/detector.service.ts"
    click Pix2PixWorker "https://github.com/sign/translate/blob/master/src/app/modules/pix2pix/pix2pix.worker.ts"
    click PoseNorm "https://github.com/sign/translate/blob/master/src/app/modules/pose/pose-normalization.service.ts"
    click NGXS "https://github.com/sign/translate/tree/master/src/app/core/modules/ngxs/store/"
    click SignWriting "https://github.com/sign/translate/tree/master/src/app/modules/sign-writing/"
    click TranslateMod "https://github.com/sign/translate/tree/master/src/app/modules/translate/"
    click HTTPClient "https://github.com/sign/translate/blob/master/src/app/core/services/http/token-interceptor.service.ts"
    click AppCheckHelper "https://github.com/sign/translate/blob/master/src/app/core/helpers/app-check/app-check.ts"
    click Analytics "https://github.com/sign/translate/blob/master/src/app/core/modules/google-analytics/google-analytics.service.ts"
    click Assets "https://github.com/sign/translate/tree/master/src/assets/models/"
    click index "https://github.com/sign/translate/blob/master/functions/src/index.ts"
    click Avatars "https://github.com/sign/translate/blob/master/functions/src/gateway/avatars.ts"
    click Me "https://github.com/sign/translate/blob/master/functions/src/gateway/me.ts"
    click SpokenToSigned "https://github.com/sign/translate/blob/master/functions/src/gateway/spoken-to-signed.ts"
    click PrerenderGW "https://github.com/sign/translate/blob/master/functions/src/gateway/prerender/controller.ts"
    click TextNormCtrl "https://github.com/sign/translate/blob/master/functions/src/text-normalization/controller.ts"
    click TextNormModel "https://github.com/sign/translate/blob/master/functions/src/text-normalization/model.ts"
    click TextToTextCtrl "https://github.com/sign/translate/blob/master/functions/src/text-to-text/controller.ts"
    click TextToTextModel "https://github.com/sign/translate/blob/master/functions/src/text-to-text/model/model.ts"
    click Middlewares "https://github.com/sign/translate/tree/master/functions/src/middlewares/"
    click Utils "https://github.com/sign/translate/blob/master/functions/src/utils/memory.ts"
    click GH "https://github.com/sign/translate/blob/master/.github/workflows/client.yml"
    click GH "https://github.com/sign/translate/blob/master/.github/workflows/server.yml"
    click GH "https://github.com/sign/translate/blob/master/.github/workflows/lighthouse/lighthouserc.json"
    click FastlaneA "https://github.com/sign/translate/tree/master/android/app/"
    click FastlaneA "https://github.com/sign/translate/tree/master/android/fastlane/"
    click FastlaneI "https://github.com/sign/translate/tree/master/ios/App/"
    click FastlaneI "https://github.com/sign/translate/tree/master/ios/App/fastlane/"
    click Docs "https://github.com/sign/translate/tree/master/docs/.vitepress/"

    %% Styles
    classDef frontendApp fill:#E0F7FA,stroke:#0288D1,color:#000
    classDef ui fill:#B3E5FC,stroke:#0277BD
    classDef ml fill:#E1F5FE,stroke:#01579B
    classDef state fill:#BBDEFB,stroke:#1A237E
    classDef service fill:#B2DFDB,stroke:#004D40
    classDef helper fill:#FFF9C4,stroke:#F57F17
    classDef assets fill:#FFECB3,stroke:#EF6C00
    classDef backend fill:#C8E6C9,stroke:#2E7D32
    classDef middleware fill:#CFD8DC,stroke:#607D8B,stroke-dasharray: 5 5
    classDef storage fill:#FFE0B2,stroke:#E65100,shape:cylinder
    classDef ci fill:#D1C4E9,stroke:#4A148C
    classDef infra fill:#E1BEE7,stroke:#6A1B9A
    classDef docs fill:#E8F5E9,stroke:#1B5E20
```
