# nvidia-ai-blueprints/digital-human architecture

> Architecture diagram of the GitHub repository nvidia-ai-blueprints/digital-human, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nvidia-ai-blueprints/digital-human
- Repository: https://github.com/nvidia-ai-blueprints/digital-human

## Overview

##### Main Components
- Digital Avatar Pipeline – This part handles 3D animation, speech synthesis, rendering, and real‐time interactions. It uses NVIDIA Tokkio technologies (e.g., Audio2Face for avatar animation, Omniverse RTX for rendering, and integrations with Riva/ElevenLabs for speech).
- RAG Pipeline – This component is responsible for processing natural language queries using LLMs and generative models. It takes care of question-answer interactions and generates the conversation context necessary for the digital human.
- API / Integration Layer – The two pipelines (digital avatar and RAG) communicate via REST APIs. Look at the “api” directory (containing the openapi.json) to define the endpoints, request/response structures, and integration points.
- Deployment & Configuration – The “deploy” directory contains various Docker Compose files, environment scripts (print_env.sh, data_loader.sh), and configuration YAML files used to automate deployment in different environments and variations of the workflow.
- Customization and Evaluation – The “customize” and “evaluate” directories indicate areas where developers can adjust the pipeline (e.g., avatar customization and RAG fine-tuning) and evaluate the system using Jupyter notebooks and additional scripts.

##### Architectural Patterns and Technologies
- Separation of concerns: the digital avatar component is distinct from the RAG pipeline, and they communicate using REST – this reflects a modular microservices style.
- External services like the NVIDIA NIM inference microservices and third-party APIs (e.g., ElevenLabs, Riva) that power parts of the digital human application.
- That deployment is containerized; include Docker Compose (multiple variations for different microservices configurations) and clearly mention that the project supports both digital avatar and customized RAG deployments.

- From there, “drill down” further by adding sub-modules where necessary – note customization options and evaluation components.

## Diagram (Mermaid)

```mermaid
flowchart TD
    User["User/Client"]:::user

    subgraph "Digital Avatar Pipeline"
        DA_Interface["Digital Avatar Interface"]:::digitalAvatar
        DA_Animation["Animation: Audio2Face"]:::digitalAvatar
        DA_Rendering["Rendering: Omniverse RTX"]:::digitalAvatar
        DA_Speech["Speech: NVIDIA Riva/ElevenLabs"]:::digitalAvatar
        DA_Interface -->|"activates"| DA_Animation
        DA_Interface -->|"activates"| DA_Rendering
        DA_Interface -->|"activates"| DA_Speech
    end

    API["API Gateway / REST API"]:::api

    subgraph "RAG Pipeline"
        RAG_Interface["RAG Interface"]:::ragPipeline
        LLM["LLM Processing"]:::ragPipeline
        Embedding["Embedding Generation"]:::ragPipeline
        Reranking["Reranking&DataProcessing"]:::ragPipeline
        Customization["Customization"]:::ragPipeline
        Evaluation["Evaluation"]:::ragPipeline
        DataResources["Data Resources"]:::ragPipeline
        RAG_Interface -->|"triggers"| LLM
        RAG_Interface -->|"triggers"| Embedding
        RAG_Interface -->|"triggers"| Reranking
        RAG_Interface -->|"triggers"| Customization
        RAG_Interface -->|"triggers"| Evaluation
        RAG_Interface -->|"triggers"| DataResources
    end

    DEP["Deployment Infrastructure"]:::deployment

    User -->|"interacts"| DA_Interface
    DA_Interface -->|"callsAPI"| API
    API -->|"forwardsQuery"| RAG_Interface
    RAG_Interface -->|"returnsResponse"| API
    API -->|"deliversResponse"| DA_Interface

    DEP ---|"provisions"| DA_Interface
    DEP ---|"provisions"| API
    DEP ---|"provisions"| RAG_Interface

    subgraph "Deployment Configurations"
        D1["Docker Compose YAML"]:::deployment
        D2["NIM Microservices Compose"]:::deployment
        D3["VectorDB Compose"]:::deployment
        D4["Common Config YAML"]:::deployment
        D5["Data Loader Script"]:::deployment
        DEP --> D1
        DEP --> D2
        DEP --> D3
        DEP --> D4
        DEP --> D5
    end

    click API "https://github.com/nvidia-ai-blueprints/digital-human/blob/main/api/openapi.json"
    click Customization "https://github.com/nvidia-ai-blueprints/digital-human/blob/main/customize/rag_peft.md"
    click Evaluation "https://github.com/nvidia-ai-blueprints/digital-human/blob/main/evaluate/rag_evals.ipynb"
    click DataResources "https://github.com/nvidia-ai-blueprints/digital-human/tree/main/data/"

    click D1 "https://github.com/nvidia-ai-blueprints/digital-human/blob/main/deploy/docker-compose.yaml"
    click D2 "https://github.com/nvidia-ai-blueprints/digital-human/blob/main/deploy/docker-compose-nim-ms.yaml"
    click D3 "https://github.com/nvidia-ai-blueprints/digital-human/blob/main/deploy/docker-compose-vectordb.yaml"
    click D4 "https://github.com/nvidia-ai-blueprints/digital-human/blob/main/deploy/common.yaml"
    click D5 "https://github.com/nvidia-ai-blueprints/digital-human/blob/main/deploy/data_loader.sh"

    classDef user fill:#277da1,stroke:#fff,stroke-width:2px;
    classDef digitalAvatar fill:#f9c74f,stroke:#000,stroke-width:2px;
    classDef api fill:#f9844a,stroke:#000,stroke-width:2px;
    classDef ragPipeline fill:#90be6d,stroke:#000,stroke-width:2px;
    classDef deployment fill:#577590,stroke:#fff,stroke-width:2px;
```
