# rmr327/llmembeddingxrvizualization architecture

> Architecture diagram of the GitHub repository rmr327/llmembeddingxrvizualization, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rmr327/llmembeddingxrvizualization
- Repository: https://github.com/rmr327/llmembeddingxrvizualization

## Overview

##### Project Type and Purpose
- This is a Python-based CLI/library tool (with an optional Streamlit UI) for fetching LLM embeddings from Hugging Face, applying dimensionality-reduction algorithms, exporting the result as a 3D model (.dae), and viewing it in XR headsets (e.g., Meta Quest).
- It’s distributed as a PyPI package and tested via GitHub Actions.

##### Key Architectural Components
a. Entry Points
- CLI (`llm-embedding-viz`)
- Streamlit dashboard (`streamlit.py`)
b. Core Library Modules (`mylib/`)
- get_hugging_face_embeding.py – connects to Hugging Face API, retrieves embeddings
- plotverse_xr_llm.py – applies dimensionality-reduction (PCA, t-SNE, UMAP, MDS, ISOMAP) and builds the 3D scene graph
- 3D Exporter – serializes scene graph into .dae files for XR consumption
c. External Services & Dependencies
- Hugging Face Model Hub (external REST API)
- Python libs: NumPy, scikit-learn/UMAP/t-SNE libraries, streamlit, pycollada (or similar) for .dae
d. Output & Visualization
- Generated .dae file consumed by any COLLADA-compatible viewer or Meta Quest XR app
- (Optional) Web preview via example website

##### Data Flow & Interactions
1. User runs CLI or Streamlit UI
2. The tool calls Hugging Face API → retrieves high-dimensional embeddings
3. Embeddings fed into selected dimensionality-reduction algorithm
4. Reduced 3D coordinates assembled into a COLLADA scene graph
5. .dae file written to disk
6. User loads .dae in XR viewer on Meta Quest or web preview

##### Architectural Patterns & Principles
- Pipeline / ETL pattern: fetch → transform → export
- Modular design: separate concerns (data access, transformation, export, UI)
- Dependency injection via CLI args or function parameters (model name, reduction method, labels CSV)
- Test suite isolates data-fetch and transform logic (`tests/`)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interfaces
    subgraph "User Interfaces"
        CLI["CLI Interface: llm-embedding-viz"]:::ui
        StreamlitUI["Streamlit UI"]:::ui
    end

    %% Core Processing
    subgraph "Core Processing"
        Fetcher["Embedding Fetcher (Hugging Face API Client)"]:::core
        subgraph "Dimensionality Reduction Engine"
            PCA["PCA"]:::core
            tSNE["t-SNE"]:::core
            UMAP["UMAP"]:::core
            MDS["MDS"]:::core
            ISOMAP["ISOMAP"]:::core
        end
        Processor["3D Scene Builder & Exporter (.dae)"]:::core
    end

    %% External Systems
    subgraph "External Systems"
        HuggingFace["Hugging Face Model Hub"]:::external
        XRViewer["XR Viewer / Web Preview"]:::external
    end

    %% Infrastructure & Testing
    subgraph "Infrastructure & Testing"
        EntryPoints["Package Entry-points"]:::infra
        Config["Config & Dependencies"]:::infra
        CI["GitHub Actions CI"]:::infra
        Tests["Test Suite"]:::infra
        DemoData["Demo Embedding Data"]:::infra
        Notebook["Demo Notebook (Web Preview)"]:::infra
    end

    %% Data Flow
    CLI -->|"invokes"| Fetcher
    StreamlitUI -->|"invokes"| Fetcher
    Fetcher -->|"fetch embeddings"| HuggingFace
    HuggingFace -->|"returns embeddings"| Fetcher
    Fetcher -->|"output high-dim data"| PCA
    Fetcher -->|"output high-dim data"| tSNE
    Fetcher -->|"output high-dim data"| UMAP
    Fetcher -->|"output high-dim data"| MDS
    Fetcher -->|"output high-dim data"| ISOMAP
    PCA -->|"3D coords"| Processor
    tSNE -->|"3D coords"| Processor
    UMAP -->|"3D coords"| Processor
    MDS -->|"3D coords"| Processor
    ISOMAP -->|"3D coords"| Processor
    Processor -->|"export .dae"| XRViewer

    %% Infrastructure Relations
    CLI --> EntryPoints
    EntryPoints --> Config
    CI -->|runs tests| Tests
    Tests --> Fetcher
    Tests --> Processor
    DemoData --> Tests
    Notebook --> Processor

    %% Click Events
    click CLI "https://github.com/rmr327/llmembeddingxrvizualization/blob/main/setup.py"
    click StreamlitUI "https://github.com/rmr327/llmembeddingxrvizualization/blob/main/mylib/streamlit.py"
    click Fetcher "https://github.com/rmr327/llmembeddingxrvizualization/blob/main/mylib/get_hugging_face_embeding.py"
    click Processor "https://github.com/rmr327/llmembeddingxrvizualization/blob/main/mylib/plotverse_xr_llm.py"
    click EntryPoints "https://github.com/rmr327/llmembeddingxrvizualization/blob/main/mylib/LlmEmbeddingXrVizualization.egg-info/entry_points.txt"
    click Tests "https://github.com/rmr327/llmembeddingxrvizualization/blob/main/tests/test_get_hugging_face_embeding.py"
    click DemoData "https://github.com/rmr327/llmembeddingxrvizualization/blob/main/tests/demo_embeddings.npy"
    click CI "https://github.com/rmr327/llmembeddingxrvizualization/blob/main/.github/workflows/python-package.yml"
    click Config "https://github.com/rmr327/llmembeddingxrvizualization/blob/main/setup.py"
    click Config "https://github.com/rmr327/llmembeddingxrvizualization/blob/main/requirements.txt"
    click Config "https://github.com/rmr327/llmembeddingxrvizualization/blob/main/MANIFEST.in"
    click Notebook "https://github.com/rmr327/llmembeddingxrvizualization/blob/main/PlotVerseXrDemo.ipynb"

    %% Styles
    classDef ui fill:#ffe4b5,stroke:#333,stroke-width:1px
    classDef core fill:#add8e6,stroke:#333,stroke-width:1px
    classDef external fill:#f0e68c,stroke:#333,stroke-width:1px,stroke-dasharray: 5 5
    classDef infra fill:#98fb98,stroke:#333,stroke-width:1px,stroke-dasharray: 5 5
```
