# miladfa7/spatiallm-gradio architecture

> Architecture diagram of the GitHub repository miladfa7/spatiallm-gradio, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/miladfa7/spatiallm-gradio
- Repository: https://github.com/miladfa7/spatiallm-gradio

## Overview

##### Project Type and Purpose
- The README clearly indicates that this is a large language model applied to 3D geometric data, with a demo interface implemented using Gradio.

##### Analyzing the File Structure
- The spatiallm directory is the heart of the project and is subdivided into clearly defined modules:
- spatiallm/model: Contains model-specific implementations (pcd_encoder.py, spatiallm_llama.py, spatiallm_qwen.py) to handle different variants of the model.
- spatiallm/layout: Contains components like entity.py and layout.py, which likely handle the conversion of raw model outputs into formatted scene descriptions or layouts.
- spatiallm/pcd: Manages point cloud data loading and registration (with pcd_loader.py and registry.py) – essentially preparing the raw 3D data for the model.
- Scripts like install.sh and script.sh indicate environment setup or auxiliary operations, but they are less central to the architectural flow.

##### Key Components
a. Input & Data Processing:
- Data Source: Point cloud files (e.g., from the pcd folder and external sources as mentioned in the README).
- Data Loader: spatiallm/pcd/pcd_loader.py along with registry – the module that handles importing and preprocessing the point cloud data.
b. Core Processing Pipelines:
- PCD Encoder: spatiallm/model/pcd_encoder.py performs feature extraction or encoding of the point cloud data.
- SpatialLM Models: The two model variants (spatiallm_llama.py and spatiallm_qwen.py) that perform inference based on the encoded input.
- Layout Processing: components in spatiallm/layout (entity.py, layout.py) that convert raw model outputs into structured scene elements.
c. Interfaces & Outputs:
- Inference Script: inference.py orchestrates processing by calling the data loader, encoder, and model components.
- Demo Interface: gradio_demo.py provides a front-end (via Gradio) to allow interactive testing and demonstration of the model.
d. External Tools & Environment:
- Dependencies like PyTorch, TorchSparse, etc. (as indicated in the README installation instructions).
- Build/installation scripts (install.sh, script.sh) that set up the environment.

##### Architectural Patterns and Relationships
- Modular Design: The project is well segmented by functionality (Data Handling, Model Inference, Layout Processing, and UI/Visualization).
Input Data (Point Cloud) → Data Loader (PCD Loader) → Feature Encoder (PCD Encoder) → Model Inference (SpatialLM-Llama / SpatialLM-Qwen) → Layout Generation (Layout Modules) → Output (Visualization/Demo)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Input & Preprocessing Layer
    subgraph "Data Input & Preprocessing"
        RD["Raw Point Cloud Data"]:::external
        PL["PCD Loader"]:::backend
        REG["Registry"]:::backend
        RD -->|"loads"| PL
        PL -->|"registers data"| REG
    end

    %% Core Inference Pipeline Layer
    subgraph "Core Inference Pipeline"
        ENC["Point Cloud Encoder"]:::backend
        MODEL_LLAMA["SpatialLM Model: Llama Variant"]:::backend
        MODEL_QWEN["SpatialLM Model: Qwen Variant"]:::backend
        LAYOUT_ENTITY["Layout Processing: Entity"]:::backend
        LAYOUT_LAYOUT["Layout Processing: Layout"]:::backend

        REG --> ENC
        ENC -->|"encodes"| MODEL_LLAMA
        ENC -->|"encodes"| MODEL_QWEN
        MODEL_LLAMA -->|"inference output"| LAYOUT_ENTITY
        MODEL_QWEN -->|"inference output"| LAYOUT_ENTITY
        LAYOUT_ENTITY -->|"formats layout"| LAYOUT_LAYOUT
    end

    %% Interface & Outputs Layer
    subgraph "Interface & Outputs"
        INF["Inference Orchestration"]:::utility
        DEMO["Gradio Demo Interface"]:::frontend
        VIS["Visualization Utilities"]:::frontend
        LAYOUT_OUT["Layout Outputs"]:::external

        LAYOUT_LAYOUT --> INF
        INF --> DEMO
        INF --> VIS
        INF --> LAYOUT_OUT
    end

    %% Overall Data Flow
    RD --- PL

    %% Click Events
    click RD "https://github.com/miladfa7/spatiallm-gradio/tree/master/pcd"
    click PL "https://github.com/miladfa7/spatiallm-gradio/blob/master/spatiallm/pcd/pcd_loader.py"
    click REG "https://github.com/miladfa7/spatiallm-gradio/blob/master/spatiallm/pcd/registry.py"
    click ENC "https://github.com/miladfa7/spatiallm-gradio/blob/master/spatiallm/model/pcd_encoder.py"
    click MODEL_LLAMA "https://github.com/miladfa7/spatiallm-gradio/blob/master/spatiallm/model/spatiallm_llama.py"
    click MODEL_QWEN "https://github.com/miladfa7/spatiallm-gradio/blob/master/spatiallm/model/spatiallm_qwen.py"
    click LAYOUT_ENTITY "https://github.com/miladfa7/spatiallm-gradio/blob/master/spatiallm/layout/entity.py"
    click LAYOUT_LAYOUT "https://github.com/miladfa7/spatiallm-gradio/blob/master/spatiallm/layout/layout.py"
    click INF "https://github.com/miladfa7/spatiallm-gradio/blob/master/inference.py"
    click DEMO "https://github.com/miladfa7/spatiallm-gradio/blob/master/gradio_demo.py"
    click VIS "https://github.com/miladfa7/spatiallm-gradio/blob/master/visualize.py"
    click LAYOUT_OUT "https://github.com/miladfa7/spatiallm-gradio/tree/master/outputs"

    %% Styles
    classDef external fill:#f9e79f,stroke:#d35400,stroke-width:2px;
    classDef backend fill:#aed6f1,stroke:#2471a3,stroke-width:2px;
    classDef frontend fill:#abebc6,stroke:#27ae60,stroke-width:2px;
    classDef utility fill:#f5cba7,stroke:#d35400,stroke-width:2px;
```
