# smse-org/smse architecture

> Architecture diagram of the GitHub repository smse-org/smse, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/smse-org/smse
- Repository: https://github.com/smse-org/smse

## Overview

SMSE is a Python library for preparing multimodal inputs, encoding them as embeddings, and comparing embeddings for cross-modal similarity. Its sampled ImageBind example orchestrates image, text, and audio preprocessing before encoding and similarity calculations; other model and pipeline capabilities are also present in the source index. The repository describes a semantic multimodal search engine, but the available evidence does not show an index, persistent store, or search service. Some subsystem relationships below rely on source-index references rather than inspected implementations.

Main entry points: [`smse/recipes/models/imagebind_embedding.py`](https://github.com/smse-org/smse/blob/main/smse/recipes/models/imagebind_embedding.py) (ImageBind workflow), [`smse/pipelines/image.py`](https://github.com/smse-org/smse/blob/main/smse/pipelines/image.py) (Image pipeline), [`smse/models/imagebind.py`](https://github.com/smse-org/smse/blob/main/smse/models/imagebind.py) (ImageBind model), [`smse/models/sentence_transformers.py`](https://github.com/smse-org/smse/blob/main/smse/models/sentence_transformers.py) (Sentence model)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Entry points"]
  node_cli["CLI example<br/>[cli.py]"]
  node_imagebind_recipe["ImageBind workflow"]
  node_clip_recipe["CLIP workflow<br/>[clip_embedding.py]"]
  node_text_recipe["Text workflow<br/>[text_embedding.py]"]
  node_image_recipe["Image recipe<br/>[image.py]"]
end

subgraph group_prep["Input pipelines"]
  node_audio_pipe["Audio pipeline<br/>[audio.py]"]
  node_image_pipe["Image pipeline<br/>[image.py]"]
  node_text_pipe["Text pipeline<br/>[text.py]"]
  node_video_pipe["Video pipeline<br/>[video.py]"]
  node_pipeline_base["Pipeline base<br/>[base.py]"]
end

subgraph group_models["Embedding models"]
  node_imagebind["ImageBind model<br/>[imagebind.py]"]
  node_sentence_model["Sentence model"]
  node_model_base["Model base<br/>[base.py]"]
end

subgraph group_shared["Shared utilities"]
  node_device["Device selection<br/>[device.py]"]
  node_transformations["Image transformations<br/>[transformations.py]"]
  node_logging["Logging<br/>[logging.py]"]
end

node_user(("Library user"))
node_embeddings["Embeddings and similarities"]

node_user -->|"runs"| node_cli
node_cli -->|"invokes"| node_imagebind_recipe
node_user -.->|"uses"| node_imagebind_recipe
node_imagebind_recipe -->|"processes images"| node_image_pipe
node_imagebind_recipe -->|"processes text"| node_text_pipe
node_imagebind_recipe -->|"processes audio"| node_audio_pipe
node_imagebind_recipe -->|"encodes inputs"| node_imagebind
node_imagebind_recipe -->|"compares embeddings"| node_embeddings
node_user -.->|"uses"| node_clip_recipe
node_clip_recipe -->|"uses"| node_sentence_model
node_user -.->|"uses"| node_text_recipe
node_text_recipe -->|"uses"| node_sentence_model
node_user -.->|"uses"| node_image_recipe
node_image_recipe -->|"uses"| node_image_pipe
node_video_pipe -->|"uses audio"| node_audio_pipe
node_video_pipe -->|"uses frames"| node_image_pipe
node_video_pipe -->|"extends"| node_pipeline_base
node_audio_pipe -->|"extends"| node_pipeline_base
node_image_pipe -->|"extends"| node_pipeline_base
node_text_pipe -->|"extends"| node_pipeline_base
node_imagebind -->|"extends"| node_model_base
node_sentence_model -->|"extends"| node_model_base
node_imagebind -->|"selects device"| node_device
node_imagebind_recipe -->|"selects device"| node_device
node_image_pipe -->|"transforms images"| node_transformations
node_imagebind -.->|"produces embeddings"| node_embeddings
node_sentence_model -.->|"produces embeddings"| node_embeddings

click node_cli "https://github.com/smse-org/smse/blob/main/smse/cli.py"
click node_imagebind_recipe "https://github.com/smse-org/smse/blob/main/smse/recipes/models/imagebind_embedding.py"
click node_clip_recipe "https://github.com/smse-org/smse/blob/main/smse/recipes/models/clip_embedding.py"
click node_text_recipe "https://github.com/smse-org/smse/blob/main/smse/recipes/models/text_embedding.py"
click node_image_recipe "https://github.com/smse-org/smse/blob/main/smse/recipes/pipelines/image.py"
click node_audio_pipe "https://github.com/smse-org/smse/blob/main/smse/pipelines/audio.py"
click node_image_pipe "https://github.com/smse-org/smse/blob/main/smse/pipelines/image.py"
click node_text_pipe "https://github.com/smse-org/smse/blob/main/smse/pipelines/text.py"
click node_video_pipe "https://github.com/smse-org/smse/blob/main/smse/pipelines/video.py"
click node_pipeline_base "https://github.com/smse-org/smse/blob/main/smse/pipelines/base.py"
click node_imagebind "https://github.com/smse-org/smse/blob/main/smse/models/imagebind.py"
click node_sentence_model "https://github.com/smse-org/smse/blob/main/smse/models/sentence_transformers.py"
click node_model_base "https://github.com/smse-org/smse/blob/main/smse/models/base.py"
click node_device "https://github.com/smse-org/smse/blob/main/smse/device.py"
click node_transformations "https://github.com/smse-org/smse/blob/main/smse/utils/transformations.py"
click node_logging "https://github.com/smse-org/smse/blob/main/smse/logging.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_imagebind_recipe,node_clip_recipe,node_text_recipe,node_image_recipe,node_user toneBlue
class node_audio_pipe,node_image_pipe,node_text_pipe,node_video_pipe,node_pipeline_base toneAmber
class node_imagebind,node_sentence_model,node_model_base toneMint
class node_device,node_transformations,node_logging toneRose
class node_embeddings toneIndigo
```

## Components

### Entry points

- [CLI example](https://github.com/smse-org/smse/blob/main/smse/cli.py): `smse/cli.py`
- [ImageBind workflow](https://github.com/smse-org/smse/blob/main/smse/recipes/models/imagebind_embedding.py): `smse/recipes/models/imagebind_embedding.py`
- [CLIP workflow](https://github.com/smse-org/smse/blob/main/smse/recipes/models/clip_embedding.py): `smse/recipes/models/clip_embedding.py`
- [Text workflow](https://github.com/smse-org/smse/blob/main/smse/recipes/models/text_embedding.py): `smse/recipes/models/text_embedding.py`
- [Image recipe](https://github.com/smse-org/smse/blob/main/smse/recipes/pipelines/image.py): `smse/recipes/pipelines/image.py`

### Input pipelines

- [Audio pipeline](https://github.com/smse-org/smse/blob/main/smse/pipelines/audio.py): `smse/pipelines/audio.py`
- [Image pipeline](https://github.com/smse-org/smse/blob/main/smse/pipelines/image.py): `smse/pipelines/image.py`
- [Text pipeline](https://github.com/smse-org/smse/blob/main/smse/pipelines/text.py): `smse/pipelines/text.py`
- [Video pipeline](https://github.com/smse-org/smse/blob/main/smse/pipelines/video.py): `smse/pipelines/video.py`
- [Pipeline base](https://github.com/smse-org/smse/blob/main/smse/pipelines/base.py): `smse/pipelines/base.py`

### Embedding models

- [ImageBind model](https://github.com/smse-org/smse/blob/main/smse/models/imagebind.py): `smse/models/imagebind.py`
- [Sentence model](https://github.com/smse-org/smse/blob/main/smse/models/sentence_transformers.py): `smse/models/sentence_transformers.py`
- [Model base](https://github.com/smse-org/smse/blob/main/smse/models/base.py): `smse/models/base.py`

### Shared utilities

- [Device selection](https://github.com/smse-org/smse/blob/main/smse/device.py): `smse/device.py`
- [Image transformations](https://github.com/smse-org/smse/blob/main/smse/utils/transformations.py): `smse/utils/transformations.py`
- [Logging](https://github.com/smse-org/smse/blob/main/smse/logging.py): `smse/logging.py`

### Other components

- **Library user**
- **Embeddings and similarities**

## Connections

- Library user → CLI example: runs (evidence: `smse/cli.py`)
- CLI example → ImageBind workflow: invokes (evidence: `smse/cli.py`)
- Library user → ImageBind workflow: uses
- ImageBind workflow → Image pipeline: processes images (evidence: `smse/recipes/models/imagebind_embedding.py`)
- ImageBind workflow → Text pipeline: processes text (evidence: `smse/recipes/models/imagebind_embedding.py`)
- ImageBind workflow → Audio pipeline: processes audio (evidence: `smse/recipes/models/imagebind_embedding.py`)
- ImageBind workflow → ImageBind model: encodes inputs (evidence: `smse/recipes/models/imagebind_embedding.py`)
- ImageBind workflow → Embeddings and similarities: compares embeddings (evidence: `smse/recipes/models/imagebind_embedding.py`)
- Library user → CLIP workflow: uses
- CLIP workflow → Sentence model: uses (evidence: `smse/recipes/models/clip_embedding.py`)
- Library user → Text workflow: uses
- Text workflow → Sentence model: uses (evidence: `smse/recipes/models/text_embedding.py`)
- Library user → Image recipe: uses
- Image recipe → Image pipeline: uses (evidence: `smse/recipes/pipelines/image.py`)
- Video pipeline → Audio pipeline: uses audio (evidence: `smse/pipelines/video.py`)
- Video pipeline → Image pipeline: uses frames (evidence: `smse/pipelines/video.py`)
- Video pipeline → Pipeline base: extends (evidence: `smse/pipelines/video.py`)
- Audio pipeline → Pipeline base: extends (evidence: `smse/pipelines/audio.py`)
- Image pipeline → Pipeline base: extends (evidence: `smse/pipelines/image.py`)
- Text pipeline → Pipeline base: extends (evidence: `smse/pipelines/text.py`)
- ImageBind model → Model base: extends (evidence: `smse/models/imagebind.py`)
- Sentence model → Model base: extends (evidence: `smse/models/sentence_transformers.py`)
- ImageBind model → Device selection: selects device (evidence: `smse/models/imagebind.py`)
- ImageBind workflow → Device selection: selects device (evidence: `smse/recipes/models/imagebind_embedding.py`)
- Image pipeline → Image transformations: transforms images (evidence: `smse/pipelines/image.py`)
- ImageBind model → Embeddings and similarities: produces embeddings
- Sentence model → Embeddings and similarities: produces embeddings
