How smse-org/smse works
SMSE is a Python library for preparing multimodal inputs, encoding them as embeddings, and comparing embeddings for cross-modal similarity. Its sampled ImageBind example orchestrates image, text, and audio preprocessing before encoding and similarity calculations; other model and pipeline capabilities are also present in the source index. The repository describes a semantic multimodal search engine, but the available evidence does not show an index, persistent store, or search service. Some subsystem relationships below rely on source-index references rather than inspected implementations.
Main entry points: imagebind_embedding.py ImageBind workflow · image.py Image pipeline · imagebind.py ImageBind model · sentence_transformers.py Sentence model
Components 18
Entry points
- CLI example
smse/cli.py - ImageBind workflow
smse/recipes/models/imagebind_embedding.py - CLIP workflow
smse/recipes/models/clip_embedding.py - Text workflow
smse/recipes/models/text_embedding.py - Image recipe
smse/recipes/pipelines/image.py
Input pipelines
- Audio pipeline
smse/pipelines/audio.py - Image pipeline
smse/pipelines/image.py - Text pipeline
smse/pipelines/text.py - Video pipeline
smse/pipelines/video.py - Pipeline base
smse/pipelines/base.py
Embedding models
- ImageBind model
smse/models/imagebind.py - Sentence model
smse/models/sentence_transformers.py - Model base
smse/models/base.py
Shared utilities
- Device selection
smse/device.py - Image transformations
smse/utils/transformations.py - Logging
smse/logging.py
Other components
- Library user
- Embeddings and similarities
Connections 27
- Library user → CLI example: runs
smse/cli.py - CLI example → ImageBind workflow: invokes
smse/cli.py - Library user → ImageBind workflow: uses
- ImageBind workflow → Image pipeline: processes images
smse/recipes/models/imagebind_embedding.py - ImageBind workflow → Text pipeline: processes text
smse/recipes/models/imagebind_embedding.py - ImageBind workflow → Audio pipeline: processes audio
smse/recipes/models/imagebind_embedding.py - ImageBind workflow → ImageBind model: encodes inputs
smse/recipes/models/imagebind_embedding.py - ImageBind workflow → Embeddings and similarities: compares embeddings
smse/recipes/models/imagebind_embedding.py - Library user → CLIP workflow: uses
- CLIP workflow → Sentence model: uses
smse/recipes/models/clip_embedding.py - Library user → Text workflow: uses
- Text workflow → Sentence model: uses
smse/recipes/models/text_embedding.py - Library user → Image recipe: uses
- Image recipe → Image pipeline: uses
smse/recipes/pipelines/image.py - Video pipeline → Audio pipeline: uses audio
smse/pipelines/video.py - Video pipeline → Image pipeline: uses frames
smse/pipelines/video.py - Video pipeline → Pipeline base: extends
smse/pipelines/video.py - Audio pipeline → Pipeline base: extends
smse/pipelines/audio.py - Image pipeline → Pipeline base: extends
smse/pipelines/image.py - Text pipeline → Pipeline base: extends
smse/pipelines/text.py - ImageBind model → Model base: extends
smse/models/imagebind.py - Sentence model → Model base: extends
smse/models/sentence_transformers.py - ImageBind model → Device selection: selects device
smse/models/imagebind.py - ImageBind workflow → Device selection: selects device
smse/recipes/models/imagebind_embedding.py - Image pipeline → Image transformations: transforms images
smse/pipelines/image.py - ImageBind model → Embeddings and similarities: produces embeddings
- Sentence model → Embeddings and similarities: produces embeddings