# augustine-umeh/mefid architecture

> Architecture diagram of the GitHub repository augustine-umeh/mefid, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/augustine-umeh/mefid
- Repository: https://github.com/augustine-umeh/mefid

## Overview

Mefid is a personal multimodal video-search product: users upload media, the system derives visual and speech signals, indexes embeddings, and returns timestamped searchable moments. The graph shows the documented text-search calls and the service boundaries suggested by the tree; upload-route orchestration and several service internals were not sampled, so those links are not asserted. Image, video, and combined-modality search are explicitly placeholders in the sampled search route.

Main entry points: [`services/api/src/routes/upload_route.py`](https://github.com/augustine-umeh/mefid/blob/main/services/api/src/routes/upload_route.py) (Upload route), [`services/api/src/routes/search_route.py`](https://github.com/augustine-umeh/mefid/blob/main/services/api/src/routes/search_route.py) (Search route), [`services/api/src/service_clients/embedder_client.py`](https://github.com/augustine-umeh/mefid/blob/main/services/api/src/service_clients/embedder_client.py) (Embedder client), [`services/api/src/service_clients/indexer_client.py`](https://github.com/augustine-umeh/mefid/blob/main/services/api/src/service_clients/indexer_client.py) (Indexer client)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["API and workflows"]
  node_upload["Upload route<br/>[upload_route.py]"]
  node_search["Search route<br/>[search_route.py]"]
  node_media_client["Media client"]
  node_caption_client["Caption client<br/>[caption_client.py]"]
  node_transcribe_client["Transcribe client"]
  node_embed_client["Embedder client<br/>[embedder_client.py]"]
  node_index_client["Indexer client<br/>[indexer_client.py]"]
  node_fusion["Moment fusion<br/>[pipeline.py]"]
end

subgraph group_processing["Media understanding"]
  node_media_processor["Frame extraction<br/>[extract.py]"]
  node_sampling["Hybrid sampling<br/>[hybrid_sampler.py]"]
  node_captioning["Scene captioning<br/>[caption_service.py]"]
  node_caption_pipeline["Caption generation"]
  node_speech["Speech transcription<br/>[whisper_service.py]"]
end

subgraph group_indexing["Embeddings and index"]
  node_embedder["CLIP embeddings<br/>[clip_service.py]"]
  node_index_add["Vector ingestion<br/>[add.py]"]
  node_index_search["Vector retrieval<br/>[search.py]"]
  node_faiss[("FAISS indexes<br/>[faiss_store.py]")]
end

subgraph group_storage["Metadata and vectors"]
  node_supabase[("Media metadata<br/>[supabaseDB.py]")]
  node_minio[("Media object store<br/>[minioDB.py]")]
end

node_user(("Media user"))

node_user -->|"uploads media"| node_upload
node_user -->|"queries moments"| node_search
node_search -->|"embeds query"| node_embed_client
node_search -->|"requests candidates"| node_index_client
node_search -->|"hydrates metadata"| node_supabase
node_search -->|"ranks moments"| node_fusion
node_embed_client -->|"requests vectors"| node_embedder
node_index_client -->|"searches vectors"| node_index_search
node_index_search -->|"reads indexes"| node_faiss
node_index_add -->|"writes vectors"| node_faiss
node_index_add -->|"persists mappings"| node_supabase
node_upload -->|"requests extraction"| node_media_client
node_media_client -->|"calls extraction"| node_media_processor
node_media_processor -->|"samples frames"| node_sampling
node_upload -->|"requests captions"| node_caption_client
node_caption_client -->|"calls captioning"| node_captioning
node_captioning -->|"generates captions"| node_caption_pipeline
node_upload -->|"requests transcription"| node_transcribe_client
node_transcribe_client -->|"calls transcription"| node_speech
node_upload -->|"requests embeddings"| node_embed_client
node_upload -->|"submits vectors"| node_index_client
node_upload -->|"stores media data"| node_supabase
node_upload -->|"stores media files"| node_minio

click node_upload "https://github.com/augustine-umeh/mefid/blob/main/services/api/src/routes/upload_route.py"
click node_search "https://github.com/augustine-umeh/mefid/blob/main/services/api/src/routes/search_route.py"
click node_media_client "https://github.com/augustine-umeh/mefid/blob/main/services/api/src/service_clients/media_processor_client.py"
click node_caption_client "https://github.com/augustine-umeh/mefid/blob/main/services/api/src/service_clients/caption_client.py"
click node_transcribe_client "https://github.com/augustine-umeh/mefid/blob/main/services/api/src/service_clients/transcribe_client.py"
click node_embed_client "https://github.com/augustine-umeh/mefid/blob/main/services/api/src/service_clients/embedder_client.py"
click node_index_client "https://github.com/augustine-umeh/mefid/blob/main/services/api/src/service_clients/indexer_client.py"
click node_media_processor "https://github.com/augustine-umeh/mefid/blob/main/services/media_processor/src/routes/extract.py"
click node_sampling "https://github.com/augustine-umeh/mefid/blob/main/services/media_processor/src/sampling/hybrid_sampler.py"
click node_captioning "https://github.com/augustine-umeh/mefid/blob/main/services/caption/src/caption_service.py"
click node_caption_pipeline "https://github.com/augustine-umeh/mefid/blob/main/services/caption/src/pipeline/caption_generation.py"
click node_speech "https://github.com/augustine-umeh/mefid/blob/main/services/transcribe/src/whisper_service.py"
click node_embedder "https://github.com/augustine-umeh/mefid/blob/main/services/embedder/src/clip_service.py"
click node_index_add "https://github.com/augustine-umeh/mefid/blob/main/services/indexer/src/routes/add.py"
click node_index_search "https://github.com/augustine-umeh/mefid/blob/main/services/indexer/src/routes/search.py"
click node_faiss "https://github.com/augustine-umeh/mefid/blob/main/services/exports/src/exports/faiss_store.py"
click node_fusion "https://github.com/augustine-umeh/mefid/blob/main/services/exports/src/exports/fusion/pipeline.py"
click node_supabase "https://github.com/augustine-umeh/mefid/blob/main/services/exports/src/exports/db_clients/supabaseDB.py"
click node_minio "https://github.com/augustine-umeh/mefid/blob/main/services/exports/src/exports/db_clients/minioDB.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_upload,node_search,node_media_client,node_caption_client,node_transcribe_client,node_embed_client,node_index_client,node_fusion,node_user toneBlue
class node_media_processor,node_sampling,node_captioning,node_caption_pipeline,node_speech toneAmber
class node_embedder,node_index_add,node_index_search,node_faiss toneMint
class node_supabase,node_minio toneRose
```

## Components

### API and workflows

- [Upload route](https://github.com/augustine-umeh/mefid/blob/main/services/api/src/routes/upload_route.py): `services/api/src/routes/upload_route.py`
- [Search route](https://github.com/augustine-umeh/mefid/blob/main/services/api/src/routes/search_route.py): `services/api/src/routes/search_route.py`
- [Media client](https://github.com/augustine-umeh/mefid/blob/main/services/api/src/service_clients/media_processor_client.py): `services/api/src/service_clients/media_processor_client.py`
- [Caption client](https://github.com/augustine-umeh/mefid/blob/main/services/api/src/service_clients/caption_client.py): `services/api/src/service_clients/caption_client.py`
- [Transcribe client](https://github.com/augustine-umeh/mefid/blob/main/services/api/src/service_clients/transcribe_client.py): `services/api/src/service_clients/transcribe_client.py`
- [Embedder client](https://github.com/augustine-umeh/mefid/blob/main/services/api/src/service_clients/embedder_client.py): `services/api/src/service_clients/embedder_client.py`
- [Indexer client](https://github.com/augustine-umeh/mefid/blob/main/services/api/src/service_clients/indexer_client.py): `services/api/src/service_clients/indexer_client.py`
- [Moment fusion](https://github.com/augustine-umeh/mefid/blob/main/services/exports/src/exports/fusion/pipeline.py): `services/exports/src/exports/fusion/pipeline.py`

### Media understanding

- [Frame extraction](https://github.com/augustine-umeh/mefid/blob/main/services/media_processor/src/routes/extract.py): `services/media_processor/src/routes/extract.py`
- [Hybrid sampling](https://github.com/augustine-umeh/mefid/blob/main/services/media_processor/src/sampling/hybrid_sampler.py): `services/media_processor/src/sampling/hybrid_sampler.py`
- [Scene captioning](https://github.com/augustine-umeh/mefid/blob/main/services/caption/src/caption_service.py): `services/caption/src/caption_service.py`
- [Caption generation](https://github.com/augustine-umeh/mefid/blob/main/services/caption/src/pipeline/caption_generation.py): `services/caption/src/pipeline/caption_generation.py`
- [Speech transcription](https://github.com/augustine-umeh/mefid/blob/main/services/transcribe/src/whisper_service.py): `services/transcribe/src/whisper_service.py`

### Embeddings and index

- [CLIP embeddings](https://github.com/augustine-umeh/mefid/blob/main/services/embedder/src/clip_service.py): `services/embedder/src/clip_service.py`
- [Vector ingestion](https://github.com/augustine-umeh/mefid/blob/main/services/indexer/src/routes/add.py): `services/indexer/src/routes/add.py`
- [Vector retrieval](https://github.com/augustine-umeh/mefid/blob/main/services/indexer/src/routes/search.py): `services/indexer/src/routes/search.py`
- [FAISS indexes](https://github.com/augustine-umeh/mefid/blob/main/services/exports/src/exports/faiss_store.py): `services/exports/src/exports/faiss_store.py`

### Metadata and vectors

- [Media metadata](https://github.com/augustine-umeh/mefid/blob/main/services/exports/src/exports/db_clients/supabaseDB.py): `services/exports/src/exports/db_clients/supabaseDB.py`
- [Media object store](https://github.com/augustine-umeh/mefid/blob/main/services/exports/src/exports/db_clients/minioDB.py): `services/exports/src/exports/db_clients/minioDB.py`

### Other components

- **Media user**

## Connections

- Media user → Upload route: uploads media
- Media user → Search route: queries moments
- Search route → Embedder client: embeds query
- Search route → Indexer client: requests candidates
- Search route → Media metadata: hydrates metadata
- Search route → Moment fusion: ranks moments
- Embedder client → CLIP embeddings: requests vectors
- Indexer client → Vector retrieval: searches vectors
- Vector retrieval → FAISS indexes: reads indexes
- Vector ingestion → FAISS indexes: writes vectors
- Vector ingestion → Media metadata: persists mappings
- Upload route → Media client: requests extraction
- Media client → Frame extraction: calls extraction
- Frame extraction → Hybrid sampling: samples frames
- Upload route → Caption client: requests captions
- Caption client → Scene captioning: calls captioning
- Scene captioning → Caption generation: generates captions
- Upload route → Transcribe client: requests transcription
- Transcribe client → Speech transcription: calls transcription
- Upload route → Embedder client: requests embeddings
- Upload route → Indexer client: submits vectors
- Upload route → Media metadata: stores media data
- Upload route → Media object store: stores media files