# tattle-made/feluda architecture

> Architecture diagram of the GitHub repository tattle-made/feluda, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tattle-made/feluda
- Repository: https://github.com/tattle-made/feluda

## Overview

##### Main Components

a. Core System Components:
- API Server (src/server.py)
- Indexer Service (src/indexer.py)
- Reporter Service (src/reporter.py)
- Worker Services (src/worker/*)
- Audio Vector Worker
- Video Vector Worker
- Media Worker
- Hash Worker
- Clustering Media Worker

b. External Services:
- Elasticsearch (for vector storage)
- RabbitMQ (for job queuing)
- S3 Storage (indicated by s3_utils.py)

c. Core Framework Components:
- Operator System
- Media Factory
- Queue Management
- Store Adapters

##### Key Relationships to Highlight

a. Data Flow:
- Client → API Server → RabbitMQ → Workers → Storage
- Search Flow: Client → API Server → Elasticsearch → Client

b. Component Integration:
- Worker Services ↔ Operators
- Store Adapters ↔ Elasticsearch
- Media Factory ↔ Workers

##### Implementation Details

Use these symbols and conventions:
- Rectangles: Services and Components
- Hexagons: Operators
- Double Lines: API Boundaries

##### Additional Elements

Annotations for:
- Configuration Management
- Environment Variables
- Docker Container Boundaries
- Scaling Points
- Language Support
- Media Type Support

##### The separation of concerns between indexing, searching, and processing

This system design should reflect Feluda's core strength as a flexible, modular system for handling multi-modal content analysis with swappable components for different use cases.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Core Engine Section
    subgraph CoreEngine["Core Engine"]
        Engine["Main Engine"]:::core
        Config["Configuration Management"]:::core
        Logger["Logging System"]:::core
    end

    %% Media Handling Section
    subgraph MediaLayer["Media Handling Layer"]
        MediaModels["Media Models"]:::media
        MediaFactory["Media Factory"]:::media
        S3Utils["S3 Integration"]:::media
    end

    %% Operators Section
    subgraph OperatorsLayer["Operators Layer"]
        subgraph ImageOps["Image Processing"]
            ResNet["Image Vec Rep ResNet"]:::operator
        end
        subgraph VideoOps["Video Processing"]
            CLIP["Video Vec Rep CLIP"]:::operator
        end
        subgraph AnalysisOps["Analysis Operators"]
            Cluster["Cluster Embeddings"]:::operator
            DimReduce["Dimension Reduction"]:::operator
        end
    end

    %% Plugin Architecture
    PluginBase["Plugin Architecture Base"]:::core

    %% Example Implementations
    subgraph Examples["Example Implementations"]
        TsnePlot["TSNE Plot Videos"]:::example
        ClusterPlot["Cluster Plot Videos"]:::example
    end

    %% Relationships
    MediaLayer --> CoreEngine
    CoreEngine --> OperatorsLayer
    Config --> MediaFactory
    Config --> OperatorsLayer
    PluginBase --> OperatorsLayer
    OperatorsLayer --> Examples
    S3Utils --> CoreEngine

    %% Click Events
    click Engine "https://github.com/tattle-made/feluda/blob/main/feluda/feluda.py"
    click Config "https://github.com/tattle-made/feluda/blob/main/feluda/config.py"
    click Logger "https://github.com/tattle-made/feluda/blob/main/feluda/logger.py"
    click MediaModels "https://github.com/tattle-made/feluda/blob/main/feluda/models/media.py"
    click MediaFactory "https://github.com/tattle-made/feluda/blob/main/feluda/models/media_factory.py"
    click S3Utils "https://github.com/tattle-made/feluda/blob/main/feluda/models/s3_utils.py"
    click ResNet "https://github.com/tattle-made/feluda/blob/main/operators/image_vec_rep_resnet/image_vec_rep_resnet.py"
    click CLIP "https://github.com/tattle-made/feluda/blob/main/operators/vid_vec_rep_clip/vid_vec_rep_clip.py"
    click Cluster "https://github.com/tattle-made/feluda/blob/main/operators/cluster_embeddings/cluster_embeddings.py"
    click DimReduce "https://github.com/tattle-made/feluda/blob/main/operators/dimension_reduction/dimension_reduction.py"
    click PluginBase "https://github.com/tattle-made/feluda/blob/main/feluda/operator.py"
    click TsnePlot "https://github.com/tattle-made/feluda/blob/main/examples/01_plot_tsne_videos.ipynb"
    click ClusterPlot "https://github.com/tattle-made/feluda/blob/main/examples/02_plot_cluster_videos.ipynb"

    %% Styling
    classDef core fill:#2374ab,stroke:#000,stroke-width:2px,color:white
    classDef media fill:#57a773,stroke:#000,stroke-width:2px,color:white
    classDef operator fill:#ff8c42,stroke:#000,stroke-width:2px,color:white
    classDef example fill:#7d7d7d,stroke:#000,stroke-width:2px,color:white

    %% Legend
    subgraph Legend
        CoreLegend["Core Components"]:::core
        MediaLegend["Media Handling"]:::media
        OperatorLegend["Operators"]:::operator
        ExampleLegend["Examples"]:::example
    end
```
