# twabi/framemq architecture

> Architecture diagram of the GitHub repository twabi/framemq, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/twabi/framemq
- Repository: https://github.com/twabi/framemq

## Overview

##### Project type and purpose
- FrameMQ is a Python library (not a standalone service) that provides real-time video frame transmission over pub/sub platforms (Apache Kafka or MQTT), with optional PSO-based parameter tuning.

##### File structure
- Top-level package: `frameMQ/`
- `agent/reader.py` and `agent/writer.py` house the Reader and Writer agents (entry points).
- `components/reader` has frame retrieval and display logic.
- `components/writer` has frame capture and transfer logic.
- `components/common/video_saver.py` for optional disk dumps.
- `optimizers/PSO` contains the PSO algorithm.
- `models/models.py` defines parameter objects.
- `utils/` holds enums and helper functions.
- External config/build: `pyproject.toml`, `requirements.txt`, `setup.py`.
- Tests mirror the component structure under `tests/`.

##### README for additional insights
- Supported transports: Kafka, MQTT.
- Two agents: Writer (produce) and Reader (consume).
- PSO optimizer can tune partitions, chunk size, quality.
- Video encoding: TurboJPEG; display: OpenCV+GStreamer.
- Clients invoke `Writer.start()` / `Reader.start()`.

##### Main components & interactions
a. Components
- Client Application (user code)
- Writer Agent (frameMQ.agent.writer.Writer)
- Writer Components
– FrameCapture (video capture + encoding)
– FrameTransfer (chunking + pub/sub publish)
– NotifConsumer (PSO feedback listener)
- PSO Optimizer (frameMQ.optimizers.PSO)
- Pub/Sub Broker (Kafka or MQTT) — external service
- Reader Agent (frameMQ.agent.reader.Reader)
- Reader Components
– FrameRetrieve (subscribe + reassemble chunks)
– FrameShow (decode + display)
- VideoSaver (optional) — common component to save to disk
- Models & Utils (parameter definitions, enums, helpers) — support classes

b. Relationships / Data flows
1. Client → Writer Agent: instantiates `Writer(params)`
2. Writer Agent → FrameCapture: pulls frames from camera
3. FrameCapture → FrameTransfer: encodes & chunks frames
4. FrameTransfer → Pub/Sub Broker: publishes on `video-stream` topic
5. (If PSO enabled) FrameTransfer → Broker Notification Topic
and NotifConsumer → PSO Optimizer → updates Writer parameters
6. Pub/Sub Broker → FrameRetrieve: Reader subscribes to `video-stream`
7. FrameRetrieve → FrameShow: reassemble + decode
8. FrameShow → Client: delivers frames (display, analysis)
9. VideoSaver can tap into any stage for persistence.

c. Architectural patterns / design principles
- Agent pattern: Reader/Writer agents encapsulate lifecycle (`start()`, `stop()`)
- Strategy/Factory: support multiple protocols (Kafka vs MQTT) via parameterized classes
- Pub/Sub messaging: decoupled, asynchronous frame streaming
- Optional plugin architecture: new transports, encoders, optimizers drop into predefined interfaces
- Thread safety: internal use of thread-safe queues.

d. Key technologies / frameworks
- Pub/Sub: kafka-python, paho-mqtt
- Video: PyTurboJPEG, OpenCV (GStreamer)
- Optimization: custom PSO implementation in `optimizers/PSO`
- Python packaging: setuptools, pyproject.toml

##### Detailed capture of essential elements
- Notification topic and PSO feedback loop.
- Parameter flow: how Models (`WriterParams`, `ReaderParams`) configure each agent.
- Threading where necessary (e.g., concurrent capture+publish).

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External
    Client["Client Application"]:::external
    CAM["Camera/Video Source"]:::external
    BK["Pub/Sub Broker\n(Kafka/MQTT)"]:::external
    DISP["Display/GUI"]:::external

    %% Writer Agent
    subgraph "Writer Agent"
        direction TB
        WA["Writer Agent"]:::core
        FC["FrameCapture"]:::core
        FT["FrameTransfer"]:::core
        NC["NotifConsumer"]:::core
    end

    %% Reader Agent
    subgraph "Reader Agent"
        direction TB
        RA["Reader Agent"]:::core
        FRV["FrameRetrieve"]:::core
        FS["FrameShow"]:::core
    end

    %% PSO Optimizer
    subgraph "PSO Optimizer"
        direction TB
        PO["PSO Algorithm"]:::optional
        PD["Particle Definitions"]:::optional
    end

    %% Support Modules
    subgraph "Support Modules"
        direction TB
        PM["Parameter Models"]:::core
        EN["Enums"]:::core
        HF["Helper Functions"]:::core
    end

    %% Optional VideoSaver
    VS["VideoSaver"]:::optional

    %% Flows
    Client -->|"new Writer(params)"| WA
    Client -->|"new Reader(params)"| RA
    Client -->|uses| PM
    PM -->|configures| WA
    PM -->|configures| RA

    CAM -->|"capture"| FC
    FC -->|"encode & chunk"| FT
    FT -->|"publish(frame)"| BK
    FT -->|"publish(params)"| BK
    BK -->|"notify(params)"| NC
    NC -->|"feedback"| PO
    PO -.->|"update params"| FT

    BK -->|"subscribe(frame)"| FRV
    FRV -->|"reassemble & decode"| FS
    FS -->|"display(frame)"| DISP

    %% VideoSaver taps
    FC -.-> VS
    FT -.-> VS
    FRV -.-> VS

    %% Utils usage
    WA -->|uses| EN
    RA -->|uses| EN
    WA -->|uses| HF
    RA -->|uses| HF

    %% Click Events
    click WA "https://github.com/twabi/framemq/blob/main/frameMQ/agent/writer.py"
    click RA "https://github.com/twabi/framemq/blob/main/frameMQ/agent/reader.py"
    click FC "https://github.com/twabi/framemq/blob/main/frameMQ/components/writer/frame_capture.py"
    click FT "https://github.com/twabi/framemq/blob/main/frameMQ/components/writer/frame_transfer.py"
    click NC "https://github.com/twabi/framemq/blob/main/frameMQ/components/writer/notif_consumer.py"
    click FRV "https://github.com/twabi/framemq/blob/main/frameMQ/components/reader/frame_retrieve.py"
    click FS "https://github.com/twabi/framemq/blob/main/frameMQ/components/reader/frame_show.py"
    click VS "https://github.com/twabi/framemq/blob/main/frameMQ/components/common/video_saver.py"
    click PO "https://github.com/twabi/framemq/blob/main/frameMQ/optimizers/PSO/pso.py"
    click PD "https://github.com/twabi/framemq/blob/main/frameMQ/optimizers/PSO/particle.py"
    click PM "https://github.com/twabi/framemq/blob/main/frameMQ/models/models.py"
    click EN "https://github.com/twabi/framemq/blob/main/frameMQ/utils/enums.py"
    click HF "https://github.com/twabi/framemq/blob/main/frameMQ/utils/helper.py"

    %% Styles
    classDef core fill:#BBDEFB,stroke:#1976D2,color:#0D47A1
    classDef external fill:#C8E6C9,stroke:#388E3C,color:#1B5E20
    classDef optional fill:#FFE0B2,stroke:#F57C00,color:#E65100
```
