# in03/proxima architecture

> Architecture diagram of the GitHub repository in03/proxima, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/in03/proxima
- Repository: https://github.com/in03/proxima

## Overview

##### Main Components

a) Client-Side Components:
- DaVinci Resolve Integration Layer
- Queuer Interface
- CLI Application
- Configuration Manager

b) Distributed System Components:
- Redis Message Broker
- Worker Nodes
- Task Distribution System (Celery)

c) Processing Components:
- FFmpeg Processing Engine
- Proxy Generation Pipeline
- Media Pool Linking System

##### Architecture Pattern
The system follows a distributed task queue architecture with these key characteristics:
- Message broker-based communication
- Producer-consumer pattern
- Distributed worker system
- Configuration-driven design

- Client Side  | --> | Message Broker | --> |  Worker Nodes  |

##### Detailed Component Breakdown

Client Side:

- Client Side             |

- CLI      |  | Settings  |   |
- Interface  |  | Manager   |   |

- Resolve   |  |   Queue   |   |
- Integration |  |  Manager  |   |

Message Broker:

- Message Broker            |

- Redis     |  |  Celery   |   |
- Server    |  |  Backend  |   |

Worker Nodes:

- Worker Nodes              |

- Celery    |  |  FFmpeg   |   |
- Workers   |  | Process   |   |

- Media     |  |  Proxy   |   |
- Linking   |  | Generator |   |

##### Data Flow Indicators

- Task submission flow from CLI to Redis
- Task distribution from Redis to Workers
- Proxy file generation flow
- Media linking process flow

##### Additional Elements

Include:
- Configuration file locations
- Network protocols (Redis://)
- File system interactions
- Monitoring components (Flower)

##### Important Notes

- Distributed nature of the system
- Ability to scale workers
- Shared filesystem requirements
- CPU-intensive operations
- Configuration dependencies

- Distributed encoding capability
- Automatic proxy linking
- Multi-machine support
- CPU-focused processing
- Configuration-driven architecture

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Client
        CLI["CLI Application"]:::client
        Settings["Configuration Manager"]:::client
        ResolveInt["DaVinci Resolve Integration"]:::client
        QueueMgr["Queue Manager"]:::client
        Core["Core Application Logic"]:::client
        JobMgr["Job Management"]:::client
        BatchProc["Batch Processing"]:::client

        CLI --> Settings
        CLI --> QueueMgr
        CLI --> ResolveInt
        QueueMgr --> Core
        Core --> JobMgr
        Core --> BatchProc
        ResolveInt --> Core
    end

    subgraph MessageBroker
        Redis["Redis Server"]:::broker
        CeleryBackend["Celery Backend"]:::broker
        
        Redis <--> CeleryBackend
    end

    subgraph WorkerNodes
        CeleryWorkers["Worker Nodes"]:::worker
        TaskDef["Task Definitions"]:::worker
        FFmpegProc["FFmpeg Processing Engine"]:::worker
        FFmpegUtils["FFmpeg Utilities"]:::worker
        MediaLink["Media Pool Linking"]:::worker
        
        CeleryWorkers --> TaskDef
        TaskDef --> FFmpegProc
        FFmpegProc --> FFmpegUtils
        TaskDef --> MediaLink
    end

    QueueMgr --> Redis
    Redis --> CeleryWorkers
    CeleryWorkers --> MediaLink
    MediaLink --> ResolveInt

    classDef client fill:#a8d1ff,stroke:#333,stroke-width:2px
    classDef broker fill:#90EE90,stroke:#333,stroke-width:2px
    classDef worker fill:#FFB347,stroke:#333,stroke-width:2px

    click CLI "https://github.com/in03/proxima/blob/main/proxima/cli/main.py"
    click Settings "https://github.com/in03/proxima/blob/main/proxima/settings/manager.py"
    click ResolveInt "https://github.com/in03/proxima/blob/main/proxima/app/resolve.py"
    click QueueMgr "https://github.com/in03/proxima/blob/main/proxima/cli/queue.py"
    click Core "https://github.com/in03/proxima/blob/main/proxima/app/core.py"
    click JobMgr "https://github.com/in03/proxima/blob/main/proxima/types/job.py"
    click BatchProc "https://github.com/in03/proxima/blob/main/proxima/types/batch.py"
    click CeleryWorkers "https://github.com/in03/proxima/blob/main/proxima/celery/launch_workers.py"
    click TaskDef "https://github.com/in03/proxima/blob/main/proxima/celery/tasks.py"
    click FFmpegProc "https://github.com/in03/proxima/blob/main/proxima/celery/ffmpeg/ffmpeg_process.py"
    click FFmpegUtils "https://github.com/in03/proxima/blob/main/proxima/celery/ffmpeg/utils.py"
    click MediaLink "https://github.com/in03/proxima/blob/main/proxima/app/link.py"
    click CeleryBackend "https://github.com/in03/proxima/blob/main/proxima/celery/celery.py"
```
