# seogenis/world-model-portal-backend architecture

> Architecture diagram of the GitHub repository seogenis/world-model-portal-backend, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/seogenis/world-model-portal-backend
- Repository: https://github.com/seogenis/world-model-portal-backend

## Overview

- FastAPI Server: At the heart of the system, hosting REST API endpoints (see app/main.py, app/api/routes.py) for video generation, parameter extraction, prompt management, and static file access.
- API Services:
- Video Service: Encapsulates interactions with NVIDIA’s Cosmos API (with rate limiting and API key rotation as seen in app/services/video_service.py).
- Batch Inference Service: Handles multi-GPU batch processing jobs (app/services/batch_inference_service.py).
- Parameter Extractor and Prompt Manager: Manage prompt enhancement and parameter extraction (app/services/parameter_extractor.py and app/services/prompt_manager.py).
- Session Manager: Manages persistent sessions backed by disk storage (app/services/session_manager.py).
- Asynchronous Request Handling & Worker Pool:
- Task Queues: Represent how incoming API requests are placed into queues.
- Worker Processes: Showing background workers that pick jobs from the queues to process them asynchronously (illustrated in the worker pool implementation).
- Filesystem-Based State Recovery:
- A mechanism using the directory structure (static/videos, sessions/) to validate and recover job status.
- External Integrations:
- NVIDIA Cosmos API: Mark the API integration points along with API key rotation logic.
- OpenAI API: Used for prompt enhancement (found in app/utils/openai_utils.py).
- Docker and external scripts (e.g., start_ngrok.sh) that enable deployment and public access.

##### Relationships/Interactions
- Client Requests → API Endpoints:
- Clients send video generation or batch inference requests via RESTful endpoints.
- Asynchronous Task Handling:
- Requests get enqueued and picked up by background worker processes.
- Workers interact with video_service using the semaphore for rate limiting, API key rotation, and retry logic.
- State Recovery & Filesystem Checks:
- After processing jobs, the workers store result files (videos) in the static directory. The status endpoints consult this directory along with in-memory job states.
- Batch Processing:
- How batch requests are decomposed into separate jobs that are distributed across a set of available GPUs (note the round-robin assignment).
- Data Flow:

##### Important Architectural Patterns
- Asynchronous Non-blocking Processing: Emphasize the separation of API request handling (FastAPI) from long-running background tasks.
- Microservices-Like Modularity: Although all components reside in one project, they are organized into clear domains (API, core configuration, services, utilities) allowing easier management and potential extension.
- Resilience and Recovery: The design incorporates robust error handling (API key rotation, exponential backoff) and filesystem-based state recovery for high availability.
- Parallel Processing: The multi-GPU batch inference mechanism is key for scalability in processing heavy workloads concurrently.

##### Relevant Technologies and Tools
- Python 3.10+ and FastAPI for asynchronous web service handling.
- Uvicorn as the ASGI server.
- Celery-like worker patterns (though implemented using asyncio).
- NVIDIA Cosmos API for video generation and OpenAI API for prompt enhancement.
- Docker, docker-compose (see docker-compose.yml) for deployment, plus ngrok for public exposure.
- Use configuration files like .env and requirements.txt to indicate dependencies.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Client Layer"
        clientReq("Client Requests"):::client
    end

    subgraph "API Layer"
        apiServer("FastAPI Server (API Endpoints)"):::api
    end

    subgraph "Service Layer"
        videoServ("Video Service"):::service
        batchServ("Batch Inference Service"):::service
        paramExtr("Parameter Extractor"):::service
        promptMan("Prompt Manager"):::service
        sessMan("Session Manager"):::service
    end

    subgraph "Asynchronous Processing"
        taskQueue("Task Queue"):::worker
        worker("Background Worker (Worker Pool)"):::worker
    end

    subgraph "External Integrations"
        nvidia("NVIDIA Cosmos API"):::external
        openai("OpenAI API"):::external
    end

    subgraph "State Recovery"
        resultFiles("Result Files"):::store
        sessionData("Session Data"):::store
    end

    clientReq --> apiServer
    apiServer -->|"enqueue_job"| taskQueue
    taskQueue --> worker

    worker -->|"process_video_job"| videoServ
    worker -->|"process_batch_job"| batchServ
    worker -->|"extract_parameters"| paramExtr
    worker -->|"manage_prompts"| promptMan
    worker -->|"handle_sessions"| sessMan

    videoServ -->|"calls_Cosmos"| nvidia
    promptMan -->|"calls_promptEnhancement"| openai

    worker -->|"store_results"| resultFiles
    worker -->|"update_status"| sessionData

    apiServer -->|"polls_status"| resultFiles
    apiServer -->|"polls_status"| sessionData

    click apiServer "https://github.com/seogenis/world-model-portal-backend/blob/master/app/main.py"
    click apiServer "https://github.com/seogenis/world-model-portal-backend/blob/master/app/api/routes.py"
    click videoServ "https://github.com/seogenis/world-model-portal-backend/blob/master/app/services/video_service.py"
    click batchServ "https://github.com/seogenis/world-model-portal-backend/blob/master/app/services/batch_inference_service.py"
    click paramExtr "https://github.com/seogenis/world-model-portal-backend/blob/master/app/services/parameter_extractor.py"
    click promptMan "https://github.com/seogenis/world-model-portal-backend/blob/master/app/services/prompt_manager.py"
    click sessMan "https://github.com/seogenis/world-model-portal-backend/blob/master/app/services/session_manager.py"
    click worker "https://github.com/seogenis/world-model-portal-backend/blob/master/app/celery_worker.py"
    click resultFiles "https://github.com/seogenis/world-model-portal-backend/tree/master/static"
    click sessionData "https://github.com/seogenis/world-model-portal-backend/tree/master/sessions"
    click openai "https://github.com/seogenis/world-model-portal-backend/blob/master/app/utils/openai_utils.py"

    classDef client fill:#f8d7da,stroke:#721c24,stroke-width:2px
    classDef api fill:#cce5ff,stroke:#004085,stroke-width:2px
    classDef service fill:#fff3cd,stroke:#856404,stroke-width:2px
    classDef worker fill:#d1ecf1,stroke:#0c5460,stroke-width:2px
    classDef external fill:#d4edda,stroke:#155724,stroke-width:2px
    classDef store fill:#e2e3e5,stroke:#6c757d,stroke-width:2px
```
