How seogenis/world-model-portal-backend works
• FastAPI Server: At the heart of the system, hosting REST API endpoints (see app/main.py, app/api/routes.py) for video generation, parameter extraction, prompt management, and static file access.
• API Services:
• Video Service: Encapsulates interactions with NVIDIA’s Cosmos API (with rate limiting and API key rotation as seen in app/services/video_service.py).
• Batch Inference Service: Handles multi-GPU batch processing jobs (app/services/batch_inference_service.py).
• Parameter Extractor and Prompt Manager: Manage prompt enhancement and parameter extraction (app/services/parameter_extractor.py and app/services/prompt_manager.py).
• Session Manager: Manages persistent sessions backed by disk storage (app/services/session_manager.py).
• Asynchronous Request Handling & Worker Pool:
• Task Queues: Represent how incoming API requests are placed into queues.
• Worker Processes: Showing background workers that pick jobs from the queues to process them asynchronously (illustrated in the worker pool implementation).
• Filesystem-Based State Recovery:
• A mechanism using the directory structure (static/videos, sessions/) to validate and recover job status.
• External Integrations:
• NVIDIA Cosmos API: Mark the API integration points along with API key rotation logic.
• OpenAI API: Used for prompt enhancement (found in app/utils/openai_utils.py).
• Docker and external scripts (e.g., start_ngrok.sh) that enable deployment and public access.
Relationships/Interactions
• Client Requests → API Endpoints:
• Clients send video generation or batch inference requests via RESTful endpoints.
• Asynchronous Task Handling:
• Requests get enqueued and picked up by background worker processes.
• Workers interact with video_service using the semaphore for rate limiting, API key rotation, and retry logic.
• State Recovery & Filesystem Checks:
• After processing jobs, the workers store result files (videos) in the static directory. The status endpoints consult this directory along with in-memory job states.
• Batch Processing:
• How batch requests are decomposed into separate jobs that are distributed across a set of available GPUs (note the round-robin assignment).
• Data Flow:
Important Architectural Patterns
• Asynchronous Non-blocking Processing: Emphasize the separation of API request handling (FastAPI) from long-running background tasks.
• Microservices-Like Modularity: Although all components reside in one project, they are organized into clear domains (API, core configuration, services, utilities) allowing easier management and potential extension.
• Resilience and Recovery: The design incorporates robust error handling (API key rotation, exponential backoff) and filesystem-based state recovery for high availability.
• Parallel Processing: The multi-GPU batch inference mechanism is key for scalability in processing heavy workloads concurrently.
Relevant Technologies and Tools
• Python 3.10+ and FastAPI for asynchronous web service handling.
• Uvicorn as the ASGI server.
• Celery-like worker patterns (though implemented using asyncio).
• NVIDIA Cosmos API for video generation and OpenAI API for prompt enhancement.
• Docker, docker-compose (see docker-compose.yml) for deployment, plus ngrok for public exposure.
• Use configuration files like .env and requirements.txt to indicate dependencies.