# rajat-wyrm/intelliview-orchestrator architecture

> Architecture diagram of the GitHub repository rajat-wyrm/intelliview-orchestrator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rajat-wyrm/intelliview-orchestrator
- Repository: https://github.com/rajat-wyrm/intelliview-orchestrator

## Overview

1. **System shape**
   - IntelliView is split into four runtime domains: a **Next.js dashboard**, a **FastAPI orchestrator API**, **Celery-based worker processes**, and a **monitoring stack** built around Prometheus/Grafana.
   - Redis sits in the middle as shared cache, queue/broker, and coordination layer; PostgreSQL is the durable source of truth for application state.

2. **Frontend boundary**
   - The `frontend/` app is a Next.js 14 single-page dashboard with route-based pages for **overview**, **sessions**, **interview**, **candidates**, **workers**, **analytics**, and **settings**.
   - UI state is split between reusable components and client-side hooks such as WebSocket subscription, keyboard navigation, debounce, toast handling, and token hydration.
   - The frontend’s main job is read-heavy visualization plus a few authenticated mutation actions through the backend API.

3. **Backend orchestration boundary**
   - `orchestrator/main.py` is the FastAPI entrypoint and exposes the REST API used by the dashboard and automation clients.
   - The orchestrator layer is not doing heavy AI or media processing itself; instead it coordinates sessions, worker assignment, validation, retries, and state synchronization.
   - Key responsibilities are separated into focused managers such as `session_manager`, `candidate_manager`, `worker_registry`, `load_balancer`, `scheduler`, `fault_manager`, and `health_monitor`.

4. **Session lifecycle as the core data flow**
   - The main business flow is: **start interview session → schedule work → stream updates → store state → display status on dashboard**.
   - Session and candidate state are tracked in the orchestrator, persisted in PostgreSQL, cached in Redis, and pushed live to the frontend through WebSockets.
   - Session tracking and moment tracking are first-class concerns, indicating the app is designed around real-time interview execution rather than batch reporting.

5. **Worker execution layer**
   - `workers/` contains the distributed processing side: `video_pipeline`, `audio_pipeline`, `evaluation_pipeline`, and `risk_engine`.
   - `worker_agent` and `worker_entrypoint` suggest each worker node runs independently, registers itself, and pulls/executes tasks from the broker.
   - Celery is the execution substrate, with Redis acting as broker/backend coordination for asynchronous processing.

6. **AI/provider abstraction**
   - AI behavior is intentionally decoupled from the UI and orchestrator via worker-side modules such as `ai_client` and `evaluation_pipeline`.
   - The README indicates multiple model providers with fallback behavior: **Gemini**, **Grok**, and **OpenAI**.
   - This means provider switching is an architectural concern inside the processing tier, not exposed as separate user-facing services.

7. **State, caching, and resiliency**
   - `database/` holds SQLAlchemy models and database access, while `orchestrator/http_cache.py`, `redis_client.py`, `retry_manager.py`, and `rate_limiter.py` show a deliberate split between durable state and transient operational state.
   - Redis is treated as a fault-prone dependency with resilience patterns like circuit breaking, caching, and retries.
   - The presence of `request_validation.py`, `audit_logger.py`, and `logging_config.py` points to a controlled API boundary with compliance-oriented logging.

8. **Realtime update path**
   - `monitoring/websocket_manager.py` and frontend `useWebSocket`/dashboard components indicate live updates are pushed from backend to browser rather than polled.
   - This path is used for session progress, worker heartbeats, health updates, and interview events.
   - The frontend’s sidebar/topbar/session detail components are built to reflect fast-changing state such as status, risk, and pipeline stage.

9. **Monitoring and observability subsystem**
   - `monitoring/` is a separate subsystem with Prometheus metrics collection and Grafana provisioning.
   - There is also a monitoring dashboard API and static dashboard page, which suggests internal operational visibility beyond the main product UI.
   - Metrics are exposed through the backend and scraped by Prometheus, while Grafana renders the operational dashboards.

10. **Data persistence and schema**
   - PostgreSQL is the authoritative persistence layer for sessions, candidates, worker state, and historical records.
   - SQLAlchemy 2.0 is the ORM layer, and the tests include a migration compatibility check, indicating schema evolution is part of the architecture.
   - Database writes are the durable record; Redis and in-memory state are treated as ephemeral coordination.

11. **Operational controls**
   - The backend includes load balancing, worker registry, heartbeat/health monitoring, fault management, and scheduler components, showing the system is designed for distributed execution with dynamic capacity.
   - Rate limiting and authentication tokens protect sensitive mutations such as starting interviews and submitting answers.
   - The dashboard also includes a screen-lock feature and settings for token management and strategy switching, which are application-level controls around operational use.

12. **Deployment and environment model**
   - The repo is designed to run with **Docker Compose** as a multi-container stack: frontend, API, Redis, PostgreSQL, Prometheus, and Grafana.
   - Environment variables control external service integration and security-sensitive values such as API tokens, AI keys, Redis URL, and PostgreSQL SSL mode.
   - This makes the deployment boundary explicit: the app is containerized, infrastructure-aware, and externally configurable.

13. **Testing and contract boundaries**
   - The tests are structured around architectural seams: API contract tests, session manager tests, worker registry tests, load balancer tests, risk engine tests, and smoke/e2e tests.
   - That test layout mirrors the subsystem boundaries and suggests the codebase is organized around behavior at service edges rather than a monolith of mixed concerns.

14. **How to draw the architecture**
   - Place the **dashboard** on the left, the **FastAPI orchestrator** in the center, **Redis/PostgreSQL** beneath or beside it as shared infrastructure, and the **worker pool** on the right.
   - Add a separate **observability lane** for Prometheus/Grafana and a **real-time lane** for WebSocket updates from backend to frontend.
   - The key arrows are: frontend API calls to orchestrator, orchestrator state changes to PostgreSQL/Redis, orchestrator task dispatch to workers, workers returning results/state, and metrics flowing to monitoring.

Main entry points: [`orchestrator/main.py`](https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/orchestrator/main.py) (FastAPI API), [`frontend/src/app`](https://github.com/rajat-wyrm/intelliview-orchestrator/tree/main/frontend/src/app) (Next.js UI), [`orchestrator/worker_registry.py`](https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/orchestrator/worker_registry.py) (Coordination Core), [`orchestrator/session_manager.py`](https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/orchestrator/session_manager.py) (Session Flow)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontend["Dashboard"]
  node_frontend_dashboard["Next.js UI<br/>nextjs dashboard"]
  node_frontend_state["Client State<br/>ui hooks"]
  node_frontend_components["UI Components<br/>presentation"]
end

subgraph group_orchestrator["API Orchestrator"]
  node_api_entry["FastAPI API<br/>rest entrypoint<br/>[main.py]"]
  node_session_manager["Session Flow<br/>session orchestration<br/>[session_manager.py]"]
  node_coordination_core["Coordination Core<br/>control services<br/>[worker_registry.py]"]
  node_state_sync["State Sync<br/>sync and cache<br/>[state_sync.py]"]
end

subgraph group_workers["Worker Pool"]
  node_celery_runtime["Celery Runtime<br/>task runtime<br/>[celery_app.py]"]
  node_worker_agent["Worker Agent<br/>worker node<br/>[worker_agent.py]"]
  node_pipeline_core["Pipelines<br/>[video_pipeline.py]"]
  node_ai_provider["AI Providers<br/>model abstraction<br/>[ai_client.py]"]
end

subgraph group_infra["Shared Infra"]
  node_database_layer[("Postgres Layer<br/>sqlalchemy storage<br/>[models.py]")]
  node_redis_layer[("Redis Layer<br/>cache and broker<br/>[redis_client.py]")]
end

subgraph group_monitoring["Observability"]
  node_metrics_api["Metrics API<br/>observability api<br/>[dashboard_api.py]"]
  node_websocket_mgr["Live Updates<br/>realtime transport"]
  node_prometheus_stack["Prometheus<br/>metrics scrape<br/>[prometheus.yml]"]
  node_grafana_stack["Grafana<br/>metrics dashboards<br/>[dashboard.yml]"]
end

node_frontend_dashboard -->|"REST calls"| node_api_entry
node_frontend_dashboard -->|"live feed"| node_websocket_mgr
node_frontend_state -->|"UI behavior"| node_frontend_dashboard
node_frontend_components -->|"rendering"| node_frontend_dashboard
node_api_entry -->|"session ops"| node_session_manager
node_api_entry -->|"control ops"| node_coordination_core
node_api_entry -->|"state access"| node_state_sync
node_session_manager -->|"persist"| node_database_layer
node_session_manager -->|"cache state"| node_redis_layer
node_coordination_core -->|"broker/coordination"| node_redis_layer
node_coordination_core -->|"state writes"| node_database_layer
node_coordination_core -->|"dispatch"| node_celery_runtime
node_celery_runtime -->|"execute"| node_worker_agent
node_worker_agent -->|"run pipelines"| node_pipeline_core
node_pipeline_core -->|"model calls"| node_ai_provider
node_pipeline_core -->|"results"| node_database_layer
node_websocket_mgr -->|"push updates"| node_frontend_dashboard
node_metrics_api -->|"expose metrics"| node_prometheus_stack
node_prometheus_stack -->|"visualize"| node_grafana_stack
node_api_entry -->|"export stats"| node_metrics_api
node_worker_agent -->|"events"| node_websocket_mgr

click node_frontend_dashboard "https://github.com/rajat-wyrm/intelliview-orchestrator/tree/main/frontend/src/app"
click node_frontend_state "https://github.com/rajat-wyrm/intelliview-orchestrator/tree/main/frontend/src/hooks"
click node_frontend_components "https://github.com/rajat-wyrm/intelliview-orchestrator/tree/main/frontend/src/components"
click node_api_entry "https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/orchestrator/main.py"
click node_session_manager "https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/orchestrator/session_manager.py"
click node_coordination_core "https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/orchestrator/worker_registry.py"
click node_state_sync "https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/orchestrator/state_sync.py"
click node_database_layer "https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/database/models.py"
click node_redis_layer "https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/orchestrator/redis_client.py"
click node_celery_runtime "https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/workers/celery_app.py"
click node_worker_agent "https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/workers/worker_agent.py"
click node_pipeline_core "https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/workers/video_pipeline.py"
click node_ai_provider "https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/workers/ai_client.py"
click node_metrics_api "https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/monitoring/dashboard_api.py"
click node_websocket_mgr "https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/monitoring/websocket_manager.py"
click node_prometheus_stack "https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/monitoring/prometheus.yml"
click node_grafana_stack "https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/monitoring/grafana/provisioning/dashboards/dashboard.yml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_frontend_dashboard,node_frontend_state,node_frontend_components toneBlue
class node_api_entry,node_session_manager,node_coordination_core,node_state_sync toneAmber
class node_celery_runtime,node_worker_agent,node_pipeline_core,node_ai_provider toneMint
class node_database_layer,node_redis_layer toneRose
class node_metrics_api,node_websocket_mgr,node_prometheus_stack,node_grafana_stack toneIndigo
```

## Components

### Dashboard

Next.js user-facing dashboard and client state

- [Next.js UI](https://github.com/rajat-wyrm/intelliview-orchestrator/tree/main/frontend/src/app): nextjs dashboard · Route-based dashboard for overview, sessions, interview, candidates, workers, analytics, and settings · `frontend/src/app`
- [Client State](https://github.com/rajat-wyrm/intelliview-orchestrator/tree/main/frontend/src/hooks): ui hooks · WebSocket, token hydration, keyboard navigation, debounce, and toast/session UI state · `frontend/src/hooks`
- [UI Components](https://github.com/rajat-wyrm/intelliview-orchestrator/tree/main/frontend/src/components): presentation · Reusable dashboard surfaces such as sidebar, topbar, session detail, lock, and palette · `frontend/src/components`

### API Orchestrator

FastAPI control plane for sessions, coordination, and state sync

- [FastAPI API](https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/orchestrator/main.py): rest entrypoint · REST entrypoint for dashboard and automation clients · `orchestrator/main.py`
- [Session Flow](https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/orchestrator/session_manager.py): session orchestration · Starts interviews, tracks lifecycle, and coordinates session state · `orchestrator/session_manager.py`
- [Coordination Core](https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/orchestrator/worker_registry.py): control services · Candidate management, worker registry, load balancing, scheduling, fault handling, and health checks · `orchestrator/worker_registry.py`
- [State Sync](https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/orchestrator/state_sync.py): sync and cache · Redis-backed cache, retries, rate limiting, request validation, and live synchronization · `orchestrator/state_sync.py`

### Worker Pool

Distributed Celery execution and AI processing

- [Celery Runtime](https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/workers/celery_app.py): task runtime · Asynchronous execution substrate for worker tasks · `workers/celery_app.py`
- [Worker Agent](https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/workers/worker_agent.py): worker node · Registers worker nodes and executes distributed jobs · `workers/worker_agent.py`
- [Pipelines](https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/workers/video_pipeline.py): processing pipelines · Video, audio, evaluation, and risk processing stages · `workers/video_pipeline.py`
- [AI Providers](https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/workers/ai_client.py): model abstraction · Provider abstraction with Gemini, Grok, and OpenAI fallback behavior · `workers/ai_client.py`

### Shared Infra

Persistent and transient infrastructure services

- [Postgres Layer](https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/database/models.py): sqlalchemy storage · SQLAlchemy models and durable application persistence · `database/models.py`
- [Redis Layer](https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/orchestrator/redis_client.py): cache and broker · Shared cache, queue/broker, and coordination backend · `orchestrator/redis_client.py`

### Observability

Metrics, dashboards, and operational visibility

- [Metrics API](https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/monitoring/dashboard_api.py): observability api · Metrics collection and dashboard API for operational visibility · `monitoring/dashboard_api.py`
- [Live Updates](https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/monitoring/websocket_manager.py): realtime transport · Backend websocket fanout for session, worker, and health events · `monitoring/websocket_manager.py`
- [Prometheus](https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/monitoring/prometheus.yml): metrics scrape · Scrapes exported metrics from the application stack · `monitoring/prometheus.yml`
- [Grafana](https://github.com/rajat-wyrm/intelliview-orchestrator/blob/main/monitoring/grafana/provisioning/dashboards/dashboard.yml): metrics dashboards · Provisioned dashboards and datasources for ops visibility · `monitoring/grafana/provisioning/dashboards/dashboard.yml`

## Connections

- Next.js UI → FastAPI API: REST calls · Dashboard reads and authenticated mutations go through the API
- Next.js UI → Live Updates: live feed · UI subscribes to real-time updates for sessions and health
- Client State → Next.js UI: UI behavior · Hooks provide hydration, navigation, debounce, and toast flow
- UI Components → Next.js UI: rendering · Shared components assemble the dashboard screens
- FastAPI API → Session Flow: session ops · Entrypoint delegates interview lifecycle control
- FastAPI API → Coordination Core: control ops · API exposes worker, scheduling, and health coordination actions
- FastAPI API → State Sync: state access · API boundary applies validation, rate limiting, retries, and cache sync
- Session Flow → Postgres Layer: persist · Session and candidate state are stored durably
- Session Flow → Redis Layer: cache state · Session progress and transient coordination use Redis
- Coordination Core → Redis Layer: broker/coordination · Worker registry, scheduling, and load balancing coordinate through Redis
- Coordination Core → Postgres Layer: state writes · Worker and candidate state updates are recorded in Postgres
- Coordination Core → Celery Runtime: dispatch · Tasks are scheduled onto the asynchronous execution layer
- Celery Runtime → Worker Agent: execute · Workers consume tasks from the broker/runtime
- Worker Agent → Pipelines: run pipelines · Worker nodes execute video, audio, evaluation, and risk stages
- Pipelines → AI Providers: model calls · Processing stages invoke provider abstractions for AI behavior
- Pipelines → Postgres Layer: results · Processed outputs and state updates are persisted
- Live Updates → Next.js UI: push updates · Live events are pushed to the browser UI
- Metrics API → Prometheus: expose metrics · Application metrics are scraped by Prometheus
- Prometheus → Grafana: visualize · Grafana renders the collected operational metrics
- FastAPI API → Metrics API: export stats · Backend exposes operational telemetry through the monitoring subsystem
- Worker Agent → Live Updates: events · Worker heartbeats and pipeline progress feed the realtime channel
