# kientm0810/clone-ctfjudge-for-sure architecture

> Architecture diagram of the GitHub repository kientm0810/clone-ctfjudge-for-sure, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kientm0810/clone-ctfjudge-for-sure
- Repository: https://github.com/kientm0810/clone-ctfjudge-for-sure

## Overview

CTFJudge combines a new visual workflow MVP with the original scripted evaluation pipeline. In the MVP, users edit and version node graphs, configure judge models, submit golden/student inputs, then inspect asynchronous run status and artifacts; the built-in RAG template chunks and indexes evidence, retrieves relevant chunks, and produces rubric-based comparison output. The graph also preserves the original writeup, trajectory, and qualitative evaluation stages. Evidence is limited to the README and indexed excerpts; unshown API and UI wiring is not asserted.

Main entry points: [`backend/app/workflow/executor.py`](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/workflow/executor.py) (Workflow executor), [`backend/app/api/workflows.py`](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/api/workflows.py) (Workflow API), [`backend/app/services/llm_service.py`](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/services/llm_service.py) (LLM service), [`backend/app/main.py`](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/main.py) (API application)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ui["Workflow UI"]
  node_workflow_ui["Workflow editor<br/>[WorkflowEditor.tsx]"]
  node_dashboard["Run dashboard"]
  node_model_ui["Model manager<br/>[ModelManager.tsx]"]
  node_frontend_api["API client<br/>[api.ts]"]
end

subgraph group_api["API and persistence"]
  node_app["API application<br/>[main.py]"]
  node_workflow_api["Workflow API<br/>[workflows.py]"]
  node_provider_api["Provider API<br/>[providers.py]"]
  node_deployment_api["Deployment API<br/>[deployments.py]"]
  node_runs_api["Runs API<br/>[runs.py]"]
  node_workflow_store["Workflow store<br/>[workflow_store.py]"]
  node_models[("Workflow and run records<br/>[workflow.py]")]
  node_provider_models[("Provider records<br/>[provider.py]")]
end

subgraph group_runtime["Workflow runtime"]
  node_worker["Workflow worker<br/>[workflow_task.py]"]
  node_executor["Workflow executor<br/>[executor.py]"]
  node_validator["Workflow validator<br/>[validator.py]"]
  node_node_registry["Node registry"]
  node_node_types["Workflow nodes"]
  node_context["Execution context<br/>[context.py]"]
  node_llm["LLM service<br/>[llm_service.py]"]
  node_vector_service["Vector service<br/>[vector_service.py]"]
  node_input_output["Workflow definitions<br/>[workflow.py]"]
  node_report["Report service<br/>[report_service.py]"]
end

subgraph group_legacy["Original evaluation"]
  node_legacy_runner["Evaluation runner<br/>[run_evaluation.py]"]
  node_writeup["Writeup summarizer"]
  node_trajectory["Trajectory summarizer"]
  node_qualitative["Qualitative evaluator"]
  node_llm_utils["LLM response utilities<br/>[llm_utils.py]"]
  node_version_manager["Output versioning<br/>[version_manager.py]"]
end

node_user(("User"))
node_postgres[("PostgreSQL")]
node_redis_celery["Redis and Celery"]
node_ollama["Ollama"]
node_registered_model["OpenAI-compatible models"]
node_vector_store[("pgvector chunks")]

node_user -->|"edits workflow"| node_workflow_ui
node_user -->|"submits inputs"| node_dashboard
node_user -->|"registers models"| node_model_ui
node_workflow_ui -.->|"saves definitions"| node_frontend_api
node_dashboard -.->|"runs and polls"| node_frontend_api
node_model_ui -.->|"manages models"| node_frontend_api
node_app -->|"registers routes"| node_workflow_api
node_app -->|"registers routes"| node_provider_api
node_app -->|"registers routes"| node_deployment_api
node_app -->|"registers routes"| node_runs_api
node_workflow_api -->|"validates definitions"| node_validator
node_workflow_api -->|"loads and versions"| node_workflow_store
node_workflow_api -->|"executes direct runs"| node_executor
node_workflow_api -->|"cleans collections"| node_vector_service
node_workflow_store -.->|"persists workflows"| node_models
node_models -->|"stores records"| node_postgres
node_provider_models -->|"stores connections"| node_postgres
node_provider_api -->|"manages connections"| node_provider_models
node_worker -->|"executes queued run"| node_executor
node_worker -->|"persists run status"| node_models
node_worker -->|"provides service"| node_llm
node_worker -->|"provides service"| node_vector_service
node_executor -->|"loads node handlers"| node_node_registry
node_executor -->|"validates graph"| node_validator
node_executor -->|"builds execution context"| node_context
node_node_registry -.->|"registers handlers"| node_node_types
node_node_types -.->|"uses services"| node_context
node_llm -->|"sends chat and embeddings"| node_ollama
node_llm -->|"sends model requests"| node_registered_model
node_llm -->|"loads registered model"| node_provider_models
node_vector_service -->|"requests embeddings"| node_llm
node_vector_service -->|"indexes and searches"| node_vector_store
node_executor -->|"consumes definitions"| node_input_output
node_legacy_runner -->|"summarizes writeups"| node_writeup
node_legacy_runner -->|"summarizes trajectories"| node_trajectory
node_legacy_runner -->|"evaluates submissions"| node_qualitative
node_legacy_runner -.->|"versions outputs"| node_version_manager
node_writeup -->|"parses responses"| node_llm_utils
node_trajectory -->|"parses responses"| node_llm_utils
node_qualitative -->|"parses responses"| node_llm_utils

click node_workflow_ui "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/frontend/src/components/workflow/WorkflowEditor.tsx"
click node_dashboard "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/frontend/src/components/workflow/WorkflowDashboard.tsx"
click node_model_ui "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/frontend/src/components/workflow/ModelManager.tsx"
click node_frontend_api "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/frontend/src/lib/api.ts"
click node_app "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/main.py"
click node_workflow_api "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/api/workflows.py"
click node_provider_api "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/api/providers.py"
click node_deployment_api "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/api/deployments.py"
click node_runs_api "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/api/runs.py"
click node_workflow_store "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/services/workflow_store.py"
click node_models "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/models/workflow.py"
click node_provider_models "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/models/provider.py"
click node_worker "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/workers/workflow_task.py"
click node_executor "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/workflow/executor.py"
click node_validator "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/workflow/validator.py"
click node_node_registry "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/nodes/default_registry.py"
click node_node_types "https://github.com/kientm0810/clone-ctfjudge-for-sure/tree/main/backend/app/nodes"
click node_context "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/workflow/context.py"
click node_llm "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/services/llm_service.py"
click node_vector_service "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/services/vector_service.py"
click node_input_output "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/schemas/workflow.py"
click node_legacy_runner "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/run_evaluation.py"
click node_writeup "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/writeup_summary_agent.py"
click node_trajectory "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/trajectory_summary_agent.py"
click node_qualitative "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/qualitative_evaluation_agent.py"
click node_llm_utils "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/llm_utils.py"
click node_version_manager "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/version_manager.py"
click node_report "https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/services/report_service.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_workflow_ui,node_dashboard,node_model_ui,node_frontend_api,node_user toneBlue
class node_app,node_workflow_api,node_provider_api,node_deployment_api,node_runs_api,node_workflow_store,node_models,node_provider_models,node_postgres,node_redis_celery,node_vector_store toneAmber
class node_worker,node_executor,node_validator,node_node_registry,node_node_types,node_context,node_llm,node_vector_service,node_input_output,node_report toneMint
class node_legacy_runner,node_writeup,node_trajectory,node_qualitative,node_llm_utils,node_version_manager toneRose
class node_ollama,node_registered_model toneIndigo
```

## Components

### Workflow UI

- [Workflow editor](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/frontend/src/components/workflow/WorkflowEditor.tsx): `frontend/src/components/workflow/WorkflowEditor.tsx`
- [Run dashboard](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/frontend/src/components/workflow/WorkflowDashboard.tsx): `frontend/src/components/workflow/WorkflowDashboard.tsx`
- [Model manager](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/frontend/src/components/workflow/ModelManager.tsx): `frontend/src/components/workflow/ModelManager.tsx`
- [API client](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/frontend/src/lib/api.ts): `frontend/src/lib/api.ts`

### API and persistence

- [API application](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/main.py): `backend/app/main.py`
- [Workflow API](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/api/workflows.py): `backend/app/api/workflows.py`
- [Provider API](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/api/providers.py): `backend/app/api/providers.py`
- [Deployment API](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/api/deployments.py): `backend/app/api/deployments.py`
- [Runs API](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/api/runs.py): `backend/app/api/runs.py`
- [Workflow store](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/services/workflow_store.py): `backend/app/services/workflow_store.py`
- [Workflow and run records](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/models/workflow.py): `backend/app/models/workflow.py`
- [Provider records](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/models/provider.py): `backend/app/models/provider.py`

### Workflow runtime

- [Workflow worker](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/workers/workflow_task.py): `backend/app/workers/workflow_task.py`
- [Workflow executor](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/workflow/executor.py): `backend/app/workflow/executor.py`
- [Workflow validator](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/workflow/validator.py): `backend/app/workflow/validator.py`
- [Node registry](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/nodes/default_registry.py): `backend/app/nodes/default_registry.py`
- [Workflow nodes](https://github.com/kientm0810/clone-ctfjudge-for-sure/tree/main/backend/app/nodes): `backend/app/nodes`
- [Execution context](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/workflow/context.py): `backend/app/workflow/context.py`
- [LLM service](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/services/llm_service.py): `backend/app/services/llm_service.py`
- [Vector service](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/services/vector_service.py): `backend/app/services/vector_service.py`
- [Workflow definitions](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/schemas/workflow.py): `backend/app/schemas/workflow.py`
- [Report service](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/backend/app/services/report_service.py): `backend/app/services/report_service.py`

### Original evaluation

- [Evaluation runner](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/run_evaluation.py): `run_evaluation.py`
- [Writeup summarizer](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/writeup_summary_agent.py): `writeup_summary_agent.py`
- [Trajectory summarizer](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/trajectory_summary_agent.py): `trajectory_summary_agent.py`
- [Qualitative evaluator](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/qualitative_evaluation_agent.py): `qualitative_evaluation_agent.py`
- [LLM response utilities](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/llm_utils.py): `llm_utils.py`
- [Output versioning](https://github.com/kientm0810/clone-ctfjudge-for-sure/blob/main/version_manager.py): `version_manager.py`

### Other components

- **User**
- **PostgreSQL**
- **Redis and Celery**
- **Ollama**
- **OpenAI-compatible models**
- **pgvector chunks**

## Connections

- User → Workflow editor: edits workflow (evidence: `README.md`)
- User → Run dashboard: submits inputs (evidence: `README.md`)
- User → Model manager: registers models (evidence: `README.md`)
- Workflow editor → API client: saves definitions
- Run dashboard → API client: runs and polls
- Model manager → API client: manages models
- API application → Workflow API: registers routes (evidence: `backend/app/main.py`)
- API application → Provider API: registers routes (evidence: `backend/app/main.py`)
- API application → Deployment API: registers routes (evidence: `backend/app/main.py`)
- API application → Runs API: registers routes (evidence: `backend/app/main.py`)
- Workflow API → Workflow validator: validates definitions (evidence: `backend/app/api/workflows.py`)
- Workflow API → Workflow store: loads and versions (evidence: `backend/app/api/workflows.py`)
- Workflow API → Workflow executor: executes direct runs (evidence: `backend/app/api/workflows.py`)
- Workflow API → Vector service: cleans collections (evidence: `backend/app/api/workflows.py`)
- Workflow store → Workflow and run records: persists workflows
- Workflow and run records → PostgreSQL: stores records (evidence: `backend/app/models/workflow.py`)
- Provider records → PostgreSQL: stores connections (evidence: `backend/app/models/provider.py`)
- Provider API → Provider records: manages connections (evidence: `backend/app/api/providers.py`)
- Workflow worker → Workflow executor: executes queued run (evidence: `backend/app/workers/workflow_task.py`)
- Workflow worker → Workflow and run records: persists run status (evidence: `backend/app/workers/workflow_task.py`)
- Workflow worker → LLM service: provides service (evidence: `backend/app/workers/workflow_task.py`)
- Workflow worker → Vector service: provides service (evidence: `backend/app/workers/workflow_task.py`)
- Workflow executor → Node registry: loads node handlers (evidence: `backend/app/workflow/executor.py`)
- Workflow executor → Workflow validator: validates graph (evidence: `backend/app/workflow/executor.py`)
- Workflow executor → Execution context: builds execution context (evidence: `backend/app/workflow/executor.py`)
- Node registry → Workflow nodes: registers handlers
- Workflow nodes → Execution context: uses services (evidence: `backend/app/nodes/database_node.py`)
- LLM service → Ollama: sends chat and embeddings (evidence: `backend/app/services/llm_service.py`)
- LLM service → OpenAI-compatible models: sends model requests (evidence: `backend/app/services/llm_service.py`)
- LLM service → Provider records: loads registered model (evidence: `backend/app/services/llm_service.py`)
- Vector service → LLM service: requests embeddings (evidence: `backend/app/services/vector_service.py`)
- Vector service → pgvector chunks: indexes and searches (evidence: `backend/app/services/vector_service.py`)
- Workflow executor → Workflow definitions: consumes definitions (evidence: `backend/app/workflow/executor.py`)
- Evaluation runner → Writeup summarizer: summarizes writeups (evidence: `run_evaluation.py`)
- Evaluation runner → Trajectory summarizer: summarizes trajectories (evidence: `run_evaluation.py`)
- Evaluation runner → Qualitative evaluator: evaluates submissions (evidence: `run_evaluation.py`)
- Evaluation runner → Output versioning: versions outputs (evidence: `run_evaluation.py`)
- Writeup summarizer → LLM response utilities: parses responses (evidence: `writeup_summary_agent.py`)
- Trajectory summarizer → LLM response utilities: parses responses (evidence: `trajectory_summary_agent.py`)
- Qualitative evaluator → LLM response utilities: parses responses (evidence: `qualitative_evaluation_agent.py`)
