# dodona-edu/universal-judge architecture

> Architecture diagram of the GitHub repository dodona-edu/universal-judge, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dodona-edu/universal-judge
- Repository: https://github.com/dodona-edu/universal-judge

## Overview

TESTed is a language-agnostic educational judge: an exercise’s configuration, test suite and submission are evaluated, then feedback is emitted in Dodona’s JSON Lines format. The documented workflow enters through the command-line tool; the judge coordinates planning, code generation, compilation/execution and oracle-based evaluation. Supporting branches cover suite DSL translation, language adapters, descriptions, linting and internationalization. The README and source index establish the product capabilities and module references; unsampled implementation details are represented only where those references support the relationship.

Main entry points: [`tested/testsuite.py`](https://github.com/dodona-edu/universal-judge/blob/master/tested/testsuite.py) (Test suite model), [`tested/judge/core.py`](https://github.com/dodona-edu/universal-judge/blob/master/tested/judge/core.py) (Judge orchestration), [`tested/languages/generation.py`](https://github.com/dodona-edu/universal-judge/blob/master/tested/languages/generation.py) (Test code generation), [`tested/main.py`](https://github.com/dodona-edu/universal-judge/blob/master/tested/main.py) (Runtime entry)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Inputs and DSL"]
  node_cli["CLI entry<br/>[cli.py]"]
  node_config["Configuration<br/>[configs.py]"]
  node_suite["Test suite model<br/>[testsuite.py]"]
  node_dsl["Suite DSL parser"]
end

subgraph group_judge["Judging workflow"]
  node_main["Runtime entry<br/>[main.py]"]
  node_core["Judge orchestration<br/>[core.py]"]
  node_planning["Execution planning<br/>[planning.py]"]
  node_compilation["Compilation<br/>[compilation.py]"]
  node_execution["Submission execution<br/>[execution.py]"]
end

subgraph group_languages["Language support"]
  node_preparation["Input preparation<br/>[preparation.py]"]
  node_generation["Test code generation<br/>[generation.py]"]
  node_adapters["Language adapters"]
  node_conventionalize["Submission adaptation<br/>[conventionalize.py]"]
end

subgraph group_feedback["Evaluation and feedback"]
  node_oracles["Output oracles"]
  node_evaluation["Result evaluation<br/>[evaluation.py]"]
  node_collector["Feedback collection<br/>[collector.py]"]
  node_dodona["Dodona output format<br/>[dodona.py]"]
  node_descriptions["Test descriptions"]
end

subgraph group_model["Shared test model"]
  node_features["Feature analysis<br/>[features.py]"]
  node_types["Data types"]
  node_serialization["Serialization model<br/>[serialisation.py]"]
  node_parser["Input parsing<br/>[parsing.py]"]
end

node_user(("Exercise author"))
node_submission["Submission"]
node_runner["Language toolchain"]
node_stdout(("JSONL feedback"))

node_user -->|"runs judge"| node_cli
node_cli -.->|"starts runtime"| node_main
node_main -->|"loads config"| node_config
node_main -->|"loads suite"| node_suite
node_user -->|"defines tests"| node_suite
node_suite -->|"translates suite"| node_dsl
node_core -->|"uses settings"| node_config
node_core -->|"uses cases"| node_suite
node_core -->|"plans units"| node_planning
node_core -->|"generates tests"| node_generation
node_core -->|"compiles code"| node_compilation
node_core -->|"runs submission"| node_execution
node_core -->|"evaluates results"| node_evaluation
node_core -->|"collects feedback"| node_collector
node_planning -->|"plans from cases"| node_suite
node_generation -->|"prepares cases"| node_preparation
node_generation -->|"adapts names"| node_conventionalize
node_generation -->|"generates per language"| node_adapters
node_submission -->|"is executed"| node_execution
node_compilation -.->|"uses toolchain"| node_runner
node_execution -.->|"invokes toolchain"| node_runner
node_evaluation -->|"checks outputs"| node_oracles
node_evaluation -->|"emits updates"| node_collector
node_collector -->|"formats updates"| node_dodona
node_dodona -->|"serializes output"| node_stdout
node_descriptions -->|"renders case context"| node_suite
node_generation -->|"uses capabilities"| node_features
node_suite -->|"models values"| node_types
node_suite -->|"uses format model"| node_serialization
node_parser -->|"parses values"| node_serialization
node_parser -->|"loads suite data"| node_suite

click node_cli "https://github.com/dodona-edu/universal-judge/blob/master/tested/cli.py"
click node_config "https://github.com/dodona-edu/universal-judge/blob/master/tested/configs.py"
click node_suite "https://github.com/dodona-edu/universal-judge/blob/master/tested/testsuite.py"
click node_dsl "https://github.com/dodona-edu/universal-judge/blob/master/tested/dsl/translate_parser.py"
click node_main "https://github.com/dodona-edu/universal-judge/blob/master/tested/main.py"
click node_core "https://github.com/dodona-edu/universal-judge/blob/master/tested/judge/core.py"
click node_planning "https://github.com/dodona-edu/universal-judge/blob/master/tested/judge/planning.py"
click node_preparation "https://github.com/dodona-edu/universal-judge/blob/master/tested/languages/preparation.py"
click node_generation "https://github.com/dodona-edu/universal-judge/blob/master/tested/languages/generation.py"
click node_adapters "https://github.com/dodona-edu/universal-judge/tree/master/tested/languages"
click node_conventionalize "https://github.com/dodona-edu/universal-judge/blob/master/tested/languages/conventionalize.py"
click node_compilation "https://github.com/dodona-edu/universal-judge/blob/master/tested/judge/compilation.py"
click node_execution "https://github.com/dodona-edu/universal-judge/blob/master/tested/judge/execution.py"
click node_oracles "https://github.com/dodona-edu/universal-judge/tree/master/tested/oracles"
click node_evaluation "https://github.com/dodona-edu/universal-judge/blob/master/tested/judge/evaluation.py"
click node_collector "https://github.com/dodona-edu/universal-judge/blob/master/tested/judge/collector.py"
click node_dodona "https://github.com/dodona-edu/universal-judge/blob/master/tested/dodona.py"
click node_descriptions "https://github.com/dodona-edu/universal-judge/tree/master/tested/descriptions"
click node_features "https://github.com/dodona-edu/universal-judge/blob/master/tested/features.py"
click node_types "https://github.com/dodona-edu/universal-judge/tree/master/tested/datatypes"
click node_serialization "https://github.com/dodona-edu/universal-judge/blob/master/tested/serialisation.py"
click node_parser "https://github.com/dodona-edu/universal-judge/blob/master/tested/parsing.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_config,node_suite,node_dsl toneBlue
class node_main,node_core,node_planning,node_compilation,node_execution toneAmber
class node_preparation,node_generation,node_adapters,node_conventionalize toneMint
class node_oracles,node_evaluation,node_collector,node_dodona,node_descriptions toneRose
class node_features,node_types,node_serialization,node_parser,node_user,node_submission,node_runner,node_stdout toneIndigo
```

## Components

### Inputs and DSL

- [CLI entry](https://github.com/dodona-edu/universal-judge/blob/master/tested/cli.py): `tested/cli.py`
- [Configuration](https://github.com/dodona-edu/universal-judge/blob/master/tested/configs.py): `tested/configs.py`
- [Test suite model](https://github.com/dodona-edu/universal-judge/blob/master/tested/testsuite.py): `tested/testsuite.py`
- [Suite DSL parser](https://github.com/dodona-edu/universal-judge/blob/master/tested/dsl/translate_parser.py): `tested/dsl/translate_parser.py`

### Judging workflow

- [Runtime entry](https://github.com/dodona-edu/universal-judge/blob/master/tested/main.py): `tested/main.py`
- [Judge orchestration](https://github.com/dodona-edu/universal-judge/blob/master/tested/judge/core.py): `tested/judge/core.py`
- [Execution planning](https://github.com/dodona-edu/universal-judge/blob/master/tested/judge/planning.py): `tested/judge/planning.py`
- [Compilation](https://github.com/dodona-edu/universal-judge/blob/master/tested/judge/compilation.py): `tested/judge/compilation.py`
- [Submission execution](https://github.com/dodona-edu/universal-judge/blob/master/tested/judge/execution.py): `tested/judge/execution.py`

### Language support

- [Input preparation](https://github.com/dodona-edu/universal-judge/blob/master/tested/languages/preparation.py): `tested/languages/preparation.py`
- [Test code generation](https://github.com/dodona-edu/universal-judge/blob/master/tested/languages/generation.py): `tested/languages/generation.py`
- [Language adapters](https://github.com/dodona-edu/universal-judge/tree/master/tested/languages): `tested/languages`
- [Submission adaptation](https://github.com/dodona-edu/universal-judge/blob/master/tested/languages/conventionalize.py): `tested/languages/conventionalize.py`

### Evaluation and feedback

- [Output oracles](https://github.com/dodona-edu/universal-judge/tree/master/tested/oracles): `tested/oracles`
- [Result evaluation](https://github.com/dodona-edu/universal-judge/blob/master/tested/judge/evaluation.py): `tested/judge/evaluation.py`
- [Feedback collection](https://github.com/dodona-edu/universal-judge/blob/master/tested/judge/collector.py): `tested/judge/collector.py`
- [Dodona output format](https://github.com/dodona-edu/universal-judge/blob/master/tested/dodona.py): `tested/dodona.py`
- [Test descriptions](https://github.com/dodona-edu/universal-judge/tree/master/tested/descriptions): `tested/descriptions`

### Shared test model

- [Feature analysis](https://github.com/dodona-edu/universal-judge/blob/master/tested/features.py): `tested/features.py`
- [Data types](https://github.com/dodona-edu/universal-judge/tree/master/tested/datatypes): `tested/datatypes`
- [Serialization model](https://github.com/dodona-edu/universal-judge/blob/master/tested/serialisation.py): `tested/serialisation.py`
- [Input parsing](https://github.com/dodona-edu/universal-judge/blob/master/tested/parsing.py): `tested/parsing.py`

### Other components

- **Exercise author**
- **Submission**
- **Language toolchain**
- **JSONL feedback**

## Connections

- Exercise author → CLI entry: runs judge
- CLI entry → Runtime entry: starts runtime
- Runtime entry → Configuration: loads config (evidence: `tested/main.py`)
- Runtime entry → Test suite model: loads suite (evidence: `tested/main.py`)
- Exercise author → Test suite model: defines tests
- Test suite model → Suite DSL parser: translates suite (evidence: `tested/dsl/translate_parser.py`)
- Judge orchestration → Configuration: uses settings (evidence: `tested/judge/core.py`)
- Judge orchestration → Test suite model: uses cases (evidence: `tested/judge/core.py`)
- Judge orchestration → Execution planning: plans units (evidence: `tested/judge/core.py`)
- Judge orchestration → Test code generation: generates tests (evidence: `tested/judge/core.py`)
- Judge orchestration → Compilation: compiles code (evidence: `tested/judge/core.py`)
- Judge orchestration → Submission execution: runs submission (evidence: `tested/judge/core.py`)
- Judge orchestration → Result evaluation: evaluates results (evidence: `tested/judge/core.py`)
- Judge orchestration → Feedback collection: collects feedback (evidence: `tested/judge/core.py`)
- Execution planning → Test suite model: plans from cases (evidence: `tested/judge/planning.py`)
- Test code generation → Input preparation: prepares cases (evidence: `tested/languages/generation.py`)
- Test code generation → Submission adaptation: adapts names (evidence: `tested/languages/generation.py`)
- Test code generation → Language adapters: generates per language (evidence: `tested/languages/generation.py`)
- Submission → Submission execution: is executed
- Compilation → Language toolchain: uses toolchain
- Submission execution → Language toolchain: invokes toolchain
- Result evaluation → Output oracles: checks outputs (evidence: `tested/judge/evaluation.py`)
- Result evaluation → Feedback collection: emits updates (evidence: `tested/judge/evaluation.py`)
- Feedback collection → Dodona output format: formats updates (evidence: `tested/judge/collector.py`)
- Dodona output format → JSONL feedback: serializes output
- Test descriptions → Test suite model: renders case context (evidence: `tested/descriptions/renderer.py`)
- Test code generation → Feature analysis: uses capabilities (evidence: `tested/languages/generation.py`)
- Test suite model → Data types: models values (evidence: `tested/testsuite.py`)
- Test suite model → Serialization model: uses format model (evidence: `tested/testsuite.py`)
- Input parsing → Serialization model: parses values (evidence: `tested/parsing.py`)
- Input parsing → Test suite model: loads suite data (evidence: `tested/parsing.py`)