# cmu-agents/assignment-2 architecture

> Architecture diagram of the GitHub repository cmu-agents/assignment-2, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cmu-agents/assignment-2
- Repository: https://github.com/cmu-agents/assignment-2

## Overview

This is an assignment toolkit for evaluating a data-visualization agent: tasks are authored and packaged, agents produce run artifacts, validators judge those runs, and human labels support scoring validator predictions. The graph separates task/run generation, validation, and evaluation while showing the model-serving integration. Sampled source confirms the validator and model calls; links involving unsampled workflow entry points and generation orchestration are omitted where their ownership cannot be verified.

Main entry points: [`artifacts`](https://github.com/cmu-agents/assignment-2/tree/main/artifacts) (Run Artifacts), [`validator/runner.py`](https://github.com/cmu-agents/assignment-2/blob/main/validator/runner.py) (Validation Runner), [`tasks`](https://github.com/cmu-agents/assignment-2/tree/main/tasks) (Authored Tasks), [`workflow/generate.py`](https://github.com/cmu-agents/assignment-2/blob/main/workflow/generate.py) (Run Generation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_task_run["Tasks and Runs"]
  node_task_defs["Authored Tasks"]
  node_task_loader["Task Validation<br/>[tasks.py]"]
  node_packaging["Task Packaging<br/>[package.py]"]
  node_harbor_tasks["Harbor Tasks"]
  node_generation["Run Generation<br/>[generate.py]"]
  node_runs["Run Artifacts"]
end

subgraph group_agent_serving["Agent and Serving"]
  node_agent_runner["Agent Execution<br/>[agent_runner.py]"]
  node_generation_gateway["Generation Gateway"]
  node_generation_models["Generation Models"]
end

subgraph group_validation["Validation"]
  node_runner["Validation Runner<br/>[runner.py]"]
  node_solution["Validator Solution<br/>[solution.py]"]
  node_baseline["Baseline Judge<br/>[baseline.py]"]
  node_model_client["Judge Model Client<br/>[model.py]"]
  node_predictions["Predictions<br/>[prediction.py]"]
end

subgraph group_evaluation["Evaluation"]
  node_evaluation["Labeling and Scoring<br/>[evaluate.py]"]
  node_submission["Submission Checks<br/>[submission.py]"]
  node_rewards["Harbor Rewards<br/>[harbor_rewards.py]"]
end

node_author(("Task Author"))
node_model_endpoint{{"Generation Endpoint"}}
node_validator_endpoint{{"Validator Endpoint"}}
node_human(("Human Reviewer"))

node_author -->|"authors"| node_task_defs
node_task_defs -->|"loads"| node_task_loader
node_task_loader -->|"provides tasks"| node_generation
node_generation -->|"dispatches runs"| node_agent_runner
node_agent_runner -->|"requests generation"| node_generation_gateway
node_generation_gateway -->|"forwards requests"| node_model_endpoint
node_generation -->|"records results"| node_runs
node_task_defs -->|"packages"| node_packaging
node_packaging -->|"creates tasks"| node_harbor_tasks
node_runner -->|"loads artifacts"| node_runs
node_runner -->|"invokes"| node_solution
node_solution -->|"uses judge"| node_baseline
node_baseline -->|"asks questions"| node_model_client
node_model_client -->|"sends requests"| node_validator_endpoint
node_runner -->|"validates and writes"| node_predictions
node_human -->|"reviews runs"| node_evaluation
node_evaluation -->|"loads artifacts"| node_runs
node_evaluation -->|"scores"| node_predictions
node_submission -->|"checks packaging"| node_harbor_tasks
node_submission -->|"checks records"| node_runs
node_rewards -->|"runs tasks"| node_harbor_tasks

click node_task_defs "https://github.com/cmu-agents/assignment-2/tree/main/tasks"
click node_task_loader "https://github.com/cmu-agents/assignment-2/blob/main/workflow/tasks.py"
click node_packaging "https://github.com/cmu-agents/assignment-2/blob/main/workflow/package.py"
click node_harbor_tasks "https://github.com/cmu-agents/assignment-2/tree/main/harbor"
click node_generation "https://github.com/cmu-agents/assignment-2/blob/main/workflow/generate.py"
click node_agent_runner "https://github.com/cmu-agents/assignment-2/blob/main/infrastructure/agent_runner.py"
click node_generation_gateway "https://github.com/cmu-agents/assignment-2/blob/main/infrastructure/generation_gateway.py"
click node_generation_models "https://github.com/cmu-agents/assignment-2/blob/main/infrastructure/generation_models.py"
click node_runs "https://github.com/cmu-agents/assignment-2/tree/main/artifacts"
click node_runner "https://github.com/cmu-agents/assignment-2/blob/main/validator/runner.py"
click node_solution "https://github.com/cmu-agents/assignment-2/blob/main/validator/solution.py"
click node_baseline "https://github.com/cmu-agents/assignment-2/blob/main/validator/baseline.py"
click node_model_client "https://github.com/cmu-agents/assignment-2/blob/main/validator/model.py"
click node_predictions "https://github.com/cmu-agents/assignment-2/blob/main/validator/prediction.py"
click node_evaluation "https://github.com/cmu-agents/assignment-2/blob/main/workflow/evaluate.py"
click node_submission "https://github.com/cmu-agents/assignment-2/blob/main/workflow/submission.py"
click node_rewards "https://github.com/cmu-agents/assignment-2/blob/main/workflow/harbor_rewards.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_task_defs,node_task_loader,node_packaging,node_harbor_tasks,node_generation,node_runs,node_human toneBlue
class node_agent_runner,node_generation_gateway,node_generation_models toneAmber
class node_runner,node_solution,node_baseline,node_model_client,node_predictions toneMint
class node_evaluation,node_submission,node_rewards,node_author toneRose
class node_model_endpoint,node_validator_endpoint toneIndigo
```

## Components

### Tasks and Runs

- [Authored Tasks](https://github.com/cmu-agents/assignment-2/tree/main/tasks): `tasks`
- [Task Validation](https://github.com/cmu-agents/assignment-2/blob/main/workflow/tasks.py): `workflow/tasks.py`
- [Task Packaging](https://github.com/cmu-agents/assignment-2/blob/main/workflow/package.py): `workflow/package.py`
- [Harbor Tasks](https://github.com/cmu-agents/assignment-2/tree/main/harbor): `harbor`
- [Run Generation](https://github.com/cmu-agents/assignment-2/blob/main/workflow/generate.py): `workflow/generate.py`
- [Run Artifacts](https://github.com/cmu-agents/assignment-2/tree/main/artifacts): `artifacts`

### Agent and Serving

- [Agent Execution](https://github.com/cmu-agents/assignment-2/blob/main/infrastructure/agent_runner.py): `infrastructure/agent_runner.py`
- [Generation Gateway](https://github.com/cmu-agents/assignment-2/blob/main/infrastructure/generation_gateway.py): `infrastructure/generation_gateway.py`
- [Generation Models](https://github.com/cmu-agents/assignment-2/blob/main/infrastructure/generation_models.py): `infrastructure/generation_models.py`

### Validation

- [Validation Runner](https://github.com/cmu-agents/assignment-2/blob/main/validator/runner.py): `validator/runner.py`
- [Validator Solution](https://github.com/cmu-agents/assignment-2/blob/main/validator/solution.py): `validator/solution.py`
- [Baseline Judge](https://github.com/cmu-agents/assignment-2/blob/main/validator/baseline.py): `validator/baseline.py`
- [Judge Model Client](https://github.com/cmu-agents/assignment-2/blob/main/validator/model.py): `validator/model.py`
- [Predictions](https://github.com/cmu-agents/assignment-2/blob/main/validator/prediction.py): `validator/prediction.py`

### Evaluation

- [Labeling and Scoring](https://github.com/cmu-agents/assignment-2/blob/main/workflow/evaluate.py): `workflow/evaluate.py`
- [Submission Checks](https://github.com/cmu-agents/assignment-2/blob/main/workflow/submission.py): `workflow/submission.py`
- [Harbor Rewards](https://github.com/cmu-agents/assignment-2/blob/main/workflow/harbor_rewards.py): `workflow/harbor_rewards.py`

### Other components

- **Task Author**
- **Generation Endpoint**
- **Validator Endpoint**
- **Human Reviewer**

## Connections

- Task Author → Authored Tasks: authors
- Authored Tasks → Task Validation: loads
- Task Validation → Run Generation: provides tasks
- Run Generation → Agent Execution: dispatches runs
- Agent Execution → Generation Gateway: requests generation
- Generation Gateway → Generation Endpoint: forwards requests
- Run Generation → Run Artifacts: records results
- Authored Tasks → Task Packaging: packages
- Task Packaging → Harbor Tasks: creates tasks
- Validation Runner → Run Artifacts: loads artifacts
- Validation Runner → Validator Solution: invokes
- Validator Solution → Baseline Judge: uses judge
- Baseline Judge → Judge Model Client: asks questions
- Judge Model Client → Validator Endpoint: sends requests
- Validation Runner → Predictions: validates and writes
- Human Reviewer → Labeling and Scoring: reviews runs
- Labeling and Scoring → Run Artifacts: loads artifacts
- Labeling and Scoring → Predictions: scores
- Submission Checks → Harbor Tasks: checks packaging
- Submission Checks → Run Artifacts: checks records
- Harbor Rewards → Harbor Tasks: runs tasks