# iwangjian/coding-tutor architecture

> Architecture diagram of the GitHub repository iwangjian/coding-tutor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/iwangjian/coding-tutor
- Repository: https://github.com/iwangjian/coding-tutor

## Overview

##### Project Type and Purpose
- This project is an experimental/focused research system in the domain of dialogue tutoring agents. It implements a pipeline (called “Traver”) that incorporates techniques such as knowledge tracing and turn-by-turn verification to support coding tutoring.
- The main goal is to evaluate and scale tutor agents (via DICT) for tutoring students (or simulated students) using multiple layers of verification.
- The system supports training and evaluation of verifiers and tutoring agents, along with performance benchmarking and simulated student analysis.

##### File Structure to Identify Main Components
- Frontend and Evaluation Interface:
- The human_eval folder contains a simple web application (app.py, templates, static files) used to interface with evaluations or possibly simulate student interactions.
- Configuration and Data:
- The config folder with deepspeed configuration files.
- The assets folder holds overview images and evaluation/simulation figures.
- The output folder stores released data and evaluation results.
- The prompt folder contains various prompt templates for agents (different levels/tutoring contexts).
- Core Agent Logic and Verifiers:
- The traver directory is the heart of the system:
◦ chatarena: This module contains the main tutor and agent logic. Notice multiple agent implementations (agent.py, agent_tutor.py) and environments for conversations and tutoring. It also integrates several “backends” (OpenAI, Anthropic, Cohere, etc.) for LLM inference.
◦ parser: Responsible for processing source code and constructing function calls. It appears to have a sub-package (pyan_zyf_v2) suggesting a custom parser utility including analysis and visualization of function call graphs.
◦ verifier: Contains modules to load, run, and evaluate verifiers (trained or otherwise) which are core to the “trace-and-verify” work.
◦ Additional utility scripts assist in managing inference (e.g., LM_inference.py, gpt_inference.py) and process completions.
- Evaluation and Benchmarking:
- The benchmark and scripts/eval directories host scripts to conduct various types of evaluations (such as eval_TOC.py, eval_TOR.py, and eval_pretest.py).
- There are additional scripts (in the scripts folder) that likely standardize testing and data extraction.

##### Extract Architectural Patterns and Design Principles from the Provided Information
- The design follows a modular architecture. Each domain (dialogue, parsing, verification) is separated into its own package, enhancing clarity and maintainability.
- There’s an integration layer in chatarena that abstracts different backend LLM providers (using a backends subfolder), which shows the use of a strategy design pattern for interfacing with multiple LLM endpoints.
- The overall workflow hints at a pipeline structure: input prompts/templates → dialogue/tutoring agent (chatarena) → parsing utility → verifier logic → evaluation and benchmarking.
- Dependency injection and configuration-driven patterns are evident from the use of JSON configurations and well-separated configuration files.

##### Recommended Components
a. Main System Components:
- Client/User Interface/Frontend: Represent the human_eval module (web interface for evaluation).
- Core Tutor Agent: Located under traver/chatarena. Illustrate the agent components – both basic conversation (arena) and tutoring variants (agent_tutor, arena_tutoring).
- LLM Backend Integration: Display the multiple backends (OpenAI, Anthropic, Cohere, etc.) that the tutoring agent or verifier uses for generating responses.
- Parsing Engine: Illustrate the parser module (especially the pyan_zyf_v2 subpackage) responsible for source code parsing and function call analysis.
- Verifier Module: Highlight the verifier package which is critical for “trace-and-verify” during tutoring dialogues.
- Configuration and Prompt Templates: Show that the tutoring content is dynamically generated via prompts stored in the prompt/templates directory.
- Evaluation/Benchmark Infrastructure: Corresponding modules from benchmark, scripts/eval, and output (for evaluation results).

b. Relationships and Interactions:
- User interactions and simulation flows are initiated at the frontend.
- The tutoring agent receives input via prompts, interacts with a parsing engine if needed, and queries different LLM backends.
- After generating responses, the verifier module checks and scales performance via feedback loops.
- Evaluators (scripts/eval and benchmark) trigger specific testing workflows that interface with the tutoring agent and verify outcomes.
- Flows from configuration and prompt management into the tutoring workflows.

c. Architectural Patterns and Principles:
- Modular and layered design: Each major functionality resides in a separate module (chatarena for agents, parser for code analysis, verifier for result checking).
- Loose coupling between LLM backends: Illustrate via abstraction layers in the backends package.
- Pipeline / data flow processing: Input from users → agent response → code parsing → verification → evaluation outputs.

##### Additional Details
- A notation that indicates external integrations (like Huggingface/Arxiv references in the README to signify academic and library dependencies).
- Optionally, add annotations for critical paths like “Inference-Time Scaling with Verifiers” to emphasize performance optimizations.
- To show any feedback loop between the tutor agent and simulated student evaluations (from human_eval into evaluation scripts).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interface/Simulation Layer
    subgraph "User Interface/Simulation"
        UI1["Human Eval Frontend"]:::ui
    end

    %% Core Processing Layer
    subgraph "Core Processing"
        CTA["Core Tutor Agent (Chatarena)"]:::core
        PAR["Parsing Engine (traver/parser)"]:::core
        PAR2["Custom Parser (pyan_zyf_v2)"]:::core
        VER["Verifier Module (traver/verifier)"]:::core
    end

    %% External LLM Integration
    subgraph "External LLM Integration"
        LLM["LLM Backend Integration"]:::llm
    end

    %% Data & Evaluation Infrastructure
    subgraph "Data & Evaluation Infrastructure"
        CONF["Configuration & Prompt Templates"]:::data
        EVAL["Evaluation & Benchmark Infrastructure"]:::data
        EVAL1["Benchmark"]:::data
        EVAL2["Scripts/Eval"]:::data
        EVAL3["Output Data Store"]:::data
    end

    %% Relationships between layers
    UI1 -->|"initiates" | CTA
    CONF -->|"configures" | CTA
    CTA -->|"queries" | LLM
    CTA -->|"sendsSourceCode"| PAR
    PAR2 -->|"analyzes"| PAR
    PAR -->|"returnsResults"| CTA
    CTA -->|"verifies"| VER
    VER -->|"feedback"| CTA
    CTA -->|"evaluationData"| EVAL
    EVAL1 -->|"triggersTests"| CTA
    EVAL2 -->|"triggersTests"| CTA
    EVAL3 -->|"storesResults"| EVAL

    %% Click Events
    click UI1 "https://github.com/iwangjian/coding-tutor/blob/main/human_eval/app.py"
    click CTA "https://github.com/iwangjian/coding-tutor/tree/main/traver/chatarena"
    click LLM "https://github.com/iwangjian/coding-tutor/tree/main/traver/chatarena/backends"
    click PAR "https://github.com/iwangjian/coding-tutor/tree/main/traver/parser"
    click PAR2 "https://github.com/iwangjian/coding-tutor/tree/main/traver/parser/pyan_zyf_v2"
    click VER "https://github.com/iwangjian/coding-tutor/tree/main/traver/verifier"
    click CONF "https://github.com/iwangjian/coding-tutor/tree/main/config"
    click EVAL1 "https://github.com/iwangjian/coding-tutor/tree/main/benchmark"
    click EVAL2 "https://github.com/iwangjian/coding-tutor/tree/main/scripts/eval"
    click EVAL3 "https://github.com/iwangjian/coding-tutor/tree/main/output"

    %% Styling
    classDef ui fill:#ffe680,stroke:#333,stroke-width:2px;
    classDef core fill:#cce5ff,stroke:#333,stroke-width:2px;
    classDef llm fill:#ccffcc,stroke:#333,stroke-width:2px;
    classDef data fill:#fff2cc,stroke:#333,stroke-width:2px;
```
