# umass-ml4ed/dialogue-kt architecture

> Architecture diagram of the GitHub repository umass-ml4ed/dialogue-kt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/umass-ml4ed/dialogue-kt
- Repository: https://github.com/umass-ml4ed/dialogue-kt

## Overview

##### Project Type and Purpose
- This is a research-oriented full-stack Python toolkit for “dialogue knowledge tracing” (KT) in tutor-student dialogues.
- It automates annotation of raw dialogue data using the OpenAI API, implements a suite of KT models (LLM-based and classic sequence models), trains/evaluates them, and visualizes results.

##### Core Architectural Components
A. Data Layer
- Raw source data (`data/src/…`): unannotated CoMTA, MathDial, ATC standards.
- Annotated data (`data/annotated/…`): pre-tagged CSV/JSON files with knowledge components (KCs) and correctness labels.
- Artifacts directories: `results/` (metrics, learning curves), `saved_models/` (trained model checkpoints).

B. Annotation Pipeline
- Entry point: `dialogue_kt/main.py` CLI command `annotate`.
- Modules:
- `dialogue_kt/annotate.py` – high-level orchestration
- `dialogue_kt/prompting.py` – constructs LLM prompts for KC & correctness
- `dialogue_kt/openai_api.py` – wraps OpenAI API calls
- `dialogue_kt/data_loading.py` & `dialogue_kt/kt_data_loading.py` – load raw dialogues
- `dialogue_kt/utils.py` – helpers and I/O

C. Model Training & Evaluation
- Entry point: `dialogue_kt/main.py` CLI commands `train`, `test`.
- Model implementations (`dialogue_kt/models/…`):
- `lm.py`, `dkt_multi_kc.py`, `dkt_sem.py`, `simplekt.py` – various KT algorithms
- Training orchestration: `dialogue_kt/training.py`
- Evaluation & human-comparison: `dialogue_kt/human_eval.py`
- Results saved to `results/` and `saved_models/`.

D. Visualization

##### External Dependencies & Environment
- Python 3.10, libraries in `requirements.txt` (likely PyTorch/TensorFlow, pandas, matplotlib).
- External service: OpenAI API (environment variable `OPENAI_API_KEY`).
- Determinism config (`CUBLAS_WORKSPACE_CONFIG`).

##### Relationships & Data Flow
1. Raw data → (data_loading) → Annotation pipeline → Annotated data
2. Annotated data → (training module) → Model checkpoints → Results (metrics)
3. Checkpoints + results → Visualization → Plots in `results/`
4. CLI (`main.py`) orchestrates each stage

##### Architectural Patterns & Principles
- Modular layering: Data → Annotation → Modeling → Evaluation → Visualization
- Clear CLI façade (`main.py`) separating concerns
- Plugin-style models directory for easy addition of KT algorithms
- Configurable hyperparameter sweep and cross-validation support

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CLI Orchestrator
    subgraph "CLI Orchestrator"
        CLI["main.py CLI"]:::process
    end

    %% Data Layer
    subgraph "Data Layer" 
        direction TB
        RawData["Raw Data\\n(data/src/)"]:::data
        AnnotData["Annotated Data\\n(data/annotated/)"]:::data
    end

    %% Annotation Pipeline
    subgraph "Annotation Pipeline" 
        direction TB
        AnnotateCLI["annotate cmd\\non main.py"]:::process
        Orchestrator["annotate.py"]:::process
        Prompting["prompting.py"]:::process
        APIWrapper["openai_api.py"]:::process
        DataLoad["data_loading.py"]:::process
        KTDataLoad["kt_data_loading.py"]:::process
        Utils["utils.py"]:::process
    end

    %% External Service
    OpenAI["OpenAI API"]:::external

    %% Model Training & Evaluation
    subgraph "Model Training & Evaluation"
        direction TB
        TrainCLI["train/test cmd\\non main.py"]:::process
        Training["training.py"]:::process
        HumanEval["human_eval.py"]:::process
        subgraph "Models"
            direction TB
            LM["lm.py"]:::process
            DKTMulti["dkt_multi_kc.py"]:::process
            DKTSEM["dkt_sem.py"]:::process
            SimpleKT["simplekt.py"]:::process
        end
    end

    %% Visualization
    subgraph "Visualization Module"
        direction TB
        VizCLI["visualize cmd\\non main.py"]:::process
        Visualize["visualize.py"]:::process
    end

    %% Artifacts
    subgraph "Artifacts"
        direction TB
        Results["Results & Learning Curves\\n(results/)"]:::artifact
        SavedModels["Model Checkpoints\\n(saved_models/)"]:::artifact
        Plots["Generated Plots\\n(results/)"]:::artifact
    end

    %% Data Flow Connections
    RawData -->|load| DataLoad
    RawData -->|load| KTDataLoad
    DataLoad --> Orchestrator
    KTDataLoad --> Orchestrator
    Orchestrator --> Prompting
    Prompting --> APIWrapper
    APIWrapper --> OpenAI
    Orchestrator -->|annotate| AnnotData

    %% CLI to stages
    CLI -->|annotate| AnnotateCLI
    AnnotateCLI --> Orchestrator

    CLI -->|train/test| TrainCLI
    TrainCLI --> Training
    Training --> LM
    Training --> DKTMulti
    Training --> DKTSEM
    Training --> SimpleKT
    Training --> HumanEval
    HumanEval -->|save metrics| Results
    Training -->|save ckpt| SavedModels

    CLI -->|visualize| VizCLI
    VizCLI --> Visualize
    Visualize -->|plots| Plots

    AnnotData -->|train data| TrainCLI
    SavedModels -->|input| Visualize
    Results -->|input| Visualize
    Visualize -->|output| Plots

    %% Click Events
    click RawData "https://github.com/umass-ml4ed/dialogue-kt/tree/main/data/src/"
    click AnnotData "https://github.com/umass-ml4ed/dialogue-kt/tree/main/data/annotated/"
    click CLI "https://github.com/umass-ml4ed/dialogue-kt/blob/main/dialogue_kt/main.py"
    click AnnotateCLI "https://github.com/umass-ml4ed/dialogue-kt/blob/main/dialogue_kt/main.py"
    click Orchestrator "https://github.com/umass-ml4ed/dialogue-kt/blob/main/dialogue_kt/annotate.py"
    click Prompting "https://github.com/umass-ml4ed/dialogue-kt/blob/main/dialogue_kt/prompting.py"
    click APIWrapper "https://github.com/umass-ml4ed/dialogue-kt/blob/main/dialogue_kt/openai_api.py"
    click DataLoad "https://github.com/umass-ml4ed/dialogue-kt/blob/main/dialogue_kt/data_loading.py"
    click KTDataLoad "https://github.com/umass-ml4ed/dialogue-kt/blob/main/dialogue_kt/kt_data_loading.py"
    click Utils "https://github.com/umass-ml4ed/dialogue-kt/blob/main/dialogue_kt/utils.py"
    click TrainCLI "https://github.com/umass-ml4ed/dialogue-kt/blob/main/dialogue_kt/main.py"
    click LM "https://github.com/umass-ml4ed/dialogue-kt/blob/main/dialogue_kt/models/lm.py"
    click DKTMulti "https://github.com/umass-ml4ed/dialogue-kt/blob/main/dialogue_kt/models/dkt_multi_kc.py"
    click DKTSEM "https://github.com/umass-ml4ed/dialogue-kt/blob/main/dialogue_kt/models/dkt_sem.py"
    click SimpleKT "https://github.com/umass-ml4ed/dialogue-kt/blob/main/dialogue_kt/models/simplekt.py"
    click Training "https://github.com/umass-ml4ed/dialogue-kt/blob/main/dialogue_kt/training.py"
    click HumanEval "https://github.com/umass-ml4ed/dialogue-kt/blob/main/dialogue_kt/human_eval.py"
    click VizCLI "https://github.com/umass-ml4ed/dialogue-kt/blob/main/dialogue_kt/main.py"
    click Visualize "https://github.com/umass-ml4ed/dialogue-kt/blob/main/dialogue_kt/visualize.py"
    click Results "https://github.com/umass-ml4ed/dialogue-kt/tree/main/results/"
    click SavedModels "https://github.com/umass-ml4ed/dialogue-kt/tree/main/saved_models/"

    %% Styles
    classDef data fill:#bbdefb,stroke:#0d47a1,stroke-width:2px
    classDef process fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px
    classDef artifact fill:#ffe0b2,stroke:#ef6c00,stroke-width:2px
    classDef external fill:#e0e0e0,stroke:#616161,stroke-width:2px,stroke-dasharray: 5 5
```
