# rasahq/tod-in-context-learning architecture

> Architecture diagram of the GitHub repository rasahq/tod-in-context-learning, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rasahq/tod-in-context-learning
- Repository: https://github.com/rasahq/tod-in-context-learning

## Overview

##### Project type and purpose
- This is a research codebase implementing and evaluating a task-oriented dialogue bot built on top of Rasa Pro. It provides three “versions” (v1, v2, v3) of the bot:
- version_1: a CALM (LLM-powered) implementation using in-context learning.
- version_2: an intent-based minimal baseline.
- version_3: an intent-based, larger-budget baseline.
- The project’s primary purpose is to train and end-to-end test these bots against a common suite of YAML-defined scenarios and compare performance for an academic paper.

##### File structure
- Top-level:
- e2e_tests/ : YAML conversations categorized by behavior (happy_path, corrections, cancellations, etc.).
- version_1/, version_2/, version_3/: three independent bot implementations, each with its own Rasa domain, data, actions, DB JSON files, and config.
- scripts/: result-processing scripts (combine_results.py, count_lines.py, describe_tests.py).
- test_results/: raw text outputs of each test suite run.
- Makefile, run_tests.sh: orchestration of install, train, and test commands.
- Each version directory:
- data/: NLU/training data, rules, stories (v2 & v3 use standard Rasa data layout; v1 uses custom .yml flows).
- actions/: custom action server modules that interact with DB JSON files, external services.
- domain/: slot/intents/entities definitions.
- endpoints.yml: Rasa endpoints (action server).
- External-service stubs/config:
- LLM API (OpenAI or alternative) in version_1/config.yml & action code.
- Duckling server URL in check_restaurant_availability.py.

##### Extract additional insights from the README
- A Rasa Pro license is required to run the code.
- Version_1 optionally calls out to OpenAI via env var OPENAI_API_KEY.
- Training and testing are invoked via `make train-version version=<n>` and `make test-version version=<n>`.
- Results are collected, then processed into `line-counts.json` and text files under test_results.

##### Final pointers
- Be explicit about version isolation: each version runs in its own container or process with its own config.yml, domain, data and endpoints.
- Flow of test data from YAML → bot → log → report.
- Don’t forget environment variables (RASA_PRO_LICENSE, OPENAI_API_KEY) as enablers for certain paths.
- Keep component labels high-level, but annotate subfolders (actions/, data/, domain/) where helpful.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Orchestration Layer
    subgraph "Orchestration" 
        direction TB
        E2E["E2E Test Definitions"]:::orchestration
        TH["Test Harness"]:::orchestration
    end

    %% Dialogue Engines
    subgraph "Version 1: CALM (LLM-powered)" 
        direction TB
        R1["Rasa Server v1"]:::dialogue
        A1["Action Server v1"]:::dialogue
        DB1["JSON DB v1"]:::data
    end

    subgraph "Version 2: Minimal Intent" 
        direction TB
        R2["Rasa Server v2"]:::dialogue
        A2["Action Server v2"]:::dialogue
        DB2["JSON DB v2"]:::data
    end

    subgraph "Version 3: Large Intent" 
        direction TB
        R3["Rasa Server v3"]:::dialogue
        A3["Action Server v3"]:::dialogue
        DB3["JSON DB v3"]:::data
    end

    %% External Services
    LLM["OpenAI LLM API"]:::external
    Duck["Duckling Service"]:::external

    %% Reporter & Result Collector
    subgraph "Reporter & Result Collector"
        direction TB
        Raw["Raw Test Outputs"]:::data
        ScriptsCombine["combine_results.py"]:::orchestration
        ScriptsCount["count_lines.py"]:::orchestration
        ScriptsDesc["describe_tests.py"]:::orchestration
        Metrics["line_counts.json"]:::data
    end

    %% Flows
    E2E -->|"YAML scenarios"| TH
    TH -->|"HTTP POST /webhooks/rest/webhook"| R1
    TH -->|"HTTP POST /webhooks/rest/webhook"| R2
    TH -->|"HTTP POST /webhooks/rest/webhook"| R3

    R1 -->|"predict action"| A1
    A1 -->|"read/write JSON"| DB1
    A1 -.->|"LLM API request"| LLM
    A1 -.->|"Duckling request"| Duck
    A1 -->|"return event"| R1
    R1 -->|"HTTP response"| TH

    R2 -->|"predict action"| A2
    A2 -->|"read/write JSON"| DB2
    A2 -->|"return event"| R2
    R2 -->|"HTTP response"| TH

    R3 -->|"predict action"| A3
    A3 -->|"read/write JSON"| DB3
    A3 -->|"return event"| R3
    R3 -->|"HTTP response"| TH

    TH -->|"raw logs"| Raw
    Raw --> ScriptsCombine
    Raw --> ScriptsCount
    Raw --> ScriptsDesc
    ScriptsCombine -->|"metrics"| Metrics
    ScriptsCount -->|"metrics"| Metrics
    ScriptsDesc -->|"metrics"| Metrics

    %% Click Events
    click E2E "https://github.com/rasahq/tod-in-context-learning/tree/main/./e2e_tests/"
    click TH "https://github.com/rasahq/tod-in-context-learning/tree/main/./Makefile"
    click TH "https://github.com/rasahq/tod-in-context-learning/blob/main/./run_tests.sh"
    click Raw "https://github.com/rasahq/tod-in-context-learning/tree/main/./test_results/"
    click ScriptsCombine "https://github.com/rasahq/tod-in-context-learning/blob/main/./scripts/combine_results.py"
    click ScriptsCount "https://github.com/rasahq/tod-in-context-learning/blob/main/./scripts/count_lines.py"
    click ScriptsDesc "https://github.com/rasahq/tod-in-context-learning/blob/main/./scripts/describe_tests.py"
    click Metrics "https://github.com/rasahq/tod-in-context-learning/blob/main/./line_counts.json"
    click R1 "https://github.com/rasahq/tod-in-context-learning/blob/main/version_1/config.yml"
    click R1 "https://github.com/rasahq/tod-in-context-learning/tree/main/version_1/data/flows/"
    click R1 "https://github.com/rasahq/tod-in-context-learning/tree/main/version_1/domain/"
    click A1 "https://github.com/rasahq/tod-in-context-learning/tree/main/version_1/actions/"
    click DB1 "https://github.com/rasahq/tod-in-context-learning/tree/main/version_1/db/"
    click R1 "https://github.com/rasahq/tod-in-context-learning/blob/main/version_1/endpoints.yml"
    click R2 "https://github.com/rasahq/tod-in-context-learning/blob/main/version_2/config.yml"
    click R2 "https://github.com/rasahq/tod-in-context-learning/tree/main/version_2/data/nlu/"
    click R2 "https://github.com/rasahq/tod-in-context-learning/tree/main/version_2/data/rules/"
    click R2 "https://github.com/rasahq/tod-in-context-learning/tree/main/version_2/domain/"
    click A2 "https://github.com/rasahq/tod-in-context-learning/tree/main/version_2/actions/"
    click DB2 "https://github.com/rasahq/tod-in-context-learning/tree/main/version_2/db/"
    click R2 "https://github.com/rasahq/tod-in-context-learning/blob/main/version_2/endpoints.yml"
    click R2 "https://github.com/rasahq/tod-in-context-learning/tree/main/version_2/.rasa/cache/"
    click R3 "https://github.com/rasahq/tod-in-context-learning/blob/main/version_3/config.yml"
    click R3 "https://github.com/rasahq/tod-in-context-learning/tree/main/version_3/data/nlu/"
    click R3 "https://github.com/rasahq/tod-in-context-learning/tree/main/version_3/data/rules/"
    click R3 "https://github.com/rasahq/tod-in-context-learning/tree/main/version_3/data/stories/"
    click R3 "https://github.com/rasahq/tod-in-context-learning/tree/main/version_3/domain/"
    click A3 "https://github.com/rasahq/tod-in-context-learning/tree/main/version_3/actions/"
    click DB3 "https://github.com/rasahq/tod-in-context-learning/tree/main/version_3/db/"
    click R3 "https://github.com/rasahq/tod-in-context-learning/blob/main/version_3/endpoints.yml"

    %% Styles
    classDef orchestration fill:#FFA500,stroke:#333,stroke-width:1px
    classDef dialogue fill:#1E90FF,stroke:#333,stroke-width:1px
    classDef data fill:#32CD32,stroke:#333,stroke-width:1px
    classDef external fill:#9370DB,stroke:#333,stroke-width:1px
```
