# dtsip/in-context-learning architecture

> Architecture diagram of the GitHub repository dtsip/in-context-learning, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dtsip/in-context-learning
- Repository: https://github.com/dtsip/in-context-learning

## Overview

##### Project Type & Purpose
- This is a research codebase supporting experiments for the paper “What Can Transformers Learn In-Context? A Case Study of Simple Function Classes”.
- Its primary components include model training, evaluation, configuration management, and plotting of results—typical of experimental ML research codebases.

##### Key Components
a. Configuration Management:
- Configuration files located in src/conf (e.g., base.yaml, decision_tree.yaml, linear_regression.yaml, toy.yaml, wandb.yaml, etc.).
- How these YAML files are consumed by train.py (and possibly eval.py) to determine experiment settings.

b. Training Pipeline:
- Main training entry point (train.py).
- That train.py uses configuration parameters to set up models (via src/models.py and src/base_models.py) and training routines.
- Role of curriculum.py and tasks.py which likely define experiment protocols or training curricula.

c. Model Definitions & Sampling:
- Modules such as models.py and base_models.py, which encapsulate the definition of learning models.
- Samplers.py as part of the components to generate or process training data.

d. Evaluation & Plotting:
- Evaluation pipeline with eval.py and the Jupyter notebook (eval.ipynb), which load pre-trained checkpoints (downloaded separately) for assessing model performance.

e. External Services & Dependencies:
- Integration with WandB through the wandb.yaml file – this implies an external experiment logging/monitoring framework.
- Optionally, indicate dependencies installed via the environment.yml file (via Conda), showing how the environment is set up for the whole experiment lifecycle.

##### Relationships and Interactions
- Configuration Files (YAMLs in src/conf) feed into both train.py and eval.py.
- train.py directs the training process by instantiating models (from models.py/base_models.py), fetching experiment details (from tasks.py and curriculum.py), and possibly interacting with external logging services (WandB).
- The evaluation components (eval.py and eval.ipynb) take as input the trained models and checkpoints (downloaded externally) to produce evaluation metrics using the plotting utilities.
- Out a feedback loop where the outcomes from evaluation might lead to further training iterations or hyperparameter tuning.

##### Architectural Patterns & Design Principles
- The project is modular in nature:
- Separation of configuration (conf directory) is a common design choice for research experiments.
- There’s a clear separation of concerns: model definition, training mechanics, evaluation, and plotting are all handled in different modules.
- Loose-coupling between configuration and the core algorithmic components (e.g., modifying a YAML file changes training behavior without touching the algorithm code).

##### Technology, Frameworks, and External Components
- Mention that the project leverages Python with YAML-based configuration.
- Frameworks such as WandB for experiment tracking should be noted.
- Since it is research-focused, treat the Jupyter notebook (eval.ipynb) as an entry point for interactive evaluation and visualization.

## Diagram (Mermaid)

```mermaid
flowchart TD
    ENV("Conda Env (environment.yml)"):::env
    Config("Configuration Manager (src/conf)"):::conf

    subgraph "Training Pipeline"
        Training("Training Engine (src/train.py)"):::train
        Model("Model Definitions: models.py"):::model
        BaseModel("Model Definitions: base_models.py"):::model
        Curriculum("Curriculum (src/curriculum.py)"):::util
        Tasks("Tasks (src/tasks.py)"):::util
        Sampler("Data Sampler (src/samplers.py)"):::util
        WandB("WandB Integration (src/conf/wandb.yaml)"):::external
    end

    subgraph "Evaluation Pipeline"
        EvalScript("Evaluation Script (src/eval.py)"):::eval
        EvalNotebook("Evaluation Notebook (src/eval.ipynb)"):::eval
        Plotter("Plotting Utilities (src/plot_utils.py)"):::util
    end

    %% Connections
    ENV -->|"env_setup"| Config
    Config -->|"feeds_training"| Training
    Config -->|"feeds_evaluation"| EvalScript
    Training -->|"uses"| Model
    Training -->|"uses"| BaseModel
    Training -->|"uses"| Curriculum
    Training -->|"uses"| Tasks
    Training -->|"uses"| Sampler
    Training -->|"logs_to"| WandB
    Training -->|"produces_ckpt"| EvalScript
    Training -->|"produces_ckpt"| EvalNotebook
    EvalScript -->|"generates"| Plotter
    EvalNotebook -->|"analyses_with"| Plotter

    %% Click Events
    click ENV "https://github.com/dtsip/in-context-learning/blob/main/environment.yml"
    click Config "https://github.com/dtsip/in-context-learning/tree/main/src/conf"
    click Training "https://github.com/dtsip/in-context-learning/blob/main/src/train.py"
    click Model "https://github.com/dtsip/in-context-learning/blob/main/src/models.py"
    click BaseModel "https://github.com/dtsip/in-context-learning/blob/main/src/base_models.py"
    click Curriculum "https://github.com/dtsip/in-context-learning/blob/main/src/curriculum.py"
    click Tasks "https://github.com/dtsip/in-context-learning/blob/main/src/tasks.py"
    click Sampler "https://github.com/dtsip/in-context-learning/blob/main/src/samplers.py"
    click WandB "https://github.com/dtsip/in-context-learning/blob/main/src/conf/wandb.yaml"
    click EvalScript "https://github.com/dtsip/in-context-learning/blob/main/src/eval.py"
    click EvalNotebook "https://github.com/dtsip/in-context-learning/blob/main/src/eval.ipynb"
    click Plotter "https://github.com/dtsip/in-context-learning/blob/main/src/plot_utils.py"

    %% Styles
    classDef env fill:#f9c74f,stroke:#333,stroke-width:2px;
    classDef conf fill:#90be6d,stroke:#333,stroke-width:2px;
    classDef train fill:#f94144,stroke:#333,stroke-width:2px;
    classDef model fill:#577590,stroke:#333,stroke-width:2px;
    classDef util fill:#f8961e,stroke:#333,stroke-width:2px;
    classDef eval fill:#277da1,stroke:#333,stroke-width:2px;
    classDef external fill:#8d99ae,stroke:#333,stroke-width:2px;
```
