# ironbar/arc24 architecture

> Architecture diagram of the GitHub repository ironbar/arc24, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ironbar/arc24
- Repository: https://github.com/ironbar/arc24

## Overview

##### Project Type and Purpose
- This repository is designed for an AI challenge (Kaggle’s ARC Prize 2024) where the goal is to create an AI that can solve reasoning tasks that it has never seen before.
- The repository is structured as a research and experimentation platform combining data processing, model training (including fine-tuning, verification, and inference), and evaluation modules. It includes rich documentation (following CRISP-DM), Jupyter notebooks detailing experiments, and multiple scripts for processing and training.

##### Key Elements in the File Structure
a. Data Management:
- The “data” folder contains various datasets (original data, external data, reversed versions, and new partitions) that serve as inputs for processing and training.
- Subdirectories indicate data augmentation, transformation, and creation of specialized datasets (for example, “reversed_arc” and “new_partitions”).

b. Documentation & Iteration History:
- The “docs” folder contains documentation on business understanding, state-of-the-art research, and detailed records of iterative model improvements. This confirms an iterative R&D process.
- Modeling iterations (Iteration_01 to Iteration_50 and beyond) show how experimental results and improvements evolved, useful for understanding decision points.

c. Experimentation & Prototyping:
- The “notebooks” folder holds sequential Jupyter notebooks that record experiments, data visualizations, fine-tuning methods, and evaluation strategies.

d. Core Application Scripts:
- The “scripts” directory contains numerous Python (and one C++ script) modules for data processing, model training, fine-tuning, inference, evaluation, and even some tools for ensemble or verification of predictions.
- Scripts are named to reflect their purpose (e.g., fine-tuning.py, inference.py, grid_feature_extraction.cpp) which points to a modular design in processing tasks.

e. Testing, Deployment, and Configuration:
- The “tests” folder includes unit tests ensuring the components of the library work properly.
- Files such as Makefile, Dockerfile, requirements.txt, and environment.yml indicate care for reproducibility and deployment.

##### Recommended Components
- Data Layer:
- Raw data sources from “data/original_data,” “data/external_data,” and “data/reversed_arc.”
- Data partitions (“data/new_partitions”) indicating preprocessed datasets for training and evaluation.

- Data Processing/Preparation Module:
- Scripts responsible for data augmentation, transformation, and partition creation.
- Connections from raw data to preprocessed data (e.g., scripts/data_augmentation.py, create_n-1_dataset.py).

- Experimentation and Training Module:
- The iterative processes documented in “docs/modeling” and executed in “notebooks” to tweak parameters and improve the AI’s performance.
- Training related scripts (e.g., fine-tuning.py, long-context-fine-tuning.py, merge_lora.py) that illustrate model training and fine-tuning pipelines.

- Inference and Evaluation Module:
- Scripts like inference.py, evaluation.py, and select_predictions.py, which handle test-time model application and the evaluation of predictions.
- Tools for searching optimal predictions (e.g., grid search, beam search – refer to vllm_beam_search.py and grid-related scripts).

- Verification and Ensemble/Selection Logic:
- Scripts handling prediction verification (e.g., easy_verification_and_evaluation.py, verify_predictions.py) and ensemble strategies (voting.py).

- Integration & Deployment:
- Docker configuration and Makefile for ensuring that the environment remains consistent.
- Setup scripts, tests, and installation tools to support robust deployment.

- External Interfaces:
- Connection to Kaggle competitions (as indicated in the README) for submission of final predictions.
- Documentation and rule sets (“rules” directory) which may serve as external reference guides.

##### Architectural Patterns & Design Principles
- Modular Design: The repository clearly separates data handling, modeling, evaluation, and documentation.
- Iterative Experimentation: The modeling documentation and versioned notebooks reflect an iterative, agile approach.
- Reproducibility: Use of environment files, Docker, and testing modules to ensure that experiments can be repeated.
- Data Pipeline Flow: Clearly defined data ingestion, processing, training, and evaluation stages with feedback loops.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Layer"
        D1["Original Data"]:::data
        D2["External Data"]:::data
        D3["Reversed Data"]:::data
        D4["Data Partitions"]:::data
    end

    %% Data Processing/Preparation Module
    subgraph "Data Processing & Preparation"
        P1["Data Augmentation Script"]:::processing
        P2["Data Script"]:::processing
        P3["Dataset Creation Script"]:::processing
    end

    %% Experimentation & Training Module
    subgraph "Experimentation & Training"
        T1["Jupyter Notebooks"]:::training
        T2["Fine-tuning Script"]:::training
        T3["Long-context Fine-tuning Script"]:::training
        T4["Merge LoRA Script"]:::training
    end

    %% Inference & Evaluation Module
    subgraph "Inference & Evaluation"
        I1["Inference Script"]:::inference
        I2["Evaluation Script"]:::inference
        I3["Select Predictions Script"]:::inference
        I4["Easy Inference & Evaluation Script"]:::inference
    end

    %% Verification & Ensemble/Selection Module
    subgraph "Verification & Ensemble"
        V1["Verify Predictions Script"]:::verification
        V2["Easy Verification & Evaluation Script"]:::verification
        V3["Voting Ensemble Script"]:::verification
    end

    %% Integration & Deployment
    subgraph "Integration & Deployment"
        Dp1["Docker Configuration"]:::deployment
        Dp2["Makefile"]:::deployment
        Dp3["Environment Configuration"]:::deployment
        Dp4["Requirements File"]:::deployment
        Dp5["Testing Modules"]:::deployment
    end

    %% External Interfaces & Documentation
    subgraph "External Interfaces & Documentation"
        E1["Documentation & Iteration History"]:::external
        E2["Rules / External References"]:::external
        E3["Kaggle Submission Info"]:::external
    end

    %% Data Layer to Data Processing
    D1 --> P1
    D2 --> P1
    D3 --> P1
    D4 --> P1

    D1 --> P2
    D2 --> P2
    D3 --> P2
    D4 --> P2

    P1 --> P3
    P2 --> P3

    %% Data Processing to Experimentation & Training
    P3 --> T1

    %% Feedback from Documentation to Training
    E1 --- T1

    %% Experimentation & Training internal flow
    T1 --> T2
    T1 --> T3
    T1 --> T4

    %% Training to Inference & Evaluation
    T1 --> I1
    T2 --> I2
    T3 --> I3
    T4 --> I4

    %% Inference & Evaluation to Verification & Ensemble
    I1 --> V1
    I2 --> V2
    I3 --> V3
    I4 --> V2

    %% Verification & Ensemble to Integration & Deployment
    V1 --> Dp1
    V2 --> Dp1
    V3 --> Dp1

    %% Integration & Deployment internal flow
    Dp1 --> Dp2
    Dp1 --> Dp3
    Dp1 --> Dp4
    Dp1 --> Dp5

    %% External link to Deployment (Kaggle submission)
    E3 --- Dp1
    E2 --- E3

    %% Styles
    classDef data fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef processing fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef training fill:#FFFF99,stroke:#000,stroke-width:2px;
    classDef inference fill:#FFA07A,stroke:#000,stroke-width:2px;
    classDef verification fill:#DDA0DD,stroke:#000,stroke-width:2px;
    classDef deployment fill:#D3D3D3,stroke:#000,stroke-width:2px;
    classDef external fill:#87CEEB,stroke:#000,stroke-width:2px;

    %% Click Events for Data Layer
    click D1 "https://github.com/ironbar/arc24/tree/main/data/original_data"
    click D2 "https://github.com/ironbar/arc24/tree/main/data/external_data"
    click D3 "https://github.com/ironbar/arc24/tree/main/data/reversed_arc"
    click D4 "https://github.com/ironbar/arc24/tree/main/data/new_partitions"

    %% Click Events for Data Processing/Preparation Module
    click P1 "https://github.com/ironbar/arc24/blob/main/scripts/arc24/data_augmentation.py"
    click P2 "https://github.com/ironbar/arc24/blob/main/scripts/arc24/data.py"
    click P3 "https://github.com/ironbar/arc24/blob/main/scripts/create_n-1_dataset.py"

    %% Click Events for Experimentation & Training Module
    click T1 "https://github.com/ironbar/arc24/tree/main/notebooks/"
    click T2 "https://github.com/ironbar/arc24/blob/main/scripts/fine-tuning.py"
    click T3 "https://github.com/ironbar/arc24/blob/main/scripts/long-context-fine-tuning.py"
    click T4 "https://github.com/ironbar/arc24/blob/main/scripts/merge_lora.py"

    %% Click Events for Inference & Evaluation Module
    click I1 "https://github.com/ironbar/arc24/blob/main/scripts/inference.py"
    click I2 "https://github.com/ironbar/arc24/blob/main/scripts/evaluation.py"
    click I3 "https://github.com/ironbar/arc24/blob/main/scripts/select_predictions.py"
    click I4 "https://github.com/ironbar/arc24/blob/main/scripts/easy_inference_and_evaluation.py"

    %% Click Events for Verification & Ensemble/Selection Module
    click V1 "https://github.com/ironbar/arc24/blob/main/scripts/verify_predictions.py"
    click V2 "https://github.com/ironbar/arc24/blob/main/scripts/easy_verification_and_evaluation.py"
    click V3 "https://github.com/ironbar/arc24/blob/main/scripts/voting.py"

    %% Click Events for Integration & Deployment
    click Dp1 "https://github.com/ironbar/arc24/tree/main/docker/Dockerfile"
    click Dp2 "https://github.com/ironbar/arc24/tree/main/Makefile"
    click Dp3 "https://github.com/ironbar/arc24/blob/main/environment.yml"
    click Dp4 "https://github.com/ironbar/arc24/blob/main/requirements.txt"
    click Dp5 "https://github.com/ironbar/arc24/tree/main/tests/"

    %% Click Events for External Interfaces & Documentation
    click E1 "https://github.com/ironbar/arc24/tree/main/docs/"
    click E2 "https://github.com/ironbar/arc24/tree/main/rules/"
    click E3 "https://github.com/ironbar/arc24/blob/main/README.md"
```
