# dinobby/reconcile architecture

> Architecture diagram of the GitHub repository dinobby/reconcile, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dinobby/reconcile
- Repository: https://github.com/dinobby/reconcile

## Overview

##### Project Type and Purpose
- This is a Python-based research framework (no GUI/frontend) for multi-round discussion and consensus answering across diverse LLMs (OpenAI/GPT, PaLM, Claude).
- It takes benchmark question sets, runs each question through a round-table process among the LLMs, aggregates a consensus answer, and saves both raw and consensus answers for later analysis.

##### File Structure Analysis
Top level:
- run.py          – CLI entry point that orchestrates the experiment.
- generation.py   – Core “round-table” discussion logic and API orchestration.
- data_utils.py   – Loading/parsing datasets.
- utils.py        – Utility helpers (e.g., consensus scoring, JSON read/write).
- claude.py       – Wrapper around Claude-API authentication & request logic.
- requirements.txt/.env – Dependencies and credential placeholders.
- convincing/     – Output storage: per-dataset, per-LLM JSON results and consensus files.
- dataset/        – Input test/development data for each benchmark.

##### Key README Insights
- Uses Python 3.10, external LLM services (OpenAI, PaLM, Claude via Claude-API).
- Expects an .env with API keys/cookies.
- CLI flags: `--num_samples` and `--dataset` to choose subset and benchmark.

a. Main Components
1. CLI Runner (run.py)
2. Dataset Loader (data_utils.py)
3. Discussion Coordinator (generation.py)
4. LLM Service Clients
- OpenAI Client (via openai library)
- PaLM Client (via Google PaLM API)
- Claude Client (claude.py + Claude-API wrapper)
5. Consensus & Scoring Utils (utils.py)
6. Results Store (convincing/ folder)

b. Relationships & Data Flow
1. User invokes run.py → parses arguments
2. run.py calls data_utils to load questions
3. For each question, run.py invokes generation.py
4. generation.py loops through discussion rounds:
– Sends prompts to each LLM client → receives individual responses
– Applies utilities to compare, score, and vote
– Builds a consensus answer after X rounds
5. generation.py calls utils to serialize both raw and consensus outputs to JSON under convincing/<Dataset>/
6. run.py optionally aggregates final metrics

c. Architectural Patterns & Principles
– Pipeline/Scripting Architecture: clear sequential stages (load → discuss → aggregate → store)
– Modularity & Single Responsibility: each .py handles a dedicated concern
– Adapter Pattern: separate client wrappers for each external LLM API
– Pluggability: adding another model requires minimal changes (new client + config)

d. Key Technologies & Libraries
– Python 3.10
– openai (OpenAI API)
– google-generativeai (PaLM)
– Claude-API wrapper (claude.py)
– pandas or built-in json/csv modules for data handling

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Configuration"
        Env[".env"]:::config
        Req["requirements.txt"]:::config
    end

    Env -->|"Load API keys"| Runner
    Req -->|"Install deps"| Runner

    Runner["run.py\nCLI Runner"]:::core
    Loader["data_utils.py\nDataset Loader"]:::core
    Coordinator["generation.py\nDiscussion Coordinator"]:::core
    Utils["utils.py\nConsensus & Scoring Utils"]:::util
    ClaudeAdapter["claude.py\nClaude Client Adapter"]:::core

    Dataset["dataset/"]:::storage
    Store["convincing/"]:::storage

    subgraph "Round-Table Loop"
        RoundLoop(("Rounds Loop")):::core
        OpenAI["OpenAI API"]:::external
        PaLM["PaLM API"]:::external
        ClaudeAPI["Claude API"]:::external

        RoundLoop -->|"Prompt"| OpenAI
        RoundLoop -->|"Prompt"| PaLM
        RoundLoop -->|"Prompt"| ClaudeAdapter
        OpenAI -->|"Response"| RoundLoop
        PaLM -->|"Response"| RoundLoop
        ClaudeAdapter -->|"Call API"| ClaudeAPI
        ClaudeAPI -->|"Response"| ClaudeAdapter
        RoundLoop -->|"Raw responses"| Utils
    end

    Runner -->|"parse flags"| Loader
    Loader -->|"Questions JSON"| Coordinator
    Coordinator -->|"load dataset"| Dataset
    Dataset --> Coordinator
    Coordinator --> RoundLoop
    RoundLoop -->|"Consensus data"| Utils
    Utils -->|"Write JSON"| Store
    Runner -->|"aggregate metrics"| Utils

    click Runner "https://github.com/dinobby/reconcile/blob/main/run.py"
    click Loader "https://github.com/dinobby/reconcile/blob/main/data_utils.py"
    click Coordinator "https://github.com/dinobby/reconcile/blob/main/generation.py"
    click Utils "https://github.com/dinobby/reconcile/blob/main/utils.py"
    click ClaudeAdapter "https://github.com/dinobby/reconcile/blob/main/claude.py"
    click Store "https://github.com/dinobby/reconcile/tree/main/convincing/"
    click Dataset "https://github.com/dinobby/reconcile/tree/main/dataset/"
    click Req "https://github.com/dinobby/reconcile/blob/main/requirements.txt"
    click Env "https://github.com/dinobby/reconcile/blob/main/.env"

    classDef core fill:#D0E6FF,stroke:#0366d6,color:#0366d6;
    classDef external fill:#E6FFEA,stroke:#28a745,color:#28a745,stroke-dasharray: 5 5;
    classDef storage fill:#FFF4E6,stroke:#D97706,color:#D97706,stroke-width:2px;
    classDef util fill:#F3F4F6,stroke:#6B7280,color:#6B7280,stroke-dasharray: 2 2;
    classDef config fill:#FFE6F0,stroke:#DB2777,color:#DB2777,stroke-width:2px;
```
