# uiuc-kang-lab/revisql architecture

> Architecture diagram of the GitHub repository uiuc-kang-lab/revisql, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/uiuc-kang-lab/revisql
- Repository: https://github.com/uiuc-kang-lab/revisql

## Overview

1. **Repository purpose**
   - This repo packages the full ReViSQL pipeline for text-to-SQL: dataset curation, RLVR fine-tuning with execution-based reward, and inference-time reconciliation/majority voting.
   - The primary execution path is: curated datasets + database schemas → training checkpoints → multi-candidate SQL generation → DB execution/evaluation → LLM-based reconciliation → final answered SQL.

2. **Top-level architectural split**
   - `data/` is the curated artifact layer: JSON/Parquet datasets plus schema SQL/metadata.
   - `data_scripts/` is the offline preprocessing layer that builds those artifacts.
   - `scripts/` contains the two operational entrypoints: training and inference.
   - `reconciliation/` is the post-generation decision layer used at inference time.
   - `tinker_cookbook/` is a bundled framework dependency that provides RL, supervised training, evaluation, tool-use, sandbox, and model-rendering infrastructure.

3. **Data layer and schema boundary**
   - The repo tracks a large set of task datasets in `data/`, including BIRD-Verified splits, BIRD evaluation sets, Spider 2-SQLite and Spider 2-Snow evaluation sets, curriculum stages, and various ablation subsets.
   - `data/ddls/` holds per-database schema definitions used to execute SQL against the correct backend.
   - For Spider 2-SQLite, the repo also includes `spider2sqlite.json` and `spider2sqlite_tables.json`, which act as schema metadata inputs for evaluation and execution.

4. **Data preparation pipeline**
   - `data_scripts/curate_final_data.py` is the main curation step that produces `bird-verified-train.parquet` and `val_test.parquet` from the verified JSON sources.
   - `curate_RL_dataset.py`, `curate_curriculum_data.py`, and the stage parquet files support earlier intermediate or curriculum-training variants.
   - `construct_db_schema.py` and `construct_spider2sqlite_db_schema.py` generate executable schema DDLs, while `generate_spider2sqlite_tables.py` builds the Spider table metadata file.
   - This preprocessing layer is offline and deterministic; it feeds the training and evaluation jobs but is not part of runtime inference logic.

5. **Training subsystem**
   - `scripts/train.py` is the main RLVR fine-tuning entrypoint.
   - It trains Qwen-family models with CISPO, using LoRA adapters, on `bird-verified-train.parquet` and validates on `val_test.parquet`.
   - The training loop depends on local database execution via `--db_path`, so the reward signal is grounded in SQL correctness against real DB backends rather than text matching.
   - Model checkpoints are written under `models/`, and the best checkpoint is selected by validation accuracy.

6. **Inference subsystem**
   - `scripts/infer.py` loads a trained checkpoint and generates multiple candidate SQL queries per question using greedy and/or temperature-based sampling.
   - The inference path is not a single-shot decode; it is a candidate-generation pipeline designed to support downstream grouping and reconciliation.
   - Outputs are written to an `output_path` for later grading, analysis, and reconciliation.
   - The same inference framework is used across BIRD, Spider 2-SQLite, and Spider 2-Snow by switching dataset and SQL engine configuration.

7. **Execution-based evaluation boundary**
   - The core correctness signal is execution against the target database, not just string equality.
   - This is why the repo needs both the question dataset and the matching per-database schema/DB files at runtime.
   - For Spider 2-Snow, the architecture extends to a Snowflake execution backend; for the others, it runs against SQLite-style local databases.
   - This execution layer is central to both training reward and inference evaluation.

8. **Reconciliation subsystem**
   - `reconciliation/` implements the paper’s inference-time “Decide” step: it inspects grouped candidate SQL queries and filters them by whether they satisfy the question constraints and any external knowledge hint.
   - `prompt.py` contains the prompt templates, while `reconciliation.py` contains the decision logic.
   - `run_reconciliation.py` and `run_reconciliation_detailed.py` are batch utilities for post-hoc decisioning and analysis.
   - This stage uses a pre-RLVR base model as a judge, so it is an external LLM service call rather than a local deterministic rule engine.

9. **Bundled framework: `tinker_cookbook/`**
   - This directory is a vendored ML/RL toolkit that supplies the reusable machinery for the project.
   - Important sublayers include:
     - `rl/` for rollout, message-environment, metrics, and training loops
     - `supervised/` for SFT-style training
     - `preference/` and `distillation/` for DPO and teacher/student workflows
     - `tool_use/`, `sandbox/`, and `recipes/` for environment-backed agent/tool execution
     - `renderers/` for model-specific chat/prompt formatting
     - `eval/` for evaluation utilities and inspectors
   - For ReViSQL, the key architectural point is that the project builds on this framework rather than implementing training/evaluation primitives from scratch.

10. **SQL-specific recipes and environments**
    - `tinker_cookbook/recipes/sql_rl/` contains SQL-oriented environments, grading, prompts, correction logic, and Snowflake helpers.
    - These modules are the bridge between generic RL infrastructure and the SQL execution domain.
    - They encapsulate database interaction, SQL grading, and correction workflows so the training loop can remain model-agnostic.
    - The presence of both `sql_env.py` and `sql_correction_env.py` indicates separate paths for raw SQL generation and correction-style interaction.

11. **External services and operational dependencies**
    - Training/evaluation logging uses Weights & Biases via `WANDB_API_KEY`.
    - Reconciliation uses external LLM providers through `TOGETHER_API_KEY`, with Groq as an optional alternative.
    - Snowflake credentials are needed only for the Spider 2-Snow path.
    - Local database artifacts are not stored in the repo; they must be mounted or downloaded and passed in via `--db_path`.

12. **How to diagram the system**
    - A useful architecture diagram would show four main flows:
      1. raw/corrected data and DDLs into data preparation,
      2. prepared parquet datasets into RL training,
      3. trained checkpoints into multi-sample inference,
      4. candidate SQL groups into reconciliation and final selection.
    - Surround these with the shared `tinker_cookbook/` framework as the training/evaluation substrate, and annotate the external dependencies on local databases, W&B, and Together/Snowflake.

Main entry points: [`scripts/train.py`](https://github.com/uiuc-kang-lab/revisql/blob/main/scripts/train.py) (Train), [`tinker_cookbook/recipes/sql_rl`](https://github.com/uiuc-kang-lab/revisql/tree/main/tinker_cookbook/recipes/sql_rl) (SQL RL), [`scripts/infer.py`](https://github.com/uiuc-kang-lab/revisql/blob/main/scripts/infer.py) (Infer), [`reconciliation/reconciliation.py`](https://github.com/uiuc-kang-lab/revisql/blob/main/reconciliation/reconciliation.py) (Reconcile)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data layer"]
  node_data_artifacts[("Curated data<br/>datasets/parquets")]
  node_schema_ddls["DB schemas<br/>ddl/sql"]
  node_spider_meta["Spider metadata<br/>schema json<br/>[spider2sqlite.json]"]
end

subgraph group_prep["Preparation"]
  node_curate_final["Final curation<br/>data script"]
  node_curate_rl["RL curation<br/>data script"]
  node_curriculum["Curriculum prep<br/>data script"]
  node_schema_builders["Schema builders<br/>data scripts"]
end

subgraph group_ops["Train & infer"]
  node_train_entry["Train<br/>rlvr entrypoint<br/>[train.py]"]
  node_infer_entry["Infer<br/>generation entrypoint<br/>[infer.py]"]
  node_recon_prompt["Judge prompt<br/>llm prompt<br/>[prompt.py]"]
  node_recon_logic["Reconcile<br/>llm judge<br/>[reconciliation.py]"]
  node_recon_batch["Batch reconcile<br/>postprocess"]
end

subgraph group_fw["Framework"]
  node_sql_rl["SQL RL<br/>recipe/substrate"]
  node_rl_core["RL core<br/>framework"]
  node_supervised["Supervised<br/>framework"]
  node_renderers["Renderers<br/>framework"]
  node_tool_use["Tool use<br/>framework"]
  node_eval_fw["Eval utils<br/>framework"]
end

node_wandb(("W&B<br/>external service"))
node_together(("LLM API<br/>external service"))
node_snowflake[("Snowflake<br/>external backend")]

node_data_artifacts -->|"source data"| node_curate_final
node_schema_ddls -->|"schemas"| node_curate_final
node_spider_meta -->|"metadata"| node_schema_builders
node_schema_builders -->|"generate"| node_schema_ddls
node_curate_rl -->|"variants"| node_data_artifacts
node_curriculum -->|"stages"| node_data_artifacts
node_curate_final -->|"train/val parquet"| node_train_entry
node_sql_rl -->|"SQL env"| node_train_entry
node_rl_core -->|"rollouts"| node_train_entry
node_supervised -->|"data/utilities"| node_train_entry
node_renderers -->|"chat format"| node_train_entry
node_tool_use -->|"tooling"| node_sql_rl
node_train_entry -.->|"logs"| node_wandb
node_train_entry -->|"checkpoint"| node_infer_entry
node_infer_entry -->|"execution"| node_sql_rl
node_infer_entry -->|"candidate groups"| node_recon_batch
node_recon_batch -->|"batch jobs"| node_recon_logic
node_recon_logic -->|"prompting"| node_recon_prompt
node_recon_logic -.->|"judge calls"| node_together
node_sql_rl -.->|"SnowSQL path"| node_snowflake
node_infer_entry -.->|"Spider 2-Snow eval"| node_snowflake
node_sql_rl -->|"grading"| node_eval_fw

click node_data_artifacts "https://github.com/uiuc-kang-lab/revisql/tree/main/data"
click node_schema_ddls "https://github.com/uiuc-kang-lab/revisql/tree/main/data/ddls"
click node_spider_meta "https://github.com/uiuc-kang-lab/revisql/blob/main/data/spider2sqlite.json"
click node_curate_final "https://github.com/uiuc-kang-lab/revisql/blob/main/data_scripts/curate_final_data.py"
click node_curate_rl "https://github.com/uiuc-kang-lab/revisql/blob/main/data_scripts/curate_RL_dataset.py"
click node_curriculum "https://github.com/uiuc-kang-lab/revisql/blob/main/data_scripts/curate_curriculum_data.py"
click node_schema_builders "https://github.com/uiuc-kang-lab/revisql/blob/main/data_scripts/construct_db_schema.py"
click node_train_entry "https://github.com/uiuc-kang-lab/revisql/blob/main/scripts/train.py"
click node_infer_entry "https://github.com/uiuc-kang-lab/revisql/blob/main/scripts/infer.py"
click node_recon_prompt "https://github.com/uiuc-kang-lab/revisql/blob/main/reconciliation/prompt.py"
click node_recon_logic "https://github.com/uiuc-kang-lab/revisql/blob/main/reconciliation/reconciliation.py"
click node_recon_batch "https://github.com/uiuc-kang-lab/revisql/blob/main/reconciliation/run_reconciliation.py"
click node_sql_rl "https://github.com/uiuc-kang-lab/revisql/tree/main/tinker_cookbook/recipes/sql_rl"
click node_rl_core "https://github.com/uiuc-kang-lab/revisql/tree/main/tinker_cookbook/rl"
click node_supervised "https://github.com/uiuc-kang-lab/revisql/tree/main/tinker_cookbook/supervised"
click node_renderers "https://github.com/uiuc-kang-lab/revisql/tree/main/tinker_cookbook/renderers"
click node_tool_use "https://github.com/uiuc-kang-lab/revisql/tree/main/tinker_cookbook/tool_use"
click node_eval_fw "https://github.com/uiuc-kang-lab/revisql/tree/main/tinker_cookbook/eval"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_data_artifacts,node_schema_ddls,node_spider_meta toneBlue
class node_curate_final,node_curate_rl,node_curriculum,node_schema_builders toneAmber
class node_train_entry,node_infer_entry,node_recon_prompt,node_recon_logic,node_recon_batch toneMint
class node_sql_rl,node_rl_core,node_supervised,node_renderers,node_tool_use,node_eval_fw toneRose
class node_wandb,node_together,node_snowflake toneNeutral
```

## Components

### Data layer

Curated datasets, schemas, and evaluation artifacts

- [Curated data](https://github.com/uiuc-kang-lab/revisql/tree/main/data): datasets/parquets · Task datasets, curriculum stages, evaluation splits, and ablations · `data`
- [DB schemas](https://github.com/uiuc-kang-lab/revisql/tree/main/data/ddls): ddl/sql · Per-database executable schema definitions for local SQL backends · `data/ddls`
- [Spider metadata](https://github.com/uiuc-kang-lab/revisql/blob/main/data/spider2sqlite.json): schema json · Spider 2-SQLite table and schema metadata inputs · `data/spider2sqlite.json`

### Preparation

Offline curation and schema construction

- [Final curation](https://github.com/uiuc-kang-lab/revisql/blob/main/data_scripts/curate_final_data.py): data script · Builds the main RLVR training and validation parquet sets · `data_scripts/curate_final_data.py`
- [RL curation](https://github.com/uiuc-kang-lab/revisql/blob/main/data_scripts/curate_RL_dataset.py): data script · Prepares RL-oriented dataset variants · `data_scripts/curate_RL_dataset.py`
- [Curriculum prep](https://github.com/uiuc-kang-lab/revisql/blob/main/data_scripts/curate_curriculum_data.py): data script · Constructs stage-wise curriculum datasets · `data_scripts/curate_curriculum_data.py`
- [Schema builders](https://github.com/uiuc-kang-lab/revisql/blob/main/data_scripts/construct_db_schema.py): data scripts · Generate executable DDLs and Spider table metadata · `data_scripts/construct_db_schema.py`

### Train & infer

Operational entrypoints and post-generation reconciliation

- [Train](https://github.com/uiuc-kang-lab/revisql/blob/main/scripts/train.py): rlvr entrypoint · Main CISPO/LoRA fine-tuning job for Qwen-family models · `scripts/train.py`
- [Infer](https://github.com/uiuc-kang-lab/revisql/blob/main/scripts/infer.py): generation entrypoint · Multi-sample SQL candidate generation for evaluation and reconciliation · `scripts/infer.py`
- [Judge prompt](https://github.com/uiuc-kang-lab/revisql/blob/main/reconciliation/prompt.py): llm prompt · Prompt templates for reconciliation decisions · `reconciliation/prompt.py`
- [Reconcile](https://github.com/uiuc-kang-lab/revisql/blob/main/reconciliation/reconciliation.py): llm judge · Decision logic that filters grouped SQL candidates · `reconciliation/reconciliation.py`
- [Batch reconcile](https://github.com/uiuc-kang-lab/revisql/blob/main/reconciliation/run_reconciliation.py): postprocess · Batch utilities for reconciliation runs and analysis · `reconciliation/run_reconciliation.py`

### Framework

Vendored training, evaluation, and SQL RL substrate

- [SQL RL](https://github.com/uiuc-kang-lab/revisql/tree/main/tinker_cookbook/recipes/sql_rl): recipe/substrate · SQL-specific RL environments, grading, correction, and Snowflake helpers · `tinker_cookbook/recipes/sql_rl`
- [RL core](https://github.com/uiuc-kang-lab/revisql/tree/main/tinker_cookbook/rl): framework · Rollouts, message environments, metrics, and training loops · `tinker_cookbook/rl`
- [Supervised](https://github.com/uiuc-kang-lab/revisql/tree/main/tinker_cookbook/supervised): framework · SFT data, evaluators, and training utilities · `tinker_cookbook/supervised`
- [Renderers](https://github.com/uiuc-kang-lab/revisql/tree/main/tinker_cookbook/renderers): framework · Model-specific chat and prompt formatting · `tinker_cookbook/renderers`
- [Tool use](https://github.com/uiuc-kang-lab/revisql/tree/main/tinker_cookbook/tool_use): framework · Agent/tool execution abstractions · `tinker_cookbook/tool_use`
- [Eval utils](https://github.com/uiuc-kang-lab/revisql/tree/main/tinker_cookbook/eval): framework · Evaluation and inspection helpers · `tinker_cookbook/eval`

### Other components

- **W&B**: external service · Training and evaluation logging
- **LLM API**: external service · External judge model provider for reconciliation
- **Snowflake**: external backend · Execution backend for Spider 2-Snow

## Connections

- Curated data → Final curation: source data
- DB schemas → Final curation: schemas
- Spider metadata → Schema builders: metadata
- Schema builders → DB schemas: generate
- RL curation → Curated data: variants
- Curriculum prep → Curated data: stages
- Final curation → Train: train/val parquet
- SQL RL → Train: SQL env
- RL core → Train: rollouts
- Supervised → Train: data/utilities
- Renderers → Train: chat format
- Tool use → SQL RL: tooling
- Train → W&B: logs
- Train → Infer: checkpoint
- Infer → SQL RL: execution
- Infer → Batch reconcile: candidate groups
- Batch reconcile → Reconcile: batch jobs
- Reconcile → Judge prompt: prompting
- Reconcile → LLM API: judge calls
- SQL RL → Snowflake: SnowSQL path
- Infer → Snowflake: Spider 2-Snow eval
- SQL RL → Eval utils: grading
