# hkustdial/nl2sql360 architecture

> Architecture diagram of the GitHub repository hkustdial/nl2sql360, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hkustdial/nl2sql360
- Repository: https://github.com/hkustdial/nl2sql360

## Overview

NL2SQL360 is a testbed for importing NL2SQL benchmarks, evaluating model predictions, and querying fine-grained performance. The main workflow starts with a user invoking the CLI, parses command-specific configuration, imports dataset samples and SQL features, evaluates predictions against benchmark databases, stores results, and produces reports. Its evaluation surface includes Spider and BIRD metrics; reporting also supports query-feature scenarios and domain breakdowns. The sampled core and Spider evaluator clarify some wiring; several BIRD metric implementations and report internals are represented at their documented capability level.

Main entry points: [`src/nl2sql360/core/core.py`](https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/core/core.py) (Evaluation Core), [`src/nl2sql360/evaluator/bird_eval`](https://github.com/hkustdial/nl2sql360/tree/master/src/nl2sql360/evaluator/bird_eval) (BIRD Evaluation), [`src/nl2sql360/cli/util.py`](https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/cli/util.py) (Command Handlers), [`src/nl2sql360/cli/cli.py`](https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/cli/cli.py) (CLI Commands)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workflow["User Workflow"]
  node_cli["CLI Commands<br/>[cli.py]"]
  node_cli_routes["Command Handlers<br/>[util.py]"]
  node_argument_parser["Config Parsing<br/>[parser.py]"]
  node_core["Evaluation Core<br/>[core.py]"]
end

subgraph group_data["Dataset And State"]
  node_dataset["Dataset Loader<br/>[dataset.py]"]
  node_sql_parser["SQL Feature Parser<br/>[sql_parser.py]"]
  node_database_model["Result Models<br/>[model.py]"]
  node_result_store[("Evaluation Store<br/>[core.py]")]
end

subgraph group_evaluation["Evaluation Metrics"]
  node_spider_metrics["Spider EX And EM<br/>[spider_ex_em.py]"]
  node_spider_suite["Spider Test Suite<br/>[evaluation.py]"]
  node_bird_eval["BIRD Evaluation"]
  node_bird_f1["Soft F1<br/>[evaluation_f1.py]"]
  node_bird_ves["Valid Efficiency"]
  node_bird_rves["Reward Efficiency"]
end

subgraph group_analysis["Performance Analysis"]
  node_filter["Scenario Filters<br/>[filter.py]"]
  node_reporting["Performance Reports<br/>[core.py]"]
  node_history["History Deletion<br/>[core.py]"]
end

node_user(("Researcher"))

node_user -->|"invokes"| node_cli
node_cli -->|"dispatches"| node_cli_routes
node_cli_routes -->|"parses config"| node_argument_parser
node_cli_routes -->|"runs operation"| node_core
node_core -->|"loads samples"| node_dataset
node_core -->|"extracts features"| node_sql_parser
node_core -->|"creates tables"| node_database_model
node_core -->|"reads and writes"| node_result_store
node_core -->|"invokes evaluation"| node_spider_metrics
node_spider_metrics -->|"evaluates SQL"| node_spider_suite
node_core -.->|"runs metrics"| node_bird_eval
node_bird_eval -->|"includes"| node_bird_f1
node_bird_eval -->|"includes"| node_bird_ves
node_bird_eval -->|"includes"| node_bird_rves
node_core -->|"uses scenarios"| node_filter
node_core -->|"queries results"| node_reporting
node_core -->|"deletes history"| node_history

click node_cli "https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/cli/cli.py"
click node_cli_routes "https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/cli/util.py"
click node_argument_parser "https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/arguments/parser.py"
click node_core "https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/core/core.py"
click node_dataset "https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/dataset/dataset.py"
click node_sql_parser "https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/parser/sql_parser.py"
click node_database_model "https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/database/model.py"
click node_result_store "https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/core/core.py"
click node_spider_metrics "https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/evaluator/spider_ex_em.py"
click node_spider_suite "https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/evaluator/test_suite_sql_eval/evaluation.py"
click node_bird_eval "https://github.com/hkustdial/nl2sql360/tree/master/src/nl2sql360/evaluator/bird_eval"
click node_bird_f1 "https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/evaluator/bird_eval/evaluation_f1.py"
click node_filter "https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/filter/filter.py"
click node_reporting "https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/core/core.py"
click node_history "https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/core/core.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_cli_routes,node_argument_parser,node_core toneBlue
class node_dataset,node_sql_parser,node_database_model,node_result_store toneAmber
class node_spider_metrics,node_spider_suite,node_bird_eval,node_bird_f1,node_bird_ves,node_bird_rves toneMint
class node_filter,node_reporting,node_history toneRose
class node_user toneIndigo
```

## Components

### User Workflow

- [CLI Commands](https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/cli/cli.py): `src/nl2sql360/cli/cli.py`
- [Command Handlers](https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/cli/util.py): `src/nl2sql360/cli/util.py`
- [Config Parsing](https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/arguments/parser.py): `src/nl2sql360/arguments/parser.py`
- [Evaluation Core](https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/core/core.py): `src/nl2sql360/core/core.py`

### Dataset And State

- [Dataset Loader](https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/dataset/dataset.py): `src/nl2sql360/dataset/dataset.py`
- [SQL Feature Parser](https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/parser/sql_parser.py): `src/nl2sql360/parser/sql_parser.py`
- [Result Models](https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/database/model.py): `src/nl2sql360/database/model.py`
- [Evaluation Store](https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/core/core.py): `src/nl2sql360/core/core.py`

### Evaluation Metrics

- [Spider EX And EM](https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/evaluator/spider_ex_em.py): `src/nl2sql360/evaluator/spider_ex_em.py`
- [Spider Test Suite](https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/evaluator/test_suite_sql_eval/evaluation.py): `src/nl2sql360/evaluator/test_suite_sql_eval/evaluation.py`
- [BIRD Evaluation](https://github.com/hkustdial/nl2sql360/tree/master/src/nl2sql360/evaluator/bird_eval): `src/nl2sql360/evaluator/bird_eval`
- [Soft F1](https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/evaluator/bird_eval/evaluation_f1.py): `src/nl2sql360/evaluator/bird_eval/evaluation_f1.py`
- **Valid Efficiency**
- **Reward Efficiency**

### Performance Analysis

- [Scenario Filters](https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/filter/filter.py): `src/nl2sql360/filter/filter.py`
- [Performance Reports](https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/core/core.py): `src/nl2sql360/core/core.py`
- [History Deletion](https://github.com/hkustdial/nl2sql360/blob/master/src/nl2sql360/core/core.py): `src/nl2sql360/core/core.py`

### Other components

- **Researcher**

## Connections

- Researcher → CLI Commands: invokes
- CLI Commands → Command Handlers: dispatches
- Command Handlers → Config Parsing: parses config
- Command Handlers → Evaluation Core: runs operation
- Evaluation Core → Dataset Loader: loads samples
- Evaluation Core → SQL Feature Parser: extracts features
- Evaluation Core → Result Models: creates tables
- Evaluation Core → Evaluation Store: reads and writes
- Evaluation Core → Spider EX And EM: invokes evaluation
- Spider EX And EM → Spider Test Suite: evaluates SQL
- Evaluation Core → BIRD Evaluation: runs metrics
- BIRD Evaluation → Soft F1: includes
- BIRD Evaluation → Valid Efficiency: includes
- BIRD Evaluation → Reward Efficiency: includes
- Evaluation Core → Scenario Filters: uses scenarios
- Evaluation Core → Performance Reports: queries results
- Evaluation Core → History Deletion: deletes history