# gildas-le-drogoff/convert-to-parquet-rust architecture

> Architecture diagram of the GitHub repository gildas-le-drogoff/convert-to-parquet-rust, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gildas-le-drogoff/convert-to-parquet-rust
- Repository: https://github.com/gildas-le-drogoff/convert-to-parquet-rust

## Overview

convert-to-parquet is a Rust data-conversion application with both CLI and GUI entry points. Its main workflow reads delimited, spreadsheet, JSON, or Parquet data; infers or applies a schema; converts and writes Parquet; and reports validation results. It also supports Parquet inspection and reverse CSV/JSONL export, plus GUI-based analysis, preview, and conversion. The README documents major formats and pipeline behavior; unsampled modules are included where the README or source index establishes their role, with uncertain links omitted.

Main entry points: [`src/conversion/mod.rs`](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/conversion/mod.rs) (Conversion pipeline), [`src/main.rs`](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/main.rs) (CLI), [`src/gui/app.rs`](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/gui/app.rs) (GUI application), [`src/gui/worker.rs`](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/gui/worker.rs) (Background jobs)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["User Interfaces"]
  node_cli["CLI<br/>[main.rs]"]
  node_gui_entry["GUI launcher<br/>[gui_convert.rs]"]
end

subgraph group_ingest["Input Handling"]
  node_csv_input["Delimited input<br/>[csv_blocks.rs]"]
  node_sheets["Spreadsheet reader<br/>[xlsx.rs]"]
  node_json_input["JSON handling<br/>[json.rs]"]
  node_json_arrow["JSON Arrow pipeline<br/>[json_arrow.rs]"]
end

subgraph group_transform["Schema and Conversion"]
  node_schema["Schema inference<br/>[schema.rs]"]
  node_analysis["Column analysis<br/>[analysis.rs]"]
  node_csv_pipeline["Conversion pipeline<br/>[mod.rs]"]
  node_json_pipeline["JSON conversion<br/>[json_pipeline.rs]"]
  node_block_pipeline["Parallel blocks<br/>[pipeline.rs]"]
  node_parquet_writer["Ordered Parquet writer<br/>[parquet_writer.rs]"]
  node_memory["Memory limits<br/>[memory.rs]"]
end

subgraph group_outputs["Outputs and Inspection"]
  node_validation["Validation report<br/>[diag.rs]"]
  node_inspect["Parquet inspection<br/>[inspect.rs]"]
  node_preview["Data preview<br/>[preview.rs]"]
  node_export["CSV and JSONL export<br/>[export.rs]"]
  node_rewrite["Parquet rewrite<br/>[parquet_rewrite.rs]"]
end

subgraph group_gui["GUI Workflow"]
  node_gui_app["GUI application<br/>[app.rs]"]
  node_gui_model["File and edit state<br/>[model.rs]"]
  node_gui_analysis["GUI analysis<br/>[analysis.rs]"]
  node_gui_worker["Background jobs<br/>[worker.rs]"]
  node_gui_view["GUI views<br/>[mod.rs]"]
  node_gui_paths["Output planning<br/>[paths.rs]"]
end

node_user(("User"))

node_user -->|"runs conversion"| node_cli
node_user -->|"opens GUI"| node_gui_entry
node_cli -->|"converts files"| node_csv_pipeline
node_csv_pipeline -->|"infers schema"| node_schema
node_csv_pipeline -->|"streams blocks"| node_csv_input
node_csv_pipeline -->|"tracks metrics"| node_analysis
node_csv_pipeline -->|"runs pipeline"| node_block_pipeline
node_block_pipeline -->|"writes batches"| node_parquet_writer
node_csv_pipeline -->|"starts writer"| node_parquet_writer
node_csv_pipeline -->|"sizes blocks"| node_memory
node_cli -->|"converts JSON"| node_json_pipeline
node_json_pipeline -->|"reads JSON"| node_json_input
node_json_pipeline -->|"decodes batches"| node_json_arrow
node_json_pipeline -->|"sends batches"| node_parquet_writer
node_csv_pipeline -->|"reports results"| node_validation
node_cli -->|"views schema"| node_inspect
node_cli -->|"exports formats"| node_export
node_cli -.->|"rewrites Parquet"| node_rewrite
node_cli -->|"reads spreadsheets"| node_sheets
node_gui_entry -.->|"starts app"| node_gui_app
node_gui_app -->|"updates state"| node_gui_model
node_gui_app -->|"renders views"| node_gui_view
node_gui_app -->|"dispatches jobs"| node_gui_worker
node_gui_app -->|"plans outputs"| node_gui_paths
node_gui_worker -->|"analyzes files"| node_gui_analysis
node_gui_worker -->|"converts sources"| node_csv_pipeline
node_gui_worker -->|"reads sheets"| node_sheets
node_gui_worker -->|"handles JSON"| node_json_input
node_gui_analysis -->|"previews data"| node_preview
node_gui_analysis -->|"inspects Parquet"| node_inspect
node_gui_view -->|"shows preview"| node_preview

click node_cli "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/main.rs"
click node_gui_entry "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/bin/gui_convert.rs"
click node_csv_input "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/conversion/csv_blocks.rs"
click node_sheets "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/xlsx.rs"
click node_json_input "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/json.rs"
click node_json_arrow "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/json_arrow.rs"
click node_schema "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/schema.rs"
click node_analysis "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/analysis.rs"
click node_csv_pipeline "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/conversion/mod.rs"
click node_json_pipeline "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/conversion/json_pipeline.rs"
click node_block_pipeline "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/conversion/pipeline.rs"
click node_parquet_writer "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/conversion/parquet_writer.rs"
click node_memory "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/memory.rs"
click node_validation "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/diag.rs"
click node_inspect "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/inspect.rs"
click node_preview "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/preview.rs"
click node_export "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/export.rs"
click node_rewrite "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/parquet_rewrite.rs"
click node_gui_app "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/gui/app.rs"
click node_gui_model "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/gui/model.rs"
click node_gui_analysis "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/gui/analysis.rs"
click node_gui_worker "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/gui/worker.rs"
click node_gui_view "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/gui/view/mod.rs"
click node_gui_paths "https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/gui/paths.rs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_gui_entry,node_user toneBlue
class node_csv_input,node_sheets,node_json_input,node_json_arrow toneAmber
class node_schema,node_analysis,node_csv_pipeline,node_json_pipeline,node_block_pipeline,node_parquet_writer,node_memory toneMint
class node_validation,node_inspect,node_preview,node_export,node_rewrite toneRose
class node_gui_app,node_gui_model,node_gui_analysis,node_gui_worker,node_gui_view,node_gui_paths toneIndigo
```

## Components

### User Interfaces

- [CLI](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/main.rs): `src/main.rs`
- [GUI launcher](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/bin/gui_convert.rs): `src/bin/gui_convert.rs`

### Input Handling

- [Delimited input](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/conversion/csv_blocks.rs): `src/conversion/csv_blocks.rs`
- [Spreadsheet reader](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/xlsx.rs): `src/xlsx.rs`
- [JSON handling](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/json.rs): `src/json.rs`
- [JSON Arrow pipeline](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/json_arrow.rs): `src/json_arrow.rs`

### Schema and Conversion

- [Schema inference](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/schema.rs): `src/schema.rs`
- [Column analysis](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/analysis.rs): `src/analysis.rs`
- [Conversion pipeline](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/conversion/mod.rs): `src/conversion/mod.rs`
- [JSON conversion](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/conversion/json_pipeline.rs): `src/conversion/json_pipeline.rs`
- [Parallel blocks](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/conversion/pipeline.rs): `src/conversion/pipeline.rs`
- [Ordered Parquet writer](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/conversion/parquet_writer.rs): `src/conversion/parquet_writer.rs`
- [Memory limits](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/memory.rs): `src/memory.rs`

### Outputs and Inspection

- [Validation report](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/diag.rs): `src/diag.rs`
- [Parquet inspection](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/inspect.rs): `src/inspect.rs`
- [Data preview](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/preview.rs): `src/preview.rs`
- [CSV and JSONL export](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/export.rs): `src/export.rs`
- [Parquet rewrite](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/parquet_rewrite.rs): `src/parquet_rewrite.rs`

### GUI Workflow

- [GUI application](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/gui/app.rs): `src/gui/app.rs`
- [File and edit state](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/gui/model.rs): `src/gui/model.rs`
- [GUI analysis](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/gui/analysis.rs): `src/gui/analysis.rs`
- [Background jobs](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/gui/worker.rs): `src/gui/worker.rs`
- [GUI views](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/gui/view/mod.rs): `src/gui/view/mod.rs`
- [Output planning](https://github.com/gildas-le-drogoff/convert-to-parquet-rust/blob/main/src/gui/paths.rs): `src/gui/paths.rs`

### Other components

- **User**

## Connections

- User → CLI: runs conversion (evidence: `README.md`)
- User → GUI launcher: opens GUI (evidence: `src/bin/gui_convert.rs`)
- CLI → Conversion pipeline: converts files (evidence: `README.md`)
- Conversion pipeline → Schema inference: infers schema (evidence: `src/conversion/mod.rs`)
- Conversion pipeline → Delimited input: streams blocks (evidence: `src/conversion/mod.rs`)
- Conversion pipeline → Column analysis: tracks metrics (evidence: `src/conversion/mod.rs`)
- Conversion pipeline → Parallel blocks: runs pipeline (evidence: `src/conversion/mod.rs`)
- Parallel blocks → Ordered Parquet writer: writes batches (evidence: `src/conversion/pipeline.rs`)
- Conversion pipeline → Ordered Parquet writer: starts writer (evidence: `src/conversion/mod.rs`)
- Conversion pipeline → Memory limits: sizes blocks (evidence: `src/conversion/mod.rs`)
- CLI → JSON conversion: converts JSON (evidence: `README.md`)
- JSON conversion → JSON handling: reads JSON (evidence: `src/conversion/json_pipeline.rs`)
- JSON conversion → JSON Arrow pipeline: decodes batches (evidence: `src/conversion/json_pipeline.rs`)
- JSON conversion → Ordered Parquet writer: sends batches (evidence: `src/conversion/json_pipeline.rs`)
- Conversion pipeline → Validation report: reports results (evidence: `src/conversion/mod.rs`)
- CLI → Parquet inspection: views schema (evidence: `README.md`)
- CLI → CSV and JSONL export: exports formats (evidence: `README.md`)
- CLI → Parquet rewrite: rewrites Parquet
- CLI → Spreadsheet reader: reads spreadsheets (evidence: `README.md`)
- GUI launcher → GUI application: starts app
- GUI application → File and edit state: updates state (evidence: `src/gui/app.rs`)
- GUI application → GUI views: renders views (evidence: `src/gui/app.rs`)
- GUI application → Background jobs: dispatches jobs (evidence: `src/gui/app.rs`)
- GUI application → Output planning: plans outputs (evidence: `src/gui/app.rs`)
- Background jobs → GUI analysis: analyzes files (evidence: `src/gui/worker.rs`)
- Background jobs → Conversion pipeline: converts sources (evidence: `src/gui/worker.rs`)
- Background jobs → Spreadsheet reader: reads sheets (evidence: `src/gui/worker.rs`)
- Background jobs → JSON handling: handles JSON (evidence: `src/gui/worker.rs`)
- GUI analysis → Data preview: previews data (evidence: `src/gui/analysis.rs`)
- GUI analysis → Parquet inspection: inspects Parquet (evidence: `src/gui/analysis.rs`)
- GUI views → Data preview: shows preview
