# ales27pm/quec-fr-ca-llm-training- architecture

> Architecture diagram of the GitHub repository ales27pm/quec-fr-ca-llm-training-, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ales27pm/quec-fr-ca-llm-training-
- Repository: https://github.com/ales27pm/quec-fr-ca-llm-training-

## Overview

Quec-fr-CA-llm-training is a data and evaluation pipeline for building Québécois French training datasets and checking their release readiness. Users invoke the qfr CLI to validate source and policy manifests, ingest and curate corpus material, split accepted records, and build/export a training pack. Separate branches acquire modern sources and generate minimal pairs; evaluation diagnostics and release gates provide quality controls. The graph reflects documented CLI capabilities and sampled imports/calls; some unsampled implementations are represented only at their exact module paths, so uncertain connections are omitted.

Main entry points: [`src/qfr_pipeline/cli.py`](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/cli.py) (qfr commands), [`src/qfr_pipeline/release_candidate.py`](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/release_candidate.py) (Release candidate), [`src/qfr_pipeline/schemas.py`](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/schemas.py) (Domain schemas), [`src/qfr_pipeline/io.py`](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/io.py) (File I/O)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["CLI and orchestration"]
  node_cli["qfr commands<br/>[cli.py]"]
  node_candidate["Release candidate"]
end

subgraph group_corpora["Corpus acquisition"]
  node_source_ingest["Corpus ingestion<br/>[corpus_sources.py]"]
  node_modern["Modern acquisition<br/>[modern_corpus.py]"]
  node_curation["Corpus curation<br/>[curation_policy.py]"]
  node_readiness["Corpus readiness"]
end

subgraph group_datasets["Dataset preparation"]
  node_split["Curated splitting<br/>[curated_split.py]"]
  node_training_pack["Training pack<br/>[training_pack.py]"]
  node_training_export["Training export<br/>[training_export.py]"]
  node_minimal_pairs["Minimal pairs<br/>[minimal_pairs.py]"]
  node_pair_quality["Pair quality"]
  node_micro_pack["LP9 micro-pack<br/>[lp9_micro_pack.py]"]
  node_data_pipeline["Legacy pipeline<br/>[data_pipeline.py]"]
  node_failure_mining["LP9 failure mining"]
end

subgraph group_quality["Evaluation and release"]
  node_lp9_eval["LP9 evaluation<br/>[lp9_eval.py]"]
  node_diagnostics["Eval diagnostics<br/>[diagnostics.py]"]
  node_legacy_diag["Legacy diagnostics"]
  node_contamination["Contamination checks<br/>[contamination.py]"]
  node_release_report["Release report<br/>[release_report.py]"]
  node_lp7_monitor["LP7 monitor<br/>[lp7_monitor.py]"]
end

subgraph group_shared["Shared runtime"]
  node_validation["Manifest validation<br/>[validation.py]"]
  node_schemas["Domain schemas<br/>[schemas.py]"]
  node_io["File I/O<br/>[io.py]"]
  node_paths["Repository paths<br/>[paths.py]"]
end

node_user(("Pipeline operator"))
node_external_data[("External corpora")]

node_user -->|"invokes"| node_cli
node_cli -->|"runs"| node_candidate
node_cli -->|"ingests"| node_source_ingest
node_cli -->|"acquires"| node_modern
node_cli -->|"curates"| node_curation
node_cli -->|"splits"| node_split
node_cli -->|"builds"| node_training_pack
node_cli -->|"exports"| node_training_export
node_cli -->|"generates"| node_minimal_pairs
node_cli -->|"validates"| node_pair_quality
node_cli -->|"generates"| node_lp9_eval
node_cli -->|"diagnoses"| node_diagnostics
node_cli -->|"diagnoses"| node_legacy_diag
node_cli -->|"checks"| node_contamination
node_cli -->|"reports"| node_release_report
node_cli -->|"validates"| node_validation
node_cli -->|"builds"| node_micro_pack
node_cli -->|"mines"| node_failure_mining
node_cli -->|"runs"| node_data_pipeline
node_cli -->|"monitors"| node_lp7_monitor
node_candidate -->|"ingests"| node_source_ingest
node_candidate -->|"checks"| node_readiness
node_candidate -->|"curates"| node_curation
node_candidate -->|"splits"| node_split
node_candidate -->|"summarizes"| node_diagnostics
node_candidate -->|"summarizes"| node_minimal_pairs
node_candidate -->|"builds"| node_training_pack
node_candidate -->|"exports"| node_training_export
node_candidate -->|"reports"| node_release_report
node_source_ingest -->|"uses"| node_io
node_source_ingest -->|"uses"| node_schemas
node_source_ingest -->|"resolves via"| node_paths
node_curation -->|"uses"| node_schemas
node_curation -->|"uses"| node_io
node_curation -->|"resolves via"| node_paths
node_readiness -->|"uses"| node_io
node_readiness -->|"resolves via"| node_paths
node_split -->|"uses"| node_io
node_split -->|"uses"| node_schemas
node_split -->|"resolves via"| node_paths
node_training_pack -->|"uses"| node_io
node_training_pack -->|"uses"| node_schemas
node_training_export -->|"uses"| node_io
node_training_export -->|"uses"| node_schemas
node_minimal_pairs -->|"uses"| node_schemas
node_minimal_pairs -->|"uses"| node_io
node_pair_quality -->|"validates"| node_minimal_pairs
node_pair_quality -->|"uses"| node_schemas

click node_cli "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/cli.py"
click node_candidate "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/release_candidate.py"
click node_source_ingest "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/corpus_sources.py"
click node_modern "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/modern_corpus.py"
click node_curation "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/curation_policy.py"
click node_readiness "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/corpus_readiness.py"
click node_split "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/curated_split.py"
click node_training_pack "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/training_pack.py"
click node_training_export "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/training_export.py"
click node_minimal_pairs "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/minimal_pairs.py"
click node_pair_quality "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/minimal_pair_quality.py"
click node_micro_pack "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/lp9_micro_pack.py"
click node_lp9_eval "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/lp9_eval.py"
click node_diagnostics "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/diagnostics.py"
click node_legacy_diag "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/legacy_diagnostics.py"
click node_contamination "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/contamination.py"
click node_release_report "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/release_report.py"
click node_validation "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/validation.py"
click node_schemas "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/schemas.py"
click node_io "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/io.py"
click node_paths "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/paths.py"
click node_data_pipeline "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/data_pipeline.py"
click node_lp7_monitor "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/lp7_monitor.py"
click node_failure_mining "https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/lp9_failure_mining.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_candidate toneBlue
class node_source_ingest,node_modern,node_curation,node_readiness,node_external_data toneAmber
class node_split,node_training_pack,node_training_export,node_minimal_pairs,node_pair_quality,node_micro_pack,node_data_pipeline,node_failure_mining toneMint
class node_lp9_eval,node_diagnostics,node_legacy_diag,node_contamination,node_release_report,node_lp7_monitor toneRose
class node_validation,node_schemas,node_io,node_paths,node_user toneIndigo
```

## Components

### CLI and orchestration

- [qfr commands](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/cli.py): `src/qfr_pipeline/cli.py`
- [Release candidate](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/release_candidate.py): `src/qfr_pipeline/release_candidate.py`

### Corpus acquisition

- [Corpus ingestion](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/corpus_sources.py): `src/qfr_pipeline/corpus_sources.py`
- [Modern acquisition](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/modern_corpus.py): `src/qfr_pipeline/modern_corpus.py`
- [Corpus curation](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/curation_policy.py): `src/qfr_pipeline/curation_policy.py`
- [Corpus readiness](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/corpus_readiness.py): `src/qfr_pipeline/corpus_readiness.py`

### Dataset preparation

- [Curated splitting](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/curated_split.py): `src/qfr_pipeline/curated_split.py`
- [Training pack](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/training_pack.py): `src/qfr_pipeline/training_pack.py`
- [Training export](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/training_export.py): `src/qfr_pipeline/training_export.py`
- [Minimal pairs](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/minimal_pairs.py): `src/qfr_pipeline/minimal_pairs.py`
- [Pair quality](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/minimal_pair_quality.py): `src/qfr_pipeline/minimal_pair_quality.py`
- [LP9 micro-pack](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/lp9_micro_pack.py): `src/qfr_pipeline/lp9_micro_pack.py`
- [Legacy pipeline](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/data_pipeline.py): `src/qfr_pipeline/data_pipeline.py`
- [LP9 failure mining](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/lp9_failure_mining.py): `src/qfr_pipeline/lp9_failure_mining.py`

### Evaluation and release

- [LP9 evaluation](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/lp9_eval.py): `src/qfr_pipeline/lp9_eval.py`
- [Eval diagnostics](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/diagnostics.py): `src/qfr_pipeline/diagnostics.py`
- [Legacy diagnostics](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/legacy_diagnostics.py): `src/qfr_pipeline/legacy_diagnostics.py`
- [Contamination checks](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/contamination.py): `src/qfr_pipeline/contamination.py`
- [Release report](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/release_report.py): `src/qfr_pipeline/release_report.py`
- [LP7 monitor](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/lp7_monitor.py): `src/qfr_pipeline/lp7_monitor.py`

### Shared runtime

- [Manifest validation](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/validation.py): `src/qfr_pipeline/validation.py`
- [Domain schemas](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/schemas.py): `src/qfr_pipeline/schemas.py`
- [File I/O](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/io.py): `src/qfr_pipeline/io.py`
- [Repository paths](https://github.com/ales27pm/quec-fr-ca-llm-training-/blob/main/src/qfr_pipeline/paths.py): `src/qfr_pipeline/paths.py`

### Other components

- **Pipeline operator**
- **External corpora**

## Connections

- Pipeline operator → qfr commands: invokes (evidence: `README.md`)
- qfr commands → Release candidate: runs (evidence: `src/qfr_pipeline/cli.py`)
- qfr commands → Corpus ingestion: ingests (evidence: `src/qfr_pipeline/cli.py`)
- qfr commands → Modern acquisition: acquires (evidence: `README.md`)
- qfr commands → Corpus curation: curates (evidence: `src/qfr_pipeline/cli.py`)
- qfr commands → Curated splitting: splits (evidence: `README.md`)
- qfr commands → Training pack: builds (evidence: `README.md`)
- qfr commands → Training export: exports (evidence: `src/qfr_pipeline/cli.py`)
- qfr commands → Minimal pairs: generates (evidence: `src/qfr_pipeline/cli.py`)
- qfr commands → Pair quality: validates (evidence: `src/qfr_pipeline/cli.py`)
- qfr commands → LP9 evaluation: generates (evidence: `src/qfr_pipeline/cli.py`)
- qfr commands → Eval diagnostics: diagnoses (evidence: `src/qfr_pipeline/cli.py`)
- qfr commands → Legacy diagnostics: diagnoses (evidence: `README.md`)
- qfr commands → Contamination checks: checks (evidence: `src/qfr_pipeline/cli.py`)
- qfr commands → Release report: reports (evidence: `src/qfr_pipeline/cli.py`)
- qfr commands → Manifest validation: validates (evidence: `src/qfr_pipeline/cli.py`)
- qfr commands → LP9 micro-pack: builds (evidence: `src/qfr_pipeline/cli.py`)
- qfr commands → LP9 failure mining: mines (evidence: `src/qfr_pipeline/cli.py`)
- qfr commands → Legacy pipeline: runs (evidence: `src/qfr_pipeline/cli.py`)
- qfr commands → LP7 monitor: monitors (evidence: `src/qfr_pipeline/cli.py`)
- Release candidate → Corpus ingestion: ingests (evidence: `src/qfr_pipeline/release_candidate.py`)
- Release candidate → Corpus readiness: checks (evidence: `src/qfr_pipeline/release_candidate.py`)
- Release candidate → Corpus curation: curates (evidence: `src/qfr_pipeline/release_candidate.py`)
- Release candidate → Curated splitting: splits (evidence: `src/qfr_pipeline/release_candidate.py`)
- Release candidate → Eval diagnostics: summarizes (evidence: `src/qfr_pipeline/release_candidate.py`)
- Release candidate → Minimal pairs: summarizes (evidence: `src/qfr_pipeline/release_candidate.py`)
- Release candidate → Training pack: builds (evidence: `src/qfr_pipeline/release_candidate.py`)
- Release candidate → Training export: exports (evidence: `src/qfr_pipeline/release_candidate.py`)
- Release candidate → Release report: reports (evidence: `src/qfr_pipeline/release_candidate.py`)
- Corpus ingestion → File I/O: uses (evidence: `src/qfr_pipeline/corpus_sources.py`)
- Corpus ingestion → Domain schemas: uses (evidence: `src/qfr_pipeline/corpus_sources.py`)
- Corpus ingestion → Repository paths: resolves via (evidence: `src/qfr_pipeline/corpus_sources.py`)
- Corpus curation → Domain schemas: uses (evidence: `src/qfr_pipeline/curation_policy.py`)
- Corpus curation → File I/O: uses (evidence: `src/qfr_pipeline/curation_policy.py`)
- Corpus curation → Repository paths: resolves via (evidence: `src/qfr_pipeline/curation_policy.py`)
- Corpus readiness → File I/O: uses (evidence: `src/qfr_pipeline/corpus_readiness.py`)
- Corpus readiness → Repository paths: resolves via (evidence: `src/qfr_pipeline/corpus_readiness.py`)
- Curated splitting → File I/O: uses (evidence: `src/qfr_pipeline/curated_split.py`)
- Curated splitting → Domain schemas: uses (evidence: `src/qfr_pipeline/curated_split.py`)
- Curated splitting → Repository paths: resolves via (evidence: `src/qfr_pipeline/curated_split.py`)
- Training pack → File I/O: uses (evidence: `src/qfr_pipeline/training_pack.py`)
- Training pack → Domain schemas: uses (evidence: `src/qfr_pipeline/training_pack.py`)
- Training export → File I/O: uses (evidence: `src/qfr_pipeline/training_export.py`)
- Training export → Domain schemas: uses (evidence: `src/qfr_pipeline/training_export.py`)
- Minimal pairs → Domain schemas: uses (evidence: `src/qfr_pipeline/minimal_pairs.py`)
- Minimal pairs → File I/O: uses (evidence: `src/qfr_pipeline/minimal_pairs.py`)
- Pair quality → Minimal pairs: validates (evidence: `src/qfr_pipeline/minimal_pair_quality.py`)
- Pair quality → Domain schemas: uses (evidence: `src/qfr_pipeline/minimal_pair_quality.py`)