# selbioinfonivedi-web/g4_ews_pipeline architecture

> Architecture diagram of the GitHub repository selbioinfonivedi-web/g4_ews_pipeline, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/selbioinfonivedi-web/g4_ews_pipeline
- Repository: https://github.com/selbioinfonivedi-web/g4_ews_pipeline

## Overview

G4-WATCH is a research framework for evaluating whether G-quadruplex loci behave differently from matched controls over viral evolutionary history, and—only if the evidence gate is open—producing surveillance scores. Its main workflow constructs an atlas, processes sequence data through QC, alignment and phylogenetic analysis, tests locus-control differences, then conditionally scores and reports. It also provides a separate operator console and a read-only dashboard/workstation. Pipeline stages and web boundaries follow the README; unsampled stage internals are represented only where their role is documented.

Main entry points: [`g4watch/cli.py`](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/cli.py) (Stage CLI), [`g4watch/pipeline/stage45_dh1.py`](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/pipeline/stage45_dh1.py) (D.H1 test), [`web/runner/app.py`](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/web/runner/app.py) (Operator console), [`g4watch/gating.py`](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/gating.py) (Scoring gate)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_atlas_data["Atlas and inputs"]
  node_atlas["G4 atlas<br/>[stage0_atlas.py]"]
  node_prediction["G4 prediction<br/>[stage0.py]"]
  node_corpus["Sequence corpus<br/>[corpus.py]"]
  node_qc_align["QC and alignment<br/>[stage1_align.py]"]
end

subgraph group_evolution["Evolution analysis"]
  node_recombination["Recombination screen"]
  node_phylo["Phylogenetic states"]
  node_variants["Variant analysis"]
end

subgraph group_evidence["Evidence and gating"]
  node_dh1["D.H1 test<br/>[stage45_dh1.py]"]
  node_metrics["Surveillance metrics"]
  node_gate["Scoring gate<br/>[gating.py]"]
  node_ledger["Verdict ledger<br/>[ledger.py]"]
end

subgraph group_outputs["Scoring and views"]
  node_scoring["Gated scoring<br/>[stage5_scoring.py]"]
  node_score_models["Score models"]
  node_report["Scored report"]
  node_dashboard_data["Dashboard builder<br/>[dashboard.py]"]
end

subgraph group_web_apps["Web applications"]
  node_runner["Operator console<br/>[app.py]"]
  node_commands["Command registry<br/>[commands.py]"]
  node_jobs["Job manager<br/>[jobs.py]"]
  node_analysis_store[("Analysis database<br/>[store.py]")]
  node_public_app["Read-only dashboard API<br/>[app.py]"]
  node_workstation["Linked-view workstation<br/>[dataset.py]"]
end

node_operator(("Research operator"))
node_cli["Stage CLI<br/>[cli.py]"]
node_pipeline_user(("Pipeline user"))
node_web_user(("Web user"))

node_operator -->|"runs stages"| node_cli
node_cli -->|"dispatches"| node_atlas
node_atlas -->|"scans genome"| node_prediction
node_cli -->|"dispatches"| node_qc_align
node_qc_align -->|"processes sequences"| node_corpus
node_cli -->|"dispatches"| node_recombination
node_recombination -->|"precedes"| node_phylo
node_cli -->|"dispatches"| node_phylo
node_cli -->|"dispatches"| node_variants
node_cli -->|"dispatches"| node_dh1
node_dh1 -->|"computes"| node_metrics
node_dh1 -->|"evaluates"| node_gate
node_dh1 -->|"appends verdicts"| node_ledger
node_cli -->|"dispatches"| node_scoring
node_scoring -->|"checks permission"| node_gate
node_gate -->|"reads verdicts"| node_ledger
node_scoring -->|"computes scores"| node_score_models
node_cli -->|"dispatches"| node_report
node_report -->|"checks permission"| node_gate
node_pipeline_user -->|"uses console"| node_runner
node_runner -->|"builds command"| node_commands
node_runner -->|"manages jobs"| node_jobs
node_runner -->|"persists analyses"| node_analysis_store
node_jobs -->|"launches commands"| node_cli
node_web_user -->|"views results"| node_public_app
node_public_app -->|"builds dashboard"| node_dashboard_data
node_dashboard_data -->|"reads verdicts"| node_ledger
node_web_user -->|"opens linked views"| node_workstation
node_workstation -->|"reads verdicts"| node_ledger

click node_cli "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/cli.py"
click node_atlas "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/pipeline/stage0_atlas.py"
click node_prediction "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/atlas/stage0.py"
click node_corpus "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/io/corpus.py"
click node_qc_align "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/pipeline/stage1_align.py"
click node_recombination "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/pipeline/stage15_recombination.py"
click node_phylo "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/phylo/ancestral_states.py"
click node_variants "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/variants/alignment_variant_caller.py"
click node_dh1 "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/pipeline/stage45_dh1.py"
click node_metrics "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/metrics/surveillance_metrics.py"
click node_gate "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/gating.py"
click node_ledger "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/ledger.py"
click node_scoring "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/pipeline/stage5_scoring.py"
click node_score_models "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/scoring/integrated_score.py"
click node_report "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/pipeline/stage6_reporting.py"
click node_dashboard_data "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/reporting/dashboard.py"
click node_runner "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/web/runner/app.py"
click node_commands "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/web/runner/commands.py"
click node_jobs "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/web/runner/jobs.py"
click node_analysis_store "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/web/store.py"
click node_public_app "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/web/backend/app.py"
click node_workstation "https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/web/workstation/dataset.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_atlas,node_prediction,node_corpus,node_qc_align,node_pipeline_user,node_web_user toneBlue
class node_recombination,node_phylo,node_variants toneAmber
class node_dh1,node_metrics,node_gate,node_ledger toneMint
class node_scoring,node_score_models,node_report,node_dashboard_data toneRose
class node_runner,node_commands,node_jobs,node_analysis_store,node_public_app,node_workstation,node_operator toneIndigo
class node_cli toneTeal
```

## Components

### Atlas and inputs

- [G4 atlas](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/pipeline/stage0_atlas.py): `g4watch/pipeline/stage0_atlas.py`
- [G4 prediction](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/atlas/stage0.py): `g4watch/atlas/stage0.py`
- [Sequence corpus](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/io/corpus.py): `g4watch/io/corpus.py`
- [QC and alignment](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/pipeline/stage1_align.py): `g4watch/pipeline/stage1_align.py`

### Evolution analysis

- [Recombination screen](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/pipeline/stage15_recombination.py): `g4watch/pipeline/stage15_recombination.py`
- [Phylogenetic states](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/phylo/ancestral_states.py): `g4watch/phylo/ancestral_states.py`
- [Variant analysis](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/variants/alignment_variant_caller.py): `g4watch/variants/alignment_variant_caller.py`

### Evidence and gating

- [D.H1 test](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/pipeline/stage45_dh1.py): `g4watch/pipeline/stage45_dh1.py`
- [Surveillance metrics](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/metrics/surveillance_metrics.py): `g4watch/metrics/surveillance_metrics.py`
- [Scoring gate](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/gating.py): `g4watch/gating.py`
- [Verdict ledger](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/ledger.py): `g4watch/ledger.py`

### Scoring and views

- [Gated scoring](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/pipeline/stage5_scoring.py): `g4watch/pipeline/stage5_scoring.py`
- [Score models](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/scoring/integrated_score.py): `g4watch/scoring/integrated_score.py`
- [Scored report](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/pipeline/stage6_reporting.py): `g4watch/pipeline/stage6_reporting.py`
- [Dashboard builder](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/reporting/dashboard.py): `g4watch/reporting/dashboard.py`

### Web applications

- [Operator console](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/web/runner/app.py): `web/runner/app.py`
- [Command registry](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/web/runner/commands.py): `web/runner/commands.py`
- [Job manager](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/web/runner/jobs.py): `web/runner/jobs.py`
- [Analysis database](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/web/store.py): `web/store.py`
- [Read-only dashboard API](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/web/backend/app.py): `web/backend/app.py`
- [Linked-view workstation](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/web/workstation/dataset.py): `web/workstation/dataset.py`

### Other components

- **Research operator**
- [Stage CLI](https://github.com/selbioinfonivedi-web/g4_ews_pipeline/blob/main/g4watch/cli.py): `g4watch/cli.py`
- **Pipeline user**
- **Web user**

## Connections

- Research operator → Stage CLI: runs stages
- Stage CLI → G4 atlas: dispatches
- G4 atlas → G4 prediction: scans genome
- Stage CLI → QC and alignment: dispatches
- QC and alignment → Sequence corpus: processes sequences
- Stage CLI → Recombination screen: dispatches
- Recombination screen → Phylogenetic states: precedes
- Stage CLI → Phylogenetic states: dispatches
- Stage CLI → Variant analysis: dispatches
- Stage CLI → D.H1 test: dispatches
- D.H1 test → Surveillance metrics: computes
- D.H1 test → Scoring gate: evaluates
- D.H1 test → Verdict ledger: appends verdicts
- Stage CLI → Gated scoring: dispatches
- Gated scoring → Scoring gate: checks permission
- Scoring gate → Verdict ledger: reads verdicts
- Gated scoring → Score models: computes scores
- Stage CLI → Scored report: dispatches
- Scored report → Scoring gate: checks permission
- Pipeline user → Operator console: uses console
- Operator console → Command registry: builds command
- Operator console → Job manager: manages jobs
- Operator console → Analysis database: persists analyses
- Job manager → Stage CLI: launches commands
- Web user → Read-only dashboard API: views results
- Read-only dashboard API → Dashboard builder: builds dashboard
- Dashboard builder → Verdict ledger: reads verdicts
- Web user → Linked-view workstation: opens linked views
- Linked-view workstation → Verdict ledger: reads verdicts