# saberzahhar/kpviz architecture

> Architecture diagram of the GitHub repository saberzahhar/kpviz, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/saberzahhar/kpviz
- Repository: https://github.com/saberzahhar/kpviz
- Video tour (about a minute): https://gitdiagram.com/saberzahhar/kpviz/video

## Overview

KPViz is a local evaluation and paper-figure workbench for keyphrase prediction experiments. Users point it at cards, documents, and inference runs; the app incrementally scans and derives aligned evaluation data into DuckDB, then presents dataset, model, architecture, and issue views. The five research-question workbenches analyze runs and produce figures, tables, and exports. The graph follows documented capabilities and indexed references; unsampled page implementations are represented by their exact tree paths, without assuming their internal call flow.

Main entry points: [`kpviz/appfactory.py`](https://github.com/saberzahhar/kpviz/blob/main/kpviz/appfactory.py) (Dash application), [`kpviz/scanner.py`](https://github.com/saberzahhar/kpviz/blob/main/kpviz/scanner.py) (Incremental scanner), [`kpviz/pages/insights.py`](https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/insights.py) (Insights hub), [`kpviz/figures.py`](https://github.com/saberzahhar/kpviz/blob/main/kpviz/figures.py) (Figure specifications)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtime and scanning"]
  node_cli["CLI entry<br/>[cli.py]"]
  node_app["Dash application<br/>[appfactory.py]"]
  node_scanner["Incremental scanner<br/>[scanner.py]"]
  node_settings["Runtime settings<br/>[config.py]"]
end

subgraph group_catalog["Data and evaluation"]
  node_cards["Card parsing<br/>[cards.py]"]
  node_ingest["File ingestion<br/>[ingest.py]"]
  node_derive["Derived analysis<br/>[derive.py]"]
  node_textproc["Text and tokenizers<br/>[textproc.py]"]
  node_costs["Run costs<br/>[costs.py]"]
  node_store[("DuckDB store<br/>[db.py]")]
  node_metrics["Evaluation metrics<br/>[metrics.py]"]
end

subgraph group_views["Catalog views"]
  node_overview["Overview<br/>[home.py]"]
  node_datasets["Dataset explorer<br/>[datasets.py]"]
  node_models["Model explorer<br/>[models.py]"]
  node_architectures["Architecture explorer<br/>[architectures.py]"]
end

subgraph group_research["Research workbenches"]
  node_insights["Insights hub<br/>[insights.py]"]
  node_rq1["Dataset agreement<br/>[rq1.py]"]
  node_rq2["Data quality<br/>[rq2.py]"]
  node_rq3["Context windows<br/>[rq3.py]"]
  node_rq4["Quality vs cost<br/>[rq4.py]"]
  node_rq5["Hyperparameters<br/>[rq5.py]"]
  node_stats["Statistical methods<br/>[stats.py]"]
end

subgraph group_publishing["Figures and export"]
  node_figures["Figure specifications<br/>[figures.py]"]
  node_ui["UI components<br/>[ui.py]"]
  node_export["Paper exports<br/>[export.py]"]
end

node_user(("Researcher"))

node_user -->|"starts"| node_cli
node_cli -->|"configures"| node_settings
node_cli -->|"scans"| node_scanner
node_cli -->|"launches"| node_app
node_user -->|"uses"| node_app
node_app -->|"starts scan"| node_scanner
node_app -->|"routes to"| node_overview
node_app -->|"routes to"| node_datasets
node_app -->|"routes to"| node_models
node_app -->|"routes to"| node_architectures
node_app -->|"routes to"| node_insights
node_scanner -->|"reads cards"| node_cards
node_scanner -->|"uses ingestion"| node_ingest
node_scanner -->|"derives data"| node_derive
node_scanner -->|"processes text"| node_textproc
node_scanner -->|"calculates costs"| node_costs
node_scanner -->|"writes index"| node_store
node_ingest -->|"stores records"| node_store
node_derive -->|"uses primitives"| node_textproc
node_overview -->|"reads catalog"| node_store
node_datasets -->|"gets scores"| node_metrics
node_datasets -->|"reads data"| node_store
node_architectures -->|"shows costs"| node_costs
node_insights -->|"dispatches"| node_rq1
node_insights -->|"dispatches"| node_rq2
node_insights -->|"dispatches"| node_rq3
node_insights -->|"dispatches"| node_rq4
node_insights -->|"dispatches"| node_rq5
node_rq1 -->|"tests results"| node_stats
node_rq2 -->|"tests results"| node_stats
node_rq3 -->|"tests results"| node_stats
node_rq4 -->|"tests results"| node_stats
node_rq5 -->|"tests results"| node_stats
node_rq1 -->|"scores runs"| node_metrics
node_rq2 -->|"builds figures"| node_figures
node_rq3 -->|"builds figures"| node_figures
node_rq4 -->|"builds figures"| node_figures
node_rq5 -->|"builds figures"| node_figures
node_ui -->|"renders specs"| node_figures
node_ui -->|"offers export"| node_export
node_export -->|"rerenders specs"| node_figures

click node_cli "https://github.com/saberzahhar/kpviz/blob/main/kpviz/cli.py"
click node_app "https://github.com/saberzahhar/kpviz/blob/main/kpviz/appfactory.py"
click node_scanner "https://github.com/saberzahhar/kpviz/blob/main/kpviz/scanner.py"
click node_settings "https://github.com/saberzahhar/kpviz/blob/main/kpviz/config.py"
click node_cards "https://github.com/saberzahhar/kpviz/blob/main/kpviz/cards.py"
click node_ingest "https://github.com/saberzahhar/kpviz/blob/main/kpviz/ingest.py"
click node_derive "https://github.com/saberzahhar/kpviz/blob/main/kpviz/derive.py"
click node_textproc "https://github.com/saberzahhar/kpviz/blob/main/kpviz/textproc.py"
click node_costs "https://github.com/saberzahhar/kpviz/blob/main/kpviz/costs.py"
click node_store "https://github.com/saberzahhar/kpviz/blob/main/kpviz/db.py"
click node_metrics "https://github.com/saberzahhar/kpviz/blob/main/kpviz/metrics.py"
click node_overview "https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/home.py"
click node_datasets "https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/datasets.py"
click node_models "https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/models.py"
click node_architectures "https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/architectures.py"
click node_insights "https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/insights.py"
click node_rq1 "https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/rq/rq1.py"
click node_rq2 "https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/rq/rq2.py"
click node_rq3 "https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/rq/rq3.py"
click node_rq4 "https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/rq/rq4.py"
click node_rq5 "https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/rq/rq5.py"
click node_stats "https://github.com/saberzahhar/kpviz/blob/main/kpviz/stats.py"
click node_figures "https://github.com/saberzahhar/kpviz/blob/main/kpviz/figures.py"
click node_ui "https://github.com/saberzahhar/kpviz/blob/main/kpviz/ui.py"
click node_export "https://github.com/saberzahhar/kpviz/blob/main/kpviz/export.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_app,node_scanner,node_settings toneBlue
class node_cards,node_ingest,node_derive,node_textproc,node_costs,node_store,node_metrics toneAmber
class node_overview,node_datasets,node_models,node_architectures toneMint
class node_insights,node_rq1,node_rq2,node_rq3,node_rq4,node_rq5,node_stats toneRose
class node_figures,node_ui,node_export,node_user toneIndigo
```

## Components

### Runtime and scanning

- [CLI entry](https://github.com/saberzahhar/kpviz/blob/main/kpviz/cli.py): `kpviz/cli.py`
- [Dash application](https://github.com/saberzahhar/kpviz/blob/main/kpviz/appfactory.py): `kpviz/appfactory.py`
- [Incremental scanner](https://github.com/saberzahhar/kpviz/blob/main/kpviz/scanner.py): `kpviz/scanner.py`
- [Runtime settings](https://github.com/saberzahhar/kpviz/blob/main/kpviz/config.py): `kpviz/config.py`

### Data and evaluation

- [Card parsing](https://github.com/saberzahhar/kpviz/blob/main/kpviz/cards.py): `kpviz/cards.py`
- [File ingestion](https://github.com/saberzahhar/kpviz/blob/main/kpviz/ingest.py): `kpviz/ingest.py`
- [Derived analysis](https://github.com/saberzahhar/kpviz/blob/main/kpviz/derive.py): `kpviz/derive.py`
- [Text and tokenizers](https://github.com/saberzahhar/kpviz/blob/main/kpviz/textproc.py): `kpviz/textproc.py`
- [Run costs](https://github.com/saberzahhar/kpviz/blob/main/kpviz/costs.py): `kpviz/costs.py`
- [DuckDB store](https://github.com/saberzahhar/kpviz/blob/main/kpviz/db.py): `kpviz/db.py`
- [Evaluation metrics](https://github.com/saberzahhar/kpviz/blob/main/kpviz/metrics.py): `kpviz/metrics.py`

### Catalog views

- [Overview](https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/home.py): `kpviz/pages/home.py`
- [Dataset explorer](https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/datasets.py): `kpviz/pages/datasets.py`
- [Model explorer](https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/models.py): `kpviz/pages/models.py`
- [Architecture explorer](https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/architectures.py): `kpviz/pages/architectures.py`

### Research workbenches

- [Insights hub](https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/insights.py): `kpviz/pages/insights.py`
- [Dataset agreement](https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/rq/rq1.py): `kpviz/pages/rq/rq1.py`
- [Data quality](https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/rq/rq2.py): `kpviz/pages/rq/rq2.py`
- [Context windows](https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/rq/rq3.py): `kpviz/pages/rq/rq3.py`
- [Quality vs cost](https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/rq/rq4.py): `kpviz/pages/rq/rq4.py`
- [Hyperparameters](https://github.com/saberzahhar/kpviz/blob/main/kpviz/pages/rq/rq5.py): `kpviz/pages/rq/rq5.py`
- [Statistical methods](https://github.com/saberzahhar/kpviz/blob/main/kpviz/stats.py): `kpviz/stats.py`

### Figures and export

- [Figure specifications](https://github.com/saberzahhar/kpviz/blob/main/kpviz/figures.py): `kpviz/figures.py`
- [UI components](https://github.com/saberzahhar/kpviz/blob/main/kpviz/ui.py): `kpviz/ui.py`
- [Paper exports](https://github.com/saberzahhar/kpviz/blob/main/kpviz/export.py): `kpviz/export.py`

### Other components

- **Researcher**

## Connections

- Researcher → CLI entry: starts (evidence: `README.md`)
- CLI entry → Runtime settings: configures (evidence: `kpviz/cli.py`)
- CLI entry → Incremental scanner: scans (evidence: `kpviz/cli.py`)
- CLI entry → Dash application: launches (evidence: `kpviz/cli.py`)
- Researcher → Dash application: uses (evidence: `README.md`)
- Dash application → Incremental scanner: starts scan (evidence: `kpviz/appfactory.py`)
- Dash application → Overview: routes to (evidence: `kpviz/appfactory.py`)
- Dash application → Dataset explorer: routes to (evidence: `kpviz/appfactory.py`)
- Dash application → Model explorer: routes to (evidence: `kpviz/appfactory.py`)
- Dash application → Architecture explorer: routes to (evidence: `kpviz/appfactory.py`)
- Dash application → Insights hub: routes to (evidence: `kpviz/appfactory.py`)
- Incremental scanner → Card parsing: reads cards (evidence: `kpviz/scanner.py`)
- Incremental scanner → File ingestion: uses ingestion (evidence: `kpviz/scanner.py`)
- Incremental scanner → Derived analysis: derives data (evidence: `kpviz/scanner.py`)
- Incremental scanner → Text and tokenizers: processes text (evidence: `kpviz/scanner.py`)
- Incremental scanner → Run costs: calculates costs (evidence: `kpviz/scanner.py`)
- Incremental scanner → DuckDB store: writes index (evidence: `kpviz/scanner.py`)
- File ingestion → DuckDB store: stores records (evidence: `kpviz/ingest.py`)
- Derived analysis → Text and tokenizers: uses primitives (evidence: `kpviz/derive.py`)
- Overview → DuckDB store: reads catalog (evidence: `kpviz/pages/home.py`)
- Dataset explorer → Evaluation metrics: gets scores (evidence: `kpviz/pages/datasets.py`)
- Dataset explorer → DuckDB store: reads data (evidence: `kpviz/pages/datasets.py`)
- Architecture explorer → Run costs: shows costs (evidence: `kpviz/pages/architectures.py`)
- Insights hub → Dataset agreement: dispatches (evidence: `kpviz/pages/insights.py`)
- Insights hub → Data quality: dispatches (evidence: `kpviz/pages/insights.py`)
- Insights hub → Context windows: dispatches (evidence: `kpviz/pages/insights.py`)
- Insights hub → Quality vs cost: dispatches (evidence: `kpviz/pages/insights.py`)
- Insights hub → Hyperparameters: dispatches (evidence: `kpviz/pages/insights.py`)
- Dataset agreement → Statistical methods: tests results (evidence: `kpviz/pages/rq/rq1.py`)
- Data quality → Statistical methods: tests results (evidence: `kpviz/pages/rq/rq2.py`)
- Context windows → Statistical methods: tests results (evidence: `kpviz/pages/rq/rq3.py`)
- Quality vs cost → Statistical methods: tests results (evidence: `kpviz/pages/rq/rq4.py`)
- Hyperparameters → Statistical methods: tests results (evidence: `kpviz/pages/rq/rq5.py`)
- Dataset agreement → Evaluation metrics: scores runs (evidence: `kpviz/pages/rq/rq1.py`)
- Data quality → Figure specifications: builds figures (evidence: `kpviz/pages/rq/rq2.py`)
- Context windows → Figure specifications: builds figures (evidence: `kpviz/pages/rq/rq3.py`)
- Quality vs cost → Figure specifications: builds figures (evidence: `kpviz/pages/rq/rq4.py`)
- Hyperparameters → Figure specifications: builds figures (evidence: `kpviz/pages/rq/rq5.py`)
- UI components → Figure specifications: renders specs (evidence: `kpviz/ui.py`)
- UI components → Paper exports: offers export (evidence: `kpviz/ui.py`)
- Paper exports → Figure specifications: rerenders specs (evidence: `kpviz/export.py`)