# usc-isi-i2/kgtk architecture

> Architecture diagram of the GitHub repository usc-isi-i2/kgtk, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/usc-isi-i2/kgtk
- Repository: https://github.com/usc-isi-i2/kgtk

## Overview

KGTK is a toolkit for building and exploiting large knowledge graphs represented as tab-separated KGTK edge files. A user invokes commands through the CLI to import sources, transform or combine graph files, then query, analyze, embed, validate, or export results. The shared reader/writer layer supports file-based command workflows; Kypher adds disk-backed query storage. The README documents the major command families, while sampled source confirms query execution and embedding entry points. Links between unsampled command families and their implementations are therefore limited to documented behavior.

Main entry points: [`kgtk/cli_entry.py`](https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/cli_entry.py) (KGTK commands), [`kgtk/io/kgtkreader.py`](https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/io/kgtkreader.py) (KGTK reader), [`kgtk/io/kgtkwriter.py`](https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/io/kgtkwriter.py) (KGTK writer), [`kgtk/cli/query.py`](https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/cli/query.py) (Kypher command)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Import and export"]
  node_imports["Graph importers"]
  node_wikidata_import["Wikidata import<br/>[import_wikidata.py]"]
  node_exports["Graph exporters<br/>[exportwikidata.py]"]
end

subgraph group_transform["Graph operations"]
  node_transform_cmds["Graph transforms"]
  node_reshape["Reshape edges"]
  node_joins["Combine graphs"]
  node_validation["Graph validation<br/>[validate.py]"]
end

subgraph group_query["Query and analytics"]
  node_query_cmd["Kypher command<br/>[query.py]"]
  node_parser["Query parser<br/>[parser.py]"]
  node_query_engine["Query translation<br/>[query.py]"]
  node_sql_store[("Disk query store<br/>[sqlstore.py]")]
  node_analytics["Graph analytics"]
end

subgraph group_enrich["Graph enrichment"]
  node_lexicalize["Node lexicalization<br/>[lexicalize.py]"]
  node_text_embed["Text embeddings<br/>[embedding_utils.py]"]
  node_graph_embed["Graph embeddings"]
  node_embedding_format["Embedding conversion"]
end

subgraph group_core["Shared data core"]
  node_kg_model["Knowledge graph model<br/>[knowledge_graph.py]"]
  node_reader["KGTK reader<br/>[kgtkreader.py]"]
  node_writer["KGTK writer<br/>[kgtkwriter.py]"]
end

node_user(("Engineer"))
node_cli["KGTK commands<br/>[cli_entry.py]"]
node_tsv["KGTK edge files"]

node_user -->|"runs commands"| node_cli
node_cli -->|"imports sources"| node_imports
node_cli -->|"imports Wikidata"| node_wikidata_import
node_cli -->|"exports graphs"| node_exports
node_cli -->|"transforms graphs"| node_transform_cmds
node_cli -->|"reshapes edges"| node_reshape
node_cli -->|"combines graphs"| node_joins
node_cli -->|"validates data"| node_validation
node_cli -->|"runs queries"| node_query_cmd
node_query_cmd -->|"parses Kypher"| node_parser
node_query_cmd -->|"translates query"| node_query_engine
node_query_cmd -->|"opens store"| node_sql_store
node_query_engine -->|"queries graph data"| node_sql_store
node_cli -->|"analyzes graphs"| node_analytics
node_cli -->|"lexicalizes nodes"| node_lexicalize
node_cli -->|"computes text vectors"| node_text_embed
node_cli -->|"computes graph vectors"| node_graph_embed
node_cli -->|"converts vectors"| node_embedding_format
node_imports -->|"reads graph data"| node_reader
node_wikidata_import -->|"writes graph data"| node_writer
node_transform_cmds -->|"reads graph data"| node_reader
node_transform_cmds -->|"writes graph data"| node_writer
node_joins -->|"reads graph data"| node_reader
node_joins -->|"writes graph data"| node_writer
node_validation -->|"reads graph data"| node_reader
node_exports -->|"reads graph data"| node_reader
node_query_cmd -->|"reads graph inputs"| node_reader
node_query_cmd -->|"writes query results"| node_writer
node_text_embed -->|"reads graph data"| node_reader
node_text_embed -->|"writes vectors"| node_writer
node_graph_embed -->|"reads graph data"| node_reader
node_graph_embed -->|"writes vectors"| node_writer
node_embedding_format -->|"reads vectors"| node_reader
node_embedding_format -->|"writes converted vectors"| node_writer
node_reader -->|"reads rows"| node_tsv
node_writer -->|"writes rows"| node_tsv
node_kg_model -->|"uses graph input"| node_reader

click node_cli "https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/cli_entry.py"
click node_imports "https://github.com/usc-isi-i2/kgtk/tree/main/kgtk/imports"
click node_wikidata_import "https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/cli/import_wikidata.py"
click node_exports "https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/exports/exportwikidata.py"
click node_transform_cmds "https://github.com/usc-isi-i2/kgtk/tree/main/kgtk/cli"
click node_reshape "https://github.com/usc-isi-i2/kgtk/tree/main/kgtk/reshape"
click node_joins "https://github.com/usc-isi-i2/kgtk/tree/main/kgtk/join"
click node_validation "https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/cli/validate.py"
click node_query_cmd "https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/cli/query.py"
click node_parser "https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/kypher/parser.py"
click node_query_engine "https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/kypher/query.py"
click node_sql_store "https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/kypher/sqlstore.py"
click node_analytics "https://github.com/usc-isi-i2/kgtk/tree/main/kgtk/graph_analysis"
click node_lexicalize "https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/cli/lexicalize.py"
click node_text_embed "https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/gt/embedding_utils.py"
click node_graph_embed "https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/graph_embeddings/graph_embeddings.py"
click node_embedding_format "https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/utils/convert_embeddings_format.py"
click node_kg_model "https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/knowledge_graph/knowledge_graph.py"
click node_reader "https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/io/kgtkreader.py"
click node_writer "https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/io/kgtkwriter.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_imports,node_wikidata_import,node_exports toneBlue
class node_transform_cmds,node_reshape,node_joins,node_validation toneAmber
class node_query_cmd,node_parser,node_query_engine,node_sql_store,node_analytics toneMint
class node_lexicalize,node_text_embed,node_graph_embed,node_embedding_format toneRose
class node_kg_model,node_reader,node_writer,node_user,node_tsv toneIndigo
class node_cli toneTeal
```

## Components

### Import and export

- [Graph importers](https://github.com/usc-isi-i2/kgtk/tree/main/kgtk/imports): `kgtk/imports`
- [Wikidata import](https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/cli/import_wikidata.py): `kgtk/cli/import_wikidata.py`
- [Graph exporters](https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/exports/exportwikidata.py): `kgtk/exports/exportwikidata.py`

### Graph operations

- [Graph transforms](https://github.com/usc-isi-i2/kgtk/tree/main/kgtk/cli): `kgtk/cli`
- [Reshape edges](https://github.com/usc-isi-i2/kgtk/tree/main/kgtk/reshape): `kgtk/reshape`
- [Combine graphs](https://github.com/usc-isi-i2/kgtk/tree/main/kgtk/join): `kgtk/join`
- [Graph validation](https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/cli/validate.py): `kgtk/cli/validate.py`

### Query and analytics

- [Kypher command](https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/cli/query.py): `kgtk/cli/query.py`
- [Query parser](https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/kypher/parser.py): `kgtk/kypher/parser.py`
- [Query translation](https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/kypher/query.py): `kgtk/kypher/query.py`
- [Disk query store](https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/kypher/sqlstore.py): `kgtk/kypher/sqlstore.py`
- [Graph analytics](https://github.com/usc-isi-i2/kgtk/tree/main/kgtk/graph_analysis): `kgtk/graph_analysis`

### Graph enrichment

- [Node lexicalization](https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/cli/lexicalize.py): `kgtk/cli/lexicalize.py`
- [Text embeddings](https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/gt/embedding_utils.py): `kgtk/gt/embedding_utils.py`
- [Graph embeddings](https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/graph_embeddings/graph_embeddings.py): `kgtk/graph_embeddings/graph_embeddings.py`
- [Embedding conversion](https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/utils/convert_embeddings_format.py): `kgtk/utils/convert_embeddings_format.py`

### Shared data core

- [Knowledge graph model](https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/knowledge_graph/knowledge_graph.py): `kgtk/knowledge_graph/knowledge_graph.py`
- [KGTK reader](https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/io/kgtkreader.py): `kgtk/io/kgtkreader.py`
- [KGTK writer](https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/io/kgtkwriter.py): `kgtk/io/kgtkwriter.py`

### Other components

- **Engineer**
- [KGTK commands](https://github.com/usc-isi-i2/kgtk/blob/main/kgtk/cli_entry.py): `kgtk/cli_entry.py`
- **KGTK edge files**

## Connections

- Engineer → KGTK commands: runs commands
- KGTK commands → Graph importers: imports sources
- KGTK commands → Wikidata import: imports Wikidata
- KGTK commands → Graph exporters: exports graphs
- KGTK commands → Graph transforms: transforms graphs
- KGTK commands → Reshape edges: reshapes edges
- KGTK commands → Combine graphs: combines graphs
- KGTK commands → Graph validation: validates data
- KGTK commands → Kypher command: runs queries
- Kypher command → Query parser: parses Kypher
- Kypher command → Query translation: translates query
- Kypher command → Disk query store: opens store
- Query translation → Disk query store: queries graph data
- KGTK commands → Graph analytics: analyzes graphs
- KGTK commands → Node lexicalization: lexicalizes nodes
- KGTK commands → Text embeddings: computes text vectors
- KGTK commands → Graph embeddings: computes graph vectors
- KGTK commands → Embedding conversion: converts vectors
- Graph importers → KGTK reader: reads graph data
- Wikidata import → KGTK writer: writes graph data
- Graph transforms → KGTK reader: reads graph data
- Graph transforms → KGTK writer: writes graph data
- Combine graphs → KGTK reader: reads graph data
- Combine graphs → KGTK writer: writes graph data
- Graph validation → KGTK reader: reads graph data
- Graph exporters → KGTK reader: reads graph data
- Kypher command → KGTK reader: reads graph inputs
- Kypher command → KGTK writer: writes query results
- Text embeddings → KGTK reader: reads graph data
- Text embeddings → KGTK writer: writes vectors
- Graph embeddings → KGTK reader: reads graph data
- Graph embeddings → KGTK writer: writes vectors
- Embedding conversion → KGTK reader: reads vectors
- Embedding conversion → KGTK writer: writes converted vectors
- KGTK reader → KGTK edge files: reads rows
- KGTK writer → KGTK edge files: writes rows
- Knowledge graph model → KGTK reader: uses graph input