# safishamsi/graphify architecture

> Architecture diagram of the GitHub repository safishamsi/graphify, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/safishamsi/graphify
- Repository: https://github.com/safishamsi/graphify

## Overview

1. **What this repository is**
   - Graphify is a Python CLI package that turns a codebase or document corpus into a navigable knowledge graph.
   - The main runtime is the `graphify` package, with `graphify/__main__.py` as the CLI entrypoint and `pyproject.toml` defining packaging, extras, and tooling.

2. **Primary pipeline**
   - The core flow is: discover files → extract entities/relationships → normalize IDs/paths → deduplicate and clean up → cluster communities → write graph artifacts.
   - The pipeline is designed to support incremental updates (`--update`), full rebuilds, and cluster-only reprocessing on an existing graph.

3. **Extraction is split into two boundary types**
   - **Local, deterministic extraction** handles source code and structured manifests via AST and parsers. This is the “offline” side of the system.
   - **Semantic extraction** handles docs, PDFs, images, videos, and some workspace formats by calling an LLM backend or transcription pipeline.
   - That boundary is important: code can be indexed without API keys, while non-code content usually depends on external model services or local transcription.

4. **Language and domain extractors**
   - `graphify/extractors/` contains language-specific parsers for edge cases such as Blade, C#, Elixir, Razor, and Zig, with `base.py` as the shared extractor contract.
   - Additional resolution modules like `python`, `java`, `ruby`, `swift`, `csharp`, `symbol_resolution.py`, and `resolver_registry.py` suggest a separate symbol-resolution layer that links cross-file references after raw extraction.
   - Domain-specific introspectors exist for Cargo, PostgreSQL, SCIP, MCP manifests, Google Workspace, and Prisma-like/manifest sources, indicating the graph is not limited to plain source files.

5. **Normalization, identity, and graph hygiene**
   - `ids.py`, `paths.py`, `file_slice.py`, `dedup.py`, `semantic_cleanup.py`, and `validate.py` form the data-shaping layer that makes heterogeneous inputs comparable.
   - The repo emphasizes stable node IDs, path normalization, and deduplication across both AST-derived and semantic nodes.
   - `cache.py` and the `graphify-out/cache/` convention show that extraction is cached as a first-class part of the architecture, especially for repeated runs and incremental rebuilds.

6. **Graph analysis and organization**
   - `cluster.py`, `_minhash.py`, and `global_graph.py` implement higher-level organization of the raw graph into communities and cross-project/global views.
   - `analyze.py`, `reflect.py`, `report.py`, and `benchmark.py` sit on top of the graph to summarize it, generate lessons, and measure quality/performance.
   - Community labeling and “god node” ranking are a central concept in the product, so graph analysis is not a side feature; it is part of the main value proposition.

7. **Outputs and presentation layer**
   - `export.py`, `tree_html.py`, `callflow_html.py`, and `wiki.py` turn the graph into different consumable forms: HTML visualizations, architecture/call-flow pages, wiki-style markdown, and portable graph exports.
   - The canonical artifacts are `graph.json`, `GRAPH_REPORT.md`, and `graph.html` under `graphify-out/`.
   - Additional exports target GraphML, SVG, Neo4j, FalkorDB, Obsidian, and markdown-based documentation workflows.

8. **Serve/query path**
   - `serve.py` exposes the graph as an MCP server over stdio or HTTP, making the graph available to assistants and external clients.
   - `querylog.py` records graph queries, while `query`/`path`/`explain` style commands give read-only access to the graph for exploration.
   - This is a separate runtime path from extraction: once the graph exists, the system becomes a query service rather than a scanner.

9. **Assistant integrations and install-time side effects**
   - A major subsystem is the platform installer layer: `hooks.py`, `install`, `uninstall`, and the many `skill-*.md` / `always_on/*` files.
   - These files wire Graphify into assistant ecosystems such as Claude Code, Codex, Cursor, Copilot, Gemini, Kilo, Trae, Aider, Amp, Devin, and others.
   - The architectural role here is “behavior injection”: the repo doesn’t just build graphs, it also changes how agents ask questions by steering them toward graph queries.

10. **Optional external services and backends**
   - The repo supports multiple LLM backends for semantic extraction and triage: Anthropic, OpenAI-compatible APIs, Gemini, Ollama, Azure OpenAI, DeepSeek, Bedrock, and Claude CLI.
   - Optional integrations extend to Google Workspace, PostgreSQL introspection, MCP, Neo4j, FalkorDB, PDFs, Office files, video transcription, and Terraform/SQL/DreamMaker parsing.
   - These are feature-gated extras in packaging, so the base architecture remains modular and capability-driven.

11. **Repository shape and quality controls**
   - `tests/` is extensive and fixture-heavy, which fits a system where many file formats, languages, and external backends need regression coverage.
   - `worked/` contains real-world example corpora and generated graph outputs, suggesting the project validates itself against representative repositories, not just unit fixtures.
   - `docs/` and `ARCHITECTURE.md` provide product-level and module-level documentation, while CI/release workflows indicate the repo is intended to ship as a distributed CLI tool.

12. **How to think about the boundaries**
   - The cleanest architecture diagram is a three-layer view:  
     1) **Input and extraction** from files, docs, media, manifests, and live systems;  
     2) **Graph processing** for identity, dedup, clustering, cleanup, and reporting;  
     3) **Consumption** via CLI, HTML/wiki exports, MCP server, assistant skills, and external graph databases.
   - The key architectural distinction is between the one-time/build-time graph generation path and the always-available query/integration path.

Main entry points: [`graphify/extract.py`](https://github.com/safishamsi/graphify/blob/v8/graphify/extract.py) (Extract), [`graphify/build.py`](https://github.com/safishamsi/graphify/blob/v8/graphify/build.py) (Build flow), [`graphify/export.py`](https://github.com/safishamsi/graphify/blob/v8/graphify/export.py) (Export), [`graphify/__main__.py`](https://github.com/safishamsi/graphify/blob/v8/graphify/__main__.py) (CLI)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_grp_cli["CLI Runtime"]
  node_n_main(("CLI<br/>entrypoint<br/>[__main__.py]"))
  node_n_build["Build flow<br/>pipeline<br/>[build.py]"]
  node_n_ingest["Ingest<br/>pipeline<br/>[ingest.py]"]
end

subgraph group_grp_extract["Extraction"]
  node_n_detect["Detect<br/>discovery<br/>[detect.py]"]
  node_n_extract["Extract<br/>[extract.py]"]
  node_n_base_ext["Extractor base<br/>contract<br/>[base.py]"]
  node_n_lang_ext["Language parsers"]
  node_n_symbol["Resolution<br/>symbol linking"]
  node_n_resolver_reg["Resolver registry"]
  node_n_semantic["Semantic ingest<br/>llm/media<br/>[llm.py]"]
end

subgraph group_grp_process["Graph Processing"]
  node_n_ids["IDs<br/>normalization<br/>[ids.py]"]
  node_n_paths["Paths<br/>normalization<br/>[paths.py]"]
  node_n_dedup["Dedup<br/>graph hygiene<br/>[dedup.py]"]
  node_n_cleanup["Cleanup<br/>graph hygiene"]
  node_n_cluster["Cluster<br/>community detection<br/>[cluster.py]"]
  node_n_global["Global graph<br/>aggregate view<br/>[global_graph.py]"]
  node_n_analyze["Analyze<br/>analysis<br/>[analyze.py]"]
  node_n_cache[("Cache<br/>persistence<br/>[cache.py]")]
end

subgraph group_grp_outputs["Outputs"]
  node_n_report["Report<br/>[report.py]"]
  node_n_export["Export<br/>[export.py]"]
  node_n_tree["Tree HTML<br/>visualization<br/>[tree_html.py]"]
  node_n_callflow["Callflow HTML<br/>visualization<br/>[callflow_html.py]"]
  node_n_wiki["Wiki<br/>documentation export<br/>[wiki.py]"]
  node_n_serve["Serve<br/>query service<br/>[serve.py]"]
  node_n_querylog["Query log<br/>telemetry<br/>[querylog.py]"]
end

subgraph group_grp_integrations["Integrations"]
  node_n_hooks["Hooks<br/>assistant install<br/>[hooks.py]"]
  node_n_skillgen["Skillgen<br/>content generator<br/>[gen.py]"]
  node_n_mcp["MCP ingest<br/>adapter<br/>[mcp_ingest.py]"]
  node_n_manifest["Manifests<br/>adapter<br/>[manifest.py]"]
  node_n_workspace["Workspace<br/>adapter"]
  node_n_pg["Postgres<br/>adapter<br/>[pg_introspect.py]"]
  node_n_cargo["Cargo<br/>adapter"]
  node_n_scip["SCIP<br/>adapter<br/>[scip_ingest.py]"]
end

node_n_main -->|"runs"| node_n_build
node_n_main -->|"serves"| node_n_serve
node_n_main -->|"exports"| node_n_export
node_n_build -->|"orchestrates"| node_n_ingest
node_n_ingest -->|"detects"| node_n_detect
node_n_detect -->|"routes to"| node_n_extract
node_n_extract -->|"implements"| node_n_base_ext
node_n_extract -->|"uses"| node_n_lang_ext
node_n_extract -->|"links"| node_n_symbol
node_n_symbol -->|"dispatches"| node_n_resolver_reg
node_n_extract -.->|"falls back to"| node_n_semantic
node_n_extract -.->|"ingests"| node_n_mcp
node_n_extract -->|"parses"| node_n_manifest
node_n_extract -.->|"supports"| node_n_workspace
node_n_extract -.->|"introspects"| node_n_pg
node_n_extract -.->|"introspects"| node_n_cargo
node_n_extract -.->|"imports"| node_n_scip
node_n_ingest -->|"uses"| node_n_cache
node_n_build -->|"normalizes"| node_n_ids
node_n_build -->|"normalizes"| node_n_paths
node_n_build -->|"cleans"| node_n_dedup
node_n_build -->|"validates"| node_n_cleanup
node_n_build -->|"organizes"| node_n_cluster
node_n_cluster -->|"feeds"| node_n_global
node_n_global -->|"summarizes"| node_n_analyze
node_n_analyze -->|"generates"| node_n_report
node_n_build -->|"writes"| node_n_export
node_n_export -->|"renders"| node_n_tree
node_n_export -->|"renders"| node_n_callflow
node_n_export -->|"publishes"| node_n_wiki
node_n_serve -->|"records"| node_n_querylog
node_n_hooks -.->|"injects"| node_n_main
node_n_skillgen -.->|"generates"| node_n_hooks

click node_n_main "https://github.com/safishamsi/graphify/blob/v8/graphify/__main__.py"
click node_n_build "https://github.com/safishamsi/graphify/blob/v8/graphify/build.py"
click node_n_ingest "https://github.com/safishamsi/graphify/blob/v8/graphify/ingest.py"
click node_n_detect "https://github.com/safishamsi/graphify/blob/v8/graphify/detect.py"
click node_n_extract "https://github.com/safishamsi/graphify/blob/v8/graphify/extract.py"
click node_n_base_ext "https://github.com/safishamsi/graphify/blob/v8/graphify/extractors/base.py"
click node_n_lang_ext "https://github.com/safishamsi/graphify/tree/v8/graphify/extractors"
click node_n_symbol "https://github.com/safishamsi/graphify/blob/v8/graphify/symbol_resolution.py"
click node_n_resolver_reg "https://github.com/safishamsi/graphify/blob/v8/graphify/resolver_registry.py"
click node_n_semantic "https://github.com/safishamsi/graphify/blob/v8/graphify/llm.py"
click node_n_ids "https://github.com/safishamsi/graphify/blob/v8/graphify/ids.py"
click node_n_paths "https://github.com/safishamsi/graphify/blob/v8/graphify/paths.py"
click node_n_dedup "https://github.com/safishamsi/graphify/blob/v8/graphify/dedup.py"
click node_n_cleanup "https://github.com/safishamsi/graphify/blob/v8/graphify/semantic_cleanup.py"
click node_n_cluster "https://github.com/safishamsi/graphify/blob/v8/graphify/cluster.py"
click node_n_global "https://github.com/safishamsi/graphify/blob/v8/graphify/global_graph.py"
click node_n_analyze "https://github.com/safishamsi/graphify/blob/v8/graphify/analyze.py"
click node_n_report "https://github.com/safishamsi/graphify/blob/v8/graphify/report.py"
click node_n_export "https://github.com/safishamsi/graphify/blob/v8/graphify/export.py"
click node_n_tree "https://github.com/safishamsi/graphify/blob/v8/graphify/tree_html.py"
click node_n_callflow "https://github.com/safishamsi/graphify/blob/v8/graphify/callflow_html.py"
click node_n_wiki "https://github.com/safishamsi/graphify/blob/v8/graphify/wiki.py"
click node_n_serve "https://github.com/safishamsi/graphify/blob/v8/graphify/serve.py"
click node_n_querylog "https://github.com/safishamsi/graphify/blob/v8/graphify/querylog.py"
click node_n_hooks "https://github.com/safishamsi/graphify/blob/v8/graphify/hooks.py"
click node_n_skillgen "https://github.com/safishamsi/graphify/blob/v8/tools/skillgen/gen.py"
click node_n_mcp "https://github.com/safishamsi/graphify/blob/v8/graphify/mcp_ingest.py"
click node_n_manifest "https://github.com/safishamsi/graphify/blob/v8/graphify/manifest.py"
click node_n_workspace "https://github.com/safishamsi/graphify/blob/v8/graphify/google_workspace.py"
click node_n_pg "https://github.com/safishamsi/graphify/blob/v8/graphify/pg_introspect.py"
click node_n_cargo "https://github.com/safishamsi/graphify/blob/v8/graphify/cargo_introspect.py"
click node_n_scip "https://github.com/safishamsi/graphify/blob/v8/graphify/scip_ingest.py"
click node_n_cache "https://github.com/safishamsi/graphify/blob/v8/graphify/cache.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n_main,node_n_build,node_n_ingest toneBlue
class node_n_detect,node_n_extract,node_n_base_ext,node_n_lang_ext,node_n_symbol,node_n_resolver_reg,node_n_semantic toneAmber
class node_n_ids,node_n_paths,node_n_dedup,node_n_cleanup,node_n_cluster,node_n_global,node_n_analyze,node_n_cache toneMint
class node_n_report,node_n_export,node_n_tree,node_n_callflow,node_n_wiki,node_n_serve,node_n_querylog toneRose
class node_n_hooks,node_n_skillgen,node_n_mcp,node_n_manifest,node_n_workspace,node_n_pg,node_n_cargo,node_n_scip toneIndigo
```

## Components

### CLI Runtime

Command entrypoints and run modes for building or querying graphs.

- [CLI](https://github.com/safishamsi/graphify/blob/v8/graphify/__main__.py): entrypoint · Package entrypoint and command dispatch for graph build, update, query, and serve flows. · `graphify/__main__.py`
- [Build flow](https://github.com/safishamsi/graphify/blob/v8/graphify/build.py): pipeline · Orchestrates discovery, extraction, normalization, clustering, and artifact emission. · `graphify/build.py`
- [Ingest](https://github.com/safishamsi/graphify/blob/v8/graphify/ingest.py): pipeline · Coordinates input ingestion and incremental update behavior across source types. · `graphify/ingest.py`

### Extraction

Deterministic parsing, semantic ingestion, and resolver layers that turn inputs into graph entities.

- [Detect](https://github.com/safishamsi/graphify/blob/v8/graphify/detect.py): discovery · Identifies likely input types and routes content into the appropriate extraction path. · `graphify/detect.py`
- [Extract](https://github.com/safishamsi/graphify/blob/v8/graphify/extract.py): core extraction · Top-level extraction orchestration for code, docs, and structured manifests. · `graphify/extract.py`
- [Extractor base](https://github.com/safishamsi/graphify/blob/v8/graphify/extractors/base.py): contract · Shared extractor interface used by language and domain-specific parsers. · `graphify/extractors/base.py`
- [Language parsers](https://github.com/safishamsi/graphify/tree/v8/graphify/extractors): parsers · Specialized source parsers for edge-case languages and syntaxes. · `graphify/extractors`
- [Resolution](https://github.com/safishamsi/graphify/blob/v8/graphify/symbol_resolution.py): symbol linking · Cross-file symbol and reference resolution after raw extraction. · `graphify/symbol_resolution.py`
- [Resolver registry](https://github.com/safishamsi/graphify/blob/v8/graphify/resolver_registry.py): registry · Selects and dispatches language-specific resolution strategies. · `graphify/resolver_registry.py`
- [Semantic ingest](https://github.com/safishamsi/graphify/blob/v8/graphify/llm.py): llm/media · LLM-backed or transcription-based extraction for docs, images, video, and workspace content. · `graphify/llm.py`

### Graph Processing

Identity, cleanup, clustering, and analysis steps that shape the graph.

- [IDs](https://github.com/safishamsi/graphify/blob/v8/graphify/ids.py): normalization · Stable identity generation for graph nodes and files. · `graphify/ids.py`
- [Paths](https://github.com/safishamsi/graphify/blob/v8/graphify/paths.py): normalization · Path canonicalization and repository-relative normalization. · `graphify/paths.py`
- [Dedup](https://github.com/safishamsi/graphify/blob/v8/graphify/dedup.py): graph hygiene · Removes repeated nodes and edges across heterogeneous inputs. · `graphify/dedup.py`
- [Cleanup](https://github.com/safishamsi/graphify/blob/v8/graphify/semantic_cleanup.py): graph hygiene · Semantic cleanup and validation of extracted graph content. · `graphify/semantic_cleanup.py`
- [Cluster](https://github.com/safishamsi/graphify/blob/v8/graphify/cluster.py): community detection · Groups related nodes into communities and higher-level structure. · `graphify/cluster.py`
- [Global graph](https://github.com/safishamsi/graphify/blob/v8/graphify/global_graph.py): aggregate view · Builds cross-project or cross-corpus graph views and rankings. · `graphify/global_graph.py`
- [Analyze](https://github.com/safishamsi/graphify/blob/v8/graphify/analyze.py): analysis · Summarizes graph structure and extracts architectural signals. · `graphify/analyze.py`
- [Cache](https://github.com/safishamsi/graphify/blob/v8/graphify/cache.py): persistence · Extraction cache used for incremental and repeated runs. · `graphify/cache.py`

### Outputs

Artifact generation, visualizations, and query-serving surfaces.

- [Report](https://github.com/safishamsi/graphify/blob/v8/graphify/report.py): reporting · Generates human-readable summaries and quality narratives from the graph. · `graphify/report.py`
- [Export](https://github.com/safishamsi/graphify/blob/v8/graphify/export.py): artifact export · Writes graph artifacts to JSON and external export formats. · `graphify/export.py`
- [Tree HTML](https://github.com/safishamsi/graphify/blob/v8/graphify/tree_html.py): visualization · Renders navigable tree-style HTML views of the graph. · `graphify/tree_html.py`
- [Callflow HTML](https://github.com/safishamsi/graphify/blob/v8/graphify/callflow_html.py): visualization · Renders call-flow oriented HTML views for navigation. · `graphify/callflow_html.py`
- [Wiki](https://github.com/safishamsi/graphify/blob/v8/graphify/wiki.py): documentation export · Produces wiki-style markdown documentation from graph data. · `graphify/wiki.py`
- [Serve](https://github.com/safishamsi/graphify/blob/v8/graphify/serve.py): query service · Exposes graph query and inspection capabilities as an MCP server over stdio or HTTP. · `graphify/serve.py`
- [Query log](https://github.com/safishamsi/graphify/blob/v8/graphify/querylog.py): telemetry · Records graph queries for later inspection and debugging. · `graphify/querylog.py`

### Integrations

Assistant platform wiring, external backends, and optional service adapters.

- [Hooks](https://github.com/safishamsi/graphify/blob/v8/graphify/hooks.py): assistant install · Installs and manages assistant-platform hooks and behavior injection. · `graphify/hooks.py`
- [Skillgen](https://github.com/safishamsi/graphify/blob/v8/tools/skillgen/gen.py): content generator · Generates assistant skill files and platform-specific reference bundles. · `tools/skillgen/gen.py`
- [MCP ingest](https://github.com/safishamsi/graphify/blob/v8/graphify/mcp_ingest.py): adapter · Imports or bridges MCP-oriented content into the graph pipeline. · `graphify/mcp_ingest.py`
- [Manifests](https://github.com/safishamsi/graphify/blob/v8/graphify/manifest.py): adapter · Parses structured manifests and workspace definitions. · `graphify/manifest.py`
- [Workspace](https://github.com/safishamsi/graphify/blob/v8/graphify/google_workspace.py): adapter · Google Workspace integration for document-oriented corpora. · `graphify/google_workspace.py`
- [Postgres](https://github.com/safishamsi/graphify/blob/v8/graphify/pg_introspect.py): adapter · Database introspection for PostgreSQL schemas and objects. · `graphify/pg_introspect.py`
- [Cargo](https://github.com/safishamsi/graphify/blob/v8/graphify/cargo_introspect.py): adapter · Rust package introspection for crate-level graph generation. · `graphify/cargo_introspect.py`
- [SCIP](https://github.com/safishamsi/graphify/blob/v8/graphify/scip_ingest.py): adapter · SCIP index ingestion for symbol-aware code graphs. · `graphify/scip_ingest.py`

## Connections

- CLI → Build flow: runs · CLI invokes the main build pipeline.
- CLI → Serve: serves · CLI can expose the graph as a service.
- CLI → Export: exports · CLI can emit graph artifacts and formats.
- Build flow → Ingest: orchestrates · Build flow starts by ingesting repository inputs.
- Ingest → Detect: detects · Input discovery identifies file and corpus types.
- Detect → Extract: routes to · Detected content is sent into extraction.
- Extract → Extractor base: implements · Extractors share a common contract.
- Extract → Language parsers: uses · Language-specific parsers handle source edge cases.
- Extract → Resolution: links · Raw extraction is followed by symbol/reference resolution.
- Resolution → Resolver registry: dispatches · Registry selects the appropriate resolver implementation.
- Extract → Semantic ingest: falls back to · Non-code and media inputs use semantic extraction.
- Extract → MCP ingest: ingests · MCP-oriented inputs can enter through dedicated adapters.
- Extract → Manifests: parses · Structured manifests are recognized as a first-class input.
- Extract → Workspace: supports · Workspace content is handled through a dedicated adapter.
- Extract → Postgres: introspects · Database schema extraction is supported.
- Extract → Cargo: introspects · Rust crate metadata can be ingested structurally.
- Extract → SCIP: imports · SCIP indexes can seed the graph with symbol data.
- Ingest → Cache: uses · Incremental runs reuse cached extraction results.
- Build flow → IDs: normalizes · Identity generation is applied during graph shaping.
- Build flow → Paths: normalizes · Paths are canonicalized across inputs.
- Build flow → Dedup: cleans · Duplicate entities and relationships are removed.
- Build flow → Cleanup: validates · Graph hygiene and semantic cleanup run before output.
- Build flow → Cluster: organizes · Communities are computed from the cleaned graph.
- Cluster → Global graph: feeds · Clustered structure informs global graph views.
- Global graph → Analyze: summarizes · Higher-level graph structure is analyzed.
- Analyze → Report: generates · Analysis produces readable reports and lessons.
- Build flow → Export: writes · The finalized graph is exported to artifacts and backends.
- Export → Tree HTML: renders · Tree-based HTML visualization is produced from the graph.
- Export → Callflow HTML: renders · Call-flow visualization is produced from the graph.
- Export → Wiki: publishes · Wiki-style documentation is generated.
- Serve → Query log: records · Query activity is logged for diagnostics and feedback.
- Hooks → CLI: injects · Assistant platform hooks steer users toward graph-centric workflows.
- Skillgen → Hooks: generates · Skill generation produces platform-specific assistant assets.
