# npc-worldwide/npcpy architecture

> Architecture diagram of the GitHub repository npc-worldwide/npcpy, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/npc-worldwide/npcpy
- Repository: https://github.com/npc-worldwide/npcpy

## Overview

npcpy is a Python library for building model-backed personas, tool-using agents, and multimodal or knowledge-grounded AI applications. A typical user creates an NPC or agent, submits a request, and receives a model response; agents can also use tools and knowledge features. The repository additionally provides multimodal data handling and generation, knowledge-graph and retrieval capabilities, fine-tuning, SQL integration, and serving. Several connections below rely on the SOURCE INDEX’s import/name references or README descriptions; unsampled implementation details are not asserted.

Main entry points: [`npcpy/npc_compiler.py`](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/npc_compiler.py) (NPC and agents), [`npcpy/llm_funcs.py`](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/llm_funcs.py) (LLM functions), [`npcpy/memory/search.py`](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/memory/search.py) (Retrieval search), [`npcpy/gen/response.py`](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/gen/response.py) (Model responses)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_agents["Agent runtime"]
  node_npc["NPC and agents<br/>[npc_compiler.py]"]
  node_llm["LLM functions<br/>[llm_funcs.py]"]
  node_serving["HTTP serving<br/>[serve.py]"]
  node_streaming["Streaming<br/>[streaming.py]"]
  node_tools["Agent tools<br/>[tools.py]"]
  node_workflows["Planning and triggers<br/>[plan.py]"]
end

subgraph group_models["Models and media"]
  node_response["Model responses<br/>[response.py]"]
  node_imagegen["Image generation<br/>[image_gen.py]"]
  node_audiogen["Audio generation<br/>[audio_gen.py]"]
  node_videogen["Video generation<br/>[video_gen.py]"]
  node_multimodal["Media loading<br/>[load.py]"]
  node_audio["Audio processing<br/>[audio.py]"]
end

subgraph group_knowledge["Knowledge and data"]
  node_web["Web search<br/>[web.py]"]
  node_search["Retrieval search<br/>[search.py]"]
  node_embeddings["Embeddings<br/>[embeddings.py]"]
  node_kg[("Knowledge graph<br/>[knowledge_graph.py]")]
  node_kg_population["Graph population<br/>[kg_population.py]"]
  node_store[("Knowledge store<br/>[knowledge_store.py]")]
  node_index[("Knowledge index<br/>[knowledge_index.py]")]
end

subgraph group_extensions["Application extensions"]
  node_finetune["Model fine-tuning<br/>[sft.py]"]
  node_diffusion["Diffusion training<br/>[diff.py]"]
  node_sql["AI SQL<br/>[npcsql.py]"]
end

node_user(("User"))
node_provider{{"Model providers"}}

node_user -->|"creates and invokes"| node_npc
node_npc -->|"requests responses"| node_llm
node_llm -->|"dispatches generation"| node_response
node_response -.->|"queries models"| node_provider
node_npc -->|"uses tools"| node_tools
node_npc -.->|"integrates serving"| node_serving
node_npc -.->|"uses streaming"| node_streaming
node_npc -.->|"plans tasks"| node_workflows
node_llm -->|"calls generation"| node_imagegen
node_llm -->|"calls generation"| node_videogen
node_response -->|"loads attachments"| node_multimodal
node_response -.->|"processes audio"| node_audio
node_search -->|"searches web"| node_web
node_search -->|"loads files"| node_multimodal
node_search -->|"embeds queries"| node_embeddings
node_search -->|"requests answers"| node_llm
node_kg_population -->|"populates graph"| node_kg
node_kg_population -->|"extracts knowledge"| node_llm
node_store -->|"uses graph"| node_kg
node_store -.->|"processes memories"| node_llm
node_npc -->|"uses knowledge graph"| node_kg
node_npc -->|"uses knowledge store"| node_store
node_npc -.->|"supports fine-tuning"| node_finetune
node_npc -.->|"supports image training"| node_diffusion
node_npc -.->|"integrates AI SQL"| node_sql
node_audiogen -->|"uses audio data"| node_audio

click node_npc "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/npc_compiler.py"
click node_llm "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/llm_funcs.py"
click node_serving "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/serve.py"
click node_streaming "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/streaming.py"
click node_tools "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/tools.py"
click node_workflows "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/work/plan.py"
click node_response "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/gen/response.py"
click node_imagegen "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/gen/image_gen.py"
click node_audiogen "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/gen/audio_gen.py"
click node_videogen "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/gen/video_gen.py"
click node_multimodal "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/data/load.py"
click node_audio "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/data/audio.py"
click node_web "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/data/web.py"
click node_search "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/memory/search.py"
click node_embeddings "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/gen/embeddings.py"
click node_kg "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/memory/knowledge_graph.py"
click node_kg_population "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/memory/kg_population.py"
click node_store "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/memory/knowledge_store.py"
click node_index "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/memory/knowledge_index.py"
click node_finetune "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/ft/sft.py"
click node_diffusion "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/ft/diff.py"
click node_sql "https://github.com/npc-worldwide/npcpy/blob/main/npcpy/sql/npcsql.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_npc,node_llm,node_serving,node_streaming,node_tools,node_workflows,node_user toneBlue
class node_response,node_imagegen,node_audiogen,node_videogen,node_multimodal,node_audio toneAmber
class node_web,node_search,node_embeddings,node_kg,node_kg_population,node_store,node_index toneMint
class node_finetune,node_diffusion,node_sql toneRose
class node_provider toneIndigo
```

## Components

### Agent runtime

- [NPC and agents](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/npc_compiler.py): `npcpy/npc_compiler.py`
- [LLM functions](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/llm_funcs.py): `npcpy/llm_funcs.py`
- [HTTP serving](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/serve.py): `npcpy/serve.py`
- [Streaming](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/streaming.py): `npcpy/streaming.py`
- [Agent tools](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/tools.py): `npcpy/tools.py`
- [Planning and triggers](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/work/plan.py): `npcpy/work/plan.py`

### Models and media

- [Model responses](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/gen/response.py): `npcpy/gen/response.py`
- [Image generation](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/gen/image_gen.py): `npcpy/gen/image_gen.py`
- [Audio generation](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/gen/audio_gen.py): `npcpy/gen/audio_gen.py`
- [Video generation](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/gen/video_gen.py): `npcpy/gen/video_gen.py`
- [Media loading](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/data/load.py): `npcpy/data/load.py`
- [Audio processing](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/data/audio.py): `npcpy/data/audio.py`

### Knowledge and data

- [Web search](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/data/web.py): `npcpy/data/web.py`
- [Retrieval search](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/memory/search.py): `npcpy/memory/search.py`
- [Embeddings](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/gen/embeddings.py): `npcpy/gen/embeddings.py`
- [Knowledge graph](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/memory/knowledge_graph.py): `npcpy/memory/knowledge_graph.py`
- [Graph population](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/memory/kg_population.py): `npcpy/memory/kg_population.py`
- [Knowledge store](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/memory/knowledge_store.py): `npcpy/memory/knowledge_store.py`
- [Knowledge index](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/memory/knowledge_index.py): `npcpy/memory/knowledge_index.py`

### Application extensions

- [Model fine-tuning](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/ft/sft.py): `npcpy/ft/sft.py`
- [Diffusion training](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/ft/diff.py): `npcpy/ft/diff.py`
- [AI SQL](https://github.com/npc-worldwide/npcpy/blob/main/npcpy/sql/npcsql.py): `npcpy/sql/npcsql.py`

### Other components

- **User**
- **Model providers**

## Connections

- User → NPC and agents: creates and invokes
- NPC and agents → LLM functions: requests responses (evidence: `npcpy/npc_compiler.py`)
- LLM functions → Model responses: dispatches generation (evidence: `npcpy/llm_funcs.py`)
- Model responses → Model providers: queries models
- NPC and agents → Agent tools: uses tools (evidence: `npcpy/npc_compiler.py`)
- NPC and agents → HTTP serving: integrates serving (evidence: `npcpy/npc_compiler.py`)
- NPC and agents → Streaming: uses streaming (evidence: `npcpy/npc_compiler.py`)
- NPC and agents → Planning and triggers: plans tasks
- LLM functions → Image generation: calls generation (evidence: `npcpy/llm_funcs.py`)
- LLM functions → Video generation: calls generation (evidence: `npcpy/llm_funcs.py`)
- Model responses → Media loading: loads attachments (evidence: `npcpy/gen/response.py`)
- Model responses → Audio processing: processes audio
- Retrieval search → Web search: searches web (evidence: `npcpy/memory/search.py`)
- Retrieval search → Media loading: loads files (evidence: `npcpy/memory/search.py`)
- Retrieval search → Embeddings: embeds queries (evidence: `npcpy/memory/search.py`)
- Retrieval search → LLM functions: requests answers (evidence: `npcpy/memory/search.py`)
- Graph population → Knowledge graph: populates graph (evidence: `npcpy/memory/kg_population.py`)
- Graph population → LLM functions: extracts knowledge (evidence: `npcpy/memory/kg_population.py`)
- Knowledge store → Knowledge graph: uses graph (evidence: `npcpy/memory/knowledge_store.py`)
- Knowledge store → LLM functions: processes memories (evidence: `npcpy/memory/knowledge_store.py`)
- NPC and agents → Knowledge graph: uses knowledge graph (evidence: `npcpy/npc_compiler.py`)
- NPC and agents → Knowledge store: uses knowledge store (evidence: `npcpy/npc_compiler.py`)
- NPC and agents → Model fine-tuning: supports fine-tuning
- NPC and agents → Diffusion training: supports image training
- NPC and agents → AI SQL: integrates AI SQL (evidence: `npcpy/sql/npcsql.py`)
- Audio generation → Audio processing: uses audio data (evidence: `npcpy/data/audio.py`)
