# k-abhishek/ai-matrix architecture

> Architecture diagram of the GitHub repository k-abhishek/ai-matrix, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/k-abhishek/ai-matrix
- Repository: https://github.com/k-abhishek/ai-matrix

## Overview

This is a crawler-management and semantic knowledge-base application with a React UI and Fastify API. Users configure crawlers, start and monitor crawl runs, index extracted website content, then search or chat against that content. The README documents the crawl-to-vector-indexing workflow; sampled backend code confirms semantic search and crawler management. Several UI pages and backend routes were not sampled, so those links are shown only where source references or README evidence support them.

Main entry points: [`server/src/services/crawl.service.ts`](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/services/crawl.service.ts) (Crawl and index pipeline), [`server/src/services/qdrant.service.ts`](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/services/qdrant.service.ts) (Vector index), [`client/src/App.tsx`](https://github.com/k-abhishek/ai-matrix/blob/main/client/src/App.tsx) (React application), [`client/src/api/client.ts`](https://github.com/k-abhishek/ai-matrix/blob/main/client/src/api/client.ts) (HTTP API client)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ui["Web interface"]
  node_app["React application<br/>[App.tsx]"]
  node_crawler_ui["Crawler management UI"]
  node_search_ui["Search interface<br/>[SearchPage.tsx]"]
  node_chat_ui["Chat interface<br/>[ChatPage.tsx]"]
  node_api_client["HTTP API client<br/>[client.ts]"]
  node_knowledge_api["Knowledge API<br/>[knowledgeBase.ts]"]
end

subgraph group_api["API layer"]
  node_api_app["Fastify application<br/>[app.ts]"]
  node_crawler_routes["Crawler routes<br/>[crawlerRoutes.ts]"]
  node_crawler_controller["Crawler controller"]
  node_search_routes["Search routes<br/>[searchRoutes.ts]"]
  node_search_controller["Search controller"]
  node_chat_routes["Chat routes<br/>[chatRoutes.ts]"]
end

subgraph group_crawl["Crawling and indexing"]
  node_crawler_service["Crawler service<br/>[crawler.service.ts]"]
  node_crawl_runs["Crawl-run management"]
  node_crawl_pipeline["Crawl and index pipeline<br/>[crawl.service.ts]"]
  node_sitemap["Sitemap discovery<br/>[sitemap.service.ts]"]
  node_fetcher["Page fetching<br/>[fetcher.service.ts]"]
  node_extractor["Content extraction"]
  node_chunking["Content chunking"]
end

subgraph group_knowledge["Knowledge and chat"]
  node_embedding["Embedding generation"]
  node_vectors[("Vector index<br/>[qdrant.service.ts]")]
  node_storage["Storage budgeting<br/>[storage.service.ts]"]
  node_search_service["Semantic search<br/>[search.service.ts]"]
  node_chat_service["Chat service<br/>[chat.service.ts]"]
end

subgraph group_platform["Runtime and persistence"]
  node_chat_store[("Chat persistence<br/>[chat.repository.ts]")]
  node_database[("PostgreSQL")]
  node_config["Environment configuration<br/>[env.ts]"]
end

node_user(("User"))

node_user -->|"uses"| node_app
node_app -.->|"routes to"| node_crawler_ui
node_app -->|"routes to"| node_search_ui
node_app -->|"routes to"| node_chat_ui
node_search_ui -->|"requests via"| node_api_client
node_search_ui -->|"calls"| node_knowledge_api
node_knowledge_api -->|"sends requests"| node_api_client
node_crawler_ui -.->|"requests via"| node_api_client
node_chat_ui -.->|"requests via"| node_api_client
node_api_app -.->|"registers"| node_crawler_routes
node_crawler_routes -->|"dispatches"| node_crawler_controller
node_crawler_controller -->|"calls"| node_crawler_service
node_crawler_service -->|"starts crawl"| node_crawl_pipeline
node_crawl_runs -.->|"persists runs"| node_database
node_crawl_pipeline -->|"uses"| node_sitemap
node_crawl_pipeline -->|"fetches pages"| node_fetcher
node_crawl_pipeline -->|"extracts content"| node_extractor
node_crawl_pipeline -->|"chunks content"| node_chunking
node_crawl_pipeline -->|"generates vectors"| node_embedding
node_crawl_pipeline -->|"indexes chunks"| node_vectors
node_crawler_service -->|"checks budget"| node_storage
node_storage -->|"reads statistics"| node_vectors
node_api_app -.->|"registers"| node_search_routes
node_search_routes -.->|"dispatches"| node_search_controller
node_search_controller -->|"calls"| node_search_service
node_search_service -->|"searches"| node_vectors
node_api_app -.->|"registers"| node_chat_routes
node_chat_routes -->|"calls"| node_chat_service
node_chat_service -->|"persists threads"| node_chat_store
node_chat_service -.->|"retrieves knowledge"| node_vectors
node_chat_store -.->|"stores threads"| node_database
node_embedding -->|"reads settings"| node_config
node_vectors -->|"embeds queries"| node_embedding

click node_app "https://github.com/k-abhishek/ai-matrix/blob/main/client/src/App.tsx"
click node_crawler_ui "https://github.com/k-abhishek/ai-matrix/blob/main/client/src/pages/CrawlerDetailPage.tsx"
click node_search_ui "https://github.com/k-abhishek/ai-matrix/blob/main/client/src/pages/SearchPage.tsx"
click node_chat_ui "https://github.com/k-abhishek/ai-matrix/blob/main/client/src/pages/ChatPage.tsx"
click node_api_client "https://github.com/k-abhishek/ai-matrix/blob/main/client/src/api/client.ts"
click node_knowledge_api "https://github.com/k-abhishek/ai-matrix/blob/main/client/src/api/knowledgeBase.ts"
click node_api_app "https://github.com/k-abhishek/ai-matrix/blob/main/server/src/app.ts"
click node_crawler_routes "https://github.com/k-abhishek/ai-matrix/blob/main/server/src/routes/crawlerRoutes.ts"
click node_crawler_controller "https://github.com/k-abhishek/ai-matrix/blob/main/server/src/modules/crawlers/crawler.controller.ts"
click node_crawler_service "https://github.com/k-abhishek/ai-matrix/blob/main/server/src/modules/crawlers/crawler.service.ts"
click node_crawl_runs "https://github.com/k-abhishek/ai-matrix/blob/main/server/src/modules/crawl-runs/crawl-run.service.ts"
click node_crawl_pipeline "https://github.com/k-abhishek/ai-matrix/blob/main/server/src/services/crawl.service.ts"
click node_sitemap "https://github.com/k-abhishek/ai-matrix/blob/main/server/src/services/sitemap.service.ts"
click node_fetcher "https://github.com/k-abhishek/ai-matrix/blob/main/server/src/services/fetcher.service.ts"
click node_extractor "https://github.com/k-abhishek/ai-matrix/blob/main/server/src/services/content-extractor.service.ts"
click node_chunking "https://github.com/k-abhishek/ai-matrix/blob/main/server/src/services/chunking.service.ts"
click node_embedding "https://github.com/k-abhishek/ai-matrix/blob/main/server/src/services/embedding.service.ts"
click node_vectors "https://github.com/k-abhishek/ai-matrix/blob/main/server/src/services/qdrant.service.ts"
click node_storage "https://github.com/k-abhishek/ai-matrix/blob/main/server/src/services/storage.service.ts"
click node_search_routes "https://github.com/k-abhishek/ai-matrix/blob/main/server/src/routes/searchRoutes.ts"
click node_search_controller "https://github.com/k-abhishek/ai-matrix/blob/main/server/src/modules/search/search.controller.ts"
click node_search_service "https://github.com/k-abhishek/ai-matrix/blob/main/server/src/modules/search/search.service.ts"
click node_chat_routes "https://github.com/k-abhishek/ai-matrix/blob/main/server/src/routes/chatRoutes.ts"
click node_chat_service "https://github.com/k-abhishek/ai-matrix/blob/main/server/src/modules/chat/chat.service.ts"
click node_chat_store "https://github.com/k-abhishek/ai-matrix/blob/main/server/src/modules/chat/chat.repository.ts"
click node_config "https://github.com/k-abhishek/ai-matrix/blob/main/server/src/config/env.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_crawler_ui,node_search_ui,node_chat_ui,node_api_client,node_knowledge_api,node_user toneBlue
class node_api_app,node_crawler_routes,node_crawler_controller,node_search_routes,node_search_controller,node_chat_routes toneAmber
class node_crawler_service,node_crawl_runs,node_crawl_pipeline,node_sitemap,node_fetcher,node_extractor,node_chunking toneMint
class node_embedding,node_vectors,node_storage,node_search_service,node_chat_service toneRose
class node_chat_store,node_database,node_config toneIndigo
```

## Components

### Web interface

- [React application](https://github.com/k-abhishek/ai-matrix/blob/main/client/src/App.tsx): `client/src/App.tsx`
- [Crawler management UI](https://github.com/k-abhishek/ai-matrix/blob/main/client/src/pages/CrawlerDetailPage.tsx): `client/src/pages/CrawlerDetailPage.tsx`
- [Search interface](https://github.com/k-abhishek/ai-matrix/blob/main/client/src/pages/SearchPage.tsx): `client/src/pages/SearchPage.tsx`
- [Chat interface](https://github.com/k-abhishek/ai-matrix/blob/main/client/src/pages/ChatPage.tsx): `client/src/pages/ChatPage.tsx`
- [HTTP API client](https://github.com/k-abhishek/ai-matrix/blob/main/client/src/api/client.ts): `client/src/api/client.ts`
- [Knowledge API](https://github.com/k-abhishek/ai-matrix/blob/main/client/src/api/knowledgeBase.ts): `client/src/api/knowledgeBase.ts`

### API layer

- [Fastify application](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/app.ts): `server/src/app.ts`
- [Crawler routes](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/routes/crawlerRoutes.ts): `server/src/routes/crawlerRoutes.ts`
- [Crawler controller](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/modules/crawlers/crawler.controller.ts): `server/src/modules/crawlers/crawler.controller.ts`
- [Search routes](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/routes/searchRoutes.ts): `server/src/routes/searchRoutes.ts`
- [Search controller](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/modules/search/search.controller.ts): `server/src/modules/search/search.controller.ts`
- [Chat routes](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/routes/chatRoutes.ts): `server/src/routes/chatRoutes.ts`

### Crawling and indexing

- [Crawler service](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/modules/crawlers/crawler.service.ts): `server/src/modules/crawlers/crawler.service.ts`
- [Crawl-run management](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/modules/crawl-runs/crawl-run.service.ts): `server/src/modules/crawl-runs/crawl-run.service.ts`
- [Crawl and index pipeline](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/services/crawl.service.ts): `server/src/services/crawl.service.ts`
- [Sitemap discovery](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/services/sitemap.service.ts): `server/src/services/sitemap.service.ts`
- [Page fetching](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/services/fetcher.service.ts): `server/src/services/fetcher.service.ts`
- [Content extraction](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/services/content-extractor.service.ts): `server/src/services/content-extractor.service.ts`
- [Content chunking](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/services/chunking.service.ts): `server/src/services/chunking.service.ts`

### Knowledge and chat

- [Embedding generation](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/services/embedding.service.ts): `server/src/services/embedding.service.ts`
- [Vector index](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/services/qdrant.service.ts): `server/src/services/qdrant.service.ts`
- [Storage budgeting](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/services/storage.service.ts): `server/src/services/storage.service.ts`
- [Semantic search](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/modules/search/search.service.ts): `server/src/modules/search/search.service.ts`
- [Chat service](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/modules/chat/chat.service.ts): `server/src/modules/chat/chat.service.ts`

### Runtime and persistence

- [Chat persistence](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/modules/chat/chat.repository.ts): `server/src/modules/chat/chat.repository.ts`
- **PostgreSQL**
- [Environment configuration](https://github.com/k-abhishek/ai-matrix/blob/main/server/src/config/env.ts): `server/src/config/env.ts`

### Other components

- **User**

## Connections

- User → React application: uses (evidence: `README.md`)
- React application → Crawler management UI: routes to
- React application → Search interface: routes to (evidence: `client/src/App.tsx`)
- React application → Chat interface: routes to (evidence: `client/src/App.tsx`)
- Search interface → HTTP API client: requests via (evidence: `client/src/pages/SearchPage.tsx`)
- Search interface → Knowledge API: calls (evidence: `client/src/pages/SearchPage.tsx`)
- Knowledge API → HTTP API client: sends requests (evidence: `client/src/api/knowledgeBase.ts`)
- Crawler management UI → HTTP API client: requests via
- Chat interface → HTTP API client: requests via
- Fastify application → Crawler routes: registers
- Crawler routes → Crawler controller: dispatches (evidence: `server/src/routes/crawlerRoutes.ts`)
- Crawler controller → Crawler service: calls (evidence: `server/src/modules/crawlers/crawler.controller.ts`)
- Crawler service → Crawl and index pipeline: starts crawl (evidence: `server/src/modules/crawlers/crawler.service.ts`)
- Crawl-run management → PostgreSQL: persists runs
- Crawl and index pipeline → Sitemap discovery: uses (evidence: `server/src/services/crawl.service.ts`)
- Crawl and index pipeline → Page fetching: fetches pages (evidence: `server/src/services/crawl.service.ts`)
- Crawl and index pipeline → Content extraction: extracts content (evidence: `server/src/services/crawl.service.ts`)
- Crawl and index pipeline → Content chunking: chunks content (evidence: `server/src/services/crawl.service.ts`)
- Crawl and index pipeline → Embedding generation: generates vectors (evidence: `server/src/services/crawl.service.ts`)
- Crawl and index pipeline → Vector index: indexes chunks (evidence: `server/src/services/crawl.service.ts`)
- Crawler service → Storage budgeting: checks budget (evidence: `server/src/modules/crawlers/crawler.service.ts`)
- Storage budgeting → Vector index: reads statistics (evidence: `server/src/services/storage.service.ts`)
- Fastify application → Search routes: registers
- Search routes → Search controller: dispatches
- Search controller → Semantic search: calls (evidence: `server/src/modules/search/search.controller.ts`)
- Semantic search → Vector index: searches (evidence: `server/src/modules/search/search.service.ts`)
- Fastify application → Chat routes: registers
- Chat routes → Chat service: calls (evidence: `server/src/routes/chatRoutes.ts`)
- Chat service → Chat persistence: persists threads (evidence: `server/src/modules/chat/chat.service.ts`)
- Chat service → Vector index: retrieves knowledge
- Chat persistence → PostgreSQL: stores threads
- Embedding generation → Environment configuration: reads settings (evidence: `server/src/services/embedding.service.ts`)
- Vector index → Embedding generation: embeds queries (evidence: `server/src/services/qdrant.service.ts`)
