How k-abhishek/ai-matrix works
This is a crawler-management and semantic knowledge-base application with a React UI and Fastify API. Users configure crawlers, start and monitor crawl runs, index extracted website content, then search or chat against that content. The README documents the crawl-to-vector-indexing workflow; sampled backend code confirms semantic search and crawler management. Several UI pages and backend routes were not sampled, so those links are shown only where source references or README evidence support them.
Main entry points: crawl.service.ts Crawl and index pipeline · qdrant.service.ts Vector index · App.tsx React application · client.ts HTTP API client
Components 28
Web interface
- React application
client/src/App.tsx - Crawler management UI
client/src/pages/CrawlerDetailPage.tsx - Search interface
client/src/pages/SearchPage.tsx - Chat interface
client/src/pages/ChatPage.tsx - HTTP API client
client/src/api/client.ts - Knowledge API
client/src/api/knowledgeBase.ts
API layer
- Fastify application
server/src/app.ts - Crawler routes
server/src/routes/crawlerRoutes.ts - Crawler controller
server/src/modules/crawlers/crawler.controller.ts - Search routes
server/src/routes/searchRoutes.ts - Search controller
server/src/modules/search/search.controller.ts - Chat routes
server/src/routes/chatRoutes.ts
Crawling and indexing
- Crawler service
server/src/modules/crawlers/crawler.service.ts - Crawl-run management
server/src/modules/crawl-runs/crawl-run.service.ts - Crawl and index pipeline
server/src/services/crawl.service.ts - Sitemap discovery
server/src/services/sitemap.service.ts - Page fetching
server/src/services/fetcher.service.ts - Content extraction
server/src/services/content-extractor.service.ts - Content chunking
server/src/services/chunking.service.ts
Knowledge and chat
- Embedding generation
server/src/services/embedding.service.ts - Vector index
server/src/services/qdrant.service.ts - Storage budgeting
server/src/services/storage.service.ts - Semantic search
server/src/modules/search/search.service.ts - Chat service
server/src/modules/chat/chat.service.ts
Runtime and persistence
- Chat persistence
server/src/modules/chat/chat.repository.ts - PostgreSQL
- Environment configuration
server/src/config/env.ts
Other components
- User
Connections 33
- User → React application: uses
README.md - React application → Crawler management UI: routes to
- React application → Search interface: routes to
client/src/App.tsx - React application → Chat interface: routes to
client/src/App.tsx - Search interface → HTTP API client: requests via
client/src/pages/SearchPage.tsx - Search interface → Knowledge API: calls
client/src/pages/SearchPage.tsx - Knowledge API → HTTP API client: sends requests
client/src/api/knowledgeBase.ts - Crawler management UI → HTTP API client: requests via
- Chat interface → HTTP API client: requests via
- Fastify application → Crawler routes: registers
- Crawler routes → Crawler controller: dispatches
server/src/routes/crawlerRoutes.ts - Crawler controller → Crawler service: calls
server/src/modules/crawlers/crawler.controller.ts - Crawler service → Crawl and index pipeline: starts crawl
server/src/modules/crawlers/crawler.service.ts - Crawl-run management → PostgreSQL: persists runs
- Crawl and index pipeline → Sitemap discovery: uses
server/src/services/crawl.service.ts - Crawl and index pipeline → Page fetching: fetches pages
server/src/services/crawl.service.ts - Crawl and index pipeline → Content extraction: extracts content
server/src/services/crawl.service.ts - Crawl and index pipeline → Content chunking: chunks content
server/src/services/crawl.service.ts - Crawl and index pipeline → Embedding generation: generates vectors
server/src/services/crawl.service.ts - Crawl and index pipeline → Vector index: indexes chunks
server/src/services/crawl.service.ts - Crawler service → Storage budgeting: checks budget
server/src/modules/crawlers/crawler.service.ts - Storage budgeting → Vector index: reads statistics
server/src/services/storage.service.ts - Fastify application → Search routes: registers
- Search routes → Search controller: dispatches
- Search controller → Semantic search: calls
server/src/modules/search/search.controller.ts - Semantic search → Vector index: searches
server/src/modules/search/search.service.ts - Fastify application → Chat routes: registers
- Chat routes → Chat service: calls
server/src/routes/chatRoutes.ts - Chat service → Chat persistence: persists threads
server/src/modules/chat/chat.service.ts - Chat service → Vector index: retrieves knowledge
- Chat persistence → PostgreSQL: stores threads
- Embedding generation → Environment configuration: reads settings
server/src/services/embedding.service.ts - Vector index → Embedding generation: embeds queries
server/src/services/qdrant.service.ts