How vtempest/ai-research-agent works
Project type and purpose
• This is a monorepo (Turborepo-managed) containing:
• Shared TypeScript/JavaScript packages (core “ai-research-agent,” crawler, neural-net, API clients, editor modules)
• Multiple applications (“apps”): a web app (SvelteKit + serverless APIs), a desktop app (Tauri & Svelte UI), a browser extension, and a Docusaurus-powered docs site
• The README describes a research-agent library (vector-space text extraction, search streaming, summarization, web crawling, etc.) plus front-ends that consume it.
File structure
• Top-level folders:
• /packages – pluggable modules published to NPM
• /apps – four distinct front-end and docs applications
• compose.yml – for local Docker services (e.g., crawler proxy)
• turbo.json – build orchestration
• Configuration files:
• package.json, pnpm-lock, tsconfig, vite.config, Rust Cargo.toml (Tauri)
• Apps:
• apps/web – SvelteKit UI + server-side API routes under /src/routes/api
• apps/desktop – Svelte UI + Tauri Rust backend for system features
• apps/extension – multi-page browser extension (background, content, sidepanel)
• apps/docs – Docusaurus documentation site
README for additional insights
• Core functions: extractor (HTML/PDF/YouTube), search-STREAM, topic modeling, weighting (WRITEFAT), autocomplete
• Data flow: Web UI → serverless API → core packages → external services (web, LLMs)
• Deployment targets: NPM packages, Cloudflare Workers, Docker proxy, GitHub Pages for docs
Main components
a. Monorepo orchestrator
• Turborepo / package manager / CI (GitHub Actions)
b. Shared libraries (NPM packages)
• ai-research-agent (core functions)
• crawler (web scraping Docker proxy)
• neural-net (ML wrappers)
• qwksearch-api-client, reason-editor
c. Applications
• Web App
– SvelteKit UI (Browser)
– SvelteKit Serverless API (Cloudflare Workers or Node)
• Desktop App
– Svelte UI (WebView)
– Tauri Rust backend (system capabilities)
• Browser Extension
– Content scripts / background service worker
• Documentation Site
– Docusaurus static site
d. External services and infrastructure
• NPM Registry (publish/consume packages)
• Docker crawler proxy (Puppeteer for Cloudflare bypass)
• LLM providers (OpenAI, Groq, Claude, etc.)
• Cloudflare Workers / Serverless runtime
Relationships and interactions
• All apps import and call shared NPM packages
• Web App’s /api routes invoke ai-research-agent functions
• Desktop UI calls core library in JS and Tauri RPC for OS integration
• Extension scripts call core library directly in the browser
• Crawler package runs in Docker to bypass bot checks—Web/Agent code routes scraping requests through it
• CI pipeline builds, tests, publishes packages; deploys web & docs to hosting
Architectural patterns and principles
• Monorepo with Turborepo for build-and-test orchestration
• Code reusability via published packages
• Serverless micro-API pattern for Web App
• Polyglot runtime: TypeScript for logic, Rust for system calls (desktop)
• Dockerized sidecar for complex scraping
Final advice
• Be exhaustive: include every “apps/…” and “packages/…” component at a high level
• Essential architectural layers: monorepo orchestration, shared libraries, UIs, APIs, external services, infrastructure
• Avoid clutter: group similar packages (e.g., all AI functions) into a single “Core AI Library” block, unless a deeper dive is needed