How modsetter/surfsense works
SurfSense is a privacy-focused desktop or self-hosted AI workspace for researching, transforming, and editing local documents. The README’s primary workflow is to add sources, then ask an agent to search, extract, or edit them. The graph preserves the documented ingestion, agent, studio, format-support, and plugin capabilities; source-level edges are limited to relationships visible in the supplied excerpts or source index. Several subsystems are represented by their exact enclosing directories because their implementation files were not sampled.
Main entry points: proprietary Scraping plugins · engines Agent engines · pipeline.py Document ingestion · turn.py Agent turns
Components 30
Local app
- Desktop UI
surfsense_local/frontend/src/App.tsx - Local API
surfsense_local/backend/api/main.py - API dependencies
surfsense_local/backend/api/dependencies.py - Local configuration
surfsense_local/backend/shared/config.py - Local database
surfsense_local/backend/shared/db.py
Documents and studio
- Document storage
surfsense_local/backend/modules/documents/storage.py - Document ingestion
surfsense_local/backend/worker/ingestion/pipeline.py - Studio pipelines
surfsense_local/backend/worker/studio - Office conversion
surfsense_local/backend/modules/office_support/engine.py
Agent and models
- Engine selection
surfsense_local/backend/modules/agent/engine_choice.py - Thread state
surfsense_local/backend/modules/agent/agent_threads - Agent engines
surfsense_local/backend/modules/agent/engines - Ready renders
surfsense_local/backend/modules/agent/agent_threads/ready_renders.py - Agent replies
surfsense_local/backend/modules/agent/agent_threads/replies.py - Agent turns
surfsense_local/backend/modules/agent/agent_threads/turn.py - Chat records
surfsense_local/backend/modules/chat/models.py
Plugin ecosystem
- Plugin CLI
backend/plugins/bundles/core/cli/surfsense_plugin_cli - Plugin manifests
backend/plugins/bundles/core/manifest/surfsense_plugin_manifest - Plugin SDK
backend/plugins/bundles/core/sdk/surfsense_plugin_sdk - Plugin host
backend/plugins/host/app.py - Usage metering
backend/plugins/host/usage_meter/usage_meter.py - MCP progress
backend/plugins/mcp_server/progress.py - MCP results
backend/plugins/mcp_server/results.py - Scraping plugins
backend/plugins/proprietary - Plugin license access
backend/plugins/proprietary/license_check/license_access.py
Services and licensing
- License service
backend/license/router.py - Backend server
backend/server/app.py
Other components
- User
- Model provider
- External websites
Connections 26
- User → Desktop UI: adds sources
README.md - User → Desktop UI: asks agent
README.md - Desktop UI → Local API: requests work
- Local API → API dependencies: resolves dependencies
- Document ingestion → Document storage: reads document bytes
- Document ingestion → Local database: records ingest state
surfsense_local/backend/worker/ingestion/pipeline.py - Document ingestion → Studio pipelines: queues figure pass
surfsense_local/backend/worker/ingestion/pipeline.py - Studio pipelines → Office conversion: uses conversion
- Engine selection → API dependencies: checks settings
surfsense_local/backend/modules/agent/engine_choice.py - Engine selection → Agent engines: selects engine
- Agent engines → Agent turns: runs turn
surfsense_local/backend/modules/agent/engines/opencode_engine.py - Agent engines → Thread state: loads thread state
surfsense_local/backend/modules/agent/engines/opencode_engine.py - Agent engines → Chat records: reads thread records
surfsense_local/backend/modules/agent/engines/opencode_engine.py - Agent turns → Ready renders: renders turn
surfsense_local/backend/modules/agent/agent_threads/turn.py - Agent turns → Agent replies: emits replies
surfsense_local/backend/modules/agent/agent_threads/turn.py - Plugin CLI → Plugin manifests: loads manifest
- Plugin CLI → Plugin SDK: invokes plugin
- Backend server → Plugin host: mounts host
backend/server/app.py - Plugin host → Usage metering: starts meter
backend/plugins/host/app.py - Plugin host → Scraping plugins: mounts plugin routes
- Scraping plugins → MCP progress: reports progress
backend/plugins/proprietary/tiktok/server.py - Scraping plugins → MCP results: formats results
backend/plugins/proprietary/tiktok/server.py - Scraping plugins → Plugin license access: checks access
backend/plugins/proprietary/tiktok/server.py - Scraping plugins → External websites: fetches public data
- Backend server → License service: mounts portal
backend/server/app.py - Model provider → Agent engines: provides model