How nichwch/nomad-hypertext works
Project Type and Purpose
• This is a full-stack application targeted at desktop (MacOS) users that combines a rich, interactive writing interface with built-in semantic search.
• The overall goal is to let users write freely while automatically indexing their content using embeddings and then linking related text pieces.
File Structure
• At the top level, you have configuration files and scripts (e.g., package.json, vite.config.js, svelte.config.js, tailwind.config.js) that set up a Svelte-based UI (using .svelte components) and the overall build environment.
• Files like appDB.cjs, appEmbeddings.cjs, db.mjs, and embedding.mjs indicate discrete modules responsible for local storage (vector database) and the AI embedding generation.
• The presence of “parsePDF.mjs” and “query.mjs” suggests additional processing capabilities (e.g., importing PDFs, executing search queries).
• Directory structure within “src” shows routes and components typical of a SvelteKit application: components (Overlay.svelte, SearchModal.svelte, etc.), organized routes (including special routes for notes, settings, and help), and reusable libraries in src/lib.
Breakdown of Main Components in the System
a. User Interface (UI) Layer:
• Built on Svelte with routes and UI components found under “src” (e.g., +page.svelte files, Overlay.svelte, SearchResultDisplay.svelte).
• This includes components for the writing interface, search modals, overlay displays, context menus, and settings.
b. Semantic Search Engine:
• OramaSearch is at the core for performing the semantic search on locally stored vectors.
• The UI components interact with this through an API layer (code in modules like query.mjs or components that trigger search events).
c. Embedding Generation Module:
• Modules such as embedding.mjs and appEmbeddings.cjs wrap the functionality for generating vector embeddings from text using Supabase’s gte-small model.
d. Local Vector Database:
• Modules such as appDB.cjs and db.mjs suggest a dedicated mechanism to store the indexed content and embeddings in a local, file-based or in-memory structure.
e. Auxiliary Tools:
• Scripts like parsePDF.mjs and importFromDirectory.mjs may serve to ingest and format external data (e.g., PDF content) into the system.
• There are also components for handling prompts (src/prompt folder) and tooltips (src/lib/tooltip.js) that add to the feature set.
Relationships & Data Flows
• The user writes in the UI, which is immediately captured by the front-end components.
• These inputs pass to modules that process the text: a parser (possibly parsePDF.mjs for file imports or direct text inputs) and then to the embedding generator (embedding.mjs/appEmbeddings.cjs) which produces vector representations.
• The vectors are saved into the local database (appDB.cjs/db.mjs) and indexed by OramaSearch.
• When the user queries or interacts with the content (for example, clicking on a paragraph), the UI component triggers a search query (query.mjs) against OramaSearch.
• Search results come back and are rendered by components such as SearchResultDisplay.svelte.
Architectural Patterns and Design Principles
• A clear separation of concerns is evident: the SvelteKit frontend handles presentation and routing, while separate modules handle backend logic related to embedding generation and data storage.
• The file-based routing in Svelte guides the UI organization, while Node.js modules (using .mjs and .cjs) encapsulate business logic.
• The application processes data in-memory and on-device, which emphasizes efficiency and offline capability.