How blacksmithgu/obsidian-dataview works
Project type and purpose
• This is an open-source Obsidian plugin (“Obsidian Dataview”) written in TypeScript that treats an Obsidian vault as a queryable database.
• It implements a data-import pipeline, an in-memory index, a query language (DQL), a JavaScript API, and UI renderers inside Obsidian.
File structure
• Top-level:
• src/ – TypeScript source for import, indexing, querying, expressions, plugin integration, and UI
• __mocks__/ – Jest mocks for isolation of data-import and Obsidian API
• docs/ – MkDocs sources for user and API documentation
• scripts/ – Release, install and build‐helper scripts
• rollup.config.js & tsconfig*.json – build configuration
• test-vault/ – sample vault used in dev mode by the plugin
• src/ is subdivided into:
• data-import/ – modules to pull metadata from markdown, YAML frontmatter, CSV, inline fields, including a web-worker implementation
• data-index/ – in-memory index builder, source resolution, resolver logic
• data-model/ – type definitions and serialized representations of pages and values
• expression/ – parser and evaluator for inline DQL expressions
• query/ – DQL parser and an engine to run queries against the index
• api/ – public JS API surface (DataArray, result wrappers, plugin API integration)
• ui/ – renderers and view definitions (table, list, calendar, task views, plus React/TSX components)
• main.ts & index.ts – Obsidian plugin entrypoint (register commands, set up file watchers, connect workers)
• Configuration and CI: jest.config.js for tests, GitHub workflows under .github/workflows, cspell.json, .prettierrc.json
README for additional insights
• Confirms two major aspects: Data (import frontmatter & inline fields) and Querying (DQL, DataviewJS, inline expressions).
• Explains build (rollup, npm scripts), development (npm run dev deploys to test-vault), docs (mkdocs).
Main components
a. Obsidian Vault (Markdown files with frontmatter & inline fields)
b. Data‐Import Pipeline
• CSV importer, Markdown file importer, inline field parser
• Web Worker subprocess to parallelize imports
c. Data Index
• Resolver & Source modules build an in-memory index of pages, fields, tasks
d. Data Model
• Type definitions and serialization layer governing how values are represented in memory
e. Query Engine & Parser
• DQL parser (pipeline DSL) and evaluator, inline expression evaluator, DataviewJS API
f. Plugin Core / API Integration
• main.ts registers file watchers, commands; plug into Obsidian’s plugin API
g. UI Renderer & Views
• React/TSX or DOM renderers for table, list, calendar, task, inline previews
h. Tests & Mocks
• Jest test suite validating each module; mocks for Obsidian API & data-import
Relationships and interactions
• Vault file change → Plugin Core dispatches event → Data-Import Pipeline (possibly in Web Worker) → Data Index updated → Query Engine runs DQL/JS queries on updated index → UI Renderer (views) re-renders
• DataviewJS and inline expressions call directly into the Query Engine & Data Model at render time
• Tests drive each layer in isolation via mocks
Important architectural patterns and principles
• Modular pipeline: each stage (import → index → query → render) is a separate module
• Separation of concerns: parsing, indexing, querying and rendering layers are decoupled
• Worker-based parallelism for heavy I/O (file parsing)
• Plugin architecture: main.ts glues into Obsidian hooks and command palette
• Pure functions in expression/query modules to maximize testability
Essential elements to capture
• Entrypoint and lifecycle: how main.ts registers watchers and kicks off the pipeline
• Worker boundary: show that data-import can run in web workers
• Index structure: sources, resolver, in-memory store
• Query modes: DQL engine vs DataviewJS API
• UI view plugins: list, table, calendar, tasks