# blacksmithgu/obsidian-dataview architecture

> Architecture diagram of the GitHub repository blacksmithgu/obsidian-dataview, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/blacksmithgu/obsidian-dataview
- Repository: https://github.com/blacksmithgu/obsidian-dataview

## Overview

##### Project type and purpose
- This is an open-source Obsidian plugin (“Obsidian Dataview”) written in TypeScript that treats an Obsidian vault as a queryable database.
- It implements a data-import pipeline, an in-memory index, a query language (DQL), a JavaScript API, and UI renderers inside Obsidian.

##### File structure
- Top-level:
- src/         – TypeScript source for import, indexing, querying, expressions, plugin integration, and UI
- __mocks__/   – Jest mocks for isolation of data-import and Obsidian API
- docs/        – MkDocs sources for user and API documentation
- scripts/     – Release, install and build‐helper scripts
- rollup.config.js  & tsconfig*.json – build configuration
- test-vault/  – sample vault used in dev mode by the plugin
- src/ is subdivided into:
- data-import/           – modules to pull metadata from markdown, YAML frontmatter, CSV, inline fields, including a web-worker implementation
- data-index/            – in-memory index builder, source resolution, resolver logic
- data-model/            – type definitions and serialized representations of pages and values
- expression/             – parser and evaluator for inline DQL expressions
- query/                  – DQL parser and an engine to run queries against the index
- api/                    – public JS API surface (DataArray, result wrappers, plugin API integration)
- ui/                     – renderers and view definitions (table, list, calendar, task views, plus React/TSX components)
- main.ts & index.ts      – Obsidian plugin entrypoint (register commands, set up file watchers, connect workers)
- Configuration and CI: jest.config.js for tests, GitHub workflows under .github/workflows, cspell.json, .prettierrc.json

##### README for additional insights
- Confirms two major aspects: Data (import frontmatter & inline fields) and Querying (DQL, DataviewJS, inline expressions).
- Explains build (rollup, npm scripts), development (npm run dev deploys to test-vault), docs (mkdocs).

##### Main components
a. Obsidian Vault (Markdown files with frontmatter & inline fields)
b. Data‐Import Pipeline
- CSV importer, Markdown file importer, inline field parser
- Web Worker subprocess to parallelize imports
c. Data Index
- Resolver & Source modules build an in-memory index of pages, fields, tasks
d. Data Model
- Type definitions and serialization layer governing how values are represented in memory
e. Query Engine & Parser
- DQL parser (pipeline DSL) and evaluator, inline expression evaluator, DataviewJS API
f. Plugin Core / API Integration
- main.ts registers file watchers, commands; plug into Obsidian’s plugin API
g. UI Renderer & Views
- React/TSX or DOM renderers for table, list, calendar, task, inline previews
h. Tests & Mocks
- Jest test suite validating each module; mocks for Obsidian API & data-import

##### Relationships and interactions
- Vault file change → Plugin Core dispatches event → Data-Import Pipeline (possibly in Web Worker) → Data Index updated → Query Engine runs DQL/JS queries on updated index → UI Renderer (views) re-renders
- DataviewJS and inline expressions call directly into the Query Engine & Data Model at render time
- Tests drive each layer in isolation via mocks

##### Important architectural patterns and principles
- Modular pipeline: each stage (import → index → query → render) is a separate module
- Separation of concerns: parsing, indexing, querying and rendering layers are decoupled
- Worker-based parallelism for heavy I/O (file parsing)
- Plugin architecture: main.ts glues into Obsidian hooks and command palette
- Pure functions in expression/query modules to maximize testability

##### Essential elements to capture
- Entrypoint and lifecycle: how main.ts registers watchers and kicks off the pipeline
- Worker boundary: show that data-import can run in web workers
- Index structure: sources, resolver, in-memory store
- Query modes: DQL engine vs DataviewJS API
- UI view plugins: list, table, calendar, tasks

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Acquisition Layer
    subgraph "Data Acquisition"
        Vault["Obsidian Vault (Markdown source files)"]:::external
        Import["Data-Import Pipeline"]:::acq
        WorkerSub["Data-Import Worker"]:::acq
        CSV["CSV Importer"]:::acq
        MDImp["Markdown Importer"]:::acq
        InlineFP["Inline Field Parser"]:::acq
        Persister["Persistence Module"]:::acq
        subgraph "Web Worker"
            WorkerEntry["Worker Entry Point"]:::acq
            WorkerImpl["Worker Implementation"]:::acq
            WorkerManager["Worker Manager"]:::acq
        end
    end

    %% Core Logic Layer
    subgraph "Data Index" 
        IndexBuilder["Index Builder"]:::core
        Resolver["Resolver Logic"]:::core
        SourceDef["Source Definitions"]:::core
    end

    subgraph "Data Model"
        CoreTypes["Core Model Types"]:::core
        Transferable["Transferable Definitions"]:::core
        MDModel["Markdown-specific Model"]:::core
        Serialized["Serialized Representations"]:::core
    end

    subgraph "Expression Parser & Evaluator"
        ExprParse["Parser"]:::core
        EvalCtx["Evaluation Context"]:::core
        FieldExpr["Field Expressions"]:::core
        BinaryOp["Binary Operators"]:::core
        Functions["Function Definitions"]:::core
    end

    subgraph "Query Engine & DQL Parser"
        QueryParse["Query Parser"]:::core
        QueryEngine["Query Engine"]:::core
        QueryEntrypoint["Query Entrypoint"]:::core
    end

    %% Integration Layer
    subgraph "Plugin Core / API Integration"
        MainTS["main.ts"]:::int
        IndexTS["index.ts"]:::int
        subgraph "API Surface"
            DataArray["DataArray & Wrappers"]:::int
            InlineAPI["Inline JS API"]:::int
            PluginAPI["Plugin Integration Helpers"]:::int
            Extensions["Extension Hooks"]:::int
            ResultTypes["Result Types"]:::int
        end
    end

    %% Presentation Layer
    subgraph "UI Renderer & Views"
        RenderUtils["Core Render Utilities"]:::pres
        LPR["LP Renderer"]:::pres
        ExportMD["Export to Markdown"]:::pres
        RefreshView["Refreshable View Base"]:::pres
        MDComp["Markdown Component"]:::pres
        subgraph "Views"
            Table["Table View"]:::pres
            List["List View"]:::pres
            Calendar["Calendar View"]:::pres
            Task["Task View"]:::pres
            InlineLive["Inline Field Live Preview"]:::pres
            InlineFieldView["Inline Field View"]:::pres
            InlineDQL["Inline DQL View"]:::pres
            JSView["JS-block View"]:::pres
        end
    end

    %% Infrastructure Layer
    subgraph "Infrastructure"
        Tests["Tests & Mocks"]:::infra
        Docs["Documentation (MkDocs)"]:::infra
        Build["Build Scripts & Config"]:::infra
    end

    %% Connections
    Vault -->|file change| MainTS
    MainTS -->|dispatch event| Import
    Import -->|spawn| WorkerSub
    WorkerSub --> WorkerEntry
    WorkerSub --> WorkerImpl
    WorkerSub --> WorkerManager
    Import --> CSV
    Import --> MDImp
    Import --> InlineFP
    Import --> Persister
    Import -->|imported data| IndexBuilder
    IndexBuilder --> Resolver
    IndexBuilder --> SourceDef
    IndexBuilder -->|build index| DataModelIndex[In-Memory Index]
    DataModelIndex --> QueryEntrypoint
    ExprParse --> QueryEngine
    EvalCtx --> QueryEngine
    FieldExpr --> QueryEngine
    BinaryOp --> QueryEngine
    Functions --> QueryEngine
    QueryParse --> QueryEngine
    QueryEngine --> QueryEntrypoint
    QueryEntrypoint -->|query results| RenderUtils
    DataArray --> QueryEntrypoint
    InlineAPI --> QueryEntrypoint
    RenderUtils --> LPR
    RenderUtils --> ExportMD
    RenderUtils --> RefreshView
    RefreshView --> MDComp
    MDComp --> Table
    MDComp --> List
    MDComp --> Calendar
    MDComp --> Task
    MDComp --> InlineLive
    MDComp --> InlineFieldView
    MDComp --> InlineDQL
    MDComp --> JSView
    Tests --> Import
    Tests --> IndexBuilder
    Tests --> ExprParse
    Tests --> QueryParse
    Tests --> DataArray
    Docs -->|generate| DocsSite[Docs Site]

    %% Click Events
    click Vault "https://github.com/blacksmithgu/obsidian-dataview/tree/master/test-vault/"
    click CSV "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/data-import/csv.ts"
    click MDImp "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/data-import/markdown-file.ts"
    click InlineFP "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/data-import/inline-field.ts"
    click Persister "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/data-import/persister.ts"
    click WorkerEntry "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/data-import/web-worker/import-entry.ts"
    click WorkerImpl "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/data-import/web-worker/import-impl.ts"
    click WorkerManager "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/data-import/web-worker/import-manager.ts"
    click IndexBuilder "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/data-index/index.ts"
    click Resolver "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/data-index/resolver.ts"
    click SourceDef "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/data-index/source.ts"
    click CoreTypes "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/data-model/value.ts"
    click Transferable "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/data-model/transferable.ts"
    click MDModel "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/data-model/markdown.ts"
    click Serialized "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/data-model/serialized/markdown.ts"
    click ExprParse "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/expression/parse.ts"
    click EvalCtx "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/expression/context.ts"
    click FieldExpr "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/expression/field.ts"
    click BinaryOp "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/expression/binaryop.ts"
    click Functions "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/expression/functions.ts"
    click QueryParse "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/query/parse.ts"
    click QueryEngine "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/query/engine.ts"
    click QueryEntrypoint "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/query/query.ts"
    click MainTS "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/main.ts"
    click IndexTS "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/index.ts"
    click DataArray "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/api/data-array.ts"
    click InlineAPI "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/api/inline-api.ts"
    click PluginAPI "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/api/plugin-api.ts"
    click Extensions "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/api/extensions.ts"
    click ResultTypes "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/api/result.ts"
    click RenderUtils "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/ui/render.ts"
    click LPR "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/ui/lp-render.ts"
    click ExportMD "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/ui/export/markdown.ts"
    click RefreshView "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/ui/refreshable-view.ts"
    click MDComp "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/ui/markdown.tsx"
    click Table "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/ui/views/table-view.tsx"
    click List "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/ui/views/list-view.tsx"
    click Calendar "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/ui/views/calendar-view.ts"
    click Task "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/ui/views/task-view.tsx"
    click InlineLive "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/ui/views/inline-field-live-preview.ts"
    click InlineFieldView "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/ui/views/inline-field.tsx"
    click InlineDQL "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/ui/views/inline-view.ts"
    click JSView "https://github.com/blacksmithgu/obsidian-dataview/blob/master/src/ui/views/js-view.ts"

    %% Styles
    classDef acq fill:#e0f7e9,stroke:#2e7d32;color:#2e7d32
    classDef core fill:#d0eaff,stroke:#1565c0;color:#1565c0
    classDef int fill:#ffe5b4,stroke:#ef6c00;color:#e65100
    classDef pres fill:#f3e5f5,stroke:#6a1b9a;color:#4a148c
    classDef infra fill:#f0f0f0,stroke:#616161;color:#424242
    classDef external fill:#fff3cd,stroke:#f9a825,color:#f57f17
```
