# blacksmithgu/datacore architecture

> Architecture diagram of the GitHub repository blacksmithgu/datacore, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/blacksmithgu/datacore
- Repository: https://github.com/blacksmithgu/datacore

## Overview

##### Main Components

a) Core System Components:
- Index Engine (src/index/)
- Expression Engine (src/expression/)
- API Layer (src/api/)
- UI Components (src/ui/)
- Web Worker System (src/index/web-worker/)

b) Data Flow Components:
- Storage System (src/index/storage/)
- Data Types Handlers (src/index/types/)
- Import System (src/index/import/)

**┌─────────────────────────────────────────────┐**
- UI Layer                   │
- ┤
- API Interface                 │
- ┤
- Expression Engine               │
- ┤
- Index Engine                   │
- ┤
- Storage Layer                  │
- ┘

##### Component Relationships

- UI Layer → API Interface (React-based views interaction)
- API Interface → Expression Engine (Query processing)
- Expression Engine → Index Engine (Data retrieval)
- Index Engine → Storage Layer (Data persistence)
- Web Worker → Index Engine (Asynchronous processing)

##### Detailed Component Breakdown

UI Layer:
- Views (cards, tables, lists)
- Field editors
- Error boundaries
- Loading states

API Layer:
- Local API
- Data Array handling
- Result processing
- Script cache

Expression Engine:
- Parser
- Evaluator
- Functions
- Filters

Index Engine:
- Datastore
- Persister
- Import handlers
- Query executor

##### Integration Points

Integration with:
- Obsidian API
- File system
- Markdown processing
- YAML handling

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Styles and classes
    classDef ui fill:#3498db,stroke:#2980b9,color:white
    classDef api fill:#2ecc71,stroke:#27ae60,color:white
    classDef expression fill:#e74c3c,stroke:#c0392b,color:white
    classDef index fill:#9b59b6,stroke:#8e44ad,color:white
    classDef storage fill:#f1c40f,stroke:#f39c12,color:black
    classDef worker fill:#e67e22,stroke:#d35400,color:white
    classDef import fill:#1abc9c,stroke:#16a085,color:white
    classDef types fill:#34495e,stroke:#2c3e50,color:white

    %% UI Layer
    subgraph UI
        Views["Views (Cards/Tables/Lists)"]:::ui
        FieldEditors["Field Editors"]:::ui
        ErrorBoundaries["Error Boundaries"]:::ui
        LoadingStates["Loading States"]:::ui
    end
    click Views "https://github.com/blacksmithgu/datacore/tree/master/src/api/ui/views/"
    click FieldEditors "https://github.com/blacksmithgu/datacore/tree/master/src/ui/fields/"
    click ErrorBoundaries "https://github.com/blacksmithgu/datacore/blob/master/src/ui/errors.css"
    click LoadingStates "https://github.com/blacksmithgu/datacore/blob/master/src/ui/loading-boundary.tsx"

    %% API Layer
    subgraph API
        LocalAPI["Local API"]:::api
        DataArray["Data Array Handler"]:::api
        ResultProcess["Result Processing"]:::api
        ScriptCache["Script Cache"]:::api
    end
    click LocalAPI "https://github.com/blacksmithgu/datacore/blob/master/src/api/local-api.tsx"
    click DataArray "https://github.com/blacksmithgu/datacore/blob/master/src/api/data-array.ts"

    %% Expression Engine
    subgraph Expression
        Parser["Parser"]:::expression
        Evaluator["Evaluator"]:::expression
        Functions["Functions"]:::expression
        Filters["Filters"]:::expression
    end
    click Parser "https://github.com/blacksmithgu/datacore/blob/master/src/expression/parser.ts"
    click Evaluator "https://github.com/blacksmithgu/datacore/blob/master/src/expression/evaluator.ts"
    click Functions "https://github.com/blacksmithgu/datacore/blob/master/src/expression/functions.ts"
    click Filters "https://github.com/blacksmithgu/datacore/blob/master/src/expression/filters.ts"

    %% Index Engine
    subgraph Index
        Datastore["Datastore"]:::index
        Persister["Persister"]:::index
        QueryExecutor["Query Executor"]:::index
    end
    click Datastore "https://github.com/blacksmithgu/datacore/blob/master/src/index/datastore.ts"
    click QueryExecutor "https://github.com/blacksmithgu/datacore/blob/master/src/index/storage/query-executor.ts"

    %% Storage Layer
    subgraph Storage
        InvertedIndex["Inverted Index"]:::storage
        Fields["Fields Storage"]:::storage
    end
    click InvertedIndex "https://github.com/blacksmithgu/datacore/blob/master/src/index/storage/inverted.ts"
    click Fields "https://github.com/blacksmithgu/datacore/blob/master/src/index/storage/fields.ts"

    %% Web Worker System
    subgraph Worker
        WebWorker["Web Worker"]:::worker
        ImportWorker["Import Worker"]:::worker
        MessageHandler["Message Handler"]:::worker
    end
    click WebWorker "https://github.com/blacksmithgu/datacore/tree/master/src/index/web-worker/"
    click ImportWorker "https://github.com/blacksmithgu/datacore/blob/master/src/index/web-worker/importer.worker.ts"
    click MessageHandler "https://github.com/blacksmithgu/datacore/blob/master/src/index/web-worker/message.ts"

    %% Data Types
    subgraph Types
        Canvas["Canvas Handler"]:::types
        Files["Files Handler"]:::types
        Markdown["Markdown Handler"]:::types
        YAML["YAML Handler"]:::types
    end
    click Canvas "https://github.com/blacksmithgu/datacore/blob/master/src/index/types/canvas.ts"
    click Files "https://github.com/blacksmithgu/datacore/blob/master/src/index/types/files.ts"
    click Markdown "https://github.com/blacksmithgu/datacore/blob/master/src/index/types/markdown.ts"
    click YAML "https://github.com/blacksmithgu/datacore/blob/master/src/index/types/yaml.ts"

    %% Import System
    subgraph Import
        CanvasImport["Canvas Import"]:::import
        InlineField["Inline Field Import"]:::import
        MarkdownImport["Markdown Import"]:::import
    end
    click CanvasImport "https://github.com/blacksmithgu/datacore/blob/master/src/index/import/canvas.ts"
    click InlineField "https://github.com/blacksmithgu/datacore/blob/master/src/index/import/inline-field.ts"
    click MarkdownImport "https://github.com/blacksmithgu/datacore/blob/master/src/index/import/markdown.ts"

    %% Relationships
    UI --> API
    API --> Expression
    Expression --> Index
    Index --> Storage
    Worker --> Index
    Types --> Index
    Import --> Index
    
    %% Legend
    subgraph Legend
        UI_L["UI Components"]:::ui
        API_L["API Components"]:::api
        EXP_L["Expression Components"]:::expression
        IDX_L["Index Components"]:::index
        STR_L["Storage Components"]:::storage
        WRK_L["Worker Components"]:::worker
        IMP_L["Import Components"]:::import
        TYP_L["Types Components"]:::types
    end
```
