# momonokiki/datacore-oss architecture

> Architecture diagram of the GitHub repository momonokiki/datacore-oss, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/momonokiki/datacore-oss
- Repository: https://github.com/momonokiki/datacore-oss

## Overview

DataCore is a personal-file data platform: authenticated uploads retain originals in content-addressed L1 storage, then background processing creates L2 extraction artifacts and optional L3 domain data for search and analysis. The README describes receipt and financial-data capabilities; the tree confirms corresponding API routes and add-ons, though their runtime wiring was not sampled. Edges reflect shown calls or documented behavior; unsampled route/add-on connections are omitted rather than inferred.

Main entry points: [`app/main.py`](https://github.com/momonokiki/datacore-oss/blob/main/app/main.py) (FastAPI app), [`app/layers/l2_extracted/embedding_img/service.py`](https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l2_extracted/embedding_img/service.py) (Image embedding), [`app/api/routes/ingest.py`](https://github.com/momonokiki/datacore-oss/blob/main/app/api/routes/ingest.py) (File ingest), [`app/layers/l2_extracted/models.py`](https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l2_extracted/models.py) (Extracted artifacts)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["API and access"]
  node_api_app["FastAPI app<br/>[main.py]"]
  node_auth["Authentication<br/>[auth.py]"]
  node_ingest["File ingest<br/>[ingest.py]"]
  node_search_api["Search API<br/>[search.py]"]
  node_finance_api["Finance API<br/>[finance.py]"]
  node_receipt_api["Receipt API<br/>[receipts.py]"]
  node_web_search["Search UI<br/>[search.py]"]
end

subgraph group_raw["Raw file layer"]
  node_raw_service["L1 ingestion<br/>[service.py]"]
  node_cas[("CAS storage<br/>[adapter.py]")]
  node_raw_models[("Raw file records<br/>[models.py]")]
end

subgraph group_extract["Extraction and search"]
  node_celery_tasks["Task dispatch<br/>[tasks.py]"]
  node_l2_tasks["Extraction tasks<br/>[tasks.py]"]
  node_media["Media extraction<br/>[service.py]"]
  node_ocr["OCR extraction<br/>[service.py]"]
  node_image_embedding["Image embedding<br/>[service.py]"]
  node_text_embedding["Text embedding<br/>[service.py]"]
  node_search_service["Search service<br/>[service.py]"]
end

subgraph group_structured["Structured data"]
  node_receipt_addon["Receipt add-on<br/>[service.py]"]
  node_finance_addon["Finance parser<br/>[service.py]"]
  node_layout_addon["YomiToku layout<br/>[service.py]"]
  node_classify_addon["Receipt classifier<br/>[service.py]"]
end

subgraph group_persistence["Persistence"]
  node_l2_models[("Extracted artifacts<br/>[models.py]")]
  node_l2_tools[("Tool registry<br/>[l2_tools.py]")]
  node_db_session[("DB sessions<br/>[session.py]")]
end

node_user(("User"))

node_api_app -->|"registers route"| node_ingest
node_api_app -->|"registers route"| node_auth
node_api_app -->|"registers route"| node_search_api
node_api_app -->|"registers route"| node_finance_api
node_api_app -->|"registers route"| node_receipt_api
node_user -->|"uploads files"| node_ingest
node_ingest -->|"checks identity"| node_auth
node_ingest -->|"ingests files"| node_raw_service
node_ingest -->|"dispatches chain"| node_celery_tasks
node_raw_service -->|"stores binaries"| node_cas
node_raw_service -->|"records uploads"| node_raw_models
node_search_api -->|"queries"| node_search_service
node_search_service -->|"uses embeddings"| node_text_embedding
node_l2_tasks -->|"runs extraction"| node_media
node_l2_tasks -->|"runs OCR"| node_ocr
node_l2_tasks -->|"runs embedding"| node_image_embedding
node_l2_tasks -->|"runs embedding"| node_text_embedding
node_media -->|"writes metadata"| node_l2_models
node_ocr -->|"writes artifacts"| node_l2_models
node_image_embedding -->|"writes vectors"| node_l2_models
node_image_embedding -->|"loads original"| node_cas
node_image_embedding -->|"reads file record"| node_raw_models
node_image_embedding -->|"registers tool"| node_l2_tools
node_text_embedding -->|"reads and writes"| node_l2_models
node_text_embedding -->|"registers tool"| node_l2_tools
node_search_service -->|"searches artifacts"| node_l2_models
node_web_search -.->|"uses API"| node_search_api

click node_api_app "https://github.com/momonokiki/datacore-oss/blob/main/app/main.py"
click node_auth "https://github.com/momonokiki/datacore-oss/blob/main/app/api/routes/auth.py"
click node_ingest "https://github.com/momonokiki/datacore-oss/blob/main/app/api/routes/ingest.py"
click node_search_api "https://github.com/momonokiki/datacore-oss/blob/main/app/api/routes/search.py"
click node_finance_api "https://github.com/momonokiki/datacore-oss/blob/main/app/api/routes/finance.py"
click node_receipt_api "https://github.com/momonokiki/datacore-oss/blob/main/app/api/routes/receipts.py"
click node_raw_service "https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l1_raw/service.py"
click node_cas "https://github.com/momonokiki/datacore-oss/blob/main/app/infrastructure/storage/adapter.py"
click node_raw_models "https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l1_raw/models.py"
click node_celery_tasks "https://github.com/momonokiki/datacore-oss/blob/main/app/tasks.py"
click node_l2_tasks "https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l2_extracted/tasks.py"
click node_media "https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l2_extracted/media_info/service.py"
click node_ocr "https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l2_extracted/ocr/service.py"
click node_image_embedding "https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l2_extracted/embedding_img/service.py"
click node_text_embedding "https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l2_extracted/embedding_text/service.py"
click node_search_service "https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l2_extracted/search/service.py"
click node_l2_models "https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l2_extracted/models.py"
click node_l2_tools "https://github.com/momonokiki/datacore-oss/blob/main/app/infrastructure/db/models/l2_tools.py"
click node_receipt_addon "https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l3_structured/addons/receipt/service.py"
click node_finance_addon "https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l3_structured/addons/finunce_parser/service.py"
click node_layout_addon "https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l3_structured/addons/yomitoku_layout/service.py"
click node_classify_addon "https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l3_structured/addons/receipt_classify_exp/service.py"
click node_db_session "https://github.com/momonokiki/datacore-oss/blob/main/app/infrastructure/db/session.py"
click node_web_search "https://github.com/momonokiki/datacore-oss/blob/main/app/webui/webui/pages/search.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_api_app,node_auth,node_ingest,node_search_api,node_finance_api,node_receipt_api,node_web_search,node_user toneBlue
class node_raw_service,node_cas,node_raw_models toneAmber
class node_celery_tasks,node_l2_tasks,node_media,node_ocr,node_image_embedding,node_text_embedding,node_search_service toneMint
class node_receipt_addon,node_finance_addon,node_layout_addon,node_classify_addon toneRose
class node_l2_models,node_l2_tools,node_db_session toneIndigo
```

## Components

### API and access

- [FastAPI app](https://github.com/momonokiki/datacore-oss/blob/main/app/main.py): `app/main.py`
- [Authentication](https://github.com/momonokiki/datacore-oss/blob/main/app/api/routes/auth.py): `app/api/routes/auth.py`
- [File ingest](https://github.com/momonokiki/datacore-oss/blob/main/app/api/routes/ingest.py): `app/api/routes/ingest.py`
- [Search API](https://github.com/momonokiki/datacore-oss/blob/main/app/api/routes/search.py): `app/api/routes/search.py`
- [Finance API](https://github.com/momonokiki/datacore-oss/blob/main/app/api/routes/finance.py): `app/api/routes/finance.py`
- [Receipt API](https://github.com/momonokiki/datacore-oss/blob/main/app/api/routes/receipts.py): `app/api/routes/receipts.py`
- [Search UI](https://github.com/momonokiki/datacore-oss/blob/main/app/webui/webui/pages/search.py): `app/webui/webui/pages/search.py`

### Raw file layer

- [L1 ingestion](https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l1_raw/service.py): `app/layers/l1_raw/service.py`
- [CAS storage](https://github.com/momonokiki/datacore-oss/blob/main/app/infrastructure/storage/adapter.py): `app/infrastructure/storage/adapter.py`
- [Raw file records](https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l1_raw/models.py): `app/layers/l1_raw/models.py`

### Extraction and search

- [Task dispatch](https://github.com/momonokiki/datacore-oss/blob/main/app/tasks.py): `app/tasks.py`
- [Extraction tasks](https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l2_extracted/tasks.py): `app/layers/l2_extracted/tasks.py`
- [Media extraction](https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l2_extracted/media_info/service.py): `app/layers/l2_extracted/media_info/service.py`
- [OCR extraction](https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l2_extracted/ocr/service.py): `app/layers/l2_extracted/ocr/service.py`
- [Image embedding](https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l2_extracted/embedding_img/service.py): `app/layers/l2_extracted/embedding_img/service.py`
- [Text embedding](https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l2_extracted/embedding_text/service.py): `app/layers/l2_extracted/embedding_text/service.py`
- [Search service](https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l2_extracted/search/service.py): `app/layers/l2_extracted/search/service.py`

### Structured data

- [Receipt add-on](https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l3_structured/addons/receipt/service.py): `app/layers/l3_structured/addons/receipt/service.py`
- [Finance parser](https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l3_structured/addons/finunce_parser/service.py): `app/layers/l3_structured/addons/finunce_parser/service.py`
- [YomiToku layout](https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l3_structured/addons/yomitoku_layout/service.py): `app/layers/l3_structured/addons/yomitoku_layout/service.py`
- [Receipt classifier](https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l3_structured/addons/receipt_classify_exp/service.py): `app/layers/l3_structured/addons/receipt_classify_exp/service.py`

### Persistence

- [Extracted artifacts](https://github.com/momonokiki/datacore-oss/blob/main/app/layers/l2_extracted/models.py): `app/layers/l2_extracted/models.py`
- [Tool registry](https://github.com/momonokiki/datacore-oss/blob/main/app/infrastructure/db/models/l2_tools.py): `app/infrastructure/db/models/l2_tools.py`
- [DB sessions](https://github.com/momonokiki/datacore-oss/blob/main/app/infrastructure/db/session.py): `app/infrastructure/db/session.py`

### Other components

- **User**

## Connections

- FastAPI app → File ingest: registers route (evidence: `app/main.py`)
- FastAPI app → Authentication: registers route (evidence: `app/main.py`)
- FastAPI app → Search API: registers route (evidence: `app/main.py`)
- FastAPI app → Finance API: registers route (evidence: `app/main.py`)
- FastAPI app → Receipt API: registers route (evidence: `app/main.py`)
- User → File ingest: uploads files (evidence: `README.md`)
- File ingest → Authentication: checks identity (evidence: `app/api/routes/ingest.py`)
- File ingest → L1 ingestion: ingests files (evidence: `app/api/routes/ingest.py`)
- File ingest → Task dispatch: dispatches chain (evidence: `app/api/routes/ingest.py`)
- L1 ingestion → CAS storage: stores binaries (evidence: `app/layers/l1_raw/service.py`)
- L1 ingestion → Raw file records: records uploads (evidence: `app/layers/l1_raw/service.py`)
- Search API → Search service: queries (evidence: `app/api/routes/search.py`)
- Search service → Text embedding: uses embeddings (evidence: `app/layers/l2_extracted/search/service.py`)
- Extraction tasks → Media extraction: runs extraction (evidence: `app/layers/l2_extracted/tasks.py`)
- Extraction tasks → OCR extraction: runs OCR (evidence: `app/layers/l2_extracted/tasks.py`)
- Extraction tasks → Image embedding: runs embedding (evidence: `app/layers/l2_extracted/tasks.py`)
- Extraction tasks → Text embedding: runs embedding (evidence: `app/layers/l2_extracted/tasks.py`)
- Media extraction → Extracted artifacts: writes metadata (evidence: `app/layers/l2_extracted/media_info/service.py`)
- OCR extraction → Extracted artifacts: writes artifacts (evidence: `app/layers/l2_extracted/ocr/service.py`)
- Image embedding → Extracted artifacts: writes vectors (evidence: `app/layers/l2_extracted/embedding_img/service.py`)
- Image embedding → CAS storage: loads original (evidence: `app/layers/l2_extracted/embedding_img/service.py`)
- Image embedding → Raw file records: reads file record (evidence: `app/layers/l2_extracted/embedding_img/service.py`)
- Image embedding → Tool registry: registers tool (evidence: `app/layers/l2_extracted/embedding_img/service.py`)
- Text embedding → Extracted artifacts: reads and writes (evidence: `app/layers/l2_extracted/embedding_text/service.py`)
- Text embedding → Tool registry: registers tool (evidence: `app/layers/l2_extracted/embedding_text/service.py`)
- Search service → Extracted artifacts: searches artifacts (evidence: `app/layers/l2_extracted/search/service.py`)
- Search UI → Search API: uses API
