# mongodb-partners/mdb-temporal-pra architecture

> Architecture diagram of the GitHub repository mongodb-partners/mdb-temporal-pra, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mongodb-partners/mdb-temporal-pra
- Repository: https://github.com/mongodb-partners/mdb-temporal-pra

## Overview

This repository is a Temporal-and-Atlas reference application for durable document ingestion and research chat. Its main path is an S3 object event, through a Temporal ingestion workflow that extracts, chunks, embeds, and indexes content, then a user question through the research API and durable agent to retrieval tools and an answer. A separate backfill path supports embedding-model changes and blue/green cutover. The UI exposes agent progress. Some subsystem links are based on README descriptions or SOURCE INDEX references; unsampled implementation details are not inferred.

Main entry points: [`pipeline/activities/ingest.py`](https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/activities/ingest.py) (Ingest activities), [`pipeline/workflows/ingest_workflow.py`](https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/workflows/ingest_workflow.py) (Ingest workflow), [`agent/agent_workflow.py`](https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/agent/agent_workflow.py) (Research workflow), [`agent/tools.py`](https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/agent/tools.py) (Agent tools)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingestion["Ingestion"]
  node_ingest_trigger["Ingestion trigger<br/>[trigger.py]"]
  node_ingest_workflow["Ingest workflow<br/>[ingest_workflow.py]"]
  node_ingest_activities["Ingest activities<br/>[ingest.py]"]
  node_extractors["File extractors"]
  node_data_contracts["Chunk contracts<br/>[models.py]"]
end

subgraph group_research["Research agent"]
  node_research_ui["Chat UI<br/>[App.tsx]"]
  node_research_api["Research API<br/>[api.py]"]
  node_agent_workflow["Research workflow<br/>[agent_workflow.py]"]
  node_agent_tools["Agent tools<br/>[tools.py]"]
  node_retrieval["Vector retrieval<br/>[retrieval.py]"]
end

subgraph group_operations["Index operations"]
  node_backfill_trigger["Backfill trigger"]
  node_backfill_workflow["Backfill workflow"]
  node_backfill_activities["Backfill activities<br/>[backfill.py]"]
  node_cutover["Active-index cutover<br/>[cutover.py]"]
  node_active_config[("Active index pointer<br/>[config_store.py]")]
end

subgraph group_services["Data and AI services"]
  node_atlas[("MongoDB Atlas")]
  node_s3[("S3 storage")]
  node_voyage["Voyage AI"]
  node_temporal["Temporal"]
  node_web_search["Hosted web search"]
end

node_source_event(("S3 object event"))
node_research_user(("Research user"))

node_source_event -->|"starts ingestion"| node_ingest_trigger
node_ingest_trigger -->|"starts workflow"| node_ingest_workflow
node_ingest_workflow -->|"runs activities"| node_ingest_activities
node_ingest_activities -->|"selects extractor"| node_extractors
node_ingest_workflow -->|"uses S3 reference"| node_data_contracts
node_ingest_activities -->|"builds document IDs"| node_data_contracts
node_ingest_activities -->|"downloads objects"| node_s3
node_ingest_activities -->|"stages and indexes chunks"| node_atlas
node_ingest_activities -->|"embeds chunks"| node_voyage
node_research_user -->|"asks question"| node_research_ui
node_research_ui -->|"starts and polls"| node_research_api
node_research_api -->|"starts workflow"| node_agent_workflow
node_agent_workflow -->|"invokes tools"| node_agent_tools
node_agent_tools -->|"runs vector search"| node_retrieval
node_retrieval -->|"reads active pointer"| node_active_config
node_retrieval -->|"searches active collection"| node_atlas
node_agent_tools -->|"reranks candidates"| node_voyage
node_agent_workflow -.->|"uses web search"| node_web_search
node_backfill_trigger -->|"starts backfill"| node_backfill_workflow
node_backfill_workflow -->|"runs activities"| node_backfill_activities
node_backfill_activities -->|"reads and writes chunks"| node_atlas
node_backfill_activities -->|"re-embeds chunks"| node_voyage
node_cutover -->|"flips active pointer"| node_active_config
node_active_config -->|"stores pointer"| node_atlas
node_temporal -.->|"orchestrates"| node_ingest_workflow
node_temporal -.->|"orchestrates"| node_agent_workflow

click node_ingest_trigger "https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/trigger.py"
click node_ingest_workflow "https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/workflows/ingest_workflow.py"
click node_ingest_activities "https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/activities/ingest.py"
click node_extractors "https://github.com/mongodb-partners/mdb-temporal-pra/tree/main/pipeline/extractors"
click node_data_contracts "https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/models.py"
click node_research_ui "https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/agent/ui/src/App.tsx"
click node_research_api "https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/agent/api.py"
click node_agent_workflow "https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/agent/agent_workflow.py"
click node_agent_tools "https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/agent/tools.py"
click node_retrieval "https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/retrieval.py"
click node_backfill_trigger "https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/trigger_backfill.py"
click node_backfill_workflow "https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/workflows/backfill_workflow.py"
click node_backfill_activities "https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/activities/backfill.py"
click node_cutover "https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/cutover.py"
click node_active_config "https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/config_store.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ingest_trigger,node_ingest_workflow,node_ingest_activities,node_extractors,node_data_contracts,node_research_user toneBlue
class node_research_ui,node_research_api,node_agent_workflow,node_agent_tools,node_retrieval toneAmber
class node_backfill_trigger,node_backfill_workflow,node_backfill_activities,node_cutover,node_active_config toneMint
class node_atlas,node_s3,node_voyage,node_temporal,node_web_search toneRose
class node_source_event toneIndigo
```

## Components

### Ingestion

- [Ingestion trigger](https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/trigger.py): `pipeline/trigger.py`
- [Ingest workflow](https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/workflows/ingest_workflow.py): `pipeline/workflows/ingest_workflow.py`
- [Ingest activities](https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/activities/ingest.py): `pipeline/activities/ingest.py`
- [File extractors](https://github.com/mongodb-partners/mdb-temporal-pra/tree/main/pipeline/extractors): `pipeline/extractors`
- [Chunk contracts](https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/models.py): `pipeline/models.py`

### Research agent

- [Chat UI](https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/agent/ui/src/App.tsx): `agent/ui/src/App.tsx`
- [Research API](https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/agent/api.py): `agent/api.py`
- [Research workflow](https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/agent/agent_workflow.py): `agent/agent_workflow.py`
- [Agent tools](https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/agent/tools.py): `agent/tools.py`
- [Vector retrieval](https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/retrieval.py): `pipeline/retrieval.py`

### Index operations

- [Backfill trigger](https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/trigger_backfill.py): `pipeline/trigger_backfill.py`
- [Backfill workflow](https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/workflows/backfill_workflow.py): `pipeline/workflows/backfill_workflow.py`
- [Backfill activities](https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/activities/backfill.py): `pipeline/activities/backfill.py`
- [Active-index cutover](https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/cutover.py): `pipeline/cutover.py`
- [Active index pointer](https://github.com/mongodb-partners/mdb-temporal-pra/blob/main/pipeline/config_store.py): `pipeline/config_store.py`

### Data and AI services

- **MongoDB Atlas**
- **S3 storage**
- **Voyage AI**
- **Temporal**
- **Hosted web search**

### Other components

- **S3 object event**
- **Research user**

## Connections

- S3 object event → Ingestion trigger: starts ingestion
- Ingestion trigger → Ingest workflow: starts workflow
- Ingest workflow → Ingest activities: runs activities (evidence: `pipeline/workflows/ingest_workflow.py`)
- Ingest activities → File extractors: selects extractor (evidence: `pipeline/activities/ingest.py`)
- Ingest workflow → Chunk contracts: uses S3 reference (evidence: `pipeline/workflows/ingest_workflow.py`)
- Ingest activities → Chunk contracts: builds document IDs (evidence: `pipeline/activities/ingest.py`)
- Ingest activities → S3 storage: downloads objects (evidence: `pipeline/activities/ingest.py`)
- Ingest activities → MongoDB Atlas: stages and indexes chunks (evidence: `pipeline/activities/ingest.py`)
- Ingest activities → Voyage AI: embeds chunks (evidence: `pipeline/activities/ingest.py`)
- Research user → Chat UI: asks question
- Chat UI → Research API: starts and polls (evidence: `agent/ui/src/App.tsx`)
- Research API → Research workflow: starts workflow
- Research workflow → Agent tools: invokes tools (evidence: `agent/agent_workflow.py`)
- Agent tools → Vector retrieval: runs vector search (evidence: `agent/tools.py`)
- Vector retrieval → Active index pointer: reads active pointer (evidence: `pipeline/retrieval.py`)
- Vector retrieval → MongoDB Atlas: searches active collection (evidence: `pipeline/retrieval.py`)
- Agent tools → Voyage AI: reranks candidates (evidence: `agent/tools.py`)
- Research workflow → Hosted web search: uses web search
- Backfill trigger → Backfill workflow: starts backfill (evidence: `pipeline/trigger_backfill.py`)
- Backfill workflow → Backfill activities: runs activities (evidence: `pipeline/workflows/backfill_workflow.py`)
- Backfill activities → MongoDB Atlas: reads and writes chunks (evidence: `pipeline/activities/backfill.py`)
- Backfill activities → Voyage AI: re-embeds chunks (evidence: `pipeline/activities/backfill.py`)
- Active-index cutover → Active index pointer: flips active pointer (evidence: `pipeline/cutover.py`)
- Active index pointer → MongoDB Atlas: stores pointer (evidence: `pipeline/config_store.py`)
- Temporal → Ingest workflow: orchestrates
- Temporal → Research workflow: orchestrates
