# uc-cdis/gen3-ai architecture

> Architecture diagram of the GitHub repository uc-cdis/gen3-ai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/uc-cdis/gen3-ai
- Repository: https://github.com/uc-cdis/gen3-ai

## Overview

The Gen3 AI monorepo contains three API services, with Gen3 Embeddings providing the clearest implemented workflow: a caller sends collection or vector operations, routes authorize and validate them, and a data-access layer reads or writes PostgreSQL; similarity search returns ranked results. Gen3 Inference accepts Open Responses requests, resolves a model, selects a protocol client, and returns a streamed or non-streamed response. The model-repository service is currently only an app skeleton. README capabilities are included where implementation excerpts are absent; those links are omitted when wiring cannot be verified.

Main entry points: [`services/gen3_embeddings/src/gen3_embeddings/database/db.py`](https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_embeddings/src/gen3_embeddings/database/db.py) (Data access layer), [`services/gen3_embeddings/src/gen3_embeddings/main.py`](https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_embeddings/src/gen3_embeddings/main.py) (Embeddings app), [`services/gen3_embeddings/src/gen3_embeddings/routes/embeddings.py`](https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_embeddings/src/gen3_embeddings/routes/embeddings.py) (Embedding operations), [`services/gen3_embeddings/src/gen3_embeddings/auth.py`](https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_embeddings/src/gen3_embeddings/auth.py) (Request authorization)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_shared["Shared services"]
  node_commonauth["Shared auth<br/>[auth.py]"]
  node_commonmetrics["Metrics utilities<br/>[metrics.py]"]
end

subgraph group_embeddings["Embeddings API"]
  node_embedapp["Embeddings app<br/>[main.py]"]
  node_embedroutes["Collection routes<br/>[collections.py]"]
  node_embedops["Embedding operations<br/>[embeddings.py]"]
  node_embedbulk["Bulk operations<br/>[embeddings_bulk.py]"]
  node_search["Similarity search<br/>[search.py]"]
  node_embedauth["Request authorization<br/>[auth.py]"]
  node_dal["Data access layer<br/>[db.py]"]
  node_dbmodels["Database models<br/>[models.py]"]
end

subgraph group_inference["Inference API"]
  node_inferapp["Inference app<br/>[main.py]"]
  node_responses["Open Responses route<br/>[responses.py]"]
  node_modelresolve["Model discovery<br/>[responses.py]"]
  node_protocolselect["Protocol selection<br/>[responses.py]"]
  node_openresponses["Open Responses client<br/>[openresponses.py]"]
  node_openai["OpenAI Chat client<br/>[openai_chat.py]"]
  node_protocolbase["Protocol interface<br/>[base.py]"]
end

subgraph group_modelrepo["Model repository"]
  node_modelapp["Model repository app<br/>[main.py]"]
end

node_caller(("API caller"))
node_pg[("PostgreSQL")]
node_arborist["Arborist policy engine"]
node_remote["Configured model hosts"]

node_caller -->|"sends API request"| node_embedapp
node_embedapp -->|"dispatches routes"| node_embedroutes
node_embedapp -->|"registers routes"| node_embedops
node_embedapp -->|"registers routes"| node_search
node_embedapp -->|"checks policy at startup"| node_embedauth
node_embedapp -->|"checks database at startup"| node_dal
node_embedroutes -->|"authorizes requests"| node_embedauth
node_embedroutes -->|"reads and writes"| node_dal
node_embedops -->|"authorizes requests"| node_embedauth
node_embedops -->|"reads and writes"| node_dal
node_embedops -->|"uses domain records"| node_dbmodels
node_embedbulk -->|"reads and writes"| node_dal
node_search -->|"queries vectors"| node_dal
node_dal -->|"queries and updates"| node_pg
node_dal -->|"maps records"| node_dbmodels
node_embedauth -->|"checks permissions"| node_arborist
node_caller -->|"sends API request"| node_inferapp
node_inferapp -->|"dispatches requests"| node_responses
node_inferapp -->|"records metrics"| node_commonmetrics
node_responses -->|"resolves requested model"| node_modelresolve
node_modelresolve -.->|"checks configured hosts"| node_remote
node_responses -->|"selects protocol client"| node_protocolselect
node_protocolselect -->|"selects compatible client"| node_openresponses
node_protocolselect -->|"selects compatible client"| node_openai
node_openresponses -->|"implements interface"| node_protocolbase
node_openai -->|"implements interface"| node_protocolbase
node_openresponses -.->|"requests model inference"| node_remote
node_openai -.->|"requests model inference"| node_remote
node_caller -->|"sends API request"| node_modelapp

click node_commonauth "https://github.com/uc-cdis/gen3-ai/blob/main/libraries/common/src/common/auth.py"
click node_commonmetrics "https://github.com/uc-cdis/gen3-ai/blob/main/libraries/common/src/common/metrics.py"
click node_embedapp "https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_embeddings/src/gen3_embeddings/main.py"
click node_embedroutes "https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_embeddings/src/gen3_embeddings/routes/collections.py"
click node_embedops "https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_embeddings/src/gen3_embeddings/routes/embeddings.py"
click node_embedbulk "https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_embeddings/src/gen3_embeddings/routes/embeddings_bulk.py"
click node_search "https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_embeddings/src/gen3_embeddings/routes/search.py"
click node_embedauth "https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_embeddings/src/gen3_embeddings/auth.py"
click node_dal "https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_embeddings/src/gen3_embeddings/database/db.py"
click node_dbmodels "https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_embeddings/src/gen3_embeddings/database/models.py"
click node_inferapp "https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_inference/src/gen3_inference/main.py"
click node_responses "https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_inference/src/gen3_inference/routes/responses.py"
click node_modelresolve "https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_inference/src/gen3_inference/routes/responses.py"
click node_protocolselect "https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_inference/src/gen3_inference/routes/responses.py"
click node_openresponses "https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_inference/src/gen3_inference/inference_protocols/openresponses.py"
click node_openai "https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_inference/src/gen3_inference/inference_protocols/openai_chat.py"
click node_protocolbase "https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_inference/src/gen3_inference/inference_protocols/base.py"
click node_modelapp "https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_ai_model_repo/src/gen3_ai_model_repo/main.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_commonauth,node_commonmetrics toneBlue
class node_embedapp,node_embedroutes,node_embedops,node_embedbulk,node_search,node_embedauth,node_dal,node_dbmodels,node_pg toneAmber
class node_inferapp,node_responses,node_modelresolve,node_protocolselect,node_openresponses,node_openai,node_protocolbase,node_caller toneMint
class node_modelapp toneRose
class node_arborist,node_remote toneIndigo
```

## Components

### Shared services

- [Shared auth](https://github.com/uc-cdis/gen3-ai/blob/main/libraries/common/src/common/auth.py): `libraries/common/src/common/auth.py`
- [Metrics utilities](https://github.com/uc-cdis/gen3-ai/blob/main/libraries/common/src/common/metrics.py): `libraries/common/src/common/metrics.py`

### Embeddings API

- [Embeddings app](https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_embeddings/src/gen3_embeddings/main.py): `services/gen3_embeddings/src/gen3_embeddings/main.py`
- [Collection routes](https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_embeddings/src/gen3_embeddings/routes/collections.py): `services/gen3_embeddings/src/gen3_embeddings/routes/collections.py`
- [Embedding operations](https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_embeddings/src/gen3_embeddings/routes/embeddings.py): `services/gen3_embeddings/src/gen3_embeddings/routes/embeddings.py`
- [Bulk operations](https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_embeddings/src/gen3_embeddings/routes/embeddings_bulk.py): `services/gen3_embeddings/src/gen3_embeddings/routes/embeddings_bulk.py`
- [Similarity search](https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_embeddings/src/gen3_embeddings/routes/search.py): `services/gen3_embeddings/src/gen3_embeddings/routes/search.py`
- [Request authorization](https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_embeddings/src/gen3_embeddings/auth.py): `services/gen3_embeddings/src/gen3_embeddings/auth.py`
- [Data access layer](https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_embeddings/src/gen3_embeddings/database/db.py): `services/gen3_embeddings/src/gen3_embeddings/database/db.py`
- [Database models](https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_embeddings/src/gen3_embeddings/database/models.py): `services/gen3_embeddings/src/gen3_embeddings/database/models.py`

### Inference API

- [Inference app](https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_inference/src/gen3_inference/main.py): `services/gen3_inference/src/gen3_inference/main.py`
- [Open Responses route](https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_inference/src/gen3_inference/routes/responses.py): `services/gen3_inference/src/gen3_inference/routes/responses.py`
- [Model discovery](https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_inference/src/gen3_inference/routes/responses.py): `services/gen3_inference/src/gen3_inference/routes/responses.py`
- [Protocol selection](https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_inference/src/gen3_inference/routes/responses.py): `services/gen3_inference/src/gen3_inference/routes/responses.py`
- [Open Responses client](https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_inference/src/gen3_inference/inference_protocols/openresponses.py): `services/gen3_inference/src/gen3_inference/inference_protocols/openresponses.py`
- [OpenAI Chat client](https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_inference/src/gen3_inference/inference_protocols/openai_chat.py): `services/gen3_inference/src/gen3_inference/inference_protocols/openai_chat.py`
- [Protocol interface](https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_inference/src/gen3_inference/inference_protocols/base.py): `services/gen3_inference/src/gen3_inference/inference_protocols/base.py`

### Model repository

- [Model repository app](https://github.com/uc-cdis/gen3-ai/blob/main/services/gen3_ai_model_repo/src/gen3_ai_model_repo/main.py): `services/gen3_ai_model_repo/src/gen3_ai_model_repo/main.py`

### Other components

- **API caller**
- **PostgreSQL**
- **Arborist policy engine**
- **Configured model hosts**

## Connections

- API caller → Embeddings app: sends API request
- Embeddings app → Collection routes: dispatches routes
- Embeddings app → Embedding operations: registers routes
- Embeddings app → Similarity search: registers routes
- Embeddings app → Request authorization: checks policy at startup
- Embeddings app → Data access layer: checks database at startup
- Collection routes → Request authorization: authorizes requests
- Collection routes → Data access layer: reads and writes
- Embedding operations → Request authorization: authorizes requests
- Embedding operations → Data access layer: reads and writes
- Embedding operations → Database models: uses domain records
- Bulk operations → Data access layer: reads and writes
- Similarity search → Data access layer: queries vectors
- Data access layer → PostgreSQL: queries and updates
- Data access layer → Database models: maps records
- Request authorization → Arborist policy engine: checks permissions
- API caller → Inference app: sends API request
- Inference app → Open Responses route: dispatches requests
- Inference app → Metrics utilities: records metrics
- Open Responses route → Model discovery: resolves requested model
- Model discovery → Configured model hosts: checks configured hosts
- Open Responses route → Protocol selection: selects protocol client
- Protocol selection → Open Responses client: selects compatible client
- Protocol selection → OpenAI Chat client: selects compatible client
- Open Responses client → Protocol interface: implements interface
- OpenAI Chat client → Protocol interface: implements interface
- Open Responses client → Configured model hosts: requests model inference
- OpenAI Chat client → Configured model hosts: requests model inference
- API caller → Model repository app: sends API request