How uc-cdis/gen3-ai works
The Gen3 AI monorepo contains three API services, with Gen3 Embeddings providing the clearest implemented workflow: a caller sends collection or vector operations, routes authorize and validate them, and a data-access layer reads or writes PostgreSQL; similarity search returns ranked results. Gen3 Inference accepts Open Responses requests, resolves a model, selects a protocol client, and returns a streamed or non-streamed response. The model-repository service is currently only an app skeleton. README capabilities are included where implementation excerpts are absent; those links are omitted when wiring cannot be verified.
Main entry points: db.py Data access layer · main.py Embeddings app · embeddings.py Embedding operations · auth.py Request authorization
Components 22
Shared services
- Shared auth
libraries/common/src/common/auth.py - Metrics utilities
libraries/common/src/common/metrics.py
Embeddings API
- Embeddings app
services/gen3_embeddings/src/gen3_embeddings/main.py - Collection routes
services/gen3_embeddings/src/gen3_embeddings/routes/collections.py - Embedding operations
services/gen3_embeddings/src/gen3_embeddings/routes/embeddings.py - Bulk operations
services/gen3_embeddings/src/gen3_embeddings/routes/embeddings_bulk.py - Similarity search
services/gen3_embeddings/src/gen3_embeddings/routes/search.py - Request authorization
services/gen3_embeddings/src/gen3_embeddings/auth.py - Data access layer
services/gen3_embeddings/src/gen3_embeddings/database/db.py - Database models
services/gen3_embeddings/src/gen3_embeddings/database/models.py
Inference API
- Inference app
services/gen3_inference/src/gen3_inference/main.py - Open Responses route
services/gen3_inference/src/gen3_inference/routes/responses.py - Model discovery
services/gen3_inference/src/gen3_inference/routes/responses.py - Protocol selection
services/gen3_inference/src/gen3_inference/routes/responses.py - Open Responses client
services/gen3_inference/src/gen3_inference/inference_protocols/openresponses.py - OpenAI Chat client
services/gen3_inference/src/gen3_inference/inference_protocols/openai_chat.py - Protocol interface
services/gen3_inference/src/gen3_inference/inference_protocols/base.py
Model repository
- Model repository app
services/gen3_ai_model_repo/src/gen3_ai_model_repo/main.py
Other components
- API caller
- PostgreSQL
- Arborist policy engine
- Configured model hosts
Connections 29
- API caller → Embeddings app: sends API request
- Embeddings app → Collection routes: dispatches routes
- Embeddings app → Embedding operations: registers routes
- Embeddings app → Similarity search: registers routes
- Embeddings app → Request authorization: checks policy at startup
- Embeddings app → Data access layer: checks database at startup
- Collection routes → Request authorization: authorizes requests
- Collection routes → Data access layer: reads and writes
- Embedding operations → Request authorization: authorizes requests
- Embedding operations → Data access layer: reads and writes
- Embedding operations → Database models: uses domain records
- Bulk operations → Data access layer: reads and writes
- Similarity search → Data access layer: queries vectors
- Data access layer → PostgreSQL: queries and updates
- Data access layer → Database models: maps records
- Request authorization → Arborist policy engine: checks permissions
- API caller → Inference app: sends API request
- Inference app → Open Responses route: dispatches requests
- Inference app → Metrics utilities: records metrics
- Open Responses route → Model discovery: resolves requested model
- Model discovery → Configured model hosts: checks configured hosts
- Open Responses route → Protocol selection: selects protocol client
- Protocol selection → Open Responses client: selects compatible client
- Protocol selection → OpenAI Chat client: selects compatible client
- Open Responses client → Protocol interface: implements interface
- OpenAI Chat client → Protocol interface: implements interface
- Open Responses client → Configured model hosts: requests model inference
- OpenAI Chat client → Configured model hosts: requests model inference
- API caller → Model repository app: sends API request