# raneem4444-h/shipp-marketplace architecture

> Architecture diagram of the GitHub repository raneem4444-h/shipp-marketplace, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/raneem4444-h/shipp-marketplace
- Repository: https://github.com/raneem4444-h/shipp-marketplace
- Video tour (about a minute): https://gitdiagram.com/raneem4444-h/shipp-marketplace/video

## Overview

SHIPP is a two-sided household-goods marketplace with a Databricks data and AI workflow. Donors and requesters use the app to publish listings or requests and browse marketplace features; operational changes feed incremental processing, trusted matching, semantic search, and AI-assisted recommendations. The agent can propose a saved item, with the user approving the operational action. The README documents the major pipeline relationships; source excerpts confirm app entry, client setup, model access, and marketplace storage, but many subsystem internals were not sampled.

Main entry points: [`agent/agent_ship/src/shipp/agent/agent.py`](https://github.com/raneem4444-h/shipp-marketplace/blob/main/agent/agent_ship/src/shipp/agent/agent.py) (SHIPP Agent), [`app.py`](https://github.com/raneem4444-h/shipp-marketplace/blob/main/app.py) (Streamlit App), [`data_pipeline/silver/current_state.py`](https://github.com/raneem4444-h/shipp-marketplace/blob/main/data_pipeline/silver/current_state.py) (Silver Current State), [`data_pipeline/silver/pairs.py`](https://github.com/raneem4444-h/shipp-marketplace/blob/main/data_pipeline/silver/pairs.py) (Eligible Candidate Pairs)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_app["Marketplace App"]
  node_appui["Streamlit App<br/>[app.py]"]
  node_pages["Marketplace Pages"]
  node_identity["Identity Service"]
  node_repo["Marketplace Repository"]
end

subgraph group_pipeline["Matching Pipeline"]
  node_bronze[("Bronze History<br/>[tables.py]")]
  node_current["Silver Current State<br/>[current_state.py]"]
  node_pairs["Eligible Candidate Pairs<br/>[pairs.py]"]
  node_ors["Route Enrichment<br/>[ors_client.py]"]
  node_route["Silver Routes<br/>[routes.py]"]
  node_scoring["Gold Match Scoring<br/>[scoring.py]"]
  node_matches[("Gold Candidate Matches<br/>[tables.py]")]
end

subgraph group_search["Search Enrichment"]
  node_vision["Vision Enrichment<br/>[vision.py]"]
  node_content["Listing Content<br/>[content.py]"]
  node_docs["Search Documents<br/>[search_docs.py]"]
  node_index[("AI Search Index<br/>[index.py]")]
  node_retrieval["Semantic Retrieval<br/>[retrieval.py]"]
end

subgraph group_agent["AI Agent"]
  node_agent["SHIPP Agent<br/>[agent.py]"]
  node_tools["Agent Tools<br/>[tools.py]"]
  node_model["Chat Model<br/>[llm.py]"]
  node_goldreader["Gold Candidate Reader<br/>[gold.py]"]
  node_searchclient["AI Search Client<br/>[search.py]"]
  node_lakebaserepo["Agent Lakebase Access<br/>[lakebase.py]"]
end

subgraph group_shared["Shared Services"]
  node_contracts["Agent Contracts<br/>[contracts.py]"]
  node_rules["Save Rules<br/>[rules.py]"]
  node_settings["Pipeline Settings<br/>[settings.py]"]
  node_tables["Table Registry<br/>[tables.py]"]
  node_geo["Geospatial Utilities<br/>[geo.py]"]
end

node_people(("Donors / Requesters"))
node_lakebase[("Lakebase")]
node_orsservice["OpenRouteService"]
node_images[("Listing Images")]

node_people -->|"use"| node_appui
node_appui -->|"serves pages"| node_pages
node_appui -->|"operational reads / writes"| node_repo
node_appui -->|"request recommendation"| node_agent
node_appui -.->|"resolve identity"| node_identity
node_repo -->|"read / write"| node_lakebase
node_repo -->|"store image files"| node_images
node_lakebase -->|"incremental changes"| node_bronze
node_bronze -->|"reconstruct state"| node_current
node_current -->|"apply eligibility"| node_pairs
node_pairs -->|"request routes"| node_ors
node_ors -->|"request matrix"| node_orsservice
node_ors -->|"persist response"| node_bronze
node_bronze -->|"read route responses"| node_route
node_route -->|"provide routes"| node_scoring
node_current -->|"provide attributes"| node_scoring
node_scoring -->|"write scores"| node_matches
node_images -->|"provide images"| node_vision
node_current -->|"provide listings"| node_content
node_vision -->|"enrich content"| node_content
node_content -->|"create documents"| node_docs
node_docs -->|"sync documents"| node_index
node_index -->|"serve search"| node_retrieval
node_agent -->|"request completion"| node_model
node_agent -->|"dispatch tools"| node_tools
node_agent -->|"read candidates"| node_goldreader
node_goldreader -->|"query matches"| node_matches
node_agent -->|"search listings"| node_searchclient
node_searchclient -->|"query context"| node_retrieval
node_agent -->|"save confirmed item"| node_lakebaserepo
node_lakebaserepo -->|"read / write"| node_lakebase
node_lakebaserepo -->|"validate save"| node_rules
node_agent -->|"use contracts"| node_contracts
node_pairs -->|"geospatial checks"| node_geo
node_pairs -->|"read settings"| node_settings
node_scoring -->|"read weights"| node_settings
node_current -.->|"use table names"| node_tables

click node_appui "https://github.com/raneem4444-h/shipp-marketplace/blob/main/app.py"
click node_pages "https://github.com/raneem4444-h/shipp-marketplace/tree/main/ui/pages"
click node_identity "https://github.com/raneem4444-h/shipp-marketplace/blob/main/services/identity_service.py"
click node_repo "https://github.com/raneem4444-h/shipp-marketplace/blob/main/repositories/marketplace_repository.py"
click node_bronze "https://github.com/raneem4444-h/shipp-marketplace/blob/main/config/tables.py"
click node_current "https://github.com/raneem4444-h/shipp-marketplace/blob/main/data_pipeline/silver/current_state.py"
click node_pairs "https://github.com/raneem4444-h/shipp-marketplace/blob/main/data_pipeline/silver/pairs.py"
click node_ors "https://github.com/raneem4444-h/shipp-marketplace/blob/main/data_pipeline/ingestion/ors_client.py"
click node_route "https://github.com/raneem4444-h/shipp-marketplace/blob/main/data_pipeline/silver/routes.py"
click node_scoring "https://github.com/raneem4444-h/shipp-marketplace/blob/main/data_pipeline/gold/scoring.py"
click node_matches "https://github.com/raneem4444-h/shipp-marketplace/blob/main/config/tables.py"
click node_vision "https://github.com/raneem4444-h/shipp-marketplace/blob/main/rag/vision.py"
click node_content "https://github.com/raneem4444-h/shipp-marketplace/blob/main/rag/content.py"
click node_docs "https://github.com/raneem4444-h/shipp-marketplace/blob/main/rag/search_docs.py"
click node_index "https://github.com/raneem4444-h/shipp-marketplace/blob/main/rag/index.py"
click node_retrieval "https://github.com/raneem4444-h/shipp-marketplace/blob/main/rag/retrieval.py"
click node_agent "https://github.com/raneem4444-h/shipp-marketplace/blob/main/agent/agent_ship/src/shipp/agent/agent.py"
click node_tools "https://github.com/raneem4444-h/shipp-marketplace/blob/main/agent/agent_ship/src/shipp/agent/tools.py"
click node_model "https://github.com/raneem4444-h/shipp-marketplace/blob/main/agent/agent_ship/src/shipp/agent/llm.py"
click node_goldreader "https://github.com/raneem4444-h/shipp-marketplace/blob/main/agent/agent_ship/src/shipp/agent/gold.py"
click node_searchclient "https://github.com/raneem4444-h/shipp-marketplace/blob/main/agent/agent_ship/src/shipp/agent/search.py"
click node_lakebaserepo "https://github.com/raneem4444-h/shipp-marketplace/blob/main/agent/agent_ship/src/shipp/agent/lakebase.py"
click node_contracts "https://github.com/raneem4444-h/shipp-marketplace/blob/main/agent/agent_ship/src/shipp/agent/contracts.py"
click node_rules "https://github.com/raneem4444-h/shipp-marketplace/blob/main/agent/agent_ship/src/shipp/agent/rules.py"
click node_settings "https://github.com/raneem4444-h/shipp-marketplace/blob/main/config/settings.py"
click node_tables "https://github.com/raneem4444-h/shipp-marketplace/blob/main/config/tables.py"
click node_geo "https://github.com/raneem4444-h/shipp-marketplace/blob/main/data_pipeline/common/geo.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_appui,node_pages,node_identity,node_repo toneBlue
class node_bronze,node_current,node_pairs,node_ors,node_route,node_scoring,node_matches,node_lakebase,node_images toneAmber
class node_vision,node_content,node_docs,node_index,node_retrieval,node_people,node_orsservice toneMint
class node_agent,node_tools,node_model,node_goldreader,node_searchclient,node_lakebaserepo toneRose
class node_contracts,node_rules,node_settings,node_tables,node_geo toneIndigo
```

## Components

### Marketplace App

- [Streamlit App](https://github.com/raneem4444-h/shipp-marketplace/blob/main/app.py): `app.py`
- [Marketplace Pages](https://github.com/raneem4444-h/shipp-marketplace/tree/main/ui/pages): `ui/pages`
- [Identity Service](https://github.com/raneem4444-h/shipp-marketplace/blob/main/services/identity_service.py): `services/identity_service.py`
- [Marketplace Repository](https://github.com/raneem4444-h/shipp-marketplace/blob/main/repositories/marketplace_repository.py): `repositories/marketplace_repository.py`

### Matching Pipeline

- [Bronze History](https://github.com/raneem4444-h/shipp-marketplace/blob/main/config/tables.py): `config/tables.py`
- [Silver Current State](https://github.com/raneem4444-h/shipp-marketplace/blob/main/data_pipeline/silver/current_state.py): `data_pipeline/silver/current_state.py`
- [Eligible Candidate Pairs](https://github.com/raneem4444-h/shipp-marketplace/blob/main/data_pipeline/silver/pairs.py): `data_pipeline/silver/pairs.py`
- [Route Enrichment](https://github.com/raneem4444-h/shipp-marketplace/blob/main/data_pipeline/ingestion/ors_client.py): `data_pipeline/ingestion/ors_client.py`
- [Silver Routes](https://github.com/raneem4444-h/shipp-marketplace/blob/main/data_pipeline/silver/routes.py): `data_pipeline/silver/routes.py`
- [Gold Match Scoring](https://github.com/raneem4444-h/shipp-marketplace/blob/main/data_pipeline/gold/scoring.py): `data_pipeline/gold/scoring.py`
- [Gold Candidate Matches](https://github.com/raneem4444-h/shipp-marketplace/blob/main/config/tables.py): `config/tables.py`

### Search Enrichment

- [Vision Enrichment](https://github.com/raneem4444-h/shipp-marketplace/blob/main/rag/vision.py): `rag/vision.py`
- [Listing Content](https://github.com/raneem4444-h/shipp-marketplace/blob/main/rag/content.py): `rag/content.py`
- [Search Documents](https://github.com/raneem4444-h/shipp-marketplace/blob/main/rag/search_docs.py): `rag/search_docs.py`
- [AI Search Index](https://github.com/raneem4444-h/shipp-marketplace/blob/main/rag/index.py): `rag/index.py`
- [Semantic Retrieval](https://github.com/raneem4444-h/shipp-marketplace/blob/main/rag/retrieval.py): `rag/retrieval.py`

### AI Agent

- [SHIPP Agent](https://github.com/raneem4444-h/shipp-marketplace/blob/main/agent/agent_ship/src/shipp/agent/agent.py): `agent/agent_ship/src/shipp/agent/agent.py`
- [Agent Tools](https://github.com/raneem4444-h/shipp-marketplace/blob/main/agent/agent_ship/src/shipp/agent/tools.py): `agent/agent_ship/src/shipp/agent/tools.py`
- [Chat Model](https://github.com/raneem4444-h/shipp-marketplace/blob/main/agent/agent_ship/src/shipp/agent/llm.py): `agent/agent_ship/src/shipp/agent/llm.py`
- [Gold Candidate Reader](https://github.com/raneem4444-h/shipp-marketplace/blob/main/agent/agent_ship/src/shipp/agent/gold.py): `agent/agent_ship/src/shipp/agent/gold.py`
- [AI Search Client](https://github.com/raneem4444-h/shipp-marketplace/blob/main/agent/agent_ship/src/shipp/agent/search.py): `agent/agent_ship/src/shipp/agent/search.py`
- [Agent Lakebase Access](https://github.com/raneem4444-h/shipp-marketplace/blob/main/agent/agent_ship/src/shipp/agent/lakebase.py): `agent/agent_ship/src/shipp/agent/lakebase.py`

### Shared Services

- [Agent Contracts](https://github.com/raneem4444-h/shipp-marketplace/blob/main/agent/agent_ship/src/shipp/agent/contracts.py): `agent/agent_ship/src/shipp/agent/contracts.py`
- [Save Rules](https://github.com/raneem4444-h/shipp-marketplace/blob/main/agent/agent_ship/src/shipp/agent/rules.py): `agent/agent_ship/src/shipp/agent/rules.py`
- [Pipeline Settings](https://github.com/raneem4444-h/shipp-marketplace/blob/main/config/settings.py): `config/settings.py`
- [Table Registry](https://github.com/raneem4444-h/shipp-marketplace/blob/main/config/tables.py): `config/tables.py`
- [Geospatial Utilities](https://github.com/raneem4444-h/shipp-marketplace/blob/main/data_pipeline/common/geo.py): `data_pipeline/common/geo.py`

### Other components

- **Donors / Requesters**
- **Lakebase**
- **OpenRouteService**
- **Listing Images**

## Connections

- Donors / Requesters → Streamlit App: use (evidence: `README.md`)
- Streamlit App → Marketplace Pages: serves pages (evidence: `app.py`)
- Streamlit App → Marketplace Repository: operational reads / writes (evidence: `README.md`)
- Streamlit App → SHIPP Agent: request recommendation (evidence: `README.md`)
- Streamlit App → Identity Service: resolve identity
- Marketplace Repository → Lakebase: read / write (evidence: `repositories/marketplace_repository.py`)
- Marketplace Repository → Listing Images: store image files (evidence: `repositories/marketplace_repository.py`)
- Lakebase → Bronze History: incremental changes (evidence: `README.md`)
- Bronze History → Silver Current State: reconstruct state (evidence: `README.md`)
- Silver Current State → Eligible Candidate Pairs: apply eligibility (evidence: `README.md`)
- Eligible Candidate Pairs → Route Enrichment: request routes (evidence: `README.md`)
- Route Enrichment → OpenRouteService: request matrix (evidence: `README.md`)
- Route Enrichment → Bronze History: persist response (evidence: `README.md`)
- Bronze History → Silver Routes: read route responses (evidence: `README.md`)
- Silver Routes → Gold Match Scoring: provide routes (evidence: `README.md`)
- Silver Current State → Gold Match Scoring: provide attributes (evidence: `README.md`)
- Gold Match Scoring → Gold Candidate Matches: write scores (evidence: `README.md`)
- Listing Images → Vision Enrichment: provide images (evidence: `README.md`)
- Silver Current State → Listing Content: provide listings (evidence: `README.md`)
- Vision Enrichment → Listing Content: enrich content (evidence: `README.md`)
- Listing Content → Search Documents: create documents (evidence: `README.md`)
- Search Documents → AI Search Index: sync documents (evidence: `README.md`)
- AI Search Index → Semantic Retrieval: serve search (evidence: `README.md`)
- SHIPP Agent → Chat Model: request completion (evidence: `agent/agent_ship/src/shipp/agent/agent.py`)
- SHIPP Agent → Agent Tools: dispatch tools (evidence: `agent/agent_ship/src/shipp/agent/agent.py`)
- SHIPP Agent → Gold Candidate Reader: read candidates (evidence: `README.md`)
- Gold Candidate Reader → Gold Candidate Matches: query matches (evidence: `README.md`)
- SHIPP Agent → AI Search Client: search listings (evidence: `README.md`)
- AI Search Client → Semantic Retrieval: query context (evidence: `README.md`)
- SHIPP Agent → Agent Lakebase Access: save confirmed item (evidence: `README.md`)
- Agent Lakebase Access → Lakebase: read / write (evidence: `README.md`)
- Agent Lakebase Access → Save Rules: validate save (evidence: `README.md`)
- SHIPP Agent → Agent Contracts: use contracts (evidence: `README.md`)
- Eligible Candidate Pairs → Geospatial Utilities: geospatial checks (evidence: `README.md`)
- Eligible Candidate Pairs → Pipeline Settings: read settings (evidence: `README.md`)
- Gold Match Scoring → Pipeline Settings: read weights (evidence: `data_pipeline/gold/scoring.py`)
- Silver Current State → Table Registry: use table names
