How raneem4444-h/shipp-marketplace works
SHIPP is a two-sided household-goods marketplace with a Databricks data and AI workflow. Donors and requesters use the app to publish listings or requests and browse marketplace features; operational changes feed incremental processing, trusted matching, semantic search, and AI-assisted recommendations. The agent can propose a saved item, with the user approving the operational action. The README documents the major pipeline relationships; source excerpts confirm app entry, client setup, model access, and marketplace storage, but many subsystem internals were not sampled.
Main entry points: agent.py SHIPP Agent · app.py Streamlit App · current_state.py Silver Current State · pairs.py Eligible Candidate Pairs
Components 31
Marketplace App
- Streamlit App
app.py - Marketplace Pages
ui/pages - Identity Service
services/identity_service.py - Marketplace Repository
repositories/marketplace_repository.py
Matching Pipeline
- Bronze History
config/tables.py - Silver Current State
data_pipeline/silver/current_state.py - Eligible Candidate Pairs
data_pipeline/silver/pairs.py - Route Enrichment
data_pipeline/ingestion/ors_client.py - Silver Routes
data_pipeline/silver/routes.py - Gold Match Scoring
data_pipeline/gold/scoring.py - Gold Candidate Matches
config/tables.py
Search Enrichment
- Vision Enrichment
rag/vision.py - Listing Content
rag/content.py - Search Documents
rag/search_docs.py - AI Search Index
rag/index.py - Semantic Retrieval
rag/retrieval.py
AI Agent
- SHIPP Agent
agent/agent_ship/src/shipp/agent/agent.py - Agent Tools
agent/agent_ship/src/shipp/agent/tools.py - Chat Model
agent/agent_ship/src/shipp/agent/llm.py - Gold Candidate Reader
agent/agent_ship/src/shipp/agent/gold.py - AI Search Client
agent/agent_ship/src/shipp/agent/search.py - Agent Lakebase Access
agent/agent_ship/src/shipp/agent/lakebase.py
Shared Services
- Agent Contracts
agent/agent_ship/src/shipp/agent/contracts.py - Save Rules
agent/agent_ship/src/shipp/agent/rules.py - Pipeline Settings
config/settings.py - Table Registry
config/tables.py - Geospatial Utilities
data_pipeline/common/geo.py
Other components
- Donors / Requesters
- Lakebase
- OpenRouteService
- Listing Images
Connections 37
- Donors / Requesters → Streamlit App: use
README.md - Streamlit App → Marketplace Pages: serves pages
app.py - Streamlit App → Marketplace Repository: operational reads / writes
README.md - Streamlit App → SHIPP Agent: request recommendation
README.md - Streamlit App → Identity Service: resolve identity
- Marketplace Repository → Lakebase: read / write
repositories/marketplace_repository.py - Marketplace Repository → Listing Images: store image files
repositories/marketplace_repository.py - Lakebase → Bronze History: incremental changes
README.md - Bronze History → Silver Current State: reconstruct state
README.md - Silver Current State → Eligible Candidate Pairs: apply eligibility
README.md - Eligible Candidate Pairs → Route Enrichment: request routes
README.md - Route Enrichment → OpenRouteService: request matrix
README.md - Route Enrichment → Bronze History: persist response
README.md - Bronze History → Silver Routes: read route responses
README.md - Silver Routes → Gold Match Scoring: provide routes
README.md - Silver Current State → Gold Match Scoring: provide attributes
README.md - Gold Match Scoring → Gold Candidate Matches: write scores
README.md - Listing Images → Vision Enrichment: provide images
README.md - Silver Current State → Listing Content: provide listings
README.md - Vision Enrichment → Listing Content: enrich content
README.md - Listing Content → Search Documents: create documents
README.md - Search Documents → AI Search Index: sync documents
README.md - AI Search Index → Semantic Retrieval: serve search
README.md - SHIPP Agent → Chat Model: request completion
agent/agent_ship/src/shipp/agent/agent.py - SHIPP Agent → Agent Tools: dispatch tools
agent/agent_ship/src/shipp/agent/agent.py - SHIPP Agent → Gold Candidate Reader: read candidates
README.md - Gold Candidate Reader → Gold Candidate Matches: query matches
README.md - SHIPP Agent → AI Search Client: search listings
README.md - AI Search Client → Semantic Retrieval: query context
README.md - SHIPP Agent → Agent Lakebase Access: save confirmed item
README.md - Agent Lakebase Access → Lakebase: read / write
README.md - Agent Lakebase Access → Save Rules: validate save
README.md - SHIPP Agent → Agent Contracts: use contracts
README.md - Eligible Candidate Pairs → Geospatial Utilities: geospatial checks
README.md - Eligible Candidate Pairs → Pipeline Settings: read settings
README.md - Gold Match Scoring → Pipeline Settings: read weights
data_pipeline/gold/scoring.py - Silver Current State → Table Registry: use table names