# sfuller14/semantic-consensus architecture

> Architecture diagram of the GitHub repository sfuller14/semantic-consensus, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sfuller14/semantic-consensus
- Repository: https://github.com/sfuller14/semantic-consensus

## Overview

This project is best diagrammed as a **single-app Streamlit frontend with AI/search integrations**, not as a classic multi-service backend architecture.

##### What this project is
From the README and file tree, **Commercial Consensus** is a **hackathon-built e-commerce semantic search and conversational recommendation app**. It uses:

- **Streamlit** for the UI
- **Pinecone** for vector search
- **OpenAI embeddings** to encode product/review text
- **Cohere rerank** to refine search results
- **OpenAI ChatCompletions** for the chat experience
- **Docker + AWS Elastic Beanstalk** for deployment

The repo is small, which strongly suggests the application logic is concentrated in **`main.py`** and wrapped by Streamlit.

##### End User / Browser
- The person using the app.
- Initiates search, view, and chat actions.

##### Streamlit App (`main.py`)
- This is the core application layer.
- Since there is no separate backend directory, treat this as the **UI + orchestration layer**.
- **Search UI**
- **Product Detail / View UI**
- **Chat UI**

##### Query Orchestration / Business Logic
- Inside `main.py` or logically adjacent to it.
- Handles:
- user input parsing
- filter construction
- Pinecone queries
- reranking
- prompt assembly for chat

##### Pinecone Vector Index
- Stores embedded product/review vectors.
- Main retrieval layer for semantic search.

##### OpenAI Embeddings API
- Used during indexing or preprocessing to create `text-embedding-ada-002` vectors.

##### Cohere Rerank API
- Used after Pinecone retrieval to reorder results.
- Important to show as a second step after initial vector search.

##### OpenAI ChatCompletions API
- Used in the chat workflow to generate natural-language answers from retrieved reviews/specs.

##### Product/Review Dataset / Indexed Corpus
- Even though the repo does not contain the ingestion scripts, the README makes clear that the system depends on:
- product specs
- customer reviews
- title/metadata fields
- This as an upstream data source that feeds the embedding/indexing pipeline.

##### Deployment Layer
- **Docker**
- **AWS Elastic Beanstalk**

##### Architectural patterns to highlight
This project uses several important design ideas:

##### a) Monolithic UI + orchestration
There is no separate backend service in the repo tree, so represent `main.py` as a **single orchestration monolith** rather than pretending there is a microservice architecture.

##### b) Retrieval-Augmented Generation style pipeline
The chat flow is essentially a **RAG-like pipeline**:
- retrieve relevant review/spec text
- rerank results
- feed into an LLM for response generation

##### c) Hybrid search
The README explicitly describes:
- **tabular/metadata filtering**
- **semantic vector search**
- **reranking**

That makes this a **hybrid search system**, not just a pure vector search app.

##### d) Offline indexing pipeline
Even though not present in the tree, the architecture depends on a preprocessing/indexing step:
- source data → embeddings → Pinecone index

##### Technologies to label
Be explicit with names, because the project relies heavily on external AI services.

##### Core stack
- **Python**
- **Streamlit**
- **Pinecone**
- **OpenAI API**
- embeddings
- chat completions
- **Cohere API**
- rerank

##### Deployment / ops
- **Docker**
- **AWS Elastic Beanstalk**

##### Supporting files
- **`requirements.txt`** → dependencies
- **`.streamlit/config.toml`** → Streamlit runtime/config
- **`docker/Dockerfile`** → container build
- **`docker/docker-compose.yaml`** → local composition/testing
- **`docker/Makefile`** → convenience build/run commands

##### Best-practice recommendation for this specific project
- detailed enough to show the AI retrieval chain,
- but honest that most logic lives in **one Streamlit app**.

##### Deployment via Docker on AWS

That will capture the real architecture much more accurately than splitting it into fake frontend/backend microservices.

> “`main.py` acts as both UI and orchestration layer; external AI services handle indexing, retrieval, reranking, and generation.”

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External actors
    User["End User / Browser"]:::user

    %% Presentation layer
    subgraph "Presentation Layer"
        UI["Streamlit App<br/>(main.py)"]:::frontend
        Config["Streamlit Runtime Config<br/>(.streamlit/config.toml)"]:::config
        Assets["UI Assets / Thumbnails<br/>(thumbnails)"]:::asset
    end

    %% Application layer
    subgraph "Application Layer"
        Search["Search Orchestration"]:::logic
        View["Product View Renderer"]:::logic
        Chat["Chat Orchestration"]:::logic
        Prompt["Prompt Assembly<br/>+ Context Packing"]:::logic
    end

    %% AI / Search layer
    subgraph "AI / Search Layer"
        Pinecone["Pinecone Vector Index"]:::storage
        Cohere["Cohere Rerank API"]:::external
        OpenAIEmb["OpenAI Embeddings API"]:::external
        OpenAIChat["OpenAI ChatCompletions API"]:::external
    end

    %% Data layer
    subgraph "Data Layer"
        Corpus["Product & Review Corpus"]:::data
        Embeddings["Offline Embedding / Indexing Pipeline"]:::pipeline
    end

    %% Infrastructure layer
    subgraph "Infrastructure Layer"
        Requirements["Python Dependencies<br/>(requirements.txt)"]:::infra
        DockerDir["Docker Packaging<br/>(docker/)"]:::infra
        Dockerfile["Dockerfile"]:::infra
        Compose["docker-compose.yaml"]:::infra
        Makefile["Makefile"]:::infra
        EB["AWS Elastic Beanstalk"]:::infra
        README["Architecture Docs<br/>(README.md)"]:::docs
    end

    %% Main interaction flow
    User -->|"search / view / chat"| UI
    UI -->|"dispatch query"| Search
    UI -->|"render product page"| View
    UI -->|"handle chat input"| Chat

    %% Search flow
    Search -->|"build filters + semantic query"| Pinecone
    Pinecone -->|"top-k matches"| Cohere
    Cohere -->|"reranked products / passages"| Search
    Search -->|"ranked results"| UI

    %% Product view flow
    View -->|"load matching specs + reviews"| Pinecone
    Pinecone -->|"evidence snippets"| View
    View -->|"display product details"| UI

    %% Chat flow
    Chat -->|"retrieve relevant context"| Pinecone
    Pinecone -->|"candidate passages"| Cohere
    Cohere -->|"refined context"| Prompt
    Prompt -->|"LLM prompt"| OpenAIChat
    OpenAIChat -->|"natural-language answer"| Chat
    Chat -->|"chat response"| UI

    %% Offline indexing flow
    Corpus -.->|"source text"| Embeddings
    Embeddings -.->|"create vectors"| OpenAIEmb
    OpenAIEmb -.->|"embedding vectors"| Pinecone

    %% Config / supporting files
    Requirements -.->|"install deps"| UI
    Config -.->|"runtime settings"| UI
    Assets -.->|"images / thumbnails"| UI
    README -.->|"documents architecture"| UI

    %% Deployment flow
    Dockerfile -.->|"build image"| DockerDir
    Compose -.->|"local orchestration"| DockerDir
    Makefile -.->|"build / run commands"| DockerDir
    DockerDir -->|"container package"| EB
    Requirements -->|"Python packages"| DockerDir

    %% Click events
    click UI "https://github.com/sfuller14/semantic-consensus/blob/master/main.py"
    click Search "https://github.com/sfuller14/semantic-consensus/blob/master/main.py"
    click View "https://github.com/sfuller14/semantic-consensus/blob/master/main.py"
    click Chat "https://github.com/sfuller14/semantic-consensus/blob/master/main.py"
    click Config "https://github.com/sfuller14/semantic-consensus/blob/master/.streamlit/config.toml"
    click Requirements "https://github.com/sfuller14/semantic-consensus/blob/master/requirements.txt"
    click DockerDir "https://github.com/sfuller14/semantic-consensus/tree/master/docker/"
    click Dockerfile "https://github.com/sfuller14/semantic-consensus/tree/master/docker/Dockerfile"
    click Compose "https://github.com/sfuller14/semantic-consensus/blob/master/docker/docker-compose.yaml"
    click Makefile "https://github.com/sfuller14/semantic-consensus/tree/master/docker/Makefile"
    click README "https://github.com/sfuller14/semantic-consensus/blob/master/README.md"
    click Assets "https://github.com/sfuller14/semantic-consensus/tree/master/thumbnails"

    %% Styles
    classDef user fill:#dbeafe,stroke:#2563eb,stroke-width:2px,color:#0f172a;
    classDef frontend fill:#bfdbfe,stroke:#1d4ed8,stroke-width:2px,color:#0f172a;
    classDef logic fill:#bbf7d0,stroke:#16a34a,stroke-width:2px,color:#052e16;
    classDef external fill:#fed7aa,stroke:#ea580c,stroke-width:2px,color:#431407;
    classDef storage fill:#e9d5ff,stroke:#7c3aed,stroke-width:2px,color:#2e1065;
    classDef data fill:#f5d0fe,stroke:#a21caf,stroke-width:2px,color:#4a044e;
    classDef pipeline fill:#fecaca,stroke:#dc2626,stroke-width:2px,color:#450a0a;
    classDef infra fill:#e5e7eb,stroke:#6b7280,stroke-width:2px,color:#111827;
    classDef docs fill:#f3f4f6,stroke:#9ca3af,stroke-width:2px,color:#111827;
    classDef config fill:#c7d2fe,stroke:#4f46e5,stroke-width:2px,color:#111827;
    classDef asset fill:#bae6fd,stroke:#0284c7,stroke-width:2px,color:#0c4a6e;
```
