# weaviate/bookrecs architecture

> Architecture diagram of the GitHub repository weaviate/bookrecs, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/weaviate/bookrecs
- Repository: https://github.com/weaviate/bookrecs

## Overview

##### Project & Purpose Overview
- This project is a full-stack web application designed to provide book recommendations based on user inputs including genres and book titles.
- It leverages vector embeddings stored in a Weaviate database that were created by processing a Kaggle dataset of 7000 books.
- The recommendation engine relies on large language models (LLMs) – using either OpenAI or Ollama – and optionally integrates Cohere for additional explanation features.
- The frontend is built with Next.js and styled with TailwindCSS, while the backend and data pipeline use NodeJS and Python respectively.

##### Key Components from the File Structure
- Frontend Application:
- Pages and API endpoints: Look at the “pages” directory (e.g., pages/index.tsx and pages/api/recommendations.ts) where the web interface and backend API are implemented.
- UI components: Under “components/ui”, several reusable components (buttons, cards, inputs, etc.) help build the application’s user interface.
- Data Pipeline:
- Located in the “data-pipeline” directory, separated into subfolders for “ollama” and “openai” approaches. This pipeline includes Python scripts for:
▪ Creating a book collection in Weaviate.
▪ Populating the vector database.
▪ Running semantic and generative searches.
- External Integrations:
- Weaviate is used as the vector database, with configurations provided in (for example) .env and vercel.json files.
- LLM integrations (using OpenAI’s API or a local Ollama instance) are invoked for generating embeddings and recommendations.
- An optional integration with Cohere for generative search explanations is configurable via environmental variables.
- Supporting Files & Configurations:
- Various config files such as next.config.js, tsconfig.json, and tailwind.config.js, along with the .env.example, set up the development and production environments.
- “blog-resources” and “components.json” demonstrate additional resources or examples that might relate to the project’s extensibility and documentation.

- Web Frontend (Next.js App):
- UI Pages and Components (pages, components/ui)
- Contains client-side interactions and loads styles from TailwindCSS.
- Backend/API Layer:
- API Endpoint (pages/api/recommendations.ts) handling user requests.
- Responsible for interfacing with the Weaviate vector database.
- Data Pipeline:
- Python Pipeline Scripts (within data-pipeline/ollama and data-pipeline/openai)
- Handles creation, population, and updating of vector embeddings.
- Vector Database:
- Weaviate instance (either local or cloud-hosted) where vector embeddings are stored.
- External LLM & Search Services:
- OpenAI / Ollama for embedding generation and semantic/generative search.
- Optional post-processing with Cohere for explanation generation.

b. Relationships and Data Flow:
- How the Backend/API layer queries the Weaviate vector database to fetch recommendations.
- Separate processes for the Data Pipeline, showing a one-time or scheduled ingestion that populates the Weaviate database.
- External integrations with LLMs – show bi-directional interactions where the Data Pipeline invokes embedding services (OpenAI/Ollama) and, where enabled, uses Cohere for additional outputs.

c. Architectural Patterns & Design Principles:
- Separation of Concerns: Clearly separate the presentation layer (Next.js UI), the business logic/API layer, and the data ingestion/processing components.
- Modular Design: Emphasize that the data pipeline, web application, and external service integrations are decoupled and can be modified independently.
- Scalability: Note considerations for scaling the Weaviate instance and API endpoints independently when load increases.

## Diagram (Mermaid)

```mermaid
flowchart TD
    U["User (Browser)"]:::external

    subgraph "Web Frontend"
        A["Web Pages (pages/index.tsx)"]:::frontend
        B["API Endpoint (/recommendations)"]:::frontend
        C["UI Components"]:::frontend
    end

    U -->|"RequestPage"| A
    A -->|"SubmitRec"| B

    subgraph "Vector Database"
        F(("Weaviate Vector DB")):::database
    end

    B -->|"QueryVectors"| F

    subgraph "Data Pipeline"
        D["Ollama Pipeline"]:::pipeline
        E["OpenAI Pipeline"]:::pipeline
    end

    D -.->|"UpdateEmbeddings"| F
    E -.->|"UpdateEmbeddings"| F
    D -.->|"GenerateEmbeddings"| G["LLM Service (OpenAI/Ollama)"]:::external
    E -.->|"GenerateEmbeddings"| G
    E -.->|"OptionalExplanation"| H["Cohere Explanation"]:::external

    subgraph "Configuration & Documentation"
        I1[".env.example"]:::config
        I2["vercel.json"]:::config
        J1["next.config.js"]:::config
        J2["tsconfig.json"]:::config
        J3["tailwind.config.js"]:::config
        K1["blog-resources"]:::config
        K2["components.json"]:::config
    end

    %% Click Events
    click A "https://github.com/weaviate/bookrecs/blob/main/pages/index.tsx"
    click B "https://github.com/weaviate/bookrecs/blob/main/pages/api/recommendations.ts"
    click C "https://github.com/weaviate/bookrecs/tree/main/components/ui"
    click D "https://github.com/weaviate/bookrecs/tree/main/data-pipeline/ollama"
    click E "https://github.com/weaviate/bookrecs/tree/main/data-pipeline/openai"
    click I1 "https://github.com/weaviate/bookrecs/blob/main/.env.example"
    click I2 "https://github.com/weaviate/bookrecs/blob/main/vercel.json"
    click J1 "https://github.com/weaviate/bookrecs/blob/main/next.config.js"
    click J2 "https://github.com/weaviate/bookrecs/blob/main/tsconfig.json"
    click J3 "https://github.com/weaviate/bookrecs/blob/main/tailwind.config.js"
    click K1 "https://github.com/weaviate/bookrecs/tree/main/blog-resources"
    click K2 "https://github.com/weaviate/bookrecs/blob/main/components.json"

    %% Styling Classes
    classDef frontend fill:#bbdefb,stroke:#0d47a1,stroke-width:2px;
    classDef pipeline fill:#fff9c4,stroke:#f9a825,stroke-width:2px;
    classDef database fill:#ffe082,stroke:#ff6f00,stroke-width:2px;
    classDef external fill:#ffcdd2,stroke:#b71c1c,stroke-width:2px;
    classDef config fill:#e1bee7,stroke:#6a1b9a,stroke-width:2px;
    classDef user fill:#d1c4e9,stroke:#512da8,stroke-width:2px;
    
    class U user
```
