# jaygupta8/openai architecture

> Architecture diagram of the GitHub repository jaygupta8/openai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jaygupta8/openai
- Repository: https://github.com/jaygupta8/openai

## Overview

##### Project type and purpose
- This is essentially a data-processing/QA pipeline implemented in a Jupyter notebook.
- Purpose: load a PDF on Vedic Astrology, break it into chunks, embed each chunk with OpenAI, index embeddings in Pinecone, then answer user questions via semantic search.

##### Main components
a. Notebook Orchestrator
- Represents the vedic_astrology.ipynb itself, which sequences all steps.
b. Data Source
- PDF Book (e.g., a file in storage).
c. Text Splitter
- RecursiveCharacterTextSplitter from LangChain.
d. Embedding Generator
- OpenAI embeddings API.
e. Vector Store
- Pinecone index (external vector database).
f. Question Answering Chain
- OpenAI completion/chat API wrapped by LangChain ConversationChain.
g. User Interface
- Jupyter Notebook cell prompts (or potential upstream application).

##### Relationships and data flow
- PDF Book → Notebook Orchestrator: load raw PDF bytes.
- Notebook Orchestrator → Text Splitter: chunk text into Document objects.
- Text Splitter → Embedding Generator: send each chunk to OpenAI embeddings API.
- Embedding Generator → Vector Store: upsert vectors into Pinecone index.
- User Question → Notebook Orchestrator → Pinecone similarity search → retrieves top-k chunks.
- Retrieved chunks + question → QA Chain → OpenAI chat/completion API → answer returned back to user.

##### Architectural patterns and principles
- Linear data pipeline: each step feeds the next in sequence.
- External service integration: embedding and QA sub-systems are decoupled via vector store.
- Modularity: each logical step (loading, splitting, embedding, indexing, querying) is a distinct component and can be replaced or scaled independently.

##### Relevant technologies & libraries
- LangChain (PDF loader, splitter, chains)
- OpenAI API (embeddings, chat/completion)
- Pinecone (vector database)
- Jupyter Notebook (orchestrator & UI)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Nodes
    ORC["Notebook Orchestrator"]:::purple
    PDF["PDF Book"]:::blue
    A["Load PDF"]:::green
    B["Text Splitter\n(RecursiveCharacterTextSplitter)"]:::green
    C["Embedding Generator\n(OpenAI Embeddings)"]:::green
    D["Vector Upsert\n(Pinecone)"]:::green
    E["Similarity Search\n(Pinecone Query)"]:::green
    F["QA Chain\n(OpenAI Chat/Completion)"]:::green
    User["User Interface\n(Jupyter Prompt)"]:::purple
    OpenAI["OpenAI API"]:::orange
    Pinecone["Pinecone Vector Store"]:::orange
    Doc["Project Documentation"]:::gray

    %% Flow
    PDF -->|"raw PDF bytes"| ORC
    ORC --> A
    A -->|"text content"| B
    B -->|"text chunks"| C
    C -->|"embeddings"| D
    D --> Pinecone
    User --> E
    E --> Pinecone
    Pinecone -->|top-k chunks| E
    E --> F
    F --> OpenAI
    OpenAI --> F
    F --> User

    %% Click events
    click ORC "https://github.com/jaygupta8/openai/blob/main/vedic_astrology.ipynb"
    click Doc "https://github.com/jaygupta8/openai/blob/main/readme.md"

    %% Styles
    classDef blue fill:#ADD8E6,stroke:#000
    classDef green fill:#90EE90,stroke:#000
    classDef orange fill:#FFA500,stroke:#000
    classDef purple fill:#DDA0DD,stroke:#000
    classDef gray fill:#D3D3D3,stroke:#000
```
