# sanuvishwakarma/agenticai_url_summarizer architecture

> Architecture diagram of the GitHub repository sanuvishwakarma/agenticai_url_summarizer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sanuvishwakarma/agenticai_url_summarizer
- Repository: https://github.com/sanuvishwakarma/agenticai_url_summarizer

## Overview

##### Project Type and Purpose
- This is a full-stack AI Pipeline application with:
- A lightweight frontend (Streamlit).
- A Python backend orchestrating a LangGraph “agent” pipeline.
- Multiple processing “nodes” (fetch, extract, summarize).
- A vector database for embeddings (ChromaDB).
- External AI services (open-source LLMs & embedding models).

##### Main Components
- Frontend: Streamlit UI (app.py)
- Orchestrator: LangGraph agent runner (main.py)
- Processing Nodes:
- fetch_url_content
- extract_content
- summarize_content
- Vector Store Abstraction: vector_store.py
- Database: ChromaDB (chroma_db folder, sqlite + binary files)
- External Services:
- HTTP/Web (requests)
- BeautifulSoup (HTML parsing)
- HuggingFace Llama-1.1B (text generation)
- Sentence-Transformers (embedding model)

##### Component Relationships & Data Flow
1. User enters a URL in Streamlit → “Summarize” button triggers:
2. app.py calls main.py to run the LangGraph pipeline.
3. main.py initializes agent and passes URL to the first node.
4. fetch_url_content node:
- Requests URL
- Downloads raw HTML
↓
5. extract_content node:
- Parses HTML with BeautifulSoup
- Extracts clean text
↓
6. summarize_content node:
- Sends text prompt to Llama-1.1B
- Receives summary
↓
7. vector_store.py:
- Embeds summary (Sentence-Transformers)
- Stores embedding + metadata in ChromaDB
8. main.py collects summary → returns to app.py
9. Streamlit displays the summary (and optionally retrieval results)

##### Architectural Patterns & Design Principles
- Modular Pipeline: Each node is a single responsibility.
- Separation of Concerns: UI vs. orchestration vs. processing vs. storage.
- Extensibility: New nodes can be dropped into the nodes/ folder.
- Local stateful storage for embeddings (ChromaDB) decouples compute from persistence.
- Use of open-source models (HuggingFace, Sentence-Transformers) and standard libraries.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Top Layer: User and Frontend
    User["User"]:::external
    User -->|"1. Enter URL"| UI["Streamlit UI<br>(app.py)"]:::frontend

    %% Backend Orchestrator
    UI -->|"2. Run pipeline"| Orchestrator["LangGraph Pipeline<br>(main.py)"]:::orchestrator

    %% Pipeline Subgraph
    subgraph "Pipeline"
        direction TB
        InitNodes["Nodes Package Init<br>(__init__.py)"]:::node
        FENode("Fetch URL Content Node"):::node
        EXNode("Extract Content Node"):::node
        SUMNode("Summarize Content Node"):::node

        InitNodes -->|"initialization"| FENode
        FENode -->|"3. raw HTML"| EXNode
        EXNode -->|"4. plain text"| SUMNode
    end

    Orchestrator -->|"init & pass URL"| InitNodes
    SUMNode -->|"5. summary"| VectorStore["Vector Store Module<br>(vector_store.py)"]:::orchestrator
    VectorStore -->|"6. embeddings + metadata"| DB["ChromaDB<br>(SQLite + files)"]:::db

    %% External Services
    subgraph "External Services"
        direction TB
        HTTP["Requests & BeautifulSoup"]:::external
        LLM["HuggingFace Llama-1.1B"]:::external
        EMB["Sentence-Transformers"]:::external

        FENode -->|"fetch HTTP"| HTTP
        SUMNode -->|"invoke model"| LLM
        VectorStore -->|"embed text"| EMB
    end

    %% Return Flow
    DB -->|"7. optional retrieval"| Orchestrator
    Orchestrator -->|"8. return summary"| UI
    UI -->|"9. display summary"| User

    %% Requirements File
    Requirements["requirements.txt"]:::infra

    %% Click Events
    click UI "https://github.com/sanuvishwakarma/agenticai_url_summarizer/blob/main/app.py"
    click Orchestrator "https://github.com/sanuvishwakarma/agenticai_url_summarizer/blob/main/main.py"
    click FENode "https://github.com/sanuvishwakarma/agenticai_url_summarizer/blob/main/nodes/fetch_url_content.py"
    click EXNode "https://github.com/sanuvishwakarma/agenticai_url_summarizer/blob/main/nodes/extract_content.py"
    click SUMNode "https://github.com/sanuvishwakarma/agenticai_url_summarizer/blob/main/nodes/summarize_content.py"
    click InitNodes "https://github.com/sanuvishwakarma/agenticai_url_summarizer/blob/main/nodes/__init__.py"
    click VectorStore "https://github.com/sanuvishwakarma/agenticai_url_summarizer/blob/main/vector_store.py"
    click DB "https://github.com/sanuvishwakarma/agenticai_url_summarizer/tree/main/chroma_db/"
    click Requirements "https://github.com/sanuvishwakarma/agenticai_url_summarizer/blob/main/requirements.txt"

    %% Styles
    classDef frontend fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef orchestrator fill:#90EE90,stroke:#333,stroke-width:1px
    classDef node fill:#FFFACD,stroke:#333,stroke-width:1px
    classDef external fill:#DDA0DD,stroke:#333,stroke-width:1px
    classDef db fill:#FFA500,stroke:#333,stroke-width:1px
    classDef infra fill:#D3D3D3,stroke:#333,stroke-width:1px
```
