# abir0/amazon-products-analytics architecture

> Architecture diagram of the GitHub repository abir0/amazon-products-analytics, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/abir0/amazon-products-analytics
- Repository: https://github.com/abir0/amazon-products-analytics

## Overview

##### Main Components

A. Client Layer:
- Streamlit Frontend (app/frontend/app.py)
- External API Consumers

B. Application Layer:
- FastAPI Backend (app/api/)
- Product Router (products.py)
- RAG Router (rag.py)
- Scheduler Router (scheduler.py)

C. Service Layer:
- Scraping Service (app/scraper/)
- RAG Service (app/rag/)
- Database Service (app/database.py)

D. Data Layer:
- PostgreSQL Database
- Weaviate Vector Database
- S3 Storage (for images)

E. External Services:
- Amazon Website (data source)
- OpenAI API
- Chrome WebDriver

**[Client Layer] → [Application Layer] → [Service Layer] → [Data Layer]**
- ↑               ↑                      ↑                ↑
- ┴──────────────────────┴────────────[External Services]

##### Detailed Component Breakdown

- Rectangles: Main services
- Hexagons: External services

##### Key Flows to Highlight

A. Scraping Flow:
**[Scheduler] → [Amazon Scraper] → [Amazon Website]**
- ↓
**[PostgreSQL] ← [Data Processing]**

B. RAG Flow:
**[User Query] → [RAG Router] → [Chatbot] → [Weaviate]**
- ↓             ↓
- [PostgreSQL]    [OpenAI API]

C. Product API Flow:
**[Client] → [Product Router] → [Database Service] → [PostgreSQL]**

##### Important Details

A. Infrastructure Components:
- Docker containers (show containerized services)
- AWS deployment architecture
- Load balancers (if applicable)

B. Data Flow Specifications:
- API endpoints (/products, /rag, /scheduler)
- Database connections
- Async operations

C. Security Layer:
- API authentication
- Database access
- External API keys

##### AWS-Specific Components

Since the README mentions AWS deployment, include:
- EC2 instances for API
- ECS for container orchestration
- RDS for PostgreSQL
- SageMaker for AI models
- S3 for storage
- CloudWatch for monitoring

##### AWS deployment architecture

Remember to include annotations explaining key architectural decisions and any potential bottlenecks or scaling considerations.

This system design should reflect the modern microservices architecture while showing the integration of AI capabilities through the RAG system, making it a comprehensive representation of this analytics platform.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Client
        FE["Streamlit Frontend"]:::frontend
        API_C["External API Consumers"]:::client
    end

    subgraph API_Layer
        API["FastAPI Backend"]:::api
        PR["Product Router"]:::router
        RR["RAG Router"]:::router
        SR["Scheduler Router"]:::router
    end

    subgraph Service_Layer
        SS["Scraping Service"]:::service
        RS["RAG Service"]:::service
        DS["Database Service"]:::service
        CB["Chatbot"]:::service
        RP["RAG Pipeline"]:::service
    end

    subgraph Data_Layer
        PG[(PostgreSQL)]:::database
        WV[(Weaviate)]:::database
        S3[(S3 Storage)]:::storage
    end

    subgraph External_Services
        AW["Amazon Website"]:::external
        OAI["OpenAI API"]:::external
        CWD["Chrome WebDriver"]:::external
    end

    %% Client Layer Connections
    FE --> API
    API_C --> API

    %% API Layer Connections
    API --> PR
    API --> RR
    API --> SR

    %% Service Layer Connections
    PR --> DS
    RR --> RS
    SR --> SS
    RS --> CB
    RS --> RP
    CB --> OAI
    SS --> CWD
    CWD --> AW

    %% Data Layer Connections
    DS --> PG
    RS --> WV
    SS --> S3
    CB --> WV
    RP --> PG

    %% Click Events
    click FE "https://github.com/abir0/amazon-products-analytics/blob/main/app/frontend/app.py"
    click PR "https://github.com/abir0/amazon-products-analytics/blob/main/app/api/routers/products.py"
    click RR "https://github.com/abir0/amazon-products-analytics/blob/main/app/api/routers/rag.py"
    click SR "https://github.com/abir0/amazon-products-analytics/blob/main/app/api/routers/scheduler.py"
    click DS "https://github.com/abir0/amazon-products-analytics/blob/main/app/database.py"
    click RS "https://github.com/abir0/amazon-products-analytics/tree/main/app/rag/"
    click CB "https://github.com/abir0/amazon-products-analytics/blob/main/app/rag/chatbot.py"
    click RP "https://github.com/abir0/amazon-products-analytics/blob/main/app/rag/pipeline.py"
    click SS "https://github.com/abir0/amazon-products-analytics/tree/main/app/scraper/"

    %% Styling
    classDef frontend fill:#6495ED
    classDef client fill:#87CEEB
    classDef api fill:#4169E1
    classDef router fill:#1E90FF
    classDef service fill:#0000CD
    classDef database fill:#32CD32
    classDef storage fill:#228B22
    classDef external fill:#FFA500

    %% Legend
    subgraph Legend
        L1["Frontend Components"]:::frontend
        L2["Client Components"]:::client
        L3["API Services"]:::api
        L4["Routers"]:::router
        L5["Core Services"]:::service
        L6["Databases"]:::database
        L7["Storage"]:::storage
        L8["External Services"]:::external
    end
```
