# svetoslavseo/google-trends-automation architecture

> Architecture diagram of the GitHub repository svetoslavseo/google-trends-automation, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/svetoslavseo/google-trends-automation
- Repository: https://github.com/svetoslavseo/google-trends-automation

## Overview

##### Main Components
a. Data Source
- Google Trends RSS feed (UK region).
b. Processing Pipeline (Notebook/Script)
- Fetch & parse RSS via feedparser.
- Transform into Pandas DataFrame.
- Time-zone conversion (pytz).
- Optional categorization step (placeholder for external Knowledge Panel or DataForSEO API calls, e.g. via requests or OpenAI).
c. Storage Backend
- Google Sheets (via gspread + oauth2client).
d. Users/Operators
- Principal interacts via Google Colab or local Jupyter environment.

##### Architectural Patterns & Principles
- Pipeline pattern: sequential fetch → transform → load (ETL).
- Idempotent appends: checks to avoid duplicate rows in sheet.
- Modular placeholders: categorize_with_knowledge_graph() can be replaced/extended.
- Separation of concerns:
- Data retrieval (feedparser)
- Business logic/transformation (pandas, pytz)
- Persistence (gspread/oauth2client)
- External API integration (requests, openai)

##### Technologies & Frameworks
- feedparser, pandas, pytz
- requests (for external APIs)
- openai or DataForSEO placeholder
- gspread + oauth2client for Google Sheets
- Hosted/edited in Google Colab

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Notebook Runtime (Colab/Jupyter)"
        Fetch["Fetch & parse RSS Feed"]:::processing
        Transform["Transform to DataFrame & TZ conversion"]:::processing
        Categorize["Optional Categorization Placeholder"]:::external
        Load["Load & Append to Google Sheets"]:::storage
    end

    GoogleTrendsRSS["Google Trends RSS (UK)"]:::data
    ExtAPI["Knowledge Panel / DataForSEO API"]:::external
    GSheets[(Google Sheets)]:::storage
    Operators(["User/Operator (Colab)"]):::user
    Creds["Service Account JSON"]:::credentials

    GoogleTrendsRSS --> Fetch
    Fetch --> Transform
    Transform --> Categorize
    Categorize --> Transform
    Transform --> Load
    Load --> GSheets
    Creds --> Load
    Operators --> Fetch

    click Fetch "https://github.com/svetoslavseo/google-trends-automation/blob/main/Google_Trends_+_Knowledge_panel_API_+_DataForSEO_DEV.ipynb"
    click Transform "https://github.com/svetoslavseo/google-trends-automation/blob/main/Google_Trends_+_Knowledge_panel_API_+_DataForSEO_DEV.ipynb"
    click Categorize "https://github.com/svetoslavseo/google-trends-automation/blob/main/Google_Trends_+_Knowledge_panel_API_+_DataForSEO_DEV.ipynb"
    click Load "https://github.com/svetoslavseo/google-trends-automation/blob/main/Google_Trends_+_Knowledge_panel_API_+_DataForSEO_DEV.ipynb"
    click Operators "https://github.com/svetoslavseo/google-trends-automation/blob/main/README.md"

    classDef data fill:#ADD8E6,stroke:#000
    classDef processing fill:#90EE90,stroke:#000
    classDef external fill:#FFA500,stroke:#000
    classDef storage fill:#FFFF00,stroke:#000
    classDef user fill:#DDA0DD,stroke:#000
    classDef credentials fill:#D3D3D3,stroke:#000
```
