# qabot-zh/robotics-arxiv-daily architecture

> Architecture diagram of the GitHub repository qabot-zh/robotics-arxiv-daily, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/qabot-zh/robotics-arxiv-daily
- Repository: https://github.com/qabot-zh/robotics-arxiv-daily

## Overview

##### Main components
- **Source Repository (GitHub)**
- Stores all code, configuration, documentation, and assets
- Hosts GitHub Actions workflows
– .github/workflows/cv-arxiv-daily.yml (daily schedule)
– .github/workflows/update_paper_links.yml (link maintenance)
- **Configuration (config.yaml)**
- Defines feed categories, output settings, and templates
- **Processing Script (daily_arxiv.py)**
- Orchestrates end-to-end data ingestion, parsing, synthesis, and file generation
- Modules inside:
– **Fetcher**: queries arXiv API for new papers
– **Parser**: extracts title, authors, abstract, categories
– **Renderer**: applies templates to produce Markdown (README.md) and JSON (docs/*.json)
- **Documentation & Site Source (docs/)**
- Jekyll-style site:
– docs/_config.yml
– docs/index.md, docs/wechat.md
– docs/cv-arxiv-daily.json, docs/cv-arxiv-daily-web.json, docs/cv-arxiv-daily-wechat.json
- Serves as both human-readable docs and machine-readable feeds
- **Assets (assets/)**
- Images, CSS, scripts used by the docs site
- **Outputs**
- **README.md**: daily paper list displayed on GitHub
- **docs/**/*.json**: JSON feeds for web and WeChat consumption
- **docs/**/*.html**: generated site pages when Jekyll builds
- **Hosting (GitHub Pages or equivalent)**
- Publishes docs/ folder as a static site

##### Interactions and data flow
1. **Schedule Trigger**: GitHub Actions “cv-arxiv-daily” fires (cron daily)
2. **Checkout & Install**: Workflow checks out repo, installs Python dependencies (requirements.txt)
3. **Run Script**: daily_arxiv.py reads config.yaml, queries arXiv API
4. **Data Generation**: Script writes/overwrites:
- README.md
- docs/cv-arxiv-daily.json, docs/cv-arxiv-daily-web.json, docs/cv-arxiv-daily-wechat.json
5. **Commit & Push**: Changes are automatically committed and pushed back to GitHub
6. **Site Build**: GitHub Pages (or Jekyll in CI) picks up docs/ and assets/ to rebuild static site
7. **Site Deployment**: Updated site goes live; updated README.md is visible on the repo front page

##### Architectural patterns and design principles
- **Separation of Concerns**:
- Fetch/parse logic (daily_arxiv.py) is distinct from site/content templates (docs/) and configuration (config.yaml).
- **Configuration-Driven**:
- All categories, output formats, and templates are in config.yaml, making the system extensible.
- **Event-Driven Automation**:
- Cron-style GitHub workflow triggers data refresh, ensuring a “live” daily feed.
- **Static-Site Generation**:
- Jekyll (docs/_config.yml) transforms markdown and JSON into HTML pages—no dynamic backend needed.
- **Continuous Deployment**:
- Single GH Action pipeline for both data generation and site publish.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "GitHub Platform"
        direction TB
        Repo["Source Repository (GitHub)"]:::github
        subgraph "Workflows"
            direction TB
            GH1["cv-arxiv-daily.yml"]:::github
            GH2["update_paper_links.yml"]:::github
        end
        Hosting["GitHub Pages"]:::hosting
    end

    External["arXiv API"]:::external

    Config["config.yaml"]:::config

    subgraph "Script: daily_arxiv.py"
        direction TB
        Script["daily_arxiv.py"]:::script
        Deps["requirements.txt"]:::script
        subgraph "Modules"
            direction TB
            Fetcher["Fetcher"]:::script
            Parser["Parser"]:::script
            Renderer["Renderer"]:::script
        end
    end

    Docs["docs/"]:::docs
    Assets["assets/"]:::assets

    Outputs1["README.md"]:::output
    subgraph "JSON Feeds"
        direction TB
        J1["cv-arxiv-daily.json"]:::output
        J2["cv-arxiv-daily-web.json"]:::output
        J3["cv-arxiv-daily-wechat.json"]:::output
    end
    HTML["HTML Site Pages"]:::output

    GH1 -->|trigger daily| Script
    Script -->|reads config| Config
    Script -->|queries API| External
    Script -->|uses deps| Deps
    Script -->|process| Fetcher
    Script -->|process| Parser
    Script -->|render| Renderer
    Script -->|generate| Outputs1
    Script -->|generate| J1
    Script -->|generate| J2
    Script -->|generate| J3
    Script -->|update docs| Docs
    Script -->|commit & push| Repo

    Docs -->|build site| HTML
    Docs -->|publish| Hosting
    Assets -->|assets for site| Hosting
    Outputs1 -->|visible in repo| Repo

    classDef github fill:#cce5ff,stroke:#004085
    classDef script fill:#d4edda,stroke:#155724
    classDef config fill:#fff3cd,stroke:#856404
    classDef external fill:#f8d7da,stroke:#721c24
    classDef docs fill:#e2e3e5,stroke:#6c757d
    classDef assets fill:#e2e3e5,stroke:#6c757d
    classDef output fill:#d6d8db,stroke:#1b1e21
    classDef hosting fill:#f3e5f5,stroke:#4a148c

    click Repo "https://github.com/qabot-zh/robotics-arxiv-daily/tree/main/.github/"
    click GH1 "https://github.com/qabot-zh/robotics-arxiv-daily/blob/main/.github/workflows/cv-arxiv-daily.yml"
    click GH2 "https://github.com/qabot-zh/robotics-arxiv-daily/blob/main/.github/workflows/update_paper_links.yml"
    click Config "https://github.com/qabot-zh/robotics-arxiv-daily/blob/main/config.yaml"
    click Script "https://github.com/qabot-zh/robotics-arxiv-daily/blob/main/daily_arxiv.py"
    click Deps "https://github.com/qabot-zh/robotics-arxiv-daily/blob/main/requirements.txt"
    click Docs "https://github.com/qabot-zh/robotics-arxiv-daily/tree/main/docs/"
    click Assets "https://github.com/qabot-zh/robotics-arxiv-daily/tree/main/assets/"
    click Outputs1 "https://github.com/qabot-zh/robotics-arxiv-daily/blob/main/README.md"
    click J1 "https://github.com/qabot-zh/robotics-arxiv-daily/blob/main/docs/cv-arxiv-daily.json"
    click J2 "https://github.com/qabot-zh/robotics-arxiv-daily/blob/main/docs/cv-arxiv-daily-web.json"
    click J3 "https://github.com/qabot-zh/robotics-arxiv-daily/blob/main/docs/cv-arxiv-daily-wechat.json"
```
