How qabot-zh/robotics-arxiv-daily works
Main components
• Source Repository (GitHub)
• Stores all code, configuration, documentation, and assets
• Hosts GitHub Actions workflows
– .github/workflows/cv-arxiv-daily.yml (daily schedule)
– .github/workflows/update_paper_links.yml (link maintenance)
• Configuration (config.yaml)
• Defines feed categories, output settings, and templates
• Processing Script (daily_arxiv.py)
• Orchestrates end-to-end data ingestion, parsing, synthesis, and file generation
• Modules inside:
– Fetcher: queries arXiv API for new papers
– Parser: extracts title, authors, abstract, categories
– Renderer: applies templates to produce Markdown (README.md) and JSON (docs/*.json)
• Documentation & Site Source (docs/)
• Jekyll-style site:
– docs/_config.yml
– docs/index.md, docs/wechat.md
– docs/cv-arxiv-daily.json, docs/cv-arxiv-daily-web.json, docs/cv-arxiv-daily-wechat.json
• Serves as both human-readable docs and machine-readable feeds
• Assets (assets/)
• Images, CSS, scripts used by the docs site
• Outputs
• README.md: daily paper list displayed on GitHub
• docs//*.json**: JSON feeds for web and WeChat consumption
• docs//*.html**: generated site pages when Jekyll builds
• Hosting (GitHub Pages or equivalent)
• Publishes docs/ folder as a static site
Interactions and data flow
1. Schedule Trigger: GitHub Actions “cv-arxiv-daily” fires (cron daily)
2. Checkout & Install: Workflow checks out repo, installs Python dependencies (requirements.txt)
3. Run Script: daily_arxiv.py reads config.yaml, queries arXiv API
4. Data Generation: Script writes/overwrites:
• README.md
• docs/cv-arxiv-daily.json, docs/cv-arxiv-daily-web.json, docs/cv-arxiv-daily-wechat.json
5. Commit & Push: Changes are automatically committed and pushed back to GitHub
6. Site Build: GitHub Pages (or Jekyll in CI) picks up docs/ and assets/ to rebuild static site
7. Site Deployment: Updated site goes live; updated README.md is visible on the repo front page
Architectural patterns and design principles
• Separation of Concerns:
• Fetch/parse logic (daily_arxiv.py) is distinct from site/content templates (docs/) and configuration (config.yaml).
• Configuration-Driven:
• All categories, output formats, and templates are in config.yaml, making the system extensible.
• Event-Driven Automation:
• Cron-style GitHub workflow triggers data refresh, ensuring a “live” daily feed.
• Static-Site Generation:
• Jekyll (docs/_config.yml) transforms markdown and JSON into HTML pages—no dynamic backend needed.
• Continuous Deployment:
• Single GH Action pipeline for both data generation and site publish.