# dmarx/papers-feed-src architecture

> Architecture diagram of the GitHub repository dmarx/papers-feed-src, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dmarx/papers-feed-src
- Repository: https://github.com/dmarx/papers-feed-src

## Overview

##### Client-Side Components
- Browser Extension (Chrome/Firefox)
- Background.js for URL monitoring
- Popup interface for user interactions
- Options page for PAT configuration

##### GitHub Infrastructure
- Papers-Feed Repository (Data Lake)
- Issues system (acting as a database)
- GitHub Pages (hosting frontend)
- Papers-Feed-Src Repository (Source Code)
- Core processing logic
- Frontend templates

##### Processing Pipeline
- GitHub Actions Workflows
- Issue processing workflow
- Paper downloading workflow
- Frontend deployment workflow

##### Component Representation
- Use rectangles for main components
- Use hexagons for processing components
- Use clouds for external services (like arXiv)

##### Flow Directions
1. Browser Extension → GitHub Issues (paper visit events)
2. GitHub Actions → arXiv API (paper fetching)
3. Processing Pipeline → Generated Frontend
4. Frontend → GitHub Pages (deployment)

##### Specific Processes
- URL monitoring flow
- Issue creation process
- Paper processing pipeline
- Frontend generation and deployment

Specific Implementation Details to Highlight:

##### Data Flow
- Browser Extension
- → GitHub Issues
- → GitHub Actions
- → Processed Data
- → Generated Frontend
- → GitHub Pages

##### Key Services
- arxiv_client.py for paper fetching
- paper_manager.py for data processing
- markdown_service.py for content conversion
- frontend/generate_html.py for site generation

##### Configuration Points
- GitHub PAT configuration
- Repository settings
- GitHub Actions workflows

- Component types
- Status indicators

- The use of GitHub Issues as a database
- The integration between browser extension and GitHub
- The automated processing pipeline
- The static site generation process

## Diagram (Mermaid)

```mermaid
graph TB
    %% Client-Side Components
    subgraph Browser
        Extension["Browser Extension"]
        Background["Background Monitor"]:::code
        Popup["Popup Interface"]:::code
        Options["Options Configuration"]:::code
        Manifest["Extension Config"]:::code
        
        Extension --> Background
        Extension --> Popup
        Extension --> Options
        Extension --> Manifest
    end

    %% Processing Pipeline
    subgraph Pipeline
        ArxivClient["arXiv Client"]:::code
        PaperManager["Paper Manager"]:::code
        MarkdownService["Markdown Service"]:::code
        GitHubClient["GitHub Integration"]:::code
        EventProcessor["Event Processing"]:::code
    end

    %% Frontend Generation
    subgraph Frontend
        HTMLGenerator["HTML Generator"]:::code
        Template["Frontend Template"]:::code
        HTMLGenerator --> Template
    end

    %% GitHub Workflows
    subgraph Workflows
        DownloadPipeline["Paper Download Pipeline"]:::code
        ProcessEvents["Event Processing Pipeline"]:::code
        DeployFrontend["Frontend Deployment"]:::code
        ConvertMarkdown["Markdown Conversion"]:::code
    end

    %% Utility Services
    subgraph Utils
        AssetManager["Asset Management"]:::code
        TexUtils["TeX Processing"]:::code
        PandocUtils["Pandoc Integration"]:::code
        DataModels["Data Models"]:::code
    end

    %% External Services
    ArxivAPI["arXiv API"]:::external

    %% GitHub Infrastructure
    GHIssues[("GitHub Issues")]:::storage
    GHPages["GitHub Pages"]:::hosting

    %% Connections
    Background --> GHIssues
    GHIssues --> EventProcessor
    EventProcessor --> PaperManager
    PaperManager --> ArxivClient
    ArxivClient --> ArxivAPI
    PaperManager --> MarkdownService
    MarkdownService --> HTMLGenerator
    HTMLGenerator --> GHPages

    %% Click Events
    click Background "https://github.com/dmarx/papers-feed-src/blob/main/extension/background.js"
    click Popup "https://github.com/dmarx/papers-feed-src/blob/main/extension/popup.html"
    click Options "https://github.com/dmarx/papers-feed-src/blob/main/extension/options.html"
    click Manifest "https://github.com/dmarx/papers-feed-src/blob/main/extension/manifest.json"
    click ArxivClient "https://github.com/dmarx/papers-feed-src/blob/main/src/papers_feed/arxiv_client.py"
    click PaperManager "https://github.com/dmarx/papers-feed-src/blob/main/src/papers_feed/paper_manager.py"
    click MarkdownService "https://github.com/dmarx/papers-feed-src/blob/main/src/papers_feed/markdown_service.py"
    click GitHubClient "https://github.com/dmarx/papers-feed-src/blob/main/src/papers_feed/github_client.py"
    click EventProcessor "https://github.com/dmarx/papers-feed-src/blob/main/src/papers_feed/process_events.py"
    click HTMLGenerator "https://github.com/dmarx/papers-feed-src/blob/main/src/papers_feed/frontend/generate_html.py"
    click Template "https://github.com/dmarx/papers-feed-src/blob/main/src/papers_feed/frontend/index.template.html"
    click DownloadPipeline "https://github.com/dmarx/papers-feed-src/blob/main/.github/workflows/download-papers.yml"
    click ProcessEvents "https://github.com/dmarx/papers-feed-src/blob/main/.github/workflows/process-events.yml"
    click DeployFrontend "https://github.com/dmarx/papers-feed-src/blob/main/.github/workflows/deploy_frontend.yaml"
    click ConvertMarkdown "https://github.com/dmarx/papers-feed-src/blob/main/.github/workflows/convert-markdown.yml"
    click AssetManager "https://github.com/dmarx/papers-feed-src/blob/main/src/papers_feed/asset_manager.py"
    click TexUtils "https://github.com/dmarx/papers-feed-src/blob/main/src/papers_feed/tex_utils.py"
    click PandocUtils "https://github.com/dmarx/papers-feed-src/blob/main/src/papers_feed/pandoc_utils.py"
    click DataModels "https://github.com/dmarx/papers-feed-src/blob/main/src/papers_feed/models.py"

    %% Styling
    classDef code fill:#2374ab
    classDef storage fill:#57a773
    classDef hosting fill:#484d6d
    classDef external fill:#9b6a6c
    
    %% Legend
    subgraph Legend
        Code["Code Component"]:::code
        Storage[("Storage")]:::storage
        Hosting["Hosting"]:::hosting
        External["External Service"]:::external
    end
```
