# tvhong/incremental-reading architecture

> Architecture diagram of the GitHub repository tvhong/incremental-reading, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tvhong/incremental-reading
- Repository: https://github.com/tvhong/incremental-reading

## Overview

##### Project type and purpose
- This is an Anki 2.1+ desktop plug-in (Python) that lets users import long-form content (web pages, RSS/Atom feeds, Pocket articles), read it inside Anki, highlight or extract bits into flashcards, and control their scheduling.

##### File structure
- Top level: CI configs, linter, Makefile, docs, tests.
- ir/ directory:
– main.py & manifest.json: plug-in entry point & metadata.
– settings.py: configuration API.
– importer/: content-ingestion pipeline (base_importer, html_cleaner, concrete importers: epub, web, rss, pocket, local_file).
– gui.py & view.py: Qt dialog glue + web view integration.
– web/: static HTML/CSS/JS assets (model.css, scroll.js, text.js, table_of_contents.js).
– schedule.py: hooks into Anki’s scheduling engine.
– util.py & text.py: core text-processing utilities.
– lib/: bundled third-party wrappers (feedparser, cgi).

##### README for architecture clues
- Describes four subsystems: Import, Read/Highlight/Extract, Scheduling, Quick-Keys/UI.
- Mentions integration points with Anki (menus, shortcuts, note editor).
- External dependencies: certain Python libraries, Anki’s Python API, Pocket service.

A. Host Environment (Anki Application)
B. IR Add-on “Shell” (ir/main.py + manifest.json)
C. Settings Manager (ir/settings.py ↔ Anki’s config store)
D. Importer Pipeline
– BaseImporter → Concrete Importers (Web, RSS/Atom, Pocket, EPUB, LocalFile)
– HTMLCleaner & Exception Handling
– Models (intermediate IR data structures)
E. UI Layer
– Native Qt Dialogs & Menus (ir/gui.py)
– Embedded WebView (HTML/CSS/JS under ir/web/)
– View Controller (ir/view.py)
F. Core Logic
– Text Processing (ir/text.py)
– Scheduling Logic (ir/schedule.py)
– Utility functions (ir/util.py)
G. Persistence & External Services
– Anki SQLite DB (notes, cards, scheduling data)
– External Feeds (RSS/Atom), Web pages, Pocket API
– Local files (EPUB, HTML)
H. Third-Party Libraries (feedparser, CGI)

##### Relationships & data flows
1. User triggers import via menu/shortcut → GUI Layer
2. GUI invokes Importer Pipeline → fetches content from External Services
3. Pipeline → HTMLCleaner → builds IR data models
4. Models passed back to GUI → WebView renders text with embedded JS/CSS
5. User selects/highlights/extracts text → View Controller captures events
6. View passes selections to Core Logic → ir/text.py & schedule.py decide card creation & scheduling
7. Core Logic uses Anki API → writes to Anki SQLite DB
8. Settings Manager persists any option changes

##### Architectural patterns & principles
- Plugin architecture: clear boundary between Anki host and IR code.
- MVC (roughly) in UI: Qt dialogs (View), view.py (Controller), JS/CSS (View).
- Pipeline pattern in importer: chain of responsibility for cleaning and parsing.
- Layered design: Importer → Core → Persistence, UI decoupled from data-ingestion.

##### Detail level advice
- Don’t over-complicate: break the add-on into these logical components rather than every single Python file.
- If necessary, expand the importer pipeline further to show base_importer vs. concrete_importers vs. html_cleaner.
- UI interaction flows clearly: user → GUI → core → DB → back to GUI (for scheduling display).
- Call out the embedding of ir/web assets in the WebView (HTML/CSS/JS) as a distinct subcomponent.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Services
    ExtWeb["Web Pages"]:::external
    ExtRSS["RSS/Atom Feeds"]:::external
    ExtPocket["Pocket API"]:::external
    LocalFS["Local Files"]:::external

    subgraph "Anki Application"
        subgraph "IR Add-on Shell"
            Shell["IR Add-on Shell\n(main.py, manifest.json)"]:::shell
            Settings["Settings Manager\n(settings.py)"]:::core

            subgraph "Importer Pipeline"
                Orchestrator["Importer Orchestrator\n(importer.py)"]:::importer
                BaseImporter["BaseImporter\n(base_importer.py)"]:::importer
                HTMLCleaner["HTMLCleaner\n(html_cleaner.py)"]:::importer
                Exceptions["Importer Exceptions\n(exceptions.py)"]:::importer
                Models["Models\n(models.py)"]:::importer

                subgraph "Concrete Importers"
                    EPUB["EPUB Importer\n(epub.py)"]:::importer
                    WebImp["Web Importer\n(web.py)"]:::importer
                    RSS["RSS/Atom Importers\n(concrete_importors.py)"]:::importer
                    Pocket["Pocket Importer\n(pocket.py)"]:::importer
                    LocalFileImp["Local File Importer\n(local_file.py)"]:::importer
                end
            end

            subgraph "UI Layer"
                GUIDlg["Qt Dialogs & Menus\n(gui.py)"]:::ui
                ViewCtrl["View Controller\n(view.py)"]:::ui
                subgraph "WebView Assets"
                    CSS["model.css"]:::ui
                    ScrollJS["scroll.js"]:::ui
                    TextJS["text.js"]:::ui
                    TOCJS["table_of_contents.js"]:::ui
                end
            end

            subgraph "Core Logic"
                TextProc["Text Processing\n(text.py)"]:::core
                Scheduling["Scheduling Logic\n(schedule.py)"]:::core
                Utils["Utilities\n(util.py)"]:::core
                Colors["Highlight Palettes\n(colors.u8)"]:::core
            end

            PersistenceDB["Anki Notes & Scheduling DB"]:::db
            SettingsStore["Anki Config Store"]:::db

            subgraph "Third-Party Libraries"
                Feedparser["feedparser.py"]:::third
                CGI["cgi.py"]:::third
            end
        end
    end

    %% Data Flows
    ExtWeb -->|"fetch content"| Orchestrator
    ExtRSS -->|"fetch feed"| Orchestrator
    ExtPocket -->|"fetch articles"| Orchestrator
    LocalFS -->|"load file"| Orchestrator

    Orchestrator -->|uses| BaseImporter
    Orchestrator -->|uses| Models
    Orchestrator -->|handles| Exceptions
    BaseImporter --> HTMLCleaner
    HTMLCleaner --> Models
    Models --> GUIDlg

    GUIDlg -->|"render IR Models"| CSS
    GUIDlg --> TextJS
    GUIDlg --> ScrollJS
    GUIDlg --> TOCJS

    GUIDlg -->|"UI events"| ViewCtrl
    ViewCtrl -->|"extract selection"| TextProc
    TextProc --> Scheduling
    Scheduling --> PersistenceDB

    GUIDlg --> Settings
    Settings --> SettingsStore

    BaseImporter --> Feedparser
    RSS --> Feedparser
    WebImp --> CGI

    %% Click Events
    click Shell "https://github.com/tvhong/incremental-reading/blob/main/ir/main.py"
    click Settings "https://github.com/tvhong/incremental-reading/blob/main/ir/settings.py"
    click Orchestrator "https://github.com/tvhong/incremental-reading/blob/main/ir/importer/importer.py"
    click BaseImporter "https://github.com/tvhong/incremental-reading/blob/main/ir/importer/base_importer.py"
    click Models "https://github.com/tvhong/incremental-reading/blob/main/ir/importer/models.py"
    click EPUB "https://github.com/tvhong/incremental-reading/blob/main/ir/importer/epub.py"
    click WebImp "https://github.com/tvhong/incremental-reading/blob/main/ir/importer/web.py"
    click RSS "https://github.com/tvhong/incremental-reading/blob/main/ir/importer/concrete_importors.py"
    click Pocket "https://github.com/tvhong/incremental-reading/blob/main/ir/importer/pocket.py"
    click LocalFileImp "https://github.com/tvhong/incremental-reading/blob/main/ir/importer/local_file.py"
    click HTMLCleaner "https://github.com/tvhong/incremental-reading/blob/main/ir/importer/html_cleaner.py"
    click Exceptions "https://github.com/tvhong/incremental-reading/blob/main/ir/importer/exceptions.py"
    click GUIDlg "https://github.com/tvhong/incremental-reading/blob/main/ir/gui.py"
    click ViewCtrl "https://github.com/tvhong/incremental-reading/blob/main/ir/view.py"
    click CSS "https://github.com/tvhong/incremental-reading/blob/main/ir/web/model.css"
    click ScrollJS "https://github.com/tvhong/incremental-reading/blob/main/ir/web/scroll.js"
    click TextJS "https://github.com/tvhong/incremental-reading/blob/main/ir/web/text.js"
    click TOCJS "https://github.com/tvhong/incremental-reading/blob/main/ir/web/table_of_contents.js"
    click TextProc "https://github.com/tvhong/incremental-reading/blob/main/ir/text.py"
    click Scheduling "https://github.com/tvhong/incremental-reading/blob/main/ir/schedule.py"
    click Utils "https://github.com/tvhong/incremental-reading/blob/main/ir/util.py"
    click Feedparser "https://github.com/tvhong/incremental-reading/blob/main/ir/lib/feedparser.py"
    click CGI "https://github.com/tvhong/incremental-reading/blob/main/ir/lib/cgi.py"
    click Colors "https://github.com/tvhong/incremental-reading/blob/main/ir/data/colors.u8"

    %% Styles
    classDef importer fill:#cce5ff,stroke:#69c
    classDef ui fill:#ffe5b4,stroke:#f90
    classDef core fill:#d4ffd4,stroke:#3c3
    classDef db fill:#fff2cc,stroke:#cc3
    classDef external fill:#e0f7ff,stroke:#1ca
    classDef third fill:#e0e0e0,stroke:#888
    classDef shell fill:#f2f2f2,stroke:#666
```
