# cdhigh/kindleear architecture

> Architecture diagram of the GitHub repository cdhigh/kindleear, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cdhigh/kindleear
- Repository: https://github.com/cdhigh/kindleear

## Overview

##### Project type and purpose
- From the file tree and README, this is a full-stack Python web application (“KindleEar”) that aggregates arbitrary web content (RSS/ATOM/Calibre recipes/custom feeds), converts it into EPUB/MOBI/MP3 via built-in ebook generation, summarization, translation and TTS engines, and then delivers it to users’ Kindles or serves it through an online reader.

##### File structure
- Top-level directories:
- application/ – core code, split into:
· back_end/ – adapters for DB, mail, task queues
· view/ – URL handlers (admin, library, deliver, reader, translator, etc.)
· lib/ – domain libraries (Calibre engine, summarizer, translator, TTS, dictionary, scraping, ebook building)
· templates/ & static/ – HTML templates and front-end assets
· images/, recipes/, translations/ – data assets
· routes.py, main.py – WSGI entrypoints and URL routing
- docker/, tools/ – Dockerfiles, CI/CD scripts, deployment helpers
- config.py, app.yaml, queue.yaml, cron.yaml, dispatch.yaml – configuration for GAE, cron jobs, task queues
- docs/, tests/ – documentation and test suite
- Architectural patterns:
- Adapter pattern in back_end (send_mail_adpt.py, task_queue_adpt.py)
- Plugin structure in lib (calibre/customize, ebook_translator/engines, ebook_tts/engines)
- Monolithic codebase with modular subpackages
- Support for multiple task queue engines (Celery, RQ, APScheduler, GAE push queue)

##### Extract additional insights from the README
- The app supports: web UI, scheduled jobs (daily pushes or on-demand), email delivery, online reader, library sharing, browser extension for recipe creation, and integration with AI summarization/translation/TTS via multiple third-party APIs.

## Diagram (Mermaid)

```mermaid
graph TB
  %% Frontend Layer
  subgraph "Frontend Layer"
    User["User Browser / Kindle"]:::frontend
  end

  %% Web/API Layer
  subgraph "Web/API Layer"
    Main["main.py"]:::frontend
    Routes["routes.py"]:::frontend
  end

  User -->|"HTTP/SMTP"| Routes
  Routes --> Main
  Main -->|calls handlers| Routes

  %% View Handlers
  subgraph "View Handlers"
    Admin["admin.py"]:::frontend
    Library["library.py"]:::frontend
    Reader["reader.py"]:::frontend
    TranslatorVH["translator.py"]:::frontend
  end
  Routes --> Admin
  Routes --> Library
  Routes --> Reader
  Routes --> TranslatorVH

  %% Template Engine + Static
  subgraph "Templates & Static"
    Templates["application/templates/"]:::frontend
    Static["application/static/"]:::frontend
  end
  Admin --> Templates
  Library --> Templates
  Reader --> Templates
  TranslatorVH --> Templates
  Admin --> Static
  Library --> Static
  Reader --> Static
  TranslatorVH --> Static

  %% Core Services
  subgraph "Core Services"
    FeedManager["Feed & Recipe Manager\n(application/lib/calibre/)"]:::core
    Recipes["Recipes Data\n(application/recipes/)"]:::core
    EbookBuilder["Ebook Builder Service\n(build_ebook.py)"]:::core
    Summarizer["AI Summarization Service\n(html_summarizer.py)"]:::core
    Translator["AI Translation Service\n(ebook_translator/)"]:::core
    TTS["AI Text-to-Speech Service\n(ebook_tts/)"]:::core
    FileUtils["File Download & Storage\n(filedownload.py, filesystem_dict.py)"]:::core
    DictService["Dictionary Lookup Service\n(application/lib/dictionary/)"]:::core
  end

  Library --> FeedManager
  Admin --> FeedManager
  FeedManager --> Recipes
  Reader --> EbookBuilder
  TranslatorVH --> Translator
  Reader --> TTS
  Library --> Summarizer
  FeedManager --> FileUtils
  FeedManager --> DictService

  %% Adapter Layer
  subgraph "Adapter Layer"
    MailAdpt["Mail Adapter\n(send_mail_adpt.py)"]:::infra
    TaskQueueAdpt["Task Queue Adapter\n(task_queue_adpt.py)"]:::infra
    subgraph "Task Queue Implementations"
      CeleryImpl["Celery\n(task_queue_celery.py)"]:::infra
      RQImpl["RQ\n(task_queue_rq.py)"]:::infra
      APSImpl["APScheduler\n(task_queue_apscheduler.py)"]:::infra
      GAEImpl["GAE Queue\n(task_queue_gae.py)"]:::infra
    end
    DBAdpt["Database Adapters\n(db_models_sql.py, db_models_nosql.py)"]:::infra
    Utils["Core Utilities\n(ke_utils.py, base_handler.py)"]:::infra
  end

  EbookBuilder --> TaskQueueAdpt
  Summarizer --> TaskQueueAdpt
  Translator --> TaskQueueAdpt
  TTS --> TaskQueueAdpt
  TaskQueueAdpt --> CeleryImpl
  TaskQueueAdpt --> RQImpl
  TaskQueueAdpt --> APSImpl
  TaskQueueAdpt --> GAEImpl
  EbookBuilder --> MailAdpt
  Routes --> Utils
  Admin --> DBAdpt
  Library --> DBAdpt

  %% Data Layer
  subgraph "Data Layer"
    DB[(Relational DB / NoSQL)]:::external
    Storage[(Filesystem / Object Storage)]:::external
  end
  DBAdpt --> DB
  FileUtils --> Storage

  %% External Services
  subgraph "External Services"
    SMTPServer((SMTP Server)):::external
    AIAPIs((AI API Endpoints)):::external
    RSSFeeds((RSS/ATOM Sources)):::external
    CalibreSources((Calibre Recipe Sources)):::external
  end
  MailAdpt -->|SMTP| SMTPServer
  Summarizer -->|API| AIAPIs
  Translator -->|API| AIAPIs
  TTS -->|API| AIAPIs
  FeedManager --> RSSFeeds
  FeedManager --> CalibreSources

  %% Deployment Infrastructure
  subgraph "Deployment"
    DockerCfg["Docker Config\n(docker/)"]:::deployment
    GAEConfig["GAE Config\n(app.yaml, cron.yaml, queue.yaml, dispatch.yaml)"]:::deployment
    CICDPipelines["CI/CD Pipelines\n(.github/workflows/)"]:::deployment
  end
  Main --> DockerCfg
  Routes --> DockerCfg
  DockerCfg --> GAEConfig
  GAEConfig --> CICDPipelines

  %% Click Events
  click Main "https://github.com/cdhigh/kindleear/blob/master/main.py"
  click Routes "https://github.com/cdhigh/kindleear/blob/master/application/routes.py"
  click Admin "https://github.com/cdhigh/kindleear/blob/master/application/view/admin.py"
  click Library "https://github.com/cdhigh/kindleear/blob/master/application/view/library.py"
  click Reader "https://github.com/cdhigh/kindleear/blob/master/application/view/reader.py"
  click TranslatorVH "https://github.com/cdhigh/kindleear/blob/master/application/view/translator.py"
  click Templates "https://github.com/cdhigh/kindleear/tree/master/application/templates/"
  click Static "https://github.com/cdhigh/kindleear/tree/master/application/static/"
  click FeedManager "https://github.com/cdhigh/kindleear/tree/master/application/lib/calibre/"
  click Recipes "https://github.com/cdhigh/kindleear/tree/master/application/recipes/"
  click EbookBuilder "https://github.com/cdhigh/kindleear/blob/master/application/lib/build_ebook.py"
  click Summarizer "https://github.com/cdhigh/kindleear/blob/master/application/lib/ebook_summarizer/html_summarizer.py"
  click Translator "https://github.com/cdhigh/kindleear/tree/master/application/lib/ebook_translator/"
  click TTS "https://github.com/cdhigh/kindleear/tree/master/application/lib/ebook_tts/"
  click FileUtils "https://github.com/cdhigh/kindleear/blob/master/application/lib/filedownload.py"
  click FileUtils "https://github.com/cdhigh/kindleear/blob/master/application/lib/filesystem_dict.py"
  click DictService "https://github.com/cdhigh/kindleear/tree/master/application/lib/dictionary/"
  click MailAdpt "https://github.com/cdhigh/kindleear/blob/master/application/back_end/send_mail_adpt.py"
  click TaskQueueAdpt "https://github.com/cdhigh/kindleear/blob/master/application/back_end/task_queue_adpt.py"
  click CeleryImpl "https://github.com/cdhigh/kindleear/blob/master/application/back_end/task_queue_celery.py"
  click RQImpl "https://github.com/cdhigh/kindleear/blob/master/application/back_end/task_queue_rq.py"
  click APSImpl "https://github.com/cdhigh/kindleear/blob/master/application/back_end/task_queue_apscheduler.py"
  click GAEImpl "https://github.com/cdhigh/kindleear/blob/master/application/back_end/task_queue_gae.py"
  click DBAdpt "https://github.com/cdhigh/kindleear/blob/master/application/back_end/db_models_sql.py"
  click DBAdpt "https://github.com/cdhigh/kindleear/blob/master/application/back_end/db_models_nosql.py"
  click Utils "https://github.com/cdhigh/kindleear/blob/master/application/ke_utils.py"
  click Utils "https://github.com/cdhigh/kindleear/blob/master/application/base_handler.py"
  click DockerCfg "https://github.com/cdhigh/kindleear/tree/master/docker/"
  click GAEConfig "https://github.com/cdhigh/kindleear/blob/master/app.yaml"
  click CICDPipelines "https://github.com/cdhigh/kindleear/blob/master/.github/workflows/docker_build_push.yaml"
  click CICDPipelines "https://github.com/cdhigh/kindleear/blob/master/.github/workflows/mailfix_build_push.yaml"

  %% Styles
  classDef frontend fill:#D0E8FF,stroke:#0366d6,color:#0366d6
  classDef core fill:#C8E6C9,stroke:#2e7d32,color:#2e7d32
  classDef infra fill:#FFE0B2,stroke:#ef6c00,color:#e65100
  classDef external fill:#E0E0E0,stroke:#424242,color:#424242
  classDef deployment fill:#E1BEE7,stroke:#6a1b9a,color:#4a148c
```
