# kokiebisu/saola architecture

> Architecture diagram of the GitHub repository kokiebisu/saola, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kokiebisu/saola
- Repository: https://github.com/kokiebisu/saola

## Overview

1. **Purpose and overall shape**
   - This repository is a small Python-based scraping pipeline for manga on `mangareader.to`.
   - The end-to-end flow is: take a manga URL, extract metadata and chapter listings, download chapter images into a local folder, then optionally compile those images into a single PDF.

2. **Top-level entrypoints**
   - `saola.py` is the main orchestration script: it appears to combine scraping and PDF compilation in one run.
   - `scrape.py` is the download-only entrypoint: it fetches manga content into a folder.
   - `compile.py` is the post-processing entrypoint: it takes already-downloaded manga images and turns them into a PDF.

3. **Core pipeline boundary**
   - The repo is structured around a clear separation between **collection** and **compilation**.
   - Collection interacts with the website and filesystem to build a local chapter/image tree.
   - Compilation operates on that local tree and does not depend on the remote site.

4. **Scraping and extraction layer**
   - `lib/extractor.py` is likely the central domain layer for parsing manga pages, finding chapters, and extracting metadata like title and cover image.
   - `lib/beautifulsoup.py` suggests HTML parsing is done with BeautifulSoup for static page content.
   - `lib/selenium.py` indicates browser automation is used where the site requires dynamic rendering or JS-driven navigation.

5. **Image acquisition layer**
   - `lib/image.py` is the image-handling subsystem: downloading chapter pages/images and preparing them for storage and later PDF generation.
   - This layer likely turns the remote image URLs into ordered local files, preserving chapter/page sequence.

6. **Shared utilities**
   - `lib/utils.py` is the cross-cutting support module for filesystem paths, naming, threading helpers, configuration loading, and general convenience functions.
   - The README’s mention of `settings.json` and desktop folder customization implies utilities also manage user-specific paths and output layout.

7. **Filesystem-centric data flow**
   - The primary data store is the local filesystem, not a database.
   - The application creates a manga-named directory on the user’s desktop, stores chapter folders/images there, and writes the final PDF into that same structure.
   - This makes the output folder both the intermediate artifact store and the final deliverable location.

8. **Concurrency model**
   - The CLI supports a thread count option, so chapter/image downloading is parallelized.
   - Concurrency is likely confined to the scraping/download stage, while PDF compilation is probably sequential because image order matters.

9. **Configuration and user inputs**
   - The runtime is configured via command-line flags: URL, thread count, start/end chapter range, and language.
   - The README also mentions optional `settings.json`, which suggests some behavior can be overridden without changing code, especially paths or default storage locations.

10. **Runtime and packaging**
   - The project is Python-based and appears designed to run in containers rather than directly on the host.
   - `Dockerfile`, `docker-compose.yml`, and `.devcontainer/` indicate a reproducible development/runtime environment, likely to ensure Selenium/browser dependencies and Python packages are available consistently.

11. **External dependency**
   - The only clearly identified external system is `mangareader.to`, which is the source of truth for manga metadata, chapter structure, and page images.
   - The repo is therefore an ingestion tool that depends heavily on the site’s HTML structure and download endpoints.

12. **Likely architecture diagram boundaries**
   - A good diagram would show:
     - User/CLI input
     - Orchestrator scripts (`saola.py`, `scrape.py`, `compile.py`)
     - Scraping/extraction modules (`beautifulsoup`, `selenium`, `extractor`)
     - Image/file handling (`image`, `utils`)
     - Local output folder on desktop
     - Optional PDF compilation step
     - External source website (`mangareader.to`)
   - The key architectural constraint is that remote fetching and local compilation are separate phases connected by filesystem artifacts rather than by in-memory objects or a service API.

Main entry points: [`lib/extractor.py`](https://github.com/kokiebisu/saola/blob/main/lib/extractor.py) (Extractor), [`lib/image.py`](https://github.com/kokiebisu/saola/blob/main/lib/image.py) (Image), [`saola.py`](https://github.com/kokiebisu/saola/blob/main/saola.py) (saola.py), [`scrape.py`](https://github.com/kokiebisu/saola/blob/main/scrape.py) (scrape.py)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_entrypoints["Entrypoints"]
  node_node_saola["saola.py<br/>orchestrator<br/>[saola.py]"]
  node_node_scrape["scrape.py<br/>scrape CLI<br/>[scrape.py]"]
  node_node_compile["compile.py<br/>compile CLI<br/>[compile.py]"]
  node_node_cli(("User input<br/>runtime input"))
end

subgraph group_group_pipeline["Pipeline"]
  node_node_extractor["Extractor<br/>parser<br/>[extractor.py]"]
  node_node_bs["BeautifulSoup<br/>html parser<br/>[beautifulsoup.py]"]
  node_node_selenium["Selenium<br/>browser automation<br/>[selenium.py]"]
  node_node_image["Image<br/>downloader<br/>[image.py]"]
  node_node_utils["Utils<br/>shared helpers<br/>[utils.py]"]
  node_node_desktop[("Output tree<br/>filesystem store")]
  node_node_pdf["PDF output<br/>compiled artifact"]
  node_node_site(("mangareader.to<br/>source website"))
  node_node_settings["settings.json<br/>config"]
end

subgraph group_group_runtime["Runtime"]
  node_node_dockerfile["Dockerfile<br/>container build"]
  node_node_compose["docker-compose.yml<br/>[docker-compose.yml]"]
  node_node_devcontainer[".devcontainer<br/>dev environment<br/>[devcontainer.json]"]
  node_node_requirements["requirements<br/>python deps<br/>[requirements.txt]"]
end

node_node_cli -->|"invoke"| node_node_saola
node_node_cli -->|"invoke"| node_node_scrape
node_node_cli -->|"invoke"| node_node_compile
node_node_saola -->|"extract"| node_node_extractor
node_node_saola -->|"download"| node_node_image
node_node_saola -->|"compile"| node_node_pdf
node_node_scrape -->|"extract"| node_node_extractor
node_node_scrape -->|"download"| node_node_image
node_node_compile -->|"build"| node_node_pdf
node_node_extractor -->|"parse html"| node_node_bs
node_node_extractor -->|"fallback"| node_node_selenium
node_node_bs -->|"read pages"| node_node_site
node_node_selenium -->|"browse"| node_node_site
node_node_image -->|"fetch images"| node_node_site
node_node_image -->|"paths"| node_node_utils
node_node_extractor -->|"helpers"| node_node_utils
node_node_image -->|"write files"| node_node_desktop
node_node_pdf -->|"read input"| node_node_desktop
node_node_settings -->|"configure"| node_node_utils
node_node_dockerfile -->|"installs"| node_node_requirements
node_node_compose -->|"runs"| node_node_dockerfile
node_node_devcontainer -->|"extends"| node_node_dockerfile

click node_node_saola "https://github.com/kokiebisu/saola/blob/main/saola.py"
click node_node_scrape "https://github.com/kokiebisu/saola/blob/main/scrape.py"
click node_node_compile "https://github.com/kokiebisu/saola/blob/main/compile.py"
click node_node_extractor "https://github.com/kokiebisu/saola/blob/main/lib/extractor.py"
click node_node_bs "https://github.com/kokiebisu/saola/blob/main/lib/beautifulsoup.py"
click node_node_selenium "https://github.com/kokiebisu/saola/blob/main/lib/selenium.py"
click node_node_image "https://github.com/kokiebisu/saola/blob/main/lib/image.py"
click node_node_utils "https://github.com/kokiebisu/saola/blob/main/lib/utils.py"
click node_node_dockerfile "https://github.com/kokiebisu/saola/tree/main/Dockerfile"
click node_node_compose "https://github.com/kokiebisu/saola/blob/main/docker-compose.yml"
click node_node_devcontainer "https://github.com/kokiebisu/saola/blob/main/.devcontainer/devcontainer.json"
click node_node_requirements "https://github.com/kokiebisu/saola/blob/main/requirements.txt"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_node_saola,node_node_scrape,node_node_compile,node_node_cli toneBlue
class node_node_extractor,node_node_bs,node_node_selenium,node_node_image,node_node_utils,node_node_desktop,node_node_pdf,node_node_site,node_node_settings toneAmber
class node_node_dockerfile,node_node_compose,node_node_devcontainer,node_node_requirements toneMint
```

## Components

### Entrypoints

CLI scripts that start scraping or compilation runs.

- [saola.py](https://github.com/kokiebisu/saola/blob/main/saola.py): orchestrator · Main end-to-end entrypoint that combines scraping and PDF compilation. · `saola.py`
- [scrape.py](https://github.com/kokiebisu/saola/blob/main/scrape.py): scrape CLI · Download-only entrypoint for fetching manga into local folders. · `scrape.py`
- [compile.py](https://github.com/kokiebisu/saola/blob/main/compile.py): compile CLI · Post-processing entrypoint that turns downloaded images into a PDF. · `compile.py`
- **User input**: runtime input · CLI parameters such as URL, thread count, chapter range, and language.

### Pipeline

Core scraping, download, and compile subsystems.

- [Extractor](https://github.com/kokiebisu/saola/blob/main/lib/extractor.py): parser · Domain layer for manga metadata, chapter discovery, and page extraction. · `lib/extractor.py`
- [BeautifulSoup](https://github.com/kokiebisu/saola/blob/main/lib/beautifulsoup.py): html parser · Parses static HTML content from manga pages. · `lib/beautifulsoup.py`
- [Selenium](https://github.com/kokiebisu/saola/blob/main/lib/selenium.py): browser automation · Handles dynamic pages and JS-driven navigation when static parsing is not enough. · `lib/selenium.py`
- [Image](https://github.com/kokiebisu/saola/blob/main/lib/image.py): downloader · Downloads chapter images and prepares them for local storage and PDF input. · `lib/image.py`
- [Utils](https://github.com/kokiebisu/saola/blob/main/lib/utils.py): shared helpers · Cross-cutting helpers for paths, naming, threading, and runtime configuration. · `lib/utils.py`
- **Output tree**: filesystem store · Local manga folder tree on disk used for intermediate chapters, images, and final PDF.
- **PDF output**: compiled artifact · Final PDF generated from the downloaded image set.
- **mangareader.to**: source website · External source of manga metadata, chapters, and image assets.
- **settings.json**: config · Optional local configuration for defaults and output locations.

### Runtime

Container and environment files used to run the tool consistently.

- [Dockerfile](https://github.com/kokiebisu/saola/tree/main/Dockerfile): container build · Container image definition for reproducible runtime setup. · `Dockerfile`
- [docker-compose.yml](https://github.com/kokiebisu/saola/blob/main/docker-compose.yml): compose · Local container orchestration for the application stack. · `docker-compose.yml`
- [.devcontainer](https://github.com/kokiebisu/saola/blob/main/.devcontainer/devcontainer.json): dev environment · Development container setup for consistent Python and browser dependencies. · `.devcontainer/devcontainer.json`
- [requirements](https://github.com/kokiebisu/saola/blob/main/requirements.txt): python deps · Python package dependencies for scraping and compilation. · `requirements.txt`

## Connections

- User input → saola.py: invoke · User launches the combined pipeline through the main script.
- User input → scrape.py: invoke · User can launch the download-only workflow.
- User input → compile.py: invoke · User can launch the PDF-only workflow.
- saola.py → Extractor: extract · Main run parses metadata and chapter structure.
- saola.py → Image: download · Main run downloads chapter images.
- saola.py → PDF output: compile · Main run can produce a PDF after scraping.
- scrape.py → Extractor: extract · Scrape workflow resolves manga and chapter information.
- scrape.py → Image: download · Scrape workflow stores chapter images locally.
- compile.py → PDF output: build · Compilation workflow assembles a PDF from local images.
- Extractor → BeautifulSoup: parse html · Static parsing is used for page and metadata extraction.
- Extractor → Selenium: fallback · Dynamic browser automation is used when needed.
- BeautifulSoup → mangareader.to: read pages · HTML parser consumes pages from the source site.
- Selenium → mangareader.to: browse · Browser automation interacts with the source site.
- Image → mangareader.to: fetch images · Image subsystem pulls page images from remote URLs.
- Image → Utils: paths · Uses shared helpers for naming, ordering, and filesystem layout.
- Extractor → Utils: helpers · Uses shared helpers for paths, config, and runtime options.
- Image → Output tree: write files · Downloads are persisted into the local manga folder tree.
- PDF output → Output tree: read input · Compiler reads the local image tree and writes the final PDF there.
- settings.json → Utils: configure · Optional settings influence default paths and runtime behavior.
- Dockerfile → requirements: installs · Container build uses Python dependencies.
- docker-compose.yml → Dockerfile: runs · Compose starts the built application container.
- .devcontainer → Dockerfile: extends · Development container builds on the same runtime image.
