# zyddnys/manga-image-translator architecture

> Architecture diagram of the GitHub repository zyddnys/manga-image-translator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zyddnys/manga-image-translator
- Repository: https://github.com/zyddnys/manga-image-translator

## Overview

##### High-Level Architecture Components

The system should be divided into these main components:

a) Input/Output Layer
- Input handlers for images and configuration
- Output handlers for translated images and formats (XCF, PSD, PDF)
- Web server interface for API access

b) Core Processing Pipeline
- Detection Module (text detection in images)
- OCR Module (text recognition)
- Translation Module (multiple translation services)
- Inpainting Module (background restoration)
- Rendering Module (text rendering)

c) Support Systems
- Model Management
- Configuration Management
- Threading/Processing Control

##### Detailed Component Breakdown

a) Detection Module:
- Multiple detector implementations (default, dbconvnext, ctd, craft)
- Panel detection subsystem
- Text region detection and processing
- Mask generation pipeline

b) OCR Module:
- Multiple OCR models (32px, 48px, 48px_ctc, mocr)
- Text preprocessing
- Language detection
- Text merging system

c) Translation Module:
- Online translators (GPT, DeepL, Google, etc.)
- Offline translators (NLLB, M2M100, etc.)
- Translation chain management
- Dictionary support (pre/post processing)

d) Inpainting Module:
- Multiple inpainting models (lama, SD, etc.)
- Mask refinement system
- Background reconstruction

e) Rendering Module:
- Text layout engine
- Font management
- Direction handling (vertical/horizontal)
- GIMP integration for advanced formats

##### Data Flow Representation

Flow of data through the system:
**Input Image → Detection → OCR → Translation → Inpainting → Rendering → Output**
- ↓          ↓         ↓           ↓            ↓
- Masks → Text Regions → Text → Clean Image → Final Image

##### Integration Points

Following integration points:
- Model loading and management system
- Configuration injection points
- API endpoints for web service
- External service connections (translation APIs)
- File system interaction points

##### Implementation Details

Technical details such as:
- Python as the primary language
- PyTorch for deep learning models
- Docker containerization support
- Web API using FastAPI
- File format support (XCF, PSD, PDF)
- GPU acceleration capabilities

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Legend
        L1[Processing Module]:::core
        L2[External Service]:::external
        L3[Storage/Models]:::storage
        L4[Input/Output]:::io
        L5[Support System]:::support
    end

    subgraph "Input/Output Layer"
        Input[Input Handler]:::io
        Output[Output Handler]:::io
        WebAPI[Web Server Interface]:::io
    end

    click WebAPI "https://github.com/zyddnys/manga-image-translator/blob/main/server/main.py"

    subgraph "Core Processing Pipeline"
        direction TB
        subgraph "Detection Module"
            Detection[Detection Controller]:::core
            DefaultDet[Default Detector - Manga Text Region Detection]:::core
            DBConvDet[DBConvNext - Text Detection Transformer]:::core
            CTDDet[CTD - Comic Text Detector]:::core
            CRAFTDet[CRAFT - Character Region Awareness]:::core
            
            Detection --> DefaultDet
            Detection --> DBConvDet
            Detection --> CTDDet
            Detection --> CRAFTDet
            
            subgraph "Detection Features"
                PanelDet[Panel Detection]:::core
                MaskGen[Mask Generation]:::core
                RegionProc[Region Processing]:::core
            end
            
            Detection --> PanelDet
            Detection --> MaskGen
            Detection --> RegionProc
        end

        click Detection "https://github.com/zyddnys/manga-image-translator/tree/main/manga_translator/detection/"
        click DefaultDet "https://github.com/zyddnys/manga-image-translator/blob/main/manga_translator/detection/default.py"
        click DBConvDet "https://github.com/zyddnys/manga-image-translator/blob/main/manga_translator/detection/dbnet_convnext.py"
        click CTDDet "https://github.com/zyddnys/manga-image-translator/blob/main/manga_translator/detection/ctd.py"
        click CRAFTDet "https://github.com/zyddnys/manga-image-translator/blob/main/manga_translator/detection/craft.py"

        subgraph "OCR Module"
            OCR[OCR Controller]:::core
            OCR32[32px - Basic Recognition]:::core
            OCR48[48px - Enhanced Recognition]:::core
            OCR48CTC[48px CTC - Connectionist Temporal]:::core
            MOCR[Manga OCR - Specialized Recognition]:::core
            
            OCR --> OCR32
            OCR --> OCR48
            OCR --> OCR48CTC
            OCR --> MOCR
            
            subgraph "OCR Features"
                PreProc[Text Preprocessing]:::core
                LangDet[Language Detection]:::core
                TextMerge[Text Merging System]:::core
            end
            
            OCR --> PreProc
            OCR --> LangDet
            OCR --> TextMerge
        end

        click OCR "https://github.com/zyddnys/manga-image-translator/tree/main/manga_translator/ocr/"
        click OCR32 "https://github.com/zyddnys/manga-image-translator/blob/main/manga_translator/ocr/model_32px.py"
        click OCR48 "https://github.com/zyddnys/manga-image-translator/blob/main/manga_translator/ocr/model_48px.py"
        click OCR48CTC "https://github.com/zyddnys/manga-image-translator/blob/main/manga_translator/ocr/model_48px_ctc.py"
        click MOCR "https://github.com/zyddnys/manga-image-translator/blob/main/manga_translator/ocr/model_manga_ocr.py"

        subgraph "Translation Module"
            Translation[Translation Controller]:::core
            OnlineTrans[Online Translators - GPT/DeepL/Google]:::external
            OfflineTrans[Offline - NLLB/M2M100]:::core
            
            Translation --> OnlineTrans
            Translation --> OfflineTrans
            
            subgraph "Translation Features"
                ChainMgmt[Translation Chain]:::core
                DictSupport[Dictionary Support]:::core
                PrePostProc[Pre/Post Processing]:::core
            end
            
            Translation --> ChainMgmt
            Translation --> DictSupport
            Translation --> PrePostProc
        end

        click Translation "https://github.com/zyddnys/manga-image-translator/tree/main/manga_translator/translators/"
        
        subgraph "Inpainting Module"
            Inpainting[Inpainting Controller]:::core
            LAMA[LAMA - Large Mask Inpainting]:::core
            SD[Stable Diffusion Inpainting]:::core
            AOT[AOT - Aggregated Contextual]:::core
            
            Inpainting --> LAMA
            Inpainting --> SD
            Inpainting --> AOT
            
            subgraph "Inpainting Features"
                MaskRef[Mask Refinement]:::core
                BgRecon[Background Reconstruction]:::core
            end
            
            Inpainting --> MaskRef
            Inpainting --> BgRecon
        end

        click Inpainting "https://github.com/zyddnys/manga-image-translator/tree/main/manga_translator/inpainting/"
        click LAMA "https://github.com/zyddnys/manga-image-translator/blob/main/manga_translator/inpainting/inpainting_lama.py"
        click SD "https://github.com/zyddnys/manga-image-translator/blob/main/manga_translator/inpainting/inpainting_sd.py"
        click AOT "https://github.com/zyddnys/manga-image-translator/blob/main/manga_translator/inpainting/inpainting_aot.py"

        subgraph "Rendering Module"
            Rendering[Rendering Controller]:::core
            TextRender[Text Layout Engine]:::core
            GIMPRender[GIMP Format Integration]:::core
            BalloonExt[Speech Balloon Extraction]:::core
            
            Rendering --> TextRender
            Rendering --> GIMPRender
            Rendering --> BalloonExt
            
            subgraph "Rendering Features"
                FontMgmt[Font Management]:::core
                DirHandler[Direction Handling]:::core
            end
            
            Rendering --> FontMgmt
            Rendering --> DirHandler
        end

        click Rendering "https://github.com/zyddnys/manga-image-translator/tree/main/manga_translator/rendering/"
        click TextRender "https://github.com/zyddnys/manga-image-translator/blob/main/manga_translator/rendering/text_render.py"
        click GIMPRender "https://github.com/zyddnys/manga-image-translator/blob/main/manga_translator/rendering/gimp_render.py"
        click BalloonExt "https://github.com/zyddnys/manga-image-translator/blob/main/manga_translator/rendering/ballon_extractor.py"

        Colorization[Colorization Module]:::core
    end

    click Colorization "https://github.com/zyddnys/manga-image-translator/blob/main/manga_translator/colorization/manga_colorization_v2.py"

    subgraph "Support Systems"
        Config[Configuration Manager]:::support
        Threading[Threading Control]:::support
        Logging[Logging System]:::support
        Utils[Generic Utils]:::support
        TextProcess[Text Processing]:::support
    end

    click Config "https://github.com/zyddnys/manga-image-translator/blob/main/manga_translator/config.py"
    click Threading "https://github.com/zyddnys/manga-image-translator/blob/main/manga_translator/utils/threading.py"
    click Logging "https://github.com/zyddnys/manga-image-translator/blob/main/manga_translator/utils/log.py"
    click Utils "https://github.com/zyddnys/manga-image-translator/blob/main/manga_translator/utils/generic.py"
    click TextProcess "https://github.com/zyddnys/manga-image-translator/blob/main/manga_translator/utils/textblock.py"

    %% Main Data Flow
    Input --> Detection
    Detection --> OCR
    OCR --> Translation
    Translation --> Inpainting
    Inpainting --> Rendering
    Rendering --> Output
    Rendering --> Colorization
    Colorization --> Output

    %% Support System Connections
    Config --> Detection
    Config --> OCR
    Config --> Translation
    Config --> Inpainting
    Config --> Rendering
    Threading --> Detection
    Threading --> OCR
    Threading --> Translation
    Threading --> Inpainting
    Threading --> Rendering
    TextProcess --> OCR
    TextProcess --> Translation

    %% Web API Connections
    WebAPI --> Input
    WebAPI --> Output

    %% Styling
    classDef core fill:#2374ab
    classDef external fill:#ffb347
    classDef storage fill:#b8b8b8
    classDef io fill:#47b347
    classDef support fill:#d3d3d3
```
