# openzim/kolibri architecture

> Architecture diagram of the GitHub repository openzim/kolibri, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/openzim/kolibri
- Repository: https://github.com/openzim/kolibri

## Overview

##### Main Components

a. Input Layer:
- Kolibri Channel Interface (handles channel ID and node IDs)
- Configuration Handler (processes command-line arguments)

b. Core Processing Components:
- Scraper Module (downloads content from Kolibri)
- Content Processors:
- Video Processor (handles webm/mp4)
- Document Processor (handles PDFs/ePubs)
- Audio Processor
- Subtitle Handler
- Thumbnail Processor

c. Frontend UI (Vue.js based):
- Topic Navigation System
- Content Viewer
- Search Interface

d. ZIM Creation Layer:
- Content Organizer
- Static HTML Generator
- ZIM File Builder

**[Input Layer]**
- ↓
**[Processing Layer]**
- ↓
**[Frontend Layer]**
- ↓
**[Output Layer]**

##### Key Relationships to Highlight

- Data flow from Kolibri Channel input through the processing pipeline
- How the Vue.js frontend (zimui) integrates with the processed content
- Demonstrate the final packaging into ZIM format

##### Technical Details

a. Technologies:
- Python 3.11 (main scraper)
- Node.js 20.x (frontend)
- FFmpeg (media processing)
- Vue.js (UI framework)
- TypeScript (frontend type system)

b. Important Files/Components:
- scraper/src/kolibri2zim/scraper.py (main scraping logic)
- scraper/src/kolibri2zim/processing.py (content processing)
- zimui/src/components/* (UI components)

##### Specific Implementation Notes

- The separation between the Python scraper and Vue.js frontend
- The content processing pipeline's modularity
- How different content types (video, audio, documents) are handled
- The final ZIM packaging process

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Input
        Config["Configuration Handler"]
        KolibriChannel["Kolibri Channel Interface"]
        click Config "https://github.com/openzim/kolibri/blob/main/scraper/src/kolibri2zim/entrypoint.py"
        click KolibriChannel "https://github.com/openzim/kolibri/blob/main/scraper/src/kolibri2zim/nodes.py"
    end

    subgraph Processing
        Scraper["Scraper Module"]
        DB[(Database)]
        Constants["Constants & Config"]
        
        subgraph ContentProcessors["Content Processors"]
            VideoProc["Video Processor"]
            DocProc["Document Processor"]
            AudioProc["Audio Processor"]
            SubHandler["Subtitle Handler"]
            ThumbProc["Thumbnail Processor"]
        end

        click Scraper "https://github.com/openzim/kolibri/blob/main/scraper/src/kolibri2zim/scraper.py"
        click VideoProc "https://github.com/openzim/kolibri/blob/main/scraper/src/kolibri2zim/templates/video.html"
        click DocProc "https://github.com/openzim/kolibri/blob/main/scraper/src/kolibri2zim/templates/document.html"
        click AudioProc "https://github.com/openzim/kolibri/blob/main/scraper/src/kolibri2zim/templates/audio.html"
        click DB "https://github.com/openzim/kolibri/blob/main/scraper/src/kolibri2zim/database.py"
        click Constants "https://github.com/openzim/kolibri/blob/main/scraper/src/kolibri2zim/constants.py"
    end

    subgraph Frontend
        VueApp["Main Vue Application"]
        Store["Store Management"]
        TopicNav["Topic Navigation"]
        TopicHome["Topic Home"]
        TopicData["Topic Data Structures"]
        ContentView["Content Viewer"]
        Search["Search Interface"]

        click VueApp "https://github.com/openzim/kolibri/blob/main/zimui/src/App.vue"
        click Store "https://github.com/openzim/kolibri/blob/main/zimui/src/stores/main.ts"
        click TopicNav "https://github.com/openzim/kolibri/blob/main/zimui/src/components/TopicSection.vue"
        click TopicHome "https://github.com/openzim/kolibri/blob/main/zimui/src/components/TopicHome.vue"
        click TopicData "https://github.com/openzim/kolibri/blob/main/zimui/src/types/Topic.ts"
    end

    subgraph Output
        HTMLGen["Static HTML Generator"]
        ZIMBuilder["ZIM File Builder"]
        
        click HTMLGen "https://github.com/openzim/kolibri/blob/main/scraper/src/kolibri2zim/templates/base.html"
    end

    %% Relationships
    Config --> Scraper
    KolibriChannel --> Scraper
    Scraper --> DB
    Scraper --> Constants
    Scraper --> ContentProcessors
    ContentProcessors --> HTMLGen
    DB --> HTMLGen
    HTMLGen --> ZIMBuilder
    HTMLGen --> Frontend
    Store --> TopicNav
    Store --> TopicHome
    TopicData --> Store
    VueApp --> ContentView
    VueApp --> Search

    %% Styling
    classDef input fill:#a8d1f0
    classDef processing fill:#90EE90
    classDef frontend fill:#FFB347
    classDef output fill:#DDA0DD
    classDef storage fill:#D3D3D3

    class Config,KolibriChannel input
    class Scraper,VideoProc,DocProc,AudioProc,SubHandler,ThumbProc,ContentProcessors processing
    class VueApp,Store,TopicNav,TopicHome,TopicData,ContentView,Search frontend
    class HTMLGen,ZIMBuilder output
    class DB storage
```
