# mwouts/jupytext architecture

> Architecture diagram of the GitHub repository mwouts/jupytext, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mwouts/jupytext
- Repository: https://github.com/mwouts/jupytext

## Overview

##### Project type and purpose
- Jupytext is an open-source tool (Python library + JupyterLab extension + CLI) that • converts Jupyter notebooks (`.ipynb`) to/from plain-text “text notebooks” (e.g. `.py`, `.md`, etc.) • synchronizes paired files • integrates with Jupyter Server and JupyterLab for on-the-fly conversion.

##### File structure overview
- Top level
- src/jupytext: core conversion engine, CLI entry point, file-format readers/writers, content manager adapters
- src/jupytext_config: configuration loader for global or per-directory settings
- jupyterlab/...: a full JupyterLab extension (TypeScript + schema + UI commands)
- docs/, demo/: documentation site + demo notebooks
- tests/: unit, functional, integration tests for every component
- CI, GitHub workflows, binder: build, test, publish pipelines
- Patterns
- Core library separated from CLI (`__main__.py`, `cli.py`)
- Jupyter Server extension via sync/async content managers
- Frontend/ backend split in JupyterLab extension (TS code + server-side install scripts)

##### Key architecture components
a. Core Conversion Engine (Python)
- Modules: `formats.py`, `cell_reader.py`, `cell_to_text.py`, `pairs.py`, `pandoc.py`, etc.
- Responsibilities: parse input (ipynb, markdown, script), map cells ↔ text, emit output in target format.
b. Configuration Loader
- `jupytext_config.py`: reads `jupytext.toml` or environment settings, exposes format mappings.
c. CLI Layer
- `cli.py` & `__main__.py`: command parsing, invokes core engine for `--to`, `--sync`, `--set-formats`, `--pipe`.
d. Jupyter Server Extension
- `sync_contentsmanager.py` & `async_contentsmanager.py`: plug into Jupyter ContentsManager API to intercept file I/O, convert on save/load.
e. JupyterLab Frontend Extension
- Located under `jupyterlab/packages/jupyterlab-jupytext-extension/src`: UI commands, toolbar buttons, right-click “Pair Notebook” actions • communicates via Jupyter REST API to server extension.
f. File System (User notebooks)
- Source files (`.ipynb`, `.py`, `.md`, …) stored on disk or in remote storage.
g. External Integrations
- Pre-commit hooks, black/flake8 integrations (`--pipe`), binder launch, CI pipelines (GitHub Actions, codecov).

##### Interactions and data flow
- CLI flow: User → CLI wrapper → Core engine → Read input file(s) → Convert → Write output file
- JupyterLab flow:
1. User opens file in JupyterLab frontend
2. Frontend command issues HTTP request to Jupyter server extension
3. Server extension uses core engine to convert on-the-fly
4. Notebook is loaded as `.ipynb` in browser; saves trigger reverse conversion
- Configuration: all flows consult the configuration loader for format rules before invoking the engine.

##### Architectural patterns and principles
- Separation of concerns: conversion core, CLI, server-extension, frontend-extension each in its own module
- Plugin pattern: Jupyter ContentsManager API and JupyterLab extension schema allow pluggable integration
- Modular design: format handlers isolated, new formats added via `formats.py` registry
- Test-driven: extensive unit, functional and integration tests demonstrate each component’s contract.

##### Final checklist
- Components above with clear labels
- Conversion pipeline, configuration lookup, and extension points

Top Layer: User Interfaces (CLI; JupyterLab UI)
Middle Layer: Integration & Extensions (CLI wrapper; Server Extension)
Core Layer: Conversion Engine & Config Loader
Bottom Layer: Storage (Notebook files; Config files)
External: Pre-commit, CI pipelines to the side.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Top Layer: User Interfaces
    subgraph "User Interfaces"
        CLI["CLI Interface"]:::python
        JLabUI["JupyterLab UI (Frontend)"]:::frontend
    end

    %% Middle Layer: Integration & Extensions
    subgraph "Integration & Extensions"
        CLI -->|"invokes"| CoreEngine
        CLI -->|"consults"| ConfigLoader
        JLabUI -->|"HTTP REST"| ServerExt
        ServerExt -->|"uses"| CoreEngine
        ServerExt -->|"consults"| ConfigLoader
    end

    %% Core Layer: Conversion Engine & Config Loader
    subgraph "Core Layer"
        CoreEngine["Core Conversion Engine"]:::python
        ConfigLoader["Configuration Loader"]:::python
    end

    %% Bottom Layer: Storage
    subgraph "Storage"
        FileSystem["Notebook Files (.ipynb, .py, .md)"]:::datastore
        ConfigFile["jupytext.toml"]:::datastore
    end

    %% Connections to Storage
    CoreEngine -->|"read/write"| FileSystem
    ConfigLoader -->|"reads"| ConfigFile
    ServerExt -->|"read/write"| FileSystem

    %% External Services
    subgraph "External Integrations"
        PreCommit["Pre-commit Hooks"]:::external
        CI["CI/CD Pipelines"]:::external
        Binder["Binder Environment"]:::external
        Docs["Docs & Demo"]:::external
    end

    CLI -->|"--pipe"| PreCommit
    CLI -->|"--pipe"| CI
    CLI -->|"launch"| Binder
    PreCommit --> CI
    CI --> Docs

    %% Click Events
    click CoreEngine "https://github.com/mwouts/jupytext/tree/main/src/jupytext/"
    click ConfigLoader "https://github.com/mwouts/jupytext/blob/main/src/jupytext_config/jupytext_config.py"
    click ConfigMain "https://github.com/mwouts/jupytext/blob/main/src/jupytext_config/__main__.py"
    click LabConfig "https://github.com/mwouts/jupytext/blob/main/src/jupytext_config/labconfig.py"
    click CLI "https://github.com/mwouts/jupytext/blob/main/src/jupytext/cli.py"
    click CLIMain "https://github.com/mwouts/jupytext/blob/main/src/jupytext/__main__.py"
    click SyncCM "https://github.com/mwouts/jupytext/blob/main/src/jupytext/sync_contentsmanager.py"
    click AsyncCM "https://github.com/mwouts/jupytext/blob/main/src/jupytext/async_contentsmanager.py"
    click JLabExt "https://github.com/mwouts/jupytext/tree/main/jupyterlab/packages/jupyterlab-jupytext-extension/src/"
    click CommandsTS "https://github.com/mwouts/jupytext/blob/main/jupyterlab/packages/jupyterlab-jupytext-extension/src/commands.ts"
    click FactoryTS "https://github.com/mwouts/jupytext/blob/main/jupyterlab/packages/jupyterlab-jupytext-extension/src/factory.ts"
    click IndexTS "https://github.com/mwouts/jupytext/blob/main/jupyterlab/packages/jupyterlab-jupytext-extension/src/index.ts"
    click RegistryTS "https://github.com/mwouts/jupytext/blob/main/jupyterlab/packages/jupyterlab-jupytext-extension/src/registry.ts"
    click TokensTS "https://github.com/mwouts/jupytext/blob/main/jupyterlab/packages/jupyterlab-jupytext-extension/src/tokens.ts"
    click UtilsTS "https://github.com/mwouts/jupytext/blob/main/jupyterlab/packages/jupyterlab-jupytext-extension/src/utils.ts"
    click PreCommitCfg "https://github.com/mwouts/jupytext/blob/main/.pre-commit-config.yaml"
    click PreCommitHooks "https://github.com/mwouts/jupytext/blob/main/.pre-commit-hooks.yaml"
    click Workflows "https://github.com/mwouts/jupytext/tree/main/.github/workflows/"
    click Binder "https://github.com/mwouts/jupytext/tree/main/binder/"
    click Docs "https://github.com/mwouts/jupytext/tree/main/docs/"
    click Demo "https://github.com/mwouts/jupytext/tree/main/demo/"

    %% Styles
    classDef python fill:#AED6F1,stroke:#1B4F72,color:#1B2631
    classDef frontend fill:#F9E79F,stroke:#B7950B,color:#6E2C00
    classDef datastore fill:#ABEBC6,stroke:#1D8348,color:#145A32
    classDef external fill:#D5D8DC,stroke:#717D7E,color:#1C2833
```
