# jupyter/nbconvert architecture

> Architecture diagram of the GitHub repository jupyter/nbconvert, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jupyter/nbconvert
- Repository: https://github.com/jupyter/nbconvert

## Overview

##### Project type and purpose
- This repository is the Jupyter “nbconvert” tool: an open-source library and CLI application that converts `.ipynb` Jupyter notebooks into static formats (HTML, LaTeX, PDF, Markdown, reStructuredText, scripts, slides, etc.) via Jinja templates.
- It provides both a `jupyter nbconvert` command-line entry point and a Python API, plus template files, plugin points (exporters, preprocessors, postprocessors, filters), and optional integration with external tools (e.g., Pandoc).

##### High-level file-structure analysis
- Top-level directories and files:
- `nbconvert/` – core Python package
- `share/templates/` – shipped Jinja template sets for each output format
- `tests/` – unit & integration tests for export pipeline, filters, preprocessors, writers
- `docs/` – Sphinx-based documentation and API examples
- CLI-related files: `__main__.py`, `nbconvertapp.py`, `pyproject.toml`, `hatch_build.py`
- Config & CI: `.github/workflows/`, `.pre-commit-config.yaml`, `readthedocs.yaml`
- Within `nbconvert/` package:
- `nbconvertapp.py` – application entry, argument parsing, pipeline orchestration
- `exporters/` – per-format exporter implementations (HTML, PDF, markdown, etc.)
- `preprocessors/` – transformations applied before rendering (execution, clearing output, sanitization)
- `postprocessors/` – operations after rendering (e.g., serving HTML, file cleanup)
- `filters/` – granular content filters used by templating stage
- `writers/` – final writers delivering output to stdout or files
- `templates/` – built-in skeletons; overridden at runtime by `share/templates/`
- `utils/` – common utilities (I/O, versioning, exceptions)

##### README insights
- Emphasizes the CLI usage: `jupyter nbconvert --to <format> <notebook>`
- Mentions Jinja templating and optional external dependency on Pandoc

a. Main components:
1) CLI / main application (nbconvertapp)
2) Notebook reader (JSON loader)
3) Preprocessor chain
4) Template renderer (Exporter + Jinja engine + Filters)
5) Postprocessor chain
6) Writer (file or stdout output)
7) Template store (`share/templates` / user templates)
8) External tools (Pandoc)
b. Relationships & data flow:
- User invokes CLI → nbconvertapp parses args → loads `.ipynb` JSON
→ passes NotebookNode into a sequence:
Preprocessors → Exporter.render() (uses Filters + Jinja templates + Resources)
→ yields rendered output → Postprocessors → Writer writes result
- Optionally, Exporter may shell out to Pandoc or other binaries
c. Architectural patterns / design principles:
- Pipeline pattern: ordered chains of preprocessors & postprocessors
- Plugin architecture: exporters, preprocessors, filters, writers are all pluggable via entry points
- Separation of concerns: parsing, transforming, templating, writing
- Configuration & extensibility: external templates override built-ins; custom preprocessors can be injected at runtime
d. Key technologies / frameworks:
- Python 3.x modules & entry-points (packaged via pyproject.toml / setuptools)
- Jinja2 templating engine
- JSON parsing (nbformat)
- Optional Pandoc integration

##### Final guidance
- Essential layers (CLI vs API, Pipeline vs I/O)
- Keep it focused on runtime conversion pipeline; omit dev/test folders unless illustrating CI/CD or docs build

## Diagram (Mermaid)

```mermaid
graph TD
    %% CLI / Main Application
    subgraph "CLI / Main Application"
        CLI_main["__main__.py"]:::core
        CLI_app["nbconvertapp.py"]:::core
    end

    %% Notebook Reader
    Reader["Notebook Reader\n(nbconvert/utils/io.py)"]:::core

    %% Preprocessor Chain
    subgraph "Preprocessor Chain"
        Pre_dir["nbconvert/preprocessors/"]:::core
        Pre_base["base.py"]:::core
    end

    %% Template Renderer
    subgraph "Template Renderer"
        Exp_api["exporter.py"]:::core
        Temp_exp["templateexporter.py"]:::core
        Filters["Filters\n(nbconvert/filters/)"]:::core
    end

    %% Postprocessor Chain
    subgraph "Postprocessor Chain"
        Post_dir["nbconvert/postprocessors/"]:::core
        Post_base["base.py"]:::core
    end

    %% Writer
    subgraph "Writer"
        Writers_dir["nbconvert/writers/"]:::core
        Files["files.py"]:::core
        Stdout["stdout.py"]:::core
    end

    %% Template Store
    Builtin["Built-in Templates\n(nbconvert/templates/)"]:::store
    Shipped["Shipped Templates\n(share/templates/)"]:::store

    %% External Tools
    subgraph "External Tools"
        Pandoc_wrapper["pandoc.py"]:::external
        Pandoc_filter["Pandoc Filter\n(nbconvert/filters/pandoc.py)"]:::external
    end

    %% Data Flow
    CLI_main -->|"calls"| CLI_app
    CLI_app -->|"loads notebook"| Reader
    Reader -->|"produces NotebookNode"| Pre_dir
    Pre_dir --> Pre_base
    Pre_base -->|"preprocessed notebook"| Exp_api
    Exp_api --> Temp_exp
    Filters --> Temp_exp
    Temp_exp -->|"rendered output"| Post_dir
    Post_dir --> Post_base
    Post_base --> Writers_dir
    Writers_dir --> Files
    Writers_dir --> Stdout
    Files -->|"writes file"| Output["Output\n(files or stdout)"]
    Stdout --> Output

    %% Template Store Overrides
    Builtin --> Temp_exp
    Shipped --> Temp_exp

    %% Optional Pandoc Integration
    Temp_exp -->|"shells out"| Pandoc_wrapper
    Pandoc_wrapper --> Pandoc_filter
    Pandoc_filter --> Temp_exp

    %% Click events
    click CLI_main "https://github.com/jupyter/nbconvert/blob/main/nbconvert/__main__.py"
    click CLI_app "https://github.com/jupyter/nbconvert/blob/main/nbconvert/nbconvertapp.py"
    click Reader "https://github.com/jupyter/nbconvert/blob/main/nbconvert/utils/io.py"
    click Pre_dir "https://github.com/jupyter/nbconvert/tree/main/nbconvert/preprocessors/"
    click Pre_base "https://github.com/jupyter/nbconvert/blob/main/nbconvert/preprocessors/base.py"
    click Exp_api "https://github.com/jupyter/nbconvert/blob/main/nbconvert/exporters/exporter.py"
    click Temp_exp "https://github.com/jupyter/nbconvert/blob/main/nbconvert/exporters/templateexporter.py"
    click Filters "https://github.com/jupyter/nbconvert/tree/main/nbconvert/filters/"
    click Post_dir "https://github.com/jupyter/nbconvert/tree/main/nbconvert/postprocessors/"
    click Post_base "https://github.com/jupyter/nbconvert/blob/main/nbconvert/postprocessors/base.py"
    click Writers_dir "https://github.com/jupyter/nbconvert/tree/main/nbconvert/writers/"
    click Files "https://github.com/jupyter/nbconvert/blob/main/nbconvert/writers/files.py"
    click Stdout "https://github.com/jupyter/nbconvert/blob/main/nbconvert/writers/stdout.py"
    click Builtin "https://github.com/jupyter/nbconvert/tree/main/nbconvert/templates/"
    click Shipped "https://github.com/jupyter/nbconvert/tree/main/share/templates/"
    click Pandoc_wrapper "https://github.com/jupyter/nbconvert/blob/main/nbconvert/utils/pandoc.py"
    click Pandoc_filter "https://github.com/jupyter/nbconvert/blob/main/nbconvert/filters/pandoc.py"

    %% Styles
    classDef core fill:#BBE1FA,stroke:#333,stroke-width:1px
    classDef store fill:#C3FDB8,stroke:#333,stroke-width:1px
    classDef external fill:#FFCC99,stroke:#333,stroke-width:1px
```
