# startrail-org/pixelrag architecture

> Architecture diagram of the GitHub repository startrail-org/pixelrag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/startrail-org/pixelrag
- Repository: https://github.com/startrail-org/pixelrag

## Overview

1. **Repo shape: a multi-package monorepo**
   - This repository is organized around several separately installable Python packages plus one Next.js frontend.
   - The main Python packages are `pixelrag_render`, `pixelrag_index`, `pixelrag_embed`, `pixelrag_serve`, and the top-level `pixelrag` CLI wrapper in `src/pixelrag`.
   - `train/` is intentionally a separate Python project with its own environment and dependency pinning; it is not part of the runtime pipeline.

2. **Core end-to-end pipeline**
   - The primary data flow is: **document source → rendered screenshot tiles → chunks/shards → embeddings → FAISS index → search API**.
   - The README and directory layout make this explicit: `render` produces images, `embed` turns images into vectors, `index` assembles/searches the index, and `serve` exposes it over HTTP.
   - This is the main architecture to draw: rendering and retrieval are separate stages with clean handoffs via files/artifacts on disk.

3. **Rendering subsystem**
   - `render/src/pixelrag_render` is the screenshot engine behind `pixelshot`.
   - It supports multiple backends: Chromium/CDP (`backends/cdp.py`, `fast_cdp.py`), PDF rendering (`backends/pdf.py`), and many strategy implementations under `strategies/`.
   - The renderer is built for high-throughput tile generation, with special attention to Chrome attachment, page tiling, and performance benchmarking.
   - `render/chrome-build/` and the Chromium patch files indicate the project can ship or patch a custom headless browser build for Linux.

4. **Index ingestion subsystem**
   - `index/src/pixelrag_index` orchestrates turning a source into an index.
   - Sources are abstracted in `sources/base.py` with concrete adapters for `local`, `pdf`, `web`, and `kiwix`.
   - This is the boundary where “what to index” is normalized into a common ingest pipeline, so the architecture supports local folders, PDFs, web content, and offline Kiwix archives.

5. **Embedding subsystem**
   - `embed/src/pixelrag_embed` is responsible for converting rendered tiles into vector embeddings.
   - The split between `embed.py`, `embed_cpu.py`, `chunk.py`, and `index.py` suggests separate concerns for chunking tiles, embedding execution, and index assembly.
   - In the product story, this layer is where screenshot images become the semantic retrieval space, and it is the bridge from visual assets to searchable vectors.

6. **Serving subsystem**
   - `serve/src/pixelrag_serve` provides the HTTP search layer used by the hosted API and local installs.
   - `api.py` is the main request surface, while `render_ondemand.py` and the ZIM server modules show that serving can also trigger rendering or stream archived web content.
   - The serving boundary is intentionally independent of training and browser rendering internals: it consumes built indexes and returns ranked documents/tiles over HTTP.

7. **User-facing interfaces**
   - The root `src/pixelrag/cli.py` exposes the umbrella CLI (`pixelrag chunk`, `embed`, `build-index`, `index`, `serve`), while `pixelshot` is the rendering-focused entrypoint.
   - `web/` is a separate Next.js app for the public site and interactive demo; it calls the backend API from the browser/server-side app routes.
   - `plugin/` and `skill/` add a Claude Code integration, where the `pixelbrowse` skill uses local `pixelshot` to give the agent visual access without an MCP backend.

8. **Training is a distinct boundary**
   - `train/` is a standalone project for LoRA fine-tuning the embedding model, with its own `pyproject.toml`, lockfile, and experiment scripts.
   - It handles dataset preparation, filtering, hard-negative mining, evaluation, and model training/export.
   - The runtime architecture depends on trained weights/adapters produced here, but not on training code at inference time.

9. **Evaluation and reproduction**
   - `eval/` contains benchmark runners, graders, retrieval helpers, and reproduction scripts for the paper’s experiments.
   - This layer sits beside the product pipeline rather than inside it: it consumes rendered tiles, embeddings, and APIs to measure retrieval/QA quality.
   - `assets/`, `docs/reproducing_paper.md`, and `eval/REPRODUCE.md` support reproducibility and paper-aligned comparisons.

10. **Deployment and operations**
   - `deploy/` and `.github/workflows/` cover packaging, CI, release, and production rollout.
   - The deploy artifacts include systemd units and an Nginx upstream config, which implies the API is run as a managed service behind a reverse proxy.
   - `history/` stores environment or rollout snapshots, suggesting operational tracking of service components like API, website, tile serving, and agent integration.

11. **External services and hosted assets**
   - The README references a hosted search API at `api.pixelrag.ai`, a website at `pixelrag.ai`, and published datasets/adapters on Hugging Face.
   - Those are not required for local architecture, but they matter because the repo supports both self-hosted indexing and a prebuilt public Wikipedia index.
   - The public service likely uses the same `serve` package and a precomputed FAISS index stored under `index/` artifacts.

12. **How to diagram the system**
   - Draw four main runtime boxes: **Render**, **Embed/Index**, **Serve/Search**, and **UI/Agent clients**.
   - Add a separate **Training** box feeding model adapters into Embed/Serve, and a separate **Deployment/CI** box controlling packaging and release.
   - The key arrows are:
     - documents/URLs/PDFs into Render,
     - screenshots/tile shards into Embed,
     - vectors into Index,
     - index into Serve,
     - Serve into Web/API clients,
     - and local `pixelshot` directly into the Claude skill.

Main entry points: [`serve/src/pixelrag_serve/api.py`](https://github.com/startrail-org/pixelrag/blob/main/serve/src/pixelrag_serve/api.py) (Search API), [`render/src/pixelrag_render/render.py`](https://github.com/startrail-org/pixelrag/blob/main/render/src/pixelrag_render/render.py) (Render core), [`embed/src/pixelrag_embed/embed.py`](https://github.com/startrail-org/pixelrag/blob/main/embed/src/pixelrag_embed/embed.py) (Embed runner), [`index/src/pixelrag_index/pipelines.py`](https://github.com/startrail-org/pixelrag/blob/main/index/src/pixelrag_index/pipelines.py) (Index pipeline)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime_render["Render"]
  node_render_core["Render core<br/>screenshot engine<br/>[render.py]"]
  node_cdp_backend["CDP backend<br/>browser capture<br/>[cdp.py]"]
  node_pdf_backend["PDF backend<br/>document render<br/>[pdf.py]"]
  node_render_strategies{{"Tile strategies<br/>render strategies<br/>[base.py]"}}
  node_chrome_build["Chrome build<br/>patched chromium"]
end

subgraph group_runtime_index["Embed & Index"]
  node_source_adapters["Source adapters<br/>ingest sources<br/>[base.py]"]
  node_pipeline_orch["Index pipeline<br/>orchestration<br/>[pipelines.py]"]
  node_chunker["Chunking<br/>tile shard split<br/>[chunk.py]"]
  node_embed_exec["Embed runner<br/>embedding execution<br/>[embed.py]"]
  node_embed_cpu["CPU embed<br/>local embed path<br/>[embed_cpu.py]"]
  node_index_assembly[("Index assembly<br/>FAISS index<br/>[index.py]")]
end

subgraph group_runtime_serve["Serve & Search"]
  node_search_api["Search API<br/>HTTP service<br/>[api.py]"]
  node_ondemand_render["On-demand render<br/>serve helper<br/>[render_ondemand.py]"]
  node_zim_server["ZIM server<br/>archive serving<br/>[zim_server.py]"]
end

subgraph group_clients_ui["Clients & UI"]
  node_cli["pixelrag CLI<br/>umbrella command<br/>[cli.py]"]
  node_pixelshot["pixelshot<br/>render entrypoint<br/>[cli.py]"]
  node_web_app["Web app<br/>Next.js frontend<br/>[page.tsx]"]
  node_agent_skill["Agent skill<br/>Claude skill<br/>[SKILL.md]"]
end

subgraph group_training["Training"]
  node_trainer["Training stack<br/>standalone project<br/>[train.py]"]
end

subgraph group_ops["Deploy & CI"]
  node_deploy_ops["Deploy & CI<br/>release ops<br/>[deploy.sh]"]
end

node_cli -->|"build/index"| node_pipeline_orch
node_cli -->|"serve"| node_search_api
node_pixelshot -->|"capture"| node_render_core
node_agent_skill -->|"uses local"| node_pixelshot
node_render_core -->|"web capture"| node_cdp_backend
node_render_core -->|"pdf capture"| node_pdf_backend
node_render_core -->|"tiles"| node_render_strategies
node_cdp_backend -->|"browser"| node_chrome_build
node_source_adapters -->|"normalize"| node_pipeline_orch
node_pipeline_orch -->|"shard"| node_chunker
node_chunker -->|"tiles"| node_embed_exec
node_embed_exec -->|"cpu path"| node_embed_cpu
node_embed_exec -->|"vectors"| node_index_assembly
node_index_assembly -->|"index"| node_search_api
node_search_api -->|"fallback render"| node_ondemand_render
node_search_api -->|"archive access"| node_zim_server
node_web_app -->|"queries"| node_search_api
node_trainer -.->|"adapters"| node_embed_exec
node_trainer -.->|"model assets"| node_search_api
node_deploy_ops -->|"rollout"| node_search_api
node_deploy_ops -->|"publish"| node_web_app

click node_cli "https://github.com/startrail-org/pixelrag/blob/main/src/pixelrag/cli.py"
click node_pixelshot "https://github.com/startrail-org/pixelrag/blob/main/src/pixelrag/cli.py"
click node_web_app "https://github.com/startrail-org/pixelrag/blob/main/web/app/page.tsx"
click node_agent_skill "https://github.com/startrail-org/pixelrag/blob/main/plugin/skills/pixelbrowse/SKILL.md"
click node_render_core "https://github.com/startrail-org/pixelrag/blob/main/render/src/pixelrag_render/render.py"
click node_cdp_backend "https://github.com/startrail-org/pixelrag/blob/main/render/src/pixelrag_render/backends/cdp.py"
click node_pdf_backend "https://github.com/startrail-org/pixelrag/blob/main/render/src/pixelrag_render/backends/pdf.py"
click node_render_strategies "https://github.com/startrail-org/pixelrag/blob/main/render/src/pixelrag_render/strategies/base.py"
click node_chrome_build "https://github.com/startrail-org/pixelrag/blob/main/render/chrome-build/pixelrag-chrome.patch"
click node_source_adapters "https://github.com/startrail-org/pixelrag/blob/main/index/src/pixelrag_index/sources/base.py"
click node_pipeline_orch "https://github.com/startrail-org/pixelrag/blob/main/index/src/pixelrag_index/pipelines.py"
click node_chunker "https://github.com/startrail-org/pixelrag/blob/main/embed/src/pixelrag_embed/chunk.py"
click node_embed_exec "https://github.com/startrail-org/pixelrag/blob/main/embed/src/pixelrag_embed/embed.py"
click node_embed_cpu "https://github.com/startrail-org/pixelrag/blob/main/embed/src/pixelrag_embed/embed_cpu.py"
click node_index_assembly "https://github.com/startrail-org/pixelrag/blob/main/embed/src/pixelrag_embed/index.py"
click node_search_api "https://github.com/startrail-org/pixelrag/blob/main/serve/src/pixelrag_serve/api.py"
click node_ondemand_render "https://github.com/startrail-org/pixelrag/blob/main/serve/src/pixelrag_serve/render_ondemand.py"
click node_zim_server "https://github.com/startrail-org/pixelrag/blob/main/serve/src/pixelrag_serve/zim_server.py"
click node_trainer "https://github.com/startrail-org/pixelrag/blob/main/train/train.py"
click node_deploy_ops "https://github.com/startrail-org/pixelrag/blob/main/deploy/deploy.sh"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_render_core,node_cdp_backend,node_pdf_backend,node_render_strategies,node_chrome_build toneBlue
class node_source_adapters,node_pipeline_orch,node_chunker,node_embed_exec,node_embed_cpu,node_index_assembly toneAmber
class node_search_api,node_ondemand_render,node_zim_server toneMint
class node_cli,node_pixelshot,node_web_app,node_agent_skill toneRose
class node_trainer toneIndigo
class node_deploy_ops toneTeal
```

## Components

### Render

Screenshot generation and browser/PDF capture pipeline.

- [Render core](https://github.com/startrail-org/pixelrag/blob/main/render/src/pixelrag_render/render.py): screenshot engine · Main rendering orchestration and capture workflow. · `render/src/pixelrag_render/render.py`
- [CDP backend](https://github.com/startrail-org/pixelrag/blob/main/render/src/pixelrag_render/backends/cdp.py): browser capture · Chromium/CDP-based rendering backend. · `render/src/pixelrag_render/backends/cdp.py`
- [PDF backend](https://github.com/startrail-org/pixelrag/blob/main/render/src/pixelrag_render/backends/pdf.py): document render · PDF rendering backend for non-web sources. · `render/src/pixelrag_render/backends/pdf.py`
- [Tile strategies](https://github.com/startrail-org/pixelrag/blob/main/render/src/pixelrag_render/strategies/base.py): render strategies · High-throughput page tiling and capture strategies. · `render/src/pixelrag_render/strategies/base.py`
- [Chrome build](https://github.com/startrail-org/pixelrag/blob/main/render/chrome-build/pixelrag-chrome.patch): patched chromium · Custom headless browser build and patching artifacts. · `render/chrome-build/pixelrag-chrome.patch`

### Embed & Index

Chunking, embedding, and FAISS index assembly.

- [Source adapters](https://github.com/startrail-org/pixelrag/blob/main/index/src/pixelrag_index/sources/base.py): ingest sources · Normalization boundary for local, PDF, web, and Kiwix inputs. · `index/src/pixelrag_index/sources/base.py`
- [Index pipeline](https://github.com/startrail-org/pixelrag/blob/main/index/src/pixelrag_index/pipelines.py): orchestration · Coordinates ingest, chunking, and index build/search flows. · `index/src/pixelrag_index/pipelines.py`
- [Chunking](https://github.com/startrail-org/pixelrag/blob/main/embed/src/pixelrag_embed/chunk.py): tile shard split · Splits rendered tiles into retrieval-ready chunks. · `embed/src/pixelrag_embed/chunk.py`
- [Embed runner](https://github.com/startrail-org/pixelrag/blob/main/embed/src/pixelrag_embed/embed.py): embedding execution · Converts chunks or tiles into vector embeddings. · `embed/src/pixelrag_embed/embed.py`
- [CPU embed](https://github.com/startrail-org/pixelrag/blob/main/embed/src/pixelrag_embed/embed_cpu.py): local embed path · CPU-specific embedding implementation. · `embed/src/pixelrag_embed/embed_cpu.py`
- [Index assembly](https://github.com/startrail-org/pixelrag/blob/main/embed/src/pixelrag_embed/index.py): FAISS index · Builds and persists the searchable vector index. · `embed/src/pixelrag_embed/index.py`

### Serve & Search

HTTP search API and on-demand rendering/archival serving.

- [Search API](https://github.com/startrail-org/pixelrag/blob/main/serve/src/pixelrag_serve/api.py): HTTP service · Main request surface for retrieval and ranked results. · `serve/src/pixelrag_serve/api.py`
- [On-demand render](https://github.com/startrail-org/pixelrag/blob/main/serve/src/pixelrag_serve/render_ondemand.py): serve helper · Triggers rendering during serving when needed. · `serve/src/pixelrag_serve/render_ondemand.py`
- [ZIM server](https://github.com/startrail-org/pixelrag/blob/main/serve/src/pixelrag_serve/zim_server.py): archive serving · Offline archive serving for ZIM content. · `serve/src/pixelrag_serve/zim_server.py`

### Clients & UI

CLI, web app, and agent-facing entrypoints.

- [pixelrag CLI](https://github.com/startrail-org/pixelrag/blob/main/src/pixelrag/cli.py): umbrella command · Top-level command wrapper for chunking, embedding, indexing, and serving. · `src/pixelrag/cli.py`
- [pixelshot](https://github.com/startrail-org/pixelrag/blob/main/src/pixelrag/cli.py): render entrypoint · Rendering-focused entrypoint used for visual capture workflows. · `src/pixelrag/cli.py`
- [Web app](https://github.com/startrail-org/pixelrag/blob/main/web/app/page.tsx): Next.js frontend · Public site and interactive search/demo interface. · `web/app/page.tsx`
- [Agent skill](https://github.com/startrail-org/pixelrag/blob/main/plugin/skills/pixelbrowse/SKILL.md): Claude skill · Local agent integration for visual browsing with pixelshot. · `plugin/skills/pixelbrowse/SKILL.md`

### Training

Standalone model fine-tuning and dataset preparation project.

- [Training stack](https://github.com/startrail-org/pixelrag/blob/main/train/train.py): standalone project · Separate fine-tuning and dataset-preparation environment. · `train/train.py`

### Deploy & CI

Packaging, release, and service rollout.

- [Deploy & CI](https://github.com/startrail-org/pixelrag/blob/main/deploy/deploy.sh): release ops · Service units, deployment scripts, and CI/release workflows. · `deploy/deploy.sh`

## Connections

- pixelrag CLI → Index pipeline: build/index · CLI drives the end-to-end ingest and indexing pipeline.
- pixelrag CLI → Search API: serve · CLI can launch the HTTP serving layer.
- pixelshot → Render core: capture · Rendering entrypoint uses the screenshot engine.
- Agent skill → pixelshot: uses local · Agent skill relies on local pixelshot for visual access.
- Render core → CDP backend: web capture · Primary browser-capture backend for rendering web content.
- Render core → PDF backend: pdf capture · Alternative backend for document-style rendering.
- Render core → Tile strategies: tiles · Strategies control page tiling and throughput behavior.
- CDP backend → Chrome build: browser · CDP backend depends on patched Chromium artifacts.
- Source adapters → Index pipeline: normalize · Source adapters feed normalized documents into the pipeline.
- Index pipeline → Chunking: shard · Pipeline breaks rendered outputs into chunks/shards.
- Chunking → Embed runner: tiles · Chunked tiles are passed to the embedding stage.
- Embed runner → CPU embed: cpu path · Embedding runner can execute through CPU-specific implementation.
- Embed runner → Index assembly: vectors · Embeddings are assembled into the searchable index.
- Index assembly → Search API: index · Search service consumes the built FAISS index artifacts.
- Search API → On-demand render: fallback render · Serve layer can trigger rendering when needed.
- Search API → ZIM server: archive access · Serve layer also exposes archived web content paths.
- Web app → Search API: queries · Frontend calls the backend search API.
- Training stack → Embed runner: adapters · Training produces weights/adapters used by embedding runtime.
- Training stack → Search API: model assets · Training outputs can influence served retrieval models.
- Deploy & CI → Search API: rollout · Deployment manages service packaging and release.
- Deploy & CI → Web app: publish · CI and deployment also cover the web frontend.