startrail-org/pixelrag

1. Repo shape: a multi-package monorepo

- This repository is organized around several separately installable Python packages plus one Next.js frontend.

- The main Python packages are pixelrag_render, pixelrag_index, pixelrag_embed, pixelrag_serve, and the top-level pixelrag CLI wrapper in src/pixelrag.

- train/ is intentionally a separate Python project with its own environment and dependency pinning; it is not part of the runtime pipeline.

2. Core end-to-end pipeline

- The primary data flow is: document source → rendered screenshot tiles → chunks/shards → embeddings → FAISS index → search API.

- The README and directory layout make this explicit: render produces images, embed turns images into vectors, index assembles/searches the index, and serve exposes it over HTTP.

- This is the main architecture to draw: rendering and retrieval are separate stages with clean handoffs via files/artifacts on disk.

3. Rendering subsystem

- render/src/pixelrag_render is the screenshot engine behind pixelshot.

- It supports multiple backends: Chromium/CDP (backends/cdp.py, fast_cdp.py), PDF rendering (backends/pdf.py), and many strategy implementations under strategies/.

- The renderer is built for high-throughput tile generation, with special attention to Chrome attachment, page tiling, and performance benchmarking.

- render/chrome-build/ and the Chromium patch files indicate the project can ship or patch a custom headless browser build for Linux.

4. Index ingestion subsystem

- index/src/pixelrag_index orchestrates turning a source into an index.

- Sources are abstracted in sources/base.py with concrete adapters for local, pdf, web, and kiwix.

- This is the boundary where “what to index” is normalized into a common ingest pipeline, so the architecture supports local folders, PDFs, web content, and offline Kiwix archives.

5. Embedding subsystem

- embed/src/pixelrag_embed is responsible for converting rendered tiles into vector embeddings.

- The split between embed.py, embed_cpu.py, chunk.py, and index.py suggests separate concerns for chunking tiles, embedding execution, and index assembly.

- In the product story, this layer is where screenshot images become the semantic retrieval space, and it is the bridge from visual assets to searchable vectors.

6. Serving subsystem

- serve/src/pixelrag_serve provides the HTTP search layer used by the hosted API and local installs.

- api.py is the main request surface, while render_ondemand.py and the ZIM server modules show that serving can also trigger rendering or stream archived web content.

- The serving boundary is intentionally independent of training and browser rendering internals: it consumes built indexes and returns ranked documents/tiles over HTTP.

7. User-facing interfaces

- The root src/pixelrag/cli.py exposes the umbrella CLI (pixelrag chunk, embed, build-index, index, serve), while pixelshot is the rendering-focused entrypoint.

- web/ is a separate Next.js app for the public site and interactive demo; it calls the backend API from the browser/server-side app routes.

- plugin/ and skill/ add a Claude Code integration, where the pixelbrowse skill uses local pixelshot to give the agent visual access without an MCP backend.

8. Training is a distinct boundary

- train/ is a standalone project for LoRA fine-tuning the embedding model, with its own pyproject.toml, lockfile, and experiment scripts.

- It handles dataset preparation, filtering, hard-negative mining, evaluation, and model training/export.

- The runtime architecture depends on trained weights/adapters produced here, but not on training code at inference time.

9. Evaluation and reproduction

- eval/ contains benchmark runners, graders, retrieval helpers, and reproduction scripts for the paper’s experiments.

- This layer sits beside the product pipeline rather than inside it: it consumes rendered tiles, embeddings, and APIs to measure retrieval/QA quality.

- assets/, docs/reproducing_paper.md, and eval/REPRODUCE.md support reproducibility and paper-aligned comparisons.

10. Deployment and operations

- deploy/ and .github/workflows/ cover packaging, CI, release, and production rollout.

- The deploy artifacts include systemd units and an Nginx upstream config, which implies the API is run as a managed service behind a reverse proxy.

- history/ stores environment or rollout snapshots, suggesting operational tracking of service components like API, website, tile serving, and agent integration.

11. External services and hosted assets

- The README references a hosted search API at api.pixelrag.ai, a website at pixelrag.ai, and published datasets/adapters on Hugging Face.

- Those are not required for local architecture, but they matter because the repo supports both self-hosted indexing and a prebuilt public Wikipedia index.

- The public service likely uses the same serve package and a precomputed FAISS index stored under index/ artifacts.

12. How to diagram the system

- Draw four main runtime boxes: Render, Embed/Index, Serve/Search, and UI/Agent clients.

- Add a separate Training box feeding model adapters into Embed/Serve, and a separate Deployment/CI box controlling packaging and release.

- The key arrows are:

- documents/URLs/PDFs into Render,

- screenshots/tile shards into Embed,

- vectors into Index,

- index into Serve,

- Serve into Web/API clients,

- and local pixelshot directly into the Claude skill.