How g-cqd/apple-docs works
Purpose and entry points
apple-docs is a Bun-based, offline Apple-documentation corpus system: it installs or builds a local indexed corpus, then exposes it through CLI, MCP, and web/static-site interfaces. Process entry points are cli.js (user CLI) and index.js (library/export surface); command parsing and dispatch live in src/cli/parser.js and src/commands/index.js.
Command orchestration
Commands separate user-facing operations from domain implementations: setup/update/sync ingest data; search/lookup/browse read it; snapshot and web-deploy package or publish it. Major command modules are src/commands/setup.js, src/commands/sync.js, and src/commands/search.js. CLI formatters isolate terminal/JSON presentation from command logic in src/cli/formatter.js.
Source ingestion and corpus pipeline
Source adapters represent Apple DocC, guidelines, WWDC, Swift, package, and archive providers behind a registry: src/sources/registry.js, src/sources/base.js, src/sources/apple-docc.js. Sync coordinates staged discovery, download, conversion, coalescing, and persistence through src/pipeline/discover.js, src/pipeline/download.js, src/pipeline/convert.js, and src/pipeline/persist.js. This is the principal external-I/O boundary, with retry/rate-limit support in src/lib/fetch-with-retry.js and src/lib/per-host-rate-limiter.js.
Content normalization and rendering
Raw HTML, Markdown, DocC, and guideline material is transformed into a common document model before storage. The boundary is centered on src/content/normalize.js, with format-specific normalization in src/content/normalize/docc.js and src/content/normalize/guidelines.js; renderers project canonical content to HTML, Markdown, text, or snippets via src/content/render-html.js and src/content/render-markdown.js. Apple-specific extraction/normalization is contained in src/apple/extractor.js and src/apple/normalizer.js.
Persistent corpus and search
SQLite-oriented storage owns schema migration, profiles, document/source repositories, files, and concurrent read access: src/storage/database.js, src/storage/migrations/index.js, src/storage/repos/documents.js, and src/storage/reader-pool.js. Search combines FTS, filtering/ranking, and local semantic-vector retrieval/fusion: src/search/fts-query-builder.js, src/search/semantic.js, src/search/fusion.js, and src/search/ranking.js. Storage profiles trade disk for serving/read performance without changing corpus semantics.
Delivery surfaces
The MCP server provides read-only stdio and HTTP tool/resource access over the same corpus, with caching, pagination, health, and metrics boundaries in src/mcp/server.js, src/mcp/http-server.js, and src/mcp/tools/docs.js. The web surface can serve dynamically or build static output: src/web/serve.js, src/web/route-registry.js, and src/web/build.js; routes render stored documents and search artifacts rather than crawling sources.
Assets, snapshots, and operations
Fonts and SF Symbols are separately synchronized, transformed, stored, and rendered because they require platform/archive/PDF processing: src/resources/apple-fonts/sync.js, src/resources/apple-symbols/sync.js, and src/resources/symbol-pdf-to-svg.js. Snapshot setup/build provides the portable distribution boundary (src/commands/setup.js, src/commands/snapshot.js). Production operations use Caddy, Cloudflare tunnels/purge, launchd, and optional Prometheus/Grafana monitoring under ops/caddy/Caddyfile.tpl, ops/cloudflared/config.yml.tpl, and ops/grafana/docker-compose.yml.
Main entry points: database.js SQLite corpus · discover.js Sync pipeline · semantic.js Search engine · normalize.js Content normalization
Components 20
Entry & Commands
- CLI entry point
cli.js - Library exports
index.js - Command parser
src/cli/parser.js - Command dispatcher · Routes setup, sync, search, snapshot, and deployment operations.
- CLI formatters
src/cli/formatter.js
Ingestion Pipeline
- Source adapters · Unifies Apple DocC and other documentation providers.
src/sources/registry.js - Sync pipeline · Discovers, downloads, converts, coalesces, and persists source material.
src/pipeline/discover.js - Resilient fetch · Applies retries and per-host rate limits.
src/lib/fetch-with-retry.js - Content normalization · Normalizes DocC, guideline, HTML, and Markdown content.
src/content/normalize.js - Apple extraction
src/apple/extractor.js - Fonts & symbols sync · Processes platform archives and symbol PDFs separately from documents.
src/resources/apple-fonts/sync.js
Corpus & Search
- SQLite corpus · Owns migrations, profiles, repositories, files, and concurrent readers.
src/storage/database.js - Search engine · Combines FTS, vector retrieval, fusion, filtering, and ranking.
src/search/semantic.js - Content renderers
src/content/render-html.js
Delivery Surfaces
- MCP server · Read-only stdio and HTTP access with caching and pagination.
src/mcp/server.js - Web serving · Routes render stored documents and search artifacts.
src/web/serve.js - Static-site build
src/web/build.js
Distribution & Operations
- Snapshot distribution
src/commands/snapshot.js - Setup command · Installs or builds a local indexed corpus.
src/commands/setup.js - Production operations · Caddy, Cloudflare tunnels, launchd, and optional Grafana monitoring.
ops/caddy/Caddyfile.tpl
Connections 24
- CLI entry point → Command parser: parses
- Command parser → Command dispatcher: dispatches
- Command dispatcher → CLI formatters: formats results
- Command dispatcher → Setup command: runs
- Command dispatcher → Sync pipeline: runs
- Command dispatcher → Search engine: queries
- Command dispatcher → Snapshot distribution: packages
- Setup command → Sync pipeline: builds corpus
- Sync pipeline → Source adapters: discovers from
- Sync pipeline → Resilient fetch: downloads through
- Sync pipeline → Content normalization: converts to
- Content normalization → Apple extraction: uses
- Content normalization → SQLite corpus: persists documents
- Fonts & symbols sync → SQLite corpus: stores assets
- SQLite corpus → Search engine: supplies corpus
- SQLite corpus → Content renderers: supplies documents
- Search engine → CLI formatters: returns results
- MCP server → SQLite corpus: reads
- MCP server → Search engine: searches
- Web serving → Content renderers: renders
- Web serving → Search engine: serves search
- Static-site build → Content renderers: generates from
- Snapshot distribution → SQLite corpus: packages
- Production operations → Web serving: deploys