How context-labs/autodoc works
Purpose and Runtime
Autodoc is a Node.js (v18+) TypeScript CLI that generates, stores, and queries LLM-produced documentation for a local Git repository. Its public executable starts at src/index.ts; command implementations live under src/cli/commands. Generated documentation and retrieval artifacts are persisted inside the target repository’s .autodoc/ directory.
CLI Control Plane
The CLI exposes initialization, indexing, estimation, querying, and user/model-management flows:
• src/cli/commands/init/index.ts
• src/cli/commands/index/index.ts
• src/cli/commands/query/index.ts
init interactively collects repository metadata, model access, and prompt choices, then writes autodoc.config.json. This configuration is the boundary between user/project policy and indexing/query execution.
Repository Indexing Pipeline
Indexing is orchestrated in src/cli/commands/index/index.ts and processes the repository depth-first through src/cli/utils/traverseFileSystem.ts. The main stages are:
1. Detect repository contents and prior generated documentation.
2. Determine changed or undocumented files.
3. Read source files and calculate token requirements.
4. Select a supported LLM/model for each input.
5. Generate file and directory documentation.
6. Persist structured JSON/Markdown documentation and rebuild retrieval data.
Core per-repository work is separated into src/cli/commands/index/processRepository.ts, while prompt construction resides in src/cli/commands/index/prompts.ts.
Documentation Artifact Boundary
The indexer emits two durable representations: structured documents for machine consumption and Markdown for repository-visible documentation. Conversion is handled by src/cli/commands/index/convertJsonToMarkdown.ts. In this repository, outputs demonstrate the layout:
• .autodoc/docs/json/src/index.json
• .autodoc/docs/markdown/src/index.md
• .autodoc/docs/data/docstore.json
These artifacts are intended to be committed with code, enabling subsequent users to query a prebuilt repository index.
Cost Estimation and Model Selection
Before indexing, the estimate command calculates likely token/cost usage based on repository contents and selectable OpenAI models:
• src/cli/commands/estimate/index.ts
• src/cli/commands/index/selectModel.ts
• src/cli/utils/LLMUtil.ts
Model choice is a material quality/cost boundary: the documented behavior selects the least expensive model whose context window fits a file. API throttling and pacing are isolated in src/cli/utils/APIRateLimit.ts and src/cli/utils/WaitUtil.ts.
Retrieval and Interactive Querying
Querying loads the locally persisted vector store, retrieves relevant generated documents, and supplies them as context to an LLM chat chain:
• src/cli/commands/query/index.ts
• src/cli/commands/query/createChatChain.ts
• src/langchain/hnswlib.ts
The vector index is stored locally (for example, .autodoc/docs/data/hnswlib.index) alongside the document store. This establishes a retrieval-augmented generation boundary: source code is documented during indexing; queries generally operate on generated documentation plus vector-retrieved context rather than repeatedly scanning the full codebase.
External Dependencies and Services
The primary external service is the OpenAI API, authenticated via OPENAI_API_KEY; current README behavior supports OpenAI models only. LangChain and HNSWLib underpin embeddings/vector retrieval, with the local HNSW index managed through src/langchain/hnswlib.ts. The tool is distributed as an npm CLI package and operates directly against local Git working trees, without a web-service runtime.
Main entry points: processRepository.ts Repository processor · LLMUtil.ts LLM client · src/index.ts Autodoc CLI · index/index.ts Index command
Components 20
CLI control plane
- Autodoc CLI
src/index.ts - Init command · Collects repository, model, and prompt choices.
src/cli/commands/init/index.ts - Project config · Captures user and project indexing/query policy.
autodoc.config.json - Estimate command · Calculates likely token and cost usage before indexing.
src/cli/commands/estimate/index.ts
Indexing pipeline
- Index command
src/cli/commands/index/index.ts - Filesystem traversal · Walks the Git working tree depth-first.
src/cli/utils/traverseFileSystem.ts - Repository processor · Determines changed files and produces file and directory documentation.
src/cli/commands/index/processRepository.ts - Index prompts
src/cli/commands/index/prompts.ts - Model selection · Chooses the least expensive model that fits the input context.
src/cli/commands/index/selectModel.ts - LLM client
src/cli/utils/LLMUtil.ts - API pacing
src/cli/utils/APIRateLimit.ts - Markdown conversion
src/cli/commands/index/convertJsonToMarkdown.ts
Local artifacts
Persisted in the target repository.
- Structured docs · Machine-consumable generated documents.
.autodoc/docs/json/src/index.json - Markdown docs · Commit-ready, repository-visible documentation.
.autodoc/docs/markdown/src/index.md - Document store
.autodoc/docs/data/docstore.json - HNSW vector index
.autodoc/docs/data/hnswlib.index
Retrieval and query
- Query command
src/cli/commands/query/index.ts - RAG chat chain · Answers using retrieved generated documentation.
src/cli/commands/query/createChatChain.ts - HNSWLib adapter · Manages the local vector index.
src/langchain/hnswlib.ts
Other components
- OpenAI API · Authenticated with OPENAI_API_KEY.
Connections 28
- Autodoc CLI → Init command: runs
- Autodoc CLI → Index command: runs
- Autodoc CLI → Estimate command: runs
- Autodoc CLI → Query command: runs
- Init command → Project config: writes
- Project config → Index command: governs
- Project config → Query command: governs
- Estimate command → Filesystem traversal: scans
- Estimate command → Model selection: prices inputs
- Index command → Filesystem traversal: scans
- Index command → Repository processor: orchestrates
- Filesystem traversal → Repository processor: source files
- Repository processor → Index prompts: builds requests
- Repository processor → Model selection: selects model
- Index prompts → LLM client: LLM request
- Model selection → LLM client: selected model
- API pacing → LLM client: paces calls
- LLM client → OpenAI API: API calls
- Repository processor → Structured docs: writes
- Structured docs → Markdown conversion: converts
- Markdown conversion → Markdown docs: writes
- Structured docs → Document store: builds corpus
- Document store → HNSWLib adapter: embeds and indexes
- HNSWLib adapter → HNSW vector index: persists
- Query command → RAG chat chain: starts
- RAG chat chain → HNSWLib adapter: retrieves context
- HNSWLib adapter → HNSW vector index: loads
- RAG chat chain → LLM client: answers with context