How kaarut/testpilot works
Project type and purpose
• This is a GitHub‐integrated code review bot (ChatGPT-powered) that can be used either as:
• A GitHub Action (runs on GitHub’s hosted runners)
• A GitHub App (Probot) self-hosted on AWS Lambda or in a Docker/PM2 container
• Purpose: Automatically review pull request diffs, generate review comments via OpenAI (or GitHub/Azure LLMs), and post results back to the PR.
File structure
• Top-level deployment configs:
• .github/workflows/cr.yml – GitHub Action workflow
• serverless.yml + src/aws-lambda.cjs – AWS Lambda (Probot App) deployment
• Dockerfile + pm2.config.cjs – Docker/PM2 self-hosting
• “action/” directory – bundled Action code (rollup output), includes:
• index.cjs, github-action.js – entry points
• worker*.js, middleware, vocab.bpe, _tiktoken_bg.wasm – tokenization pipeline
• “src/” directory – TypeScript source for core bot logic:
• bot.ts, chat.ts, log.ts, github-action.cjs, aws-lambda.cjs
• “api/github/webhooks/index.ts” – optional webhook handler for a custom API
• Test suite and fixtures in “test/”
• Config files: package.json, tsconfig.json, rollup.config.ts, jest.config.js
README insights
• Explains two integration modes: GitHub Action vs. self-hosted Probot App
• Describes environment variables (OPENAI_API_KEY, MODEL, USE_GITHUB_MODELS, etc.)
• Deployment instructions for AWS Lambda (via serverless) and Docker/PM2
Main components and interactions
A. GitHub Platform
– Triggers: pull_request events (opened, reopened, synchronize)
– Destination: either a workflow that runs the Action or a webhook delivery to the Probot App
B. GitHub Actions Runner (if using Action)
– Checks out code, loads anc95/ChatGPT-CodeReview@main, and invokes the Action container
C. Action Container
– Node.js process running bundled code in “action/”
– Parses PR diff, passes it through the tokenization pipeline (vocab.bpe + WASM)
– Calls Bot Core Logic
D. Bot Core Logic
– chat.ts builds the prompt, selects model endpoint (OpenAI, Azure, or GitHub models)
– bot.ts orchestrates API call and formats the response
– log.ts records metrics/errors
– middleware.ts handles multipart parsing if needed
E. LLM Endpoint (external)
– OpenAI API or GitHub models endpoint, returning review suggestions
F. GitHub API
– Posts comments to the PR timeline or file-diff annotations
G. Alternate self-host mode (Probot App)
– An HTTP server (src/index.ts) listens for webhooks
– Deployed via AWS Lambda (serverless.yml) or Docker (Dockerfile + PM2)
– Same Bot Core Logic invoked inside the Lambda handler or Express app
Architectural patterns and design principles
• Separation of concerns: tokenization, prompt construction, API invocation, and logging are modular
• Polyglot deployment: single codebase works as Action or Probot App
• Serverless-first with fallback to containerized hosting
• Use of WASM for efficient on-the-fly token encoding