How myaggression/flaky-guard works
1. Overall shape
- Flaky Guard is a Next.js + PostgreSQL application centered on GitHub App webhook ingestion.
- The system has two main faces: a server-side ingestion/analytics pipeline and a read-only dashboard/API layer for viewing repository health, test insights, regressions, and risk.
2. Primary runtime boundary
- The repo is mostly the web/ app, which contains the Next.js server, API routes, pages, and all processing logic.
- Persistence is through Prisma to PostgreSQL using the schema in prisma/schema.prisma and migrations under prisma/migrations.
3. Event ingestion path
- The key entrypoint is POST /webhooks/github.
- GitHub sends workflow_run events when Actions runs complete; the handler verifies the webhook signature, rejects oversized payloads, and then starts ingestion.
- This is the main boundary where external GitHub activity enters the system.
4. Workflow-run processing pipeline
- On a completed run, the app upserts the Repository and WorkflowRun records.
- If the run is tied to a pull request, it resolves and stores the PR number.
- It then downloads workflow artifacts from GitHub, unpacks them, and parses JUnit XML to persist TestRun and TestCaseResult data.
5. Analytics engines
- After raw test data lands in the database, the system recomputes derived intelligence using several internal engines in web/src/lib/:
- JUnit parsing for test result extraction
- Flaky detection for pass/fail switching behavior
- Regression detection for new broken/flaky/performance issues
- Insight engine for classifying tests into categories like broken, flaky, timing unstable, risky, stable
- Risk engine for repo-level risk snapshots and trends
- These are server-side computation modules, not client-side logic.
6. PR comment automation
- If the webhook originated from a pull request workflow run, the app builds a summary comment containing workflow status, regressions, and flaky tests.
- It upserts that comment back to GitHub using a stable marker (<!-- flaky-guard -->) so repeated runs update the same PR comment instead of creating duplicates.
7. Read model / dashboard APIs
- The UI does not read raw tables directly; it uses API routes under web/src/app/api/.
- Main endpoints are:
- /api/health for service/DB health
- /api/health/:owner/:repo for the repository dashboard payload
- /api/health/:owner/:repo/tests for test-level flaky data
- /api/insights/:owner/:repo
- /api/regressions/:owner/:repo
- /api/risk/:owner/:repo
- /api/repositories for multi-repo listing
- This split creates a clear boundary between stored operational data and the API-shaped data used by the UI.
8. UI surfaces
- The dashboard is delivered by Next.js pages under web/src/app/, including:
- /:owner/:repo for the repository view
- /repos for the multi-repo listing
- These pages are likely server-rendered or data-fed via the API layer, keeping the presentation layer thin compared with the backend analytics.
9. Data model
- The schema and migrations indicate a history-oriented model built around:
- repositories
- workflow runs and attempts
- test runs
- individual test case results
- test case insights
- regression records
- risk snapshots
- This supports time-series style analytics such as trends, last-48h regressions, and per-branch/workflow comparisons.
10. External integrations
- The only major external service is GitHub:
- GitHub App installation permissions
- Webhook delivery
- Artifact downloads
- PR comment updates
- Repository/workflow metadata
- The database is the other core dependency; there is no separate message queue or worker service in this repo.
11. Operational/deployment model
- The app can run locally or in Docker, with docker-compose.yml providing the app and Postgres.
- Health checks are exposed for orchestration, and the readme emphasizes running Prisma migrations before startup.
- This suggests a single deployable Node/Next.js service with database-backed state, rather than a distributed microservice architecture.
12. Important architectural boundary
- There is no user login layer in the current design.
- Access is repo-scoped and based on GitHub App installation plus webhook-fed data, so the security boundary is primarily GitHub permissions + webhook verification + database isolation by repository.
Main entry points: schema.prisma Prisma schema · route.ts GitHub webhook · page.tsx Repo page · junit-parser.ts JUnit parser
Components 23
Next.js app
Single deployable web/runtime boundary containing routes, pages, and server-side processing.
- GitHub webhook · Ingress for completed workflow_run events; verifies signature and starts ingestion.
web/src/app/webhooks/github/route.ts - Server core · Main server bootstrap and request handling boundary for the Next.js app.
web/src/server.ts - GitHub client · Wrapper for GitHub API calls, artifact download, and PR comment updates.
web/src/github.ts - Artifact processor · Downloads and unpacks workflow artifacts before parsing test outputs.
web/src/lib/artifact-processor.ts - JUnit parser · Extracts structured test cases from JUnit XML.
web/src/lib/junit-parser.ts - Flaky detector · Detects pass/fail switching and other flaky behavior.
web/src/lib/test-flaky.ts - Regression detector · Identifies new broken, flaky, or performance regressions.
web/src/lib/regression-detector.ts - Insight engine · Classifies tests into broken, flaky, timing-unstable, risky, or stable.
web/src/lib/insight-engine.ts - Risk engine · Computes repository-level risk snapshots and trends.
web/src/lib/risk-engine.ts - PR comment · Builds and upserts the stable-marker pull request summary comment.
web/src/lib/pr-comment.ts - Health API · Service and database health endpoint.
web/src/app/api/health/route.ts - Repo health API · Repository dashboard payload endpoint.
web/src/app/api/health/[owner]/[repo]/route.ts - Tests API · Test-level flaky data endpoint for a repository.
web/src/app/api/health/[owner]/[repo]/tests/route.ts - Insights API · Repository insight summaries for the dashboard.
web/src/app/api/insights/[owner]/[repo]/route.ts - Regressions API · Regression summaries and recent issue feeds.
web/src/app/api/regressions/[owner]/[repo]/route.ts - Risk API · Repository risk snapshot and trend endpoint.
web/src/app/api/risk/[owner]/[repo]/route.ts - Repos API · Multi-repository listing endpoint.
web/src/app/api/repositories/route.ts - Repo page · Repository dashboard surface for a single owner/repo.
web/src/app/[owner]/[repo]/page.tsx - Repos page · Multi-repository listing surface.
web/src/app/repos/page.tsx
Persistence
Prisma schema, migrations, and database access layer.
- Prisma schema · Database model for repositories, runs, tests, insights, regressions, and risk.
prisma/schema.prisma - Migrations · Schema evolution history for the PostgreSQL-backed data model.
prisma/migrations
External systems
GitHub and PostgreSQL dependencies outside the repo.
- PostgreSQL · Primary persistence store for raw and derived repository health data.
- GitHub · Webhook source, artifact host, and pull request target.
Connections 27
- GitHub webhook → Server core: handled by · Webhook requests enter the Next.js server runtime.
- GitHub webhook → GitHub: delivered from · GitHub sends workflow_run events.
- GitHub webhook → GitHub client: calls · Uses GitHub API for metadata and artifact access.
- GitHub webhook → Artifact processor: ingests · Completed runs trigger artifact retrieval and unpacking.
- Artifact processor → JUnit parser: parses with · Extracted artifacts are turned into structured test records.
- JUnit parser → Flaky detector: feeds · Parsed test outcomes are analyzed for flakiness.
- JUnit parser → Regression detector: feeds · Test results drive regression detection.
- Flaky detector → Insight engine: categorizes into · Flaky signals contribute to the insight classifications.
- Regression detector → Insight engine: categorizes into · Regression signals contribute to the insight classifications.
- JUnit parser → Risk engine: feeds · Historical test data informs risk snapshots and trends.
- Insight engine → Risk engine: feeds · Insight classifications contribute to repository risk.
- GitHub webhook → PR comment: may write · Pull request runs trigger summary comment updates.
- PR comment → GitHub client: upserts via · Uses a stable marker to update the same PR comment.
- Repo health API → Prisma schema: reads · Repository dashboard data is shaped from persisted records.
- Tests API → Prisma schema: reads · Test-level API reads database state.
- Insights API → Prisma schema: reads · Insight API reads computed classification data.
- Regressions API → Prisma schema: reads · Regression API reads regression records.
- Risk API → Prisma schema: reads · Risk API reads repository risk snapshots.
- Repos API → Prisma schema: reads · Listing endpoint reads repository inventory.
- Repo page → Repo health API: loads · Repository page consumes the dashboard payload endpoint.
- Repo page → Tests API: loads · Repository page fetches test-level detail.
- Repo page → Insights API: loads · Repository page fetches insights.
- Repo page → Regressions API: loads · Repository page fetches regressions.
- Repo page → Risk API: loads · Repository page fetches risk data.
- Repos page → Repos API: loads · Repos page consumes the multi-repo listing.
- Prisma schema → PostgreSQL: persists to · Prisma maps the schema to PostgreSQL.
- Migrations → Prisma schema: evolves · Migrations track schema changes.