How yahya-debug/john_log works
John Log accepts structured logs over HTTP, validates and buffers batches, then transactionally stores raw logs and minute/hour rollups in Postgres. Operators use the dashboard to ingest, search, aggregate, tail, inspect dead letters, and view service/database statistics; retention manages daily partitions. Edges are limited to relationships evidenced by the README, excerpts, or SOURCE INDEX references; some route internals and dashboard page-to-API calls are unsampled.
Main entry points: App.tsx Dashboard shell · api.ts Dashboard API client · app.ts HTTP service · logs.ts Log storage queries
Components 26
Ingestion
- HTTP service
src/http/app.ts - Ingest endpoint
src/http/routes/logs.ts - Batch validation
src/ingestion/validate.ts - Write buffer
src/ingestion/writeBuffer.ts
Query and aggregation
- Aggregate endpoint
src/http/routes/aggregate.ts - Aggregate routing
src/query/aggregateQuery.ts - Query filters
src/query/filters.ts - Live aggregates
src/query/liveAggregate.ts - Aggregate cache
src/query/aggregateCache.ts - Log search
src/query/logsQuery.ts
Postgres and retention
- Log storage queries
src/db/logs.ts - Postgres
- Partition retention
src/retention/partitions.ts - Retention scheduler
src/retention/job.ts
Operator dashboard
- Admin endpoints
src/http/routes/admin.ts - Database statistics
src/db/stats.ts - Dashboard shell
dashboard/src/App.tsx - Dashboard API client
dashboard/src/api.ts - Ingest console
dashboard/src/pages/IngestPage.tsx - Log explorer
dashboard/src/pages/LogsPage.tsx - Aggregate charts
dashboard/src/pages/AggregatePage.tsx - Live tail
dashboard/src/pages/TailPage.tsx - Dead-letter console
dashboard/src/pages/DeadLetterPage.tsx - Stats console
dashboard/src/pages/StatsPage.tsx
Other components
- Log producers
- Operators
Connections 36
- Log producers → HTTP service: submit logs
README.md - HTTP service → Ingest endpoint: routes requests
src/http/app.ts - Ingest endpoint → Batch validation: validates batch
src/http/routes/logs.ts - Ingest endpoint → Write buffer: queues accepted logs
src/http/routes/logs.ts - Write buffer → Log storage queries: writes transaction
src/ingestion/writeBuffer.ts - Log storage queries → Postgres: reads and writes
README.md - Write buffer → Live aggregates: records committed logs
src/ingestion/writeBuffer.ts - Aggregate endpoint → Aggregate routing: runs query
src/http/routes/aggregate.ts - Aggregate routing → Live aggregates: checks live buckets
src/query/aggregateQuery.ts - Aggregate routing → Log storage queries: loads rollups or logs
src/query/aggregateQuery.ts - Aggregate routing → Query filters: builds scan conditions
src/query/aggregateQuery.ts - Aggregate routing → Aggregate cache: caches results
src/query/aggregateQuery.ts - HTTP service → Aggregate endpoint: routes requests
src/http/app.ts - HTTP service → Retention scheduler: starts schedule
src/http/app.ts - Retention scheduler → Partition retention: runs retention
src/retention/job.ts - Partition retention → Postgres: maintains partitions
src/retention/partitions.ts - HTTP service → Admin endpoints: routes requests
src/http/app.ts - Admin endpoints → Database statistics: loads statistics
src/http/routes/admin.ts - Database statistics → Postgres: queries statistics
src/db/stats.ts - Admin endpoints → Log storage queries: manages dead letters
src/http/routes/admin.ts - Operators → Dashboard shell: uses console
README.md - Dashboard shell → Ingest console: selects tab
dashboard/src/App.tsx - Dashboard shell → Log explorer: selects tab
dashboard/src/App.tsx - Dashboard shell → Aggregate charts: selects tab
dashboard/src/App.tsx - Dashboard shell → Live tail: selects tab
dashboard/src/App.tsx - Dashboard shell → Dead-letter console: selects tab
dashboard/src/App.tsx - Dashboard shell → Stats console: selects tab
dashboard/src/App.tsx - Ingest console → Dashboard API client: submits logs
dashboard/src/pages/IngestPage.tsx - Log explorer → Dashboard API client: requests logs
dashboard/src/pages/LogsPage.tsx - Aggregate charts → Dashboard API client: requests aggregates
dashboard/src/pages/AggregatePage.tsx - Live tail → Dashboard API client: requests tail
dashboard/src/pages/TailPage.tsx - Dead-letter console → Dashboard API client: manages dead letters
dashboard/src/pages/DeadLetterPage.tsx - Stats console → Dashboard API client: requests statistics
dashboard/src/pages/StatsPage.tsx - Dashboard API client → HTTP service: calls service endpoints
dashboard/src/api.ts - Ingest endpoint → Log storage queries: queries logs
- Log search → Log storage queries: queries matching logs
src/query/logsQuery.ts