How flaggigbo/llms-txt-ogechukwu-flagg-igbo works
This is an llms.txt directory: users search and browse generated entries or submit a website, which is queued for scraping, generation, validation, and storage before the API serves its files. A separate cadence service checks for site changes and schedules refreshes. The graph includes the major README capabilities and sampled runtime modules; some subsystem links are documented rather than visible in excerpts, so those are dashed where needed.
Main entry points: generateConsumer.ts Generation consumer · index.ts PostgreSQL database · App.tsx Tabbed application · entries.ts Entry endpoints
Components 28
Directory web app
- Tabbed application
apps/web/src/App.tsx - Search and browse
apps/web/src/pages/HomePage.tsx - Search input
apps/web/src/components/SearchBar.tsx - Directory table
apps/web/src/components/DirectoryTable.tsx - Pagination controls
apps/web/src/components/PaginationControls.tsx - Add entry page
apps/web/src/pages/AddEntryPage.tsx - Submission form
apps/web/src/components/SubmissionForm.tsx - Updates page
apps/web/src/pages/UpdatesPage.tsx - Entry API hook
apps/web/src/hooks/useEntries.ts
HTTP API
- HTTP gateway
apps/api/src/index.ts - Entry endpoints
apps/api/src/routes/entries.ts - Search endpoint
apps/api/src/routes/search.ts - Search service
apps/api/src/services/searchService.ts - Public URL validation
apps/api/src/services/urlValidator.ts - File serving
apps/api/src/services/storageService.ts
Generation pipeline
- Generation consumer
apps/worker/src/consumers/generateConsumer.ts - Website scraper
apps/worker/src/services/scraper.ts - LLM file generator
apps/worker/src/services/llmsGenerator.ts - Standard validator
apps/worker/src/services/llmsValidator.ts - Generated file upload
apps/worker/src/services/storageUpload.ts
Scheduling and queue
- Cadence updater
apps/cron-updater/src/jobs/checkStaleSites.ts - Generation queue
packages/queue/index.ts
Persistence and storage
- PostgreSQL database
packages/database/index.ts - SQL query definitions
packages/database/queries.ts
Other components
- Directory visitor
- R2 or MinIO
- Google Gemini
- Submitted websites
Connections 34
- Directory visitor → Tabbed application: uses
- Tabbed application → Search and browse: shows
apps/web/src/App.tsx - Tabbed application → Add entry page: shows
apps/web/src/App.tsx - Tabbed application → Updates page: shows
apps/web/src/App.tsx - Tabbed application → Entry API hook: loads entries
apps/web/src/App.tsx - Search and browse → Search input: renders
apps/web/src/pages/HomePage.tsx - Search and browse → Directory table: renders
apps/web/src/pages/HomePage.tsx - Search and browse → Pagination controls: renders
apps/web/src/pages/HomePage.tsx - Add entry page → Submission form: renders
apps/web/src/pages/AddEntryPage.tsx - Updates page → Directory table: renders
apps/web/src/pages/UpdatesPage.tsx - Entry API hook → HTTP gateway: fetches entries
apps/web/src/hooks/useEntries.ts - HTTP gateway → Entry endpoints: routes entries
apps/api/src/index.ts - HTTP gateway → Search endpoint: routes search
apps/api/src/index.ts - Entry endpoints → Public URL validation: validates submissions
apps/api/src/routes/entries.ts - Entry endpoints → PostgreSQL database: reads and writes
apps/api/src/routes/entries.ts - Entry endpoints → Generation queue: enqueues generation
apps/api/src/routes/entries.ts - Search endpoint → Search service: delegates search
apps/api/src/routes/search.ts - Search endpoint → Entry endpoints: formats file URLs
apps/api/src/routes/search.ts - Search service → PostgreSQL database: searches entries
apps/api/src/services/searchService.ts - HTTP gateway → File serving: serves stored files
apps/api/src/index.ts - File serving → R2 or MinIO: fetches file content
- Generation queue → PostgreSQL database: polls jobs
packages/queue/index.ts - Generation consumer → Generation queue: processes jobs
apps/worker/src/consumers/generateConsumer.ts - Generation consumer → PostgreSQL database: updates entries
apps/worker/src/consumers/generateConsumer.ts - Generation consumer → Website scraper: scrapes site
apps/worker/src/consumers/generateConsumer.ts - Website scraper → Submitted websites: crawls pages
- Generation consumer → LLM file generator: generates files
apps/worker/src/consumers/generateConsumer.ts - LLM file generator → Google Gemini: categorizes links
- Generation consumer → Standard validator: validates output
apps/worker/src/consumers/generateConsumer.ts - Generation consumer → Generated file upload: uploads files
apps/worker/src/consumers/generateConsumer.ts - Generated file upload → R2 or MinIO: writes objects
- Cadence updater → PostgreSQL database: claims due sites
apps/cron-updater/src/jobs/checkStaleSites.ts - Cadence updater → Generation queue: schedules refreshes
apps/cron-updater/src/jobs/checkStaleSites.ts - PostgreSQL database → SQL query definitions: uses SQL builders
packages/database/index.ts