How sandboxnu/searchneu works
SearchNEU is Northeastern University’s course-information platform. Its central workflow is to browse or search course offerings, inspect sections and meeting details, and use those offerings in a schedule; the app also supports degree planning, saved schedules, and seat notifications. The data lifecycle begins with scraping Banner, validating and caching catalog data, then uploading it to PostgreSQL for application use. The graph preserves those documented capabilities, but several web-to-database and UI-to-API links are only shown as unverified because sampled route excerpts do not expose their wiring.
Main entry points: neon.ts PostgreSQL · generate/main.ts Banner scraping · route.ts Schedule APIs · upload/main.ts Catalog upload
Components 25
Catalog and scheduling
- Course catalog
apps/searchneu/app/catalog/[term]/[course]/page.tsx - Catalog APIs
apps/searchneu/app/api/catalog/courses/route.ts - Schedule generator
apps/searchneu/app/scheduler/generator/page.tsx - Schedule APIs
apps/searchneu/app/api/scheduler/generate-schedules/route.ts - Schedule generation
apps/searchneu/lib/scheduler/generateSchedules.ts
Plans and identity
- Degree plans
apps/searchneu/components/graduate/PlanClient.tsx - Audit APIs
apps/searchneu/app/api/audit/plan/route.ts - Audit utilities
apps/searchneu/lib/graduate/auditUtils.ts - Authentication
apps/searchneu/lib/auth/auth.ts
Catalog ingestion
- Banner scraping
packages/scraper/src/generate/main.ts - Fetch engine
packages/scraper/src/generate/fetch.ts - Scrape steps
packages/scraper/src/generate/steps/sections.ts - Cache validation
packages/scraper/src/schemas/scraper/banner-cache.ts - Generate command
apps/cli/src/commands/generate.ts - Upload command
apps/cli/src/commands/upload.ts - Catalog upload
packages/scraper/src/upload/main.ts - Section persistence
packages/scraper/src/upload/steps/sections.ts
Persistent data
- Database schema
packages/db/src/schema/catalog.ts - PostgreSQL
packages/db/src/neon.ts
Seat updates
- Seat trackers
apps/searchneu/app/api/trackers/route.ts - SMS notifications
apps/searchneu/lib/trackers/twilio.ts - Scheduled updates
apps/searchneu/app/api/update/route.ts - Term comparison
packages/scraper/src/update/main.ts
Other components
- Student
- Banner
Connections 25
- Student → Course catalog: browses
README.md - Student → Schedule generator: builds schedules
README.md - Course catalog → Catalog APIs: loads catalog
- Schedule generator → Schedule APIs: requests schedules
- Schedule APIs → Schedule generation: generates schedules
- Student → Degree plans: plans degree
README.md - Degree plans → Audit APIs: manages plans
- Student → Authentication: signs in
- Student → Seat trackers: tracks sections
- Scheduled updates → Term comparison: updates term
apps/searchneu/app/api/update/route.ts - Term comparison → Scrape steps: scrapes sections
packages/scraper/src/update/main.ts - Scheduled updates → SMS notifications: sends alerts
apps/searchneu/app/api/update/route.ts - Generate command → Banner scraping: runs scrape
README.md - Banner scraping → Scrape steps: scrapes sections
packages/scraper/src/generate/main.ts - Scrape steps → Fetch engine: fetches Banner
packages/scraper/src/generate/steps/sections.ts - Banner scraping → Banner: requests catalog
- Banner scraping → Cache validation: produces cache
packages/scraper/src/generate/main.ts - Upload command → Catalog upload: runs upload
apps/cli/src/commands/upload.ts - Catalog upload → Section persistence: upserts sections
packages/scraper/src/upload/main.ts - Catalog upload → PostgreSQL: writes catalog
packages/scraper/src/upload/main.ts - Section persistence → Cache validation: reads cache
packages/scraper/src/upload/steps/sections.ts - Database schema → PostgreSQL: defines tables
- Catalog APIs → PostgreSQL: reads catalog
- Schedule APIs → PostgreSQL: stores schedules
- Audit APIs → PostgreSQL: stores plans