How varun-satyasanjeev2008/aeroindex works
AeroIndex collects domestic flight fares, cleans observations, computes a weighted daily price index, and optionally narrates significant changes. Its main workflow is orchestrated by main.py; the REST API and optional read-only MCP server expose stored results independently. The README documents calendar matching and Groq narration, while the sampled pipeline excerpt confirms the orchestrator’s stage calls. Some subsystem details, including API history/explanation queries and MCP tools, are documented but only partially sampled.
Main entry points: main.py Pipeline runner · connection.py Database helpers · spike_explainer.py Spike explanation · fetch.py Fare scraper
Components 19
Fare collection
- Route and scrape config
config.py - Robots policy check
scraper/robots_check.py - Fare scraper
scraper/fetch.py
Index and explanation
- Pipeline runner
main.py - Fare cleaning
pipeline/clean.py - Index computation
pipeline/index.py - Spike explanation
pipeline/spike_explainer.py - Event calendar
data/calendar.json
Persistence
- Fare persistence
pipeline/store.py - Database helpers
db/connection.py - Database schema
db/schema.sql - PostgreSQL
Other components
- Pipeline operator
- Fare websites
- Groq LLM
- API consumer
- MCP client
Connections 25
- Pipeline operator → Pipeline runner: runs
README.md - Pipeline runner → Route and scrape config: loads settings
main.py - Pipeline runner → Database helpers: bootstraps database
main.py - Pipeline runner → Robots policy check: filters targets
main.py - Pipeline runner → Fare scraper: fetches fares
main.py - Robots policy check → Fare websites: checks robots policy
scraper/robots_check.py - Fare scraper → Robots policy check: checks permission
scraper/fetch.py - Fare scraper → Fare websites: scrapes fares
scraper/fetch.py - Pipeline runner → Fare cleaning: cleans observations
main.py - Pipeline runner → Fare persistence: stores cleaned fares
main.py - Pipeline runner → Index computation: computes and writes index
main.py - Pipeline runner → Spike explanation: checks for spike
main.py - Fare persistence → PostgreSQL: inserts fares
pipeline/store.py - Index computation → PostgreSQL: reads and writes index data
pipeline/index.py - Spike explanation → Event calendar: matches events
README.md - Spike explanation → Groq LLM: requests narration
README.md - Spike explanation → PostgreSQL: stores explanation
pipeline/spike_explainer.py - Database helpers → PostgreSQL: connects and applies schema
db/connection.py - Database helpers → Database schema: executes DDL
db/connection.py - API consumer → REST API: requests index data
README.md - REST API → Database helpers: opens connection
api/main.py - REST API → PostgreSQL: queries index and explanations
api/main.py - MCP client → MCP tools: invokes tools
README.md - MCP tools → Database helpers: opens connection
mcp_server/server.py - MCP tools → PostgreSQL: queries index and explanations
mcp_server/server.py