How allenai/infinigram-api works
Project type and purpose
• This is a microservice‐based, full‑stack “API + worker” application wrapping the Infini‑gram language model for high‑scale, asynchronous inference.
• The top‑level services are:
• API (FastAPI/Uvicorn)
• Attribution Worker (Python worker)
• Shared library (infini‑gram‑processor)
• Web Proxy (Nginx)
• Queue (Redis)
• Infini‑gram engine (external binary/service)
• Index storage (S3/Elasticsearch or custom “indexes”)
File structure
• “api/”: REST API, FastAPI entrypoint (app.py), routers/services for attribution and documents, Redis cache, config.
• “attribution_worker/”: standalone worker that pulls from the same Redis queue, processes “attribution” jobs, writes back.
• “packages/infini‑gram‑processor/”: Python library shared by API and worker to invoke the Infini‑gram engine, handle tokenization, models, configs.
• “proxy/”: Nginx configs (local, prod) to reverse‑proxy external traffic to the API.
• “.skiff/” & “.devcontainer/”: Kubernetes/Skiff and local dev container deployment definitions.
• “load‑test/”, “compute_stats/”, “indexing/”: auxiliary scripts and jobs for performance testing, statistics, and index management (not core runtime).
• “otel-collector/”: OpenTelemetry collector config for tracing/metrics.
README for architecture insights
API ⇄ Queue ⇄ Worker
API & Worker → Infini‑gram → Indexes
Proxy → API
• Deployment note: API and worker are separate k8s deployments, independently scalable.
Elements
• Clear labels: “API”, “Worker”, “Redis Queue”, “Infini‑gram Engine”, “Indexes”, “Proxy”, “otel-collector”.
• Rounded rectangles for services (API, Worker)