How run-llama/sec-insights works
1. Overall shape
- This is a two-tier RAG application with a hard split between frontend/ and backend/.
- The frontend is a Next.js/React client for browsing SEC documents and chatting.
- The backend is a FastAPI service that owns retrieval, LLM orchestration, document metadata, and vector search.
2. User-facing frontend
- frontend/src/pages/ defines the main routes: the landing page, a conversation page (conversation/[id].tsx), and app-level shell files.
- frontend/src/components/ is organized by feature area: conversations, landing page, PDF viewer, modals, and basic UI primitives.
- The frontend’s primary responsibilities are document selection, rendering conversation state, showing streamed answers, and displaying PDFs with citation highlights.
3. Frontend state and data access
- frontend/src/hooks/ and frontend/src/context/ hold most of the client-side orchestration: message loading, PDF viewer state, document selection, modal state, and local-storage persistence.
- frontend/src/api/ and frontend/src/types/backend/ define the client-side boundary to the backend API.
- The frontend is not doing retrieval itself; it acts as a presentation and interaction layer over backend responses.
4. Backend API boundary
- backend/app/main.py is the service entry point, with backend/app/api/ exposing routes and dependency wiring.
- The main route groups are documents, conversations/chat, and health checks under backend/app/api/endpoints/.
- backend/app/api/crud.py and backend/app/api/deps.py suggest a conventional FastAPI layering for request handling, DB access, and dependency injection.
5. Chat/RAG engine
- backend/app/chat/ is the core runtime subsystem.
- engine.py is the central orchestration layer for answering questions, likely coordinating retrieval, tool use, response synthesis, and streaming.
- Supporting modules such as pg_vector.py, qa_response_synth.py, messaging.py, tools.py, and utils.py separate vector retrieval, answer formation, message streaming, and tool invocation.
- The README’s “sub-questions” and “token-level streaming” imply the backend streams partial reasoning and final text back to the frontend, likely over SSE.
6. Retrieval and persistence
- The backend uses PostgreSQL plus pgvector for embedding storage and similarity search.
- backend/app/models/, backend/app/db/, and backend/app/schema.py define the SQLAlchemy data model and database session layer.
- Alembic migrations under backend/alembic/versions/ show the schema is managed as an evolving persistent store, not an ephemeral cache.
- The database is the system of record for documents, conversations, and vector-backed retrieval artifacts.
7. Document ingestion and indexing
- backend/scripts/ contains a separate ingestion and maintenance toolchain: downloading SEC PDFs, seeding the DB, building vector tables, deduplicating vectors, and upserting documents.
- Files like build_vector_tables.py, upsert_document.py, upsert_db_sec_documents.py, and seed_storage_context.py indicate the index can be rebuilt and updated independently of the serving path.
- This is an important boundary: ingestion/indexing is offline/operational work, while the API serves live queries from the indexed data.
8. File and document storage
- backend/app/loader_io.py and the script set suggest document files and parsed content are handled separately from relational metadata.
- The README references AWS S3 and CloudFront for production storage/distribution, while backend/docker-compose.yml and localstack-cors-config.json support local emulation.
- The frontend also includes example PDFs in frontend/public/, which are useful for demoing the PDF viewer and citation highlighting flow.
9. External model and tool dependencies
- The system is built around OpenAI models for both generation and embeddings, as noted in the README.
- backend/app/chat/tools.py and the README’s polygon.io mention indicate the assistant can call external APIs for quantitative questions, so answer generation is not limited to document retrieval.
- This makes the chat subsystem a hybrid of RAG + tool-augmented QA.
10. Deployment and runtime boundaries
- The project is designed for production deployment on Render for the backend and Vercel for the frontend.
- render.yaml and the backend Docker files indicate containerized backend deployment with managed Postgres.
- The repo also includes .devcontainer/ for a reproducible Codespaces/devcontainer environment, so local development mirrors the deployed stack closely.
11. Observability and operational tooling
- The README calls out Sentry for monitoring/profiling, Arize Phoenix for LLM observability, and Loader.io for load testing.
- These tools are not just documentation extras; they reflect a production-oriented architecture where inference quality, latency, and errors are actively observed.
12. Main data flow to diagram
- A user selects a document or ticker in the frontend, opens a conversation, and submits a question.
- The frontend sends the request to FastAPI, which uses the chat engine to retrieve relevant chunks from Postgres/pgvector and may invoke external tools.
- The backend streams back reasoning updates and token-level answer text, plus citations.
- The frontend renders the streamed conversation and uses citation metadata to highlight the relevant passages in the PDF viewer.
13. Key architectural boundary to preserve
- The important separation is between “presentation and interaction” in Next.js and “retrieval, storage, and LLM orchestration” in FastAPI.
- A second boundary exists between online query serving and offline ingestion/index-building scripts.
- A third boundary is between local development emulation and cloud deployment/storage services.
Main entry points: endpoints API Routes · engine.py Chat Engine · api API Client · db Database
Components 20
Frontend
Next.js client for browsing SEC docs and chatting
- Pages · App shell, landing page, and conversation route
frontend/src/pages - UI Components · Conversation, landing page, modal, and PDF viewer components
frontend/src/components - Client State · Document selection, messages, PDF viewer state, and local persistence
frontend/src/hooks - API Client · Typed client boundary for backend requests and response handling
frontend/src/api - Client Types · Frontend models for backend documents, conversations, and selection
frontend/src/types/backend
Backend
FastAPI service for retrieval, chat orchestration, and data access
- App Entry · Backend service startup and application wiring
backend/app/main.py - API Routes · Documents, conversation/chat, and health endpoints
backend/app/api/endpoints - Deps & CRUD · FastAPI dependency injection and database access helpers
backend/app/api - Chat Engine · Coordinates retrieval, tool use, synthesis, and streaming
backend/app/chat/engine.py - Vector Search · Similarity search and retrieval over embedded document chunks
backend/app/chat/pg_vector.py - Answer Synth · Formats answers, citations, and streamed responses
backend/app/chat/qa_response_synth.py - Tools · External tool integration for hybrid QA
backend/app/chat/tools.py - Database · System of record for documents, conversations, and embeddings
backend/app/db - Models · Persistent entity definitions and ORM base classes
backend/app/models - Schema · Shared backend data shapes and API contracts
backend/app/schema.py
Ops
Offline ingestion, deployment, and local development boundaries
- Ingestion · Download, seed, dedupe, and upsert document/index data
backend/scripts - Migrations · Evolving database schema and index structure
backend/alembic/versions - Dev Runtime · Local development and containerized backend/runtime setup
.devcontainer/devcontainer.json - Deploy Config · Production deployment wiring for frontend and backend services
render.yaml - External Services · Model providers, object storage, and optional third-party data APIs
Connections 23
- Pages → UI Components: renders · Routes compose the main user interface
- Pages → Client State: uses · Pages delegate interaction state to hooks
- UI Components → API Client: calls · UI requests conversation and document data
- Client State → API Client: fetches · Client state hooks load and sync backend data
- API Client → Client Types: types · Client requests are shaped by backend DTOs
- API Client → API Routes: http · Frontend talks to FastAPI endpoints
- App Entry → API Routes: mounts · Application entry registers route groups
- API Routes → Deps & CRUD: depends on · Endpoints use dependency wiring and CRUD helpers
- API Routes → Chat Engine: invokes · Conversation routes delegate answering to the chat engine
- Chat Engine → Vector Search: retrieves · Engine pulls relevant chunks from vector search
- Chat Engine → Answer Synth: synthesizes · Engine turns retrieved context into streamed answers
- Chat Engine → Tools: tools · Engine may call out to external tools for hybrid QA
- Vector Search → Database: queries · Similarity search runs against Postgres/pgvector
- Deps & CRUD → Models: uses · CRUD helpers operate on ORM entities
- Models → Database: persists · Models map onto relational and vector-backed storage
- Schema → API Routes: contracts · Schemas define request and response payloads
- Ingestion → Database: writes · Offline jobs seed and update documents and vectors
- Ingestion → External Services: downloads · Ingestion can fetch PDFs and other upstream data
- Migrations → Database: evolves · Alembic updates the persistent schema
- Dev Runtime → Deploy Config: mirrors · Local container setup approximates deployment runtime
- Tools → External Services: calls · Tool-augmented answers may query third-party APIs
- Deploy Config → Pages: serves · Frontend is deployed as a separate runtime boundary
- Deploy Config → App Entry: hosts · Backend service is deployed independently