How mithunjack/extract_chain_of_events_from_documents works
1. Narrative Atlas is a local-first, full-stack document analysis system. The architecture splits cleanly into a FastAPI backend that ingests and analyzes PDFs, and a React/Vite frontend that turns the results into a cinematic narrative UI.
2. The backend API surface is small and purpose-built: documents for upload and document lifecycle, timelines for entity/story-beat views, chat for grounded Q&A, and health for readiness checks. That makes the main integration boundary easy to diagram as “frontend calls backend HTTP API.”
3. PDF ingestion starts with file upload in the UI, then the backend stores the raw PDF locally and extracts text page-by-page. The repo is designed for local runtime storage plus SQLite-backed persistence for metadata and analysis state, so raw files and structured results stay on the same machine by default.
4. The core analysis path is a service pipeline in backend/app/services. pdf_parser handles text extraction, chunking breaks content into retrievable segments, entity_extraction finds narrative entities, and timeline_builder orders evidence into story beats. analysis_pipeline orchestrates these steps so the route handlers stay thin.
5. Retrieval and chat are separated from extraction. embeddings, retrieval, and qa form the grounded answer path: user questions are matched against stored chunks and timeline evidence, and responses are expected to include citations. This is the main “RAG-like” boundary in the architecture.
6. Data contracts are defined explicitly in models.py and schemas.py. Those files sit between persistence, services, and the API routes, which is important for diagramming because the backend is not just free-form logic; it has structured document, entity, chunk, and timeline representations.
7. The frontend is a componentized React app centered on a few high-level panels: UploadPanel for document submission, DocumentRail for browsing loaded documents, EntitySpotlight for choosing a character/entity, CinematicTimeline for the ordered narrative view, and ChatPanel for Q&A. EmptyState and StatusPill support the interaction flow and loading/status feedback.
8. Frontend-to-backend communication is isolated in frontend/src/lib/api.ts with shared types in frontend/src/lib/types.ts. That makes the UI a consumer of a stable API contract rather than directly coupling components to backend details.
9. The main user flow is: upload PDF → backend parses and analyzes → entities and beats are persisted → frontend lists the document and entities → selecting an entity renders a narrative timeline → chat answers questions with evidence. In diagram terms, the central loop is “document to structured narrative to interactive exploration.”
10. Deployment is intentionally simple. docker-compose.yml ties together backend, frontend, and Ollama; run.sh can bootstrap either Docker-based or local development. The backend is Python/FastAPI, the frontend is TypeScript/React/Vite, and Ollama is the default local model provider, with OpenAI configuration present but not required for the current local-first path.
11. The repo also includes focused test coverage that maps to the architecture boundaries: backend tests for chunking, entity extraction, timeline building, document API, and timeline API; frontend tests for core interaction components like upload and timeline rendering. That suggests the system is organized around separable service and UI contracts rather than a monolith.
12. Finally, the docs/superpowers/specs and docs/superpowers/plans directories capture the design intent behind the rebuild, including the hybrid local RAG direction and the premium narrative timeline concept. Those docs are useful for understanding future extension points, especially stronger extraction, alias resolution, and larger-book support.
Main entry points: analysis_pipeline.py Pipeline · App.tsx UI shell · api.ts API client · main.py API app
Components 26
Backend
FastAPI API, persistence, and analysis pipeline
- API app · Creates the backend application and wires routes/configuration.
backend/app/main.py - Documents · Upload and document lifecycle endpoints.
backend/app/api/routes/documents.py - Timelines · Entity and story-beat timeline endpoints.
backend/app/api/routes/timelines.py - Chat · Grounded question-answering endpoints.
backend/app/api/routes/chat.py - Health · Readiness and liveness checks for the backend.
backend/app/api/routes/health.py - Pipeline · Coordinates parsing, chunking, extraction, and timeline building.
backend/app/services/analysis_pipeline.py - PDF parser · Reads PDFs and extracts page-level text.
backend/app/services/pdf_parser.py - Chunking · Breaks document text into retrievable chunks.
backend/app/services/chunking.py - Entities · Finds characters, people, or other story entities.
backend/app/services/entity_extraction.py - Timeline builder · Orders evidence into narrative beats and timelines.
backend/app/services/timeline_builder.py - Embeddings · Turns chunks and evidence into searchable vectors.
backend/app/services/embeddings.py - Retrieval + QA · Matches questions to evidence and generates cited answers.
backend/app/services/retrieval.py - Local storage · Stores uploaded PDFs and local artifacts on disk.
backend/app/services/storage.py - SQLite DB · Persists document state, analysis results, and structured records.
backend/app/db.py - Contracts · Shared document, entity, chunk, and timeline structures.
backend/app/models.py
Frontend
React/Vite narrative UI and API client
- UI shell · Composes the main narrative analysis workspace.
frontend/src/App.tsx - Upload panel · Handles PDF submission into the analysis flow.
frontend/src/components/UploadPanel.tsx - Document rail · Browses loaded documents and analysis state.
frontend/src/components/DocumentRail.tsx - Entity spotlight · Selects the active narrative entity.
frontend/src/components/EntitySpotlight.tsx - Timeline view · Renders the ordered narrative beat sequence.
frontend/src/components/CinematicTimeline.tsx - Chat panel · Provides grounded Q&A over the analyzed document.
frontend/src/components/ChatPanel.tsx - API client · Centralized backend request layer and shared API access.
frontend/src/lib/api.ts - Shared types · Type definitions shared across UI components and API calls.
frontend/src/lib/types.ts
Runtime & docs
Container setup and design intent
- Compose stack · Starts backend, frontend, and Ollama together.
docker-compose.yml - Run script · Bootstraps local or Docker-based development.
run.sh - Design docs · Spec and plan documents for the local RAG and narrative timeline direction.
docs/superpowers/specs/2026-05-06-hybrid-local-rag-design.md
Connections 34
- API app → Documents: mounts · Registers document routes in the API application.
- API app → Timelines: mounts · Registers timeline routes in the API application.
- API app → Chat: mounts · Registers chat routes in the API application.
- API app → Health: mounts · Registers health routes in the API application.
- Documents → Pipeline: starts · Document upload triggers the analysis pipeline.
- Pipeline → PDF parser: reads · Parses uploaded PDFs into text.
- Pipeline → Chunking: segments · Breaks extracted text into chunks.
- Pipeline → Entities: extracts · Finds narrative entities from chunks.
- Pipeline → Timeline builder: builds · Synthesizes evidence into story beats.
- Chunking → Embeddings: vectors · Produces retrievable representations for search.
- Entities → Timeline builder: feeds · Entity evidence informs timeline construction.
- Timeline builder → Retrieval + QA: evidence · Timeline evidence is reused for grounded answers.
- Embeddings → Retrieval + QA: retrieves · Vector search supports question answering.
- Pipeline → Local storage: persists · Stores raw PDFs and local artifacts.
- Pipeline → SQLite DB: records · Writes document state and analysis metadata.
- SQLite DB → Contracts: conforms to · Persistence follows shared data contracts.
- Timelines → Timeline builder: serves · Timeline endpoints expose synthesized beats.
- Chat → Retrieval + QA: serves · Chat endpoints expose grounded answers.
- UI shell → Upload panel: contains · Main UI composes the upload flow.
- UI shell → Document rail: contains · Main UI composes document browsing.
- UI shell → Entity spotlight: contains · Main UI composes entity selection.
- UI shell → Timeline view: contains · Main UI composes timeline visualization.
- UI shell → Chat panel: contains · Main UI composes the chat experience.
- UI shell → API client: uses · UI actions go through the shared API layer.
- API client → Shared types: typed by · Client requests and responses use shared types.
- API client → Documents: calls · Upload and lifecycle requests hit document endpoints.
- API client → Timelines: calls · Timeline views request analysis data.
- API client → Chat: calls · Chat requests hit the grounded Q&A API.
- Compose stack → API app: runs · Compose starts the backend service.
- Compose stack → UI shell: runs · Compose starts the frontend service.
- Compose stack → Local storage: shares · Local deployment expects filesystem-backed artifacts.
- Run script → Compose stack: boots · Bootstrap script can launch the container stack.
- Design docs → Pipeline: describes · Docs capture the intended analysis architecture.
- Design docs → Retrieval + QA: describes · Docs capture the local RAG design direction.