# mithunjack/extract_chain_of_events_from_documents architecture

> Architecture diagram of the GitHub repository mithunjack/extract_chain_of_events_from_documents, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mithunjack/extract_chain_of_events_from_documents
- Repository: https://github.com/mithunjack/extract_chain_of_events_from_documents

## Overview

1. Narrative Atlas is a local-first, full-stack document analysis system. The architecture splits cleanly into a FastAPI backend that ingests and analyzes PDFs, and a React/Vite frontend that turns the results into a cinematic narrative UI.

2. The backend API surface is small and purpose-built: `documents` for upload and document lifecycle, `timelines` for entity/story-beat views, `chat` for grounded Q&A, and `health` for readiness checks. That makes the main integration boundary easy to diagram as “frontend calls backend HTTP API.”

3. PDF ingestion starts with file upload in the UI, then the backend stores the raw PDF locally and extracts text page-by-page. The repo is designed for local runtime storage plus SQLite-backed persistence for metadata and analysis state, so raw files and structured results stay on the same machine by default.

4. The core analysis path is a service pipeline in `backend/app/services`. `pdf_parser` handles text extraction, `chunking` breaks content into retrievable segments, `entity_extraction` finds narrative entities, and `timeline_builder` orders evidence into story beats. `analysis_pipeline` orchestrates these steps so the route handlers stay thin.

5. Retrieval and chat are separated from extraction. `embeddings`, `retrieval`, and `qa` form the grounded answer path: user questions are matched against stored chunks and timeline evidence, and responses are expected to include citations. This is the main “RAG-like” boundary in the architecture.

6. Data contracts are defined explicitly in `models.py` and `schemas.py`. Those files sit between persistence, services, and the API routes, which is important for diagramming because the backend is not just free-form logic; it has structured document, entity, chunk, and timeline representations.

7. The frontend is a componentized React app centered on a few high-level panels: `UploadPanel` for document submission, `DocumentRail` for browsing loaded documents, `EntitySpotlight` for choosing a character/entity, `CinematicTimeline` for the ordered narrative view, and `ChatPanel` for Q&A. `EmptyState` and `StatusPill` support the interaction flow and loading/status feedback.

8. Frontend-to-backend communication is isolated in `frontend/src/lib/api.ts` with shared types in `frontend/src/lib/types.ts`. That makes the UI a consumer of a stable API contract rather than directly coupling components to backend details.

9. The main user flow is: upload PDF → backend parses and analyzes → entities and beats are persisted → frontend lists the document and entities → selecting an entity renders a narrative timeline → chat answers questions with evidence. In diagram terms, the central loop is “document to structured narrative to interactive exploration.”

10. Deployment is intentionally simple. `docker-compose.yml` ties together backend, frontend, and Ollama; `run.sh` can bootstrap either Docker-based or local development. The backend is Python/FastAPI, the frontend is TypeScript/React/Vite, and Ollama is the default local model provider, with OpenAI configuration present but not required for the current local-first path.

11. The repo also includes focused test coverage that maps to the architecture boundaries: backend tests for chunking, entity extraction, timeline building, document API, and timeline API; frontend tests for core interaction components like upload and timeline rendering. That suggests the system is organized around separable service and UI contracts rather than a monolith.

12. Finally, the `docs/superpowers/specs` and `docs/superpowers/plans` directories capture the design intent behind the rebuild, including the hybrid local RAG direction and the premium narrative timeline concept. Those docs are useful for understanding future extension points, especially stronger extraction, alias resolution, and larger-book support.

Main entry points: [`backend/app/services/analysis_pipeline.py`](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/services/analysis_pipeline.py) (Pipeline), [`frontend/src/App.tsx`](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/frontend/src/App.tsx) (UI shell), [`frontend/src/lib/api.ts`](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/frontend/src/lib/api.ts) (API client), [`backend/app/main.py`](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/main.py) (API app)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_backend["Backend"]
  node_backend_main["API app<br/>FastAPI entrypoint<br/>[main.py]"]
  node_api_documents["Documents<br/>HTTP route<br/>[documents.py]"]
  node_api_timelines["Timelines<br/>HTTP route<br/>[timelines.py]"]
  node_api_chat["Chat<br/>HTTP route<br/>[chat.py]"]
  node_api_health["Health<br/>HTTP route<br/>[health.py]"]
  node_analysis_pipeline["Pipeline<br/>analysis orchestrator"]
  node_pdf_parser["PDF parser<br/>extraction service<br/>[pdf_parser.py]"]
  node_chunking["Chunking<br/>segmentation service<br/>[chunking.py]"]
  node_entity_extraction(("Entities<br/>narrative extraction"))
  node_timeline_builder["Timeline builder<br/>story-beat synthesis"]
  node_embeddings(("Embeddings<br/>vectorization service<br/>[embeddings.py]"))
  node_retrieval_qa["Retrieval + QA<br/>RAG answer path<br/>[retrieval.py]"]
  node_storage[("Local storage<br/>file persistence<br/>[storage.py]")]
  node_db[("SQLite DB<br/>metadata store<br/>[db.py]")]
  node_models_schemas["Contracts<br/>data models<br/>[models.py]"]
end

subgraph group_frontend["Frontend"]
  node_frontend_app["UI shell<br/>React app<br/>[App.tsx]"]
  node_upload_panel["Upload panel<br/>UI component<br/>[UploadPanel.tsx]"]
  node_document_rail["Document rail<br/>UI component<br/>[DocumentRail.tsx]"]
  node_entity_spotlight(("Entity spotlight<br/>UI component"))
  node_cinematic_timeline["Timeline view<br/>UI component"]
  node_chat_panel["Chat panel<br/>UI component<br/>[ChatPanel.tsx]"]
  node_api_client["API client<br/>frontend HTTP client<br/>[api.ts]"]
  node_shared_types["Shared types<br/>frontend contracts<br/>[types.ts]"]
end

subgraph group_ops["Runtime & docs"]
  node_docker_compose["Compose stack<br/>deployment config<br/>[docker-compose.yml]"]
  node_run_script["Run script<br/>bootstrap script<br/>[run.sh]"]
  node_design_docs["Design docs<br/>architecture docs"]
end

node_backend_main -->|"mounts"| node_api_documents
node_backend_main -->|"mounts"| node_api_timelines
node_backend_main -->|"mounts"| node_api_chat
node_backend_main -->|"mounts"| node_api_health
node_api_documents -->|"starts"| node_analysis_pipeline
node_analysis_pipeline -->|"reads"| node_pdf_parser
node_analysis_pipeline -->|"segments"| node_chunking
node_analysis_pipeline -->|"extracts"| node_entity_extraction
node_analysis_pipeline -->|"builds"| node_timeline_builder
node_chunking -->|"vectors"| node_embeddings
node_entity_extraction -->|"feeds"| node_timeline_builder
node_timeline_builder -->|"evidence"| node_retrieval_qa
node_embeddings -->|"retrieves"| node_retrieval_qa
node_analysis_pipeline -->|"persists"| node_storage
node_analysis_pipeline -->|"records"| node_db
node_db -.->|"conforms to"| node_models_schemas
node_api_timelines -->|"serves"| node_timeline_builder
node_api_chat -->|"serves"| node_retrieval_qa
node_frontend_app -->|"contains"| node_upload_panel
node_frontend_app -->|"contains"| node_document_rail
node_frontend_app -->|"contains"| node_entity_spotlight
node_frontend_app -->|"contains"| node_cinematic_timeline
node_frontend_app -->|"contains"| node_chat_panel
node_frontend_app -->|"uses"| node_api_client
node_api_client -.->|"typed by"| node_shared_types
node_api_client -->|"calls"| node_api_documents
node_api_client -->|"calls"| node_api_timelines
node_api_client -->|"calls"| node_api_chat
node_docker_compose -->|"runs"| node_backend_main
node_docker_compose -->|"runs"| node_frontend_app
node_docker_compose -.->|"shares"| node_storage
node_run_script -->|"boots"| node_docker_compose
node_design_docs -.->|"describes"| node_analysis_pipeline
node_design_docs -.->|"describes"| node_retrieval_qa

click node_backend_main "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/main.py"
click node_api_documents "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/api/routes/documents.py"
click node_api_timelines "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/api/routes/timelines.py"
click node_api_chat "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/api/routes/chat.py"
click node_api_health "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/api/routes/health.py"
click node_analysis_pipeline "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/services/analysis_pipeline.py"
click node_pdf_parser "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/services/pdf_parser.py"
click node_chunking "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/services/chunking.py"
click node_entity_extraction "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/services/entity_extraction.py"
click node_timeline_builder "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/services/timeline_builder.py"
click node_embeddings "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/services/embeddings.py"
click node_retrieval_qa "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/services/retrieval.py"
click node_storage "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/services/storage.py"
click node_db "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/db.py"
click node_models_schemas "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/models.py"
click node_frontend_app "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/frontend/src/App.tsx"
click node_upload_panel "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/frontend/src/components/UploadPanel.tsx"
click node_document_rail "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/frontend/src/components/DocumentRail.tsx"
click node_entity_spotlight "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/frontend/src/components/EntitySpotlight.tsx"
click node_cinematic_timeline "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/frontend/src/components/CinematicTimeline.tsx"
click node_chat_panel "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/frontend/src/components/ChatPanel.tsx"
click node_api_client "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/frontend/src/lib/api.ts"
click node_shared_types "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/frontend/src/lib/types.ts"
click node_docker_compose "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/docker-compose.yml"
click node_run_script "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/run.sh"
click node_design_docs "https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/docs/superpowers/specs/2026-05-06-hybrid-local-rag-design.md"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_backend_main,node_api_documents,node_api_timelines,node_api_chat,node_api_health,node_analysis_pipeline,node_pdf_parser,node_chunking,node_entity_extraction,node_timeline_builder,node_embeddings,node_retrieval_qa,node_storage,node_db,node_models_schemas toneBlue
class node_frontend_app,node_upload_panel,node_document_rail,node_entity_spotlight,node_cinematic_timeline,node_chat_panel,node_api_client,node_shared_types toneAmber
class node_docker_compose,node_run_script,node_design_docs toneMint
```

## Components

### Backend

FastAPI API, persistence, and analysis pipeline

- [API app](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/main.py): FastAPI entrypoint · Creates the backend application and wires routes/configuration. · `backend/app/main.py`
- [Documents](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/api/routes/documents.py): HTTP route · Upload and document lifecycle endpoints. · `backend/app/api/routes/documents.py`
- [Timelines](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/api/routes/timelines.py): HTTP route · Entity and story-beat timeline endpoints. · `backend/app/api/routes/timelines.py`
- [Chat](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/api/routes/chat.py): HTTP route · Grounded question-answering endpoints. · `backend/app/api/routes/chat.py`
- [Health](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/api/routes/health.py): HTTP route · Readiness and liveness checks for the backend. · `backend/app/api/routes/health.py`
- [Pipeline](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/services/analysis_pipeline.py): analysis orchestrator · Coordinates parsing, chunking, extraction, and timeline building. · `backend/app/services/analysis_pipeline.py`
- [PDF parser](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/services/pdf_parser.py): extraction service · Reads PDFs and extracts page-level text. · `backend/app/services/pdf_parser.py`
- [Chunking](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/services/chunking.py): segmentation service · Breaks document text into retrievable chunks. · `backend/app/services/chunking.py`
- [Entities](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/services/entity_extraction.py): narrative extraction · Finds characters, people, or other story entities. · `backend/app/services/entity_extraction.py`
- [Timeline builder](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/services/timeline_builder.py): story-beat synthesis · Orders evidence into narrative beats and timelines. · `backend/app/services/timeline_builder.py`
- [Embeddings](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/services/embeddings.py): vectorization service · Turns chunks and evidence into searchable vectors. · `backend/app/services/embeddings.py`
- [Retrieval + QA](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/services/retrieval.py): RAG answer path · Matches questions to evidence and generates cited answers. · `backend/app/services/retrieval.py`
- [Local storage](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/services/storage.py): file persistence · Stores uploaded PDFs and local artifacts on disk. · `backend/app/services/storage.py`
- [SQLite DB](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/db.py): metadata store · Persists document state, analysis results, and structured records. · `backend/app/db.py`
- [Contracts](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/backend/app/models.py): data models · Shared document, entity, chunk, and timeline structures. · `backend/app/models.py`

### Frontend

React/Vite narrative UI and API client

- [UI shell](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/frontend/src/App.tsx): React app · Composes the main narrative analysis workspace. · `frontend/src/App.tsx`
- [Upload panel](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/frontend/src/components/UploadPanel.tsx): UI component · Handles PDF submission into the analysis flow. · `frontend/src/components/UploadPanel.tsx`
- [Document rail](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/frontend/src/components/DocumentRail.tsx): UI component · Browses loaded documents and analysis state. · `frontend/src/components/DocumentRail.tsx`
- [Entity spotlight](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/frontend/src/components/EntitySpotlight.tsx): UI component · Selects the active narrative entity. · `frontend/src/components/EntitySpotlight.tsx`
- [Timeline view](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/frontend/src/components/CinematicTimeline.tsx): UI component · Renders the ordered narrative beat sequence. · `frontend/src/components/CinematicTimeline.tsx`
- [Chat panel](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/frontend/src/components/ChatPanel.tsx): UI component · Provides grounded Q&A over the analyzed document. · `frontend/src/components/ChatPanel.tsx`
- [API client](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/frontend/src/lib/api.ts): frontend HTTP client · Centralized backend request layer and shared API access. · `frontend/src/lib/api.ts`
- [Shared types](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/frontend/src/lib/types.ts): frontend contracts · Type definitions shared across UI components and API calls. · `frontend/src/lib/types.ts`

### Runtime & docs

Container setup and design intent

- [Compose stack](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/docker-compose.yml): deployment config · Starts backend, frontend, and Ollama together. · `docker-compose.yml`
- [Run script](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/run.sh): bootstrap script · Bootstraps local or Docker-based development. · `run.sh`
- [Design docs](https://github.com/mithunjack/extract_chain_of_events_from_documents/blob/main/docs/superpowers/specs/2026-05-06-hybrid-local-rag-design.md): architecture docs · Spec and plan documents for the local RAG and narrative timeline direction. · `docs/superpowers/specs/2026-05-06-hybrid-local-rag-design.md`

## Connections

- API app → Documents: mounts · Registers document routes in the API application.
- API app → Timelines: mounts · Registers timeline routes in the API application.
- API app → Chat: mounts · Registers chat routes in the API application.
- API app → Health: mounts · Registers health routes in the API application.
- Documents → Pipeline: starts · Document upload triggers the analysis pipeline.
- Pipeline → PDF parser: reads · Parses uploaded PDFs into text.
- Pipeline → Chunking: segments · Breaks extracted text into chunks.
- Pipeline → Entities: extracts · Finds narrative entities from chunks.
- Pipeline → Timeline builder: builds · Synthesizes evidence into story beats.
- Chunking → Embeddings: vectors · Produces retrievable representations for search.
- Entities → Timeline builder: feeds · Entity evidence informs timeline construction.
- Timeline builder → Retrieval + QA: evidence · Timeline evidence is reused for grounded answers.
- Embeddings → Retrieval + QA: retrieves · Vector search supports question answering.
- Pipeline → Local storage: persists · Stores raw PDFs and local artifacts.
- Pipeline → SQLite DB: records · Writes document state and analysis metadata.
- SQLite DB → Contracts: conforms to · Persistence follows shared data contracts.
- Timelines → Timeline builder: serves · Timeline endpoints expose synthesized beats.
- Chat → Retrieval + QA: serves · Chat endpoints expose grounded answers.
- UI shell → Upload panel: contains · Main UI composes the upload flow.
- UI shell → Document rail: contains · Main UI composes document browsing.
- UI shell → Entity spotlight: contains · Main UI composes entity selection.
- UI shell → Timeline view: contains · Main UI composes timeline visualization.
- UI shell → Chat panel: contains · Main UI composes the chat experience.
- UI shell → API client: uses · UI actions go through the shared API layer.
- API client → Shared types: typed by · Client requests and responses use shared types.
- API client → Documents: calls · Upload and lifecycle requests hit document endpoints.
- API client → Timelines: calls · Timeline views request analysis data.
- API client → Chat: calls · Chat requests hit the grounded Q&A API.
- Compose stack → API app: runs · Compose starts the backend service.
- Compose stack → UI shell: runs · Compose starts the frontend service.
- Compose stack → Local storage: shares · Local deployment expects filesystem-backed artifacts.
- Run script → Compose stack: boots · Bootstrap script can launch the container stack.
- Design docs → Pipeline: describes · Docs capture the intended analysis architecture.
- Design docs → Retrieval + QA: describes · Docs capture the local RAG design direction.
