# scriptedbyarun47/kmrl_docsummarizer architecture

> Architecture diagram of the GitHub repository scriptedbyarun47/kmrl_docsummarizer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/scriptedbyarun47/kmrl_docsummarizer
- Repository: https://github.com/scriptedbyarun47/kmrl_docsummarizer

## Overview

1. **Repository purpose**
   - This is a document summarization system for policy and HR/resume PDFs. The repo is organized as a pipeline that ingests source documents, extracts text, generates summaries, and then serves or presents those summaries through a separate backend/frontend stack.

2. **Source document layer**
   - `app/data/raw/` holds the original PDFs.
   - `app/data/processed/` holds extracted text versions of those PDFs.
   - `app/data/summaries/` holds the generated summary text files.
   - This makes the filesystem itself an important artifact store in the architecture, not just a temporary workspace.

3. **Python processing pipeline**
   - The core document workflow lives under `app/scripts/`.
   - `ingestion.py` likely handles loading files into the pipeline.
   - `extraction.py` is the text extraction stage from PDF to plain text.
   - `document_summarizer.py` is the summarization stage.
   - `utils.py` and `config.py` provide shared helpers and environment/configuration.

4. **Python entrypoint and orchestration**
   - `app/main.py` is the likely orchestration entrypoint for the Python side.
   - The `app` package structure suggests the document workflow can run as a local batch process or CLI-style job, rather than only as a service.

5. **Persistence and model boundary**
   - `app/scripts/db.py` indicates the Python pipeline can interact with some form of storage layer beyond the filesystem.
   - On the Node side, `backend/models/Summary.js` suggests a persisted summary record model, likely used to store or retrieve summary metadata/content through the backend.
   - Together, these point to two storage surfaces: file-based artifacts for processing, and a backend data model for application access.

6. **Node.js backend service**
   - `backend/server.js` is the runtime server layer.
   - Its role is likely to expose the summarized document data to clients, separating presentation/API concerns from the Python processing pipeline.
   - The backend appears to be the bridge between stored summaries and the frontend.

7. **Frontend presentation layer**
   - `frontend/index.html`, `frontend/app.js`, and `frontend/styles.css` form a lightweight browser UI.
   - This layer is responsible for displaying documents or summaries and likely calls the Node backend for data.
   - It is a distinct consumer of the backend rather than part of the document-processing pipeline.

8. **Operational artifacts**
   - `logs/process.log` captures pipeline or application execution logs.
   - This indicates the summarization workflow is operationalized enough to track processing outcomes, errors, or document runs.

9. **Experimentation and validation**
   - `notebooks/test_summary_generation.ipynb` shows that summary quality was explored interactively.
   - This notebook sits outside the production path and is likely used to tune or validate extraction/summarization behavior.

10. **Technology split and dependency boundaries**
   - The repo uses Python for document processing (`requirements.txt`) and Node.js for the application server (`package.json`, `package-lock.json`).
   - That split is a key architectural boundary: offline or batch-style text processing in Python, application serving and UI delivery in JavaScript.

11. **End-to-end data flow**
   - The most likely flow is: PDF enters `app/data/raw/` → Python extracts text into `app/data/processed/` → Python generates summary into `app/data/summaries/` → backend serves summary data via the model/server layer → frontend displays it to the user.
   - That flow is the central diagram to draw for this repository.

Main entry points: [`app/scripts/document_summarizer.py`](https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/app/scripts/document_summarizer.py) (Summarizer), [`app/scripts/ingestion.py`](https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/app/scripts/ingestion.py) (Ingestion), [`app/data/processed`](https://github.com/scriptedbyarun47/kmrl_docsummarizer/tree/main/app/data/processed) (Extracted text), [`app/scripts/utils.py`](https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/app/scripts/utils.py) (Utilities)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_python["Python pipeline"]
  node_node_ingestion["Ingestion<br/>python stage<br/>[ingestion.py]"]
  node_node_extraction["Extraction<br/>python stage<br/>[extraction.py]"]
  node_node_summarizer["Summarizer<br/>python stage"]
  node_node_orchestrator["Main runner<br/>python entrypoint<br/>[main.py]"]
  node_node_config["Config<br/>python support<br/>[config.py]"]
  node_node_utils["Utilities<br/>python support<br/>[utils.py]"]
  node_node_db[("DB access<br/>python storage<br/>[db.py]")]
end

subgraph group_group_storage["Artifact store"]
  node_node_raw_pdfs[("Raw PDFs<br/>artifact store")]
  node_node_processed_txt["Extracted text<br/>artifact store"]
  node_node_summaries_txt["Summaries<br/>artifact store"]
end

subgraph group_group_backend["Backend service"]
  node_node_summary_model[("Summary model<br/>node model<br/>[Summary.js]")]
  node_node_server["Server<br/>node runtime<br/>[server.js]"]
end

subgraph group_group_frontend["Frontend UI"]
  node_node_ui["Browser UI<br/>frontend app<br/>[app.js]"]
  node_node_html["Shell<br/>frontend markup<br/>[index.html]"]
  node_node_styles["Styles<br/>frontend css<br/>[styles.css]"]
end

subgraph group_group_ops["Ops and experiments"]
  node_node_log["Process log<br/>ops log<br/>[process.log]"]
  node_node_notebook["Summary notebook<br/>experiment"]
end

node_node_orchestrator -->|"starts"| node_node_ingestion
node_node_ingestion -->|"reads"| node_node_raw_pdfs
node_node_ingestion -->|"hands off"| node_node_extraction
node_node_extraction -->|"writes"| node_node_processed_txt
node_node_processed_txt -->|"feeds"| node_node_summarizer
node_node_summarizer -->|"writes"| node_node_summaries_txt
node_node_config -->|"configures"| node_node_orchestrator
node_node_utils -->|"supports"| node_node_ingestion
node_node_utils -->|"supports"| node_node_extraction
node_node_utils -->|"supports"| node_node_summarizer
node_node_summarizer -->|"persists"| node_node_db
node_node_db -->|"maps to"| node_node_summary_model
node_node_summary_model -->|"served by"| node_node_server
node_node_server -->|"api/data"| node_node_ui
node_node_ui -->|"renders in"| node_node_html
node_node_ui -->|"styled by"| node_node_styles
node_node_orchestrator -->|"logs"| node_node_log
node_node_notebook -.->|"validates"| node_node_summarizer
node_node_notebook -.->|"inspects"| node_node_processed_txt
node_node_raw_pdfs -->|"transforms to"| node_node_processed_txt
node_node_summaries_txt -->|"exposed via"| node_node_server

click node_node_raw_pdfs "https://github.com/scriptedbyarun47/kmrl_docsummarizer/tree/main/app/data/raw"
click node_node_processed_txt "https://github.com/scriptedbyarun47/kmrl_docsummarizer/tree/main/app/data/processed"
click node_node_summaries_txt "https://github.com/scriptedbyarun47/kmrl_docsummarizer/tree/main/app/data/summaries"
click node_node_ingestion "https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/app/scripts/ingestion.py"
click node_node_extraction "https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/app/scripts/extraction.py"
click node_node_summarizer "https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/app/scripts/document_summarizer.py"
click node_node_orchestrator "https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/app/main.py"
click node_node_config "https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/app/scripts/config.py"
click node_node_utils "https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/app/scripts/utils.py"
click node_node_db "https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/app/scripts/db.py"
click node_node_summary_model "https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/backend/models/Summary.js"
click node_node_server "https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/backend/server.js"
click node_node_ui "https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/frontend/app.js"
click node_node_html "https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/frontend/index.html"
click node_node_styles "https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/frontend/styles.css"
click node_node_log "https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/logs/process.log"
click node_node_notebook "https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/notebooks/test_summary_generation.ipynb"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_node_ingestion,node_node_extraction,node_node_summarizer,node_node_orchestrator,node_node_config,node_node_utils,node_node_db toneBlue
class node_node_raw_pdfs,node_node_processed_txt,node_node_summaries_txt toneAmber
class node_node_summary_model,node_node_server toneMint
class node_node_ui,node_node_html,node_node_styles toneRose
class node_node_log,node_node_notebook toneIndigo
```

## Components

### Python pipeline

Document ingestion, extraction, summarization, and local orchestration.

- [Ingestion](https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/app/scripts/ingestion.py): python stage · Loads source documents into the processing flow. · `app/scripts/ingestion.py`
- [Extraction](https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/app/scripts/extraction.py): python stage · Converts PDFs into text. · `app/scripts/extraction.py`
- [Summarizer](https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/app/scripts/document_summarizer.py): python stage · Generates summaries from extracted text. · `app/scripts/document_summarizer.py`
- [Main runner](https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/app/main.py): python entrypoint · Batch orchestration entrypoint for the pipeline. · `app/main.py`
- [Config](https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/app/scripts/config.py): python support · Shared configuration and environment settings. · `app/scripts/config.py`
- [Utilities](https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/app/scripts/utils.py): python support · Shared helpers used across pipeline stages. · `app/scripts/utils.py`
- [DB access](https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/app/scripts/db.py): python storage · Python-side persistence or storage integration. · `app/scripts/db.py`

### Artifact store

Filesystem-based inputs, intermediates, summaries, and logs.

- [Raw PDFs](https://github.com/scriptedbyarun47/kmrl_docsummarizer/tree/main/app/data/raw): artifact store · Source policy and resume documents. · `app/data/raw`
- [Extracted text](https://github.com/scriptedbyarun47/kmrl_docsummarizer/tree/main/app/data/processed): artifact store · Plain-text outputs from PDF extraction. · `app/data/processed`
- [Summaries](https://github.com/scriptedbyarun47/kmrl_docsummarizer/tree/main/app/data/summaries): artifact store · Generated summary text files. · `app/data/summaries`

### Backend service

Node runtime and persisted summary model.

- [Summary model](https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/backend/models/Summary.js): node model · Persisted summary record used by the backend. · `backend/models/Summary.js`
- [Server](https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/backend/server.js): node runtime · Application server exposing summary data. · `backend/server.js`

### Frontend UI

Browser presentation layer for summaries.

- [Browser UI](https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/frontend/app.js): frontend app · Client-side display for documents and summaries. · `frontend/app.js`
- [Shell](https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/frontend/index.html): frontend markup · Static page container for the UI. · `frontend/index.html`
- [Styles](https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/frontend/styles.css): frontend css · Presentation styling for the browser UI. · `frontend/styles.css`

### Ops and experiments

Logging and notebook-based validation.

- [Process log](https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/logs/process.log): ops log · Execution log for pipeline or app runs. · `logs/process.log`
- [Summary notebook](https://github.com/scriptedbyarun47/kmrl_docsummarizer/blob/main/notebooks/test_summary_generation.ipynb): experiment · Interactive notebook for testing summary generation. · `notebooks/test_summary_generation.ipynb`

## Connections

- Main runner → Ingestion: starts · Main runner kicks off document ingestion.
- Ingestion → Raw PDFs: reads · Ingestion consumes source PDFs.
- Ingestion → Extraction: hands off · Loaded documents move into extraction.
- Extraction → Extracted text: writes · Extraction stores plain-text artifacts.
- Extracted text → Summarizer: feeds · Extracted text becomes summarizer input.
- Summarizer → Summaries: writes · Summaries are persisted as text artifacts.
- Config → Main runner: configures · Shared settings shape pipeline execution.
- Utilities → Ingestion: supports · Common helpers are reused during ingest.
- Utilities → Extraction: supports · Common helpers are reused during extraction.
- Utilities → Summarizer: supports · Common helpers are reused during summarization.
- Summarizer → DB access: persists · Summaries can be stored through the data layer.
- DB access → Summary model: maps to · Python storage boundary aligns with backend summary records.
- Summary model → Server: served by · Backend server exposes persisted summary data.
- Server → Browser UI: api/data · Frontend consumes summary data from the backend.
- Browser UI → Shell: renders in · Browser app is mounted in the HTML shell.
- Browser UI → Styles: styled by · Stylesheet controls presentation.
- Main runner → Process log: logs · Pipeline execution is recorded in process logs.
- Summary notebook → Summarizer: validates · Notebook experiments inform summary generation.
- Summary notebook → Extracted text: inspects · Notebook likely inspects extracted text artifacts.
- Raw PDFs → Extracted text: transforms to · PDFs are converted into extracted text.
- Summaries → Server: exposed via · Generated summaries are surfaced through the backend.
