# kevinjose-ma/ai-document-classification-summarization architecture

> Architecture diagram of the GitHub repository kevinjose-ma/ai-document-classification-summarization, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kevinjose-ma/ai-document-classification-summarization
- Repository: https://github.com/kevinjose-ma/ai-document-classification-summarization

## Overview

This repository is a two-tier AI document platform with a React frontend and a FastAPI backend, plus local file-based artifacts for uploads and vector indexes.

At a high level, the architecture is:

User browser → React/Vite frontend → FastAPI API layer → ingestion/retrieval/LLM services → MongoDB + local file storage + FAISS indexes

Main subsystems:

1. Frontend application
- Located under `frontend/`.
- Built with React, Vite, and Tailwind CSS.
- `src/main.jsx` and `src/App.jsx` provide the application entry and route composition.
- `src/context/AuthContext.jsx` indicates client-side auth state management, likely storing login/session state and protecting routes.
- `src/api/axios.js` is the HTTP client boundary to the backend API.
- The UI is organized into route-based pages:
  - authentication and onboarding: `Login.jsx`, `Register.jsx`, `Onboarding.jsx`
  - document workflow: `Dashboard.jsx`, `DocumentPage.jsx`, `DocumentViewer.jsx`, `UploadPage.jsx`, `IngestionPage.jsx`
  - admin/operational views: `Analytics.jsx`, `UsersPage.jsx`, `Settings.jsx`, `Emailsettings.jsx`
- Reusable UI components in `src/components/` implement the app shell and document-centric interactions: sidebar, topbar, table/cards, filters, upload form, confirmation dialogs, status badges, skeleton loaders, and toast notifications.

Architecturally, the frontend is a presentation and interaction layer only. It does not appear to own business logic; instead it drives backend APIs and renders document, ingestion, and account state.

2. Backend API layer
- Located under `backend/app/api/`.
- FastAPI is the backend runtime, with `backend/app/main.py` as the application entry point.
- The API is split into domain routers:
  - `auth.py` for login/register/authentication flow
  - `documents.py` for document CRUD, querying, and viewing
  - `ingestion_sources.py` for managing where documents come from
- This layer is the boundary between the frontend and the system’s internal processing pipeline. It likely validates requests, enforces auth, and dispatches work to services.

3. Core configuration and runtime concerns
- `backend/app/core/config.py` centralizes environment/configuration values.
- `backend/app/core/logger.py` indicates structured logging setup.
- The presence of `backend/logs/` shows runtime logs are persisted separately from app code.
- Deployment files such as `Dockerfile`, `railway.toml`, and `nixpacks.toml` show the app is designed for containerized/cloud deployment.

4. Document processing and AI service layer
This is the main backend value proposition and is split into several cooperating services under `backend/app/services/`:

- `ingestion.py` orchestrates the end-to-end ingest pipeline.
- `extractor.py` handles getting raw text/content from uploaded files.
- `parser.py` likely normalizes text and splits it into smaller semantic units/clauses.
- `storage.py` abstracts persistence of files and derived artifacts.
- `retriever.py` performs semantic lookup against FAISS indexes.
- `llm.py` wraps calls to the LLM provider.
- `prompts.py` centralizes prompt templates and structured prompt construction.
- `summarizer.py` generates summaries or condensed outputs from extracted content.

The README identifies Google Gemini as the LLM provider and FAISS as the vector retrieval layer. That means the AI flow is not “LLM only”; it is retrieval-augmented:
- the document is first processed and indexed
- user questions are then answered by combining semantic retrieval from FAISS with a generated LLM response

5. Data/model layer
- Located under `backend/app/models/`.
- `ingestion_source.py` suggests a model for configured document sources or source metadata.
- `models.py` likely contains the main document/user/schema definitions.
- The README also mentions document versioning and duplicate detection, so the persistence model likely stores metadata such as file hash, version, classification, and ownership/user scope.

6. Utility and maintenance layer
- `backend/app/utils/utils.py` contains shared helpers, likely including SHA256 hashing and text/file utility functions.
- Top-level maintenance scripts indicate operational support:
  - `clean_qa_cache.py` clears cached Q&A artifacts
  - `rebuild_all_faiss_indices.py` regenerates vector indexes from stored documents
- These scripts imply that some derived data is reproducible and can be rebuilt independently of the primary database.

Primary data flows:

A. Authentication flow
- The user signs in or registers through the frontend.
- The frontend sends credentials to `api/auth.py`.
- The backend authenticates and returns a session/token.
- The frontend stores auth state in `AuthContext` and attaches it via the Axios client.

B. Document upload and ingestion flow
- A user uploads a file from the frontend upload workflow.
- The backend receives it through the documents/ingestion APIs.
- The file is stored under `backend/uploads/`.
- The ingestion pipeline computes a file hash for duplicate detection.
- The document text is extracted, parsed into clauses/chunks, and classified.
- Chunks are indexed into FAISS for semantic retrieval.
- Metadata is stored in MongoDB.
- Temporary intermediate files are kept under `backend/uploads/__temp__/`.
- The categorized folders under `backend/uploads/` (`Engineering`, `Finance`, `General`, `Hr`, `Legal`) show the system is already organized around document domains or tenants.

C. Question answering / retrieval flow
- The user asks a question in the UI.
- The request goes through the document API to the retriever and LLM services.
- The retriever searches FAISS for relevant clauses.
- The LLM service uses prompts from `prompts.py` to generate an answer grounded in retrieved content.
- The result is returned to the frontend, which displays it in document or viewer pages.

D. Document browsing and administration flow
- The frontend dashboard and document pages read metadata from the backend.
- Filters, tables, and document cards are purely presentation layers over backend data.
- Admin-like screens such as users, analytics, settings, and ingestion sources suggest the backend exposes operational metadata, not just raw document access.

Important architectural boundaries:

- Browser/UI boundary: React pages and components only talk to the backend through the Axios client.
- API boundary: FastAPI routers separate authentication, document operations, and ingestion-source management.
- Processing boundary: ingestion/extraction/parsing/retrieval/LLM concerns are isolated into services instead of being embedded in route handlers.
- Storage boundary: raw files live on disk, metadata lives in MongoDB, and semantic indexes live in FAISS artifacts on disk.
- External AI boundary: Gemini is an external dependency for classification and Q&A generation.
- Maintenance boundary: cache cleanup and index rebuilds are separate scripts, showing the system can recover derived artifacts without rebuilding the entire application.

In short, this repo is an AI document intelligence platform where the frontend handles document-centric UX, the backend coordinates authentication and APIs, and the service layer turns uploaded files into searchable, classifiable, LLM-grounded knowledge using MongoDB for metadata and FAISS for vector retrieval.

Main entry points: [`backend/app/main.py`](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/backend/app/main.py) (API app), [`backend/app/services/ingestion.py`](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/backend/app/services/ingestion.py) (Ingestion pipeline), [`backend/app/api/documents.py`](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/backend/app/api/documents.py) (Documents API), [`frontend/src/App.jsx`](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/frontend/src/App.jsx) (App routes)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph "Frontend"
  frontend_main["App entry<br/>react shell<br/>[main.jsx]"]
  frontend_app["App routes<br/>route shell<br/>[App.jsx]"]
  auth_context["Auth state<br/>client auth<br/>[AuthContext.jsx]"]
  axios_client["API client<br/>http boundary<br/>[axios.js]"]
  ui_pages["Pages<br/>presentation views"]
  ui_components["UI components<br/>shared widgets<br/>[Layout.jsx]"]
end

subgraph "API Layer"
  backend_main["API app<br/>fastapi entrypoint<br/>[main.py]"]
  api_auth["Auth API<br/>router<br/>[auth.py]"]
  api_documents["Documents API<br/>router<br/>[documents.py]"]
  api_sources["Sources API<br/>router"]
end

subgraph "Services"
  ingestion_service["Ingestion pipeline<br/>service orchestrator<br/>[ingestion.py]"]
  extractor_service["Extractor<br/>content extraction<br/>[extractor.py]"]
  parser_service["Parser<br/>text normalization<br/>[parser.py]"]
  retriever_service["Retriever<br/>vector search<br/>[retriever.py]"]
  llm_service["LLM service<br/>gemini wrapper<br/>[llm.py]"]
  prompts_service["Prompts<br/>prompt templates<br/>[prompts.py]"]
  storage_service["Storage<br/>[storage.py]"]
  summarizer_service["Summarizer<br/>content summary<br/>[summarizer.py]"]
end

subgraph "Core"
  config_core["Config<br/>settings<br/>[config.py]"]
  logger_core["Logger<br/>logging setup<br/>[logger.py]"]
end

subgraph "Models"
  models_domain["Domain models<br/>schemas<br/>[models.py]"]
  model_sources["Source model<br/>schema"]
end

subgraph "Runtime Storage"
  uploads_disk[("Uploads<br/>local files")]
  mongo_db[("MongoDB<br/>metadata store")]
  faiss_indexes[("FAISS indexes<br/>vector store")]
end

subgraph "Ops"
  maintenance_scripts["Maintenance<br/>ops scripts"]
  deploy_files["Deployment<br/>cloud config"]
end

subgraph "External"
  gemini_api(("Gemini<br/>external llm"))
end

browser(("Browser<br/>user client"))

browser -->|"opens"| frontend_main
frontend_main -->|"boots"| frontend_app
frontend_app -->|"uses"| auth_context
frontend_app -->|"calls"| axios_client
frontend_app -->|"renders"| ui_pages
ui_pages -->|"composes"| ui_components
axios_client -->|"requests"| backend_main
backend_main -->|"routes to"| api_auth
backend_main -->|"routes to"| api_documents
backend_main -->|"routes to"| api_sources
api_auth -->|"validates"| models_domain
api_documents -->|"orchestrates"| ingestion_service
api_documents -->|"queries"| retriever_service
api_documents -->|"summarizes"| summarizer_service
api_sources -->|"manages"| model_sources
ingestion_service -->|"extracts"| extractor_service
ingestion_service -->|"parses"| parser_service
ingestion_service -->|"stores"| storage_service
ingestion_service -->|"indexes"| retriever_service
ingestion_service -->|"writes metadata"| models_domain
extractor_service -->|"reads"| uploads_disk
storage_service -->|"writes"| uploads_disk
retriever_service -->|"searches"| faiss_indexes
retriever_service -->|"loads metadata"| mongo_db
llm_service -->|"invokes"| gemini_api
llm_service -->|"formats"| prompts_service
summarizer_service -->|"uses"| llm_service
backend_main -->|"configures"| config_core
backend_main -->|"logs"| logger_core
api_documents -->|"reads/writes"| mongo_db
maintenance_scripts -->|"rebuilds"| faiss_indexes
maintenance_scripts -->|"scans"| uploads_disk
deploy_files -->|"deploys"| backend_main

click frontend_main "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/frontend/src/main.jsx"
click frontend_app "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/frontend/src/App.jsx"
click auth_context "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/frontend/src/context/AuthContext.jsx"
click axios_client "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/frontend/src/api/axios.js"
click ui_components "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/frontend/src/components/Layout.jsx"
click backend_main "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/backend/app/main.py"
click api_auth "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/backend/app/api/auth.py"
click api_documents "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/backend/app/api/documents.py"
click api_sources "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/backend/app/api/ingestion_sources.py"
click ingestion_service "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/backend/app/services/ingestion.py"
click extractor_service "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/backend/app/services/extractor.py"
click parser_service "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/backend/app/services/parser.py"
click retriever_service "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/backend/app/services/retriever.py"
click llm_service "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/backend/app/services/llm.py"
click prompts_service "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/backend/app/services/prompts.py"
click storage_service "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/backend/app/services/storage.py"
click summarizer_service "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/backend/app/services/summarizer.py"
click config_core "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/backend/app/core/config.py"
click logger_core "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/backend/app/core/logger.py"
click models_domain "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/backend/app/models/models.py"
click model_sources "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/backend/app/models/ingestion_source.py"
click uploads_disk "https://github.com/kevinjose-ma/ai-document-classification-summarization/tree/main/backend/uploads"
click maintenance_scripts "https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/main/backend/rebuild_all_faiss_indices.py"
click deploy_files "https://github.com/kevinjose-ma/ai-document-classification-summarization/tree/main/Dockerfile"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class frontend_main,frontend_app,auth_context,axios_client,ui_pages,ui_components,maintenance_scripts,deploy_files toneBlue
class backend_main,api_auth,api_documents,api_sources,gemini_api toneAmber
class ingestion_service,extractor_service,parser_service,retriever_service,llm_service,prompts_service,storage_service,summarizer_service toneMint
class config_core,logger_core toneRose
class models_domain,model_sources toneIndigo
class uploads_disk,mongo_db,faiss_indexes toneTeal
class browser toneNeutral
```

## Components

### Frontend

React/Vite UI and client-side state

- [App entry](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/frontend/src/main.jsx): react shell · Bootstraps the React app and route composition · `frontend/src/main.jsx`
- [App routes](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/frontend/src/App.jsx): route shell · Top-level app composition and navigation · `frontend/src/App.jsx`
- [Auth state](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/frontend/src/context/AuthContext.jsx): client auth · Holds browser-side session and route protection state · `frontend/src/context/AuthContext.jsx`
- [API client](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/frontend/src/api/axios.js): http boundary · Shared frontend HTTP client for backend calls · `frontend/src/api/axios.js`
- **Pages**: presentation views · Login, dashboard, upload, document, analytics, and settings screens
- [UI components](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/frontend/src/components/Layout.jsx): shared widgets · Sidebar, tables, cards, forms, dialogs, toasts, and status indicators · `frontend/src/components/Layout.jsx`

### API Layer

FastAPI entrypoint and routers

- [API app](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/backend/app/main.py): fastapi entrypoint · Backend application bootstrap and router wiring · `backend/app/main.py`
- [Auth API](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/backend/app/api/auth.py): router · Login and registration endpoints · `backend/app/api/auth.py`
- [Documents API](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/backend/app/api/documents.py): router · Document CRUD, browsing, and question-answering endpoints · `backend/app/api/documents.py`
- [Sources API](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/backend/app/api/ingestion_sources.py): router · Ingestion-source management endpoints · `backend/app/api/ingestion_sources.py`

### Services

Document ingestion, retrieval, and LLM orchestration

- [Ingestion pipeline](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/backend/app/services/ingestion.py): service orchestrator · Coordinates upload processing, parsing, storage, and indexing · `backend/app/services/ingestion.py`
- [Extractor](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/backend/app/services/extractor.py): content extraction · Pulls raw text and content from uploaded files · `backend/app/services/extractor.py`
- [Parser](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/backend/app/services/parser.py): text normalization · Normalizes and chunks extracted content for downstream use · `backend/app/services/parser.py`
- [Retriever](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/backend/app/services/retriever.py): vector search · Searches FAISS indexes for semantically relevant passages · `backend/app/services/retriever.py`
- [LLM service](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/backend/app/services/llm.py): gemini wrapper · Wraps external model calls for answers and summaries · `backend/app/services/llm.py`
- [Prompts](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/backend/app/services/prompts.py): prompt templates · Centralized prompt construction for generation tasks · `backend/app/services/prompts.py`
- [Storage](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/backend/app/services/storage.py): artifact storage · Abstracts local file and derived artifact persistence · `backend/app/services/storage.py`
- [Summarizer](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/backend/app/services/summarizer.py): content summary · Produces condensed document outputs · `backend/app/services/summarizer.py`

### Core

Configuration, logging, and shared utilities

- [Config](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/backend/app/core/config.py): settings · Environment and application configuration · `backend/app/core/config.py`
- [Logger](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/backend/app/core/logger.py): logging setup · Structured logging configuration · `backend/app/core/logger.py`

### Models

Domain schemas and persistence models

- [Domain models](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/backend/app/models/models.py): schemas · Main document and user data models · `backend/app/models/models.py`
- [Source model](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/backend/app/models/ingestion_source.py): schema · Ingestion source metadata model · `backend/app/models/ingestion_source.py`

### Runtime Storage

Local artifacts and external backing stores

- [Uploads](https://github.com/kevinjose-ma/ai-document-classification-summarization/tree/HEAD/backend/uploads): local files · Raw documents and temporary artifacts on disk · `backend/uploads`
- **MongoDB**: metadata store · Persistent metadata and application records
- **FAISS indexes**: vector store · On-disk semantic indexes for retrieval

### Ops

Maintenance scripts and deployment

- [Maintenance](https://github.com/kevinjose-ma/ai-document-classification-summarization/blob/HEAD/backend/rebuild_all_faiss_indices.py): ops scripts · Cache cleanup and FAISS rebuild utilities · `backend/rebuild_all_faiss_indices.py`
- [Deployment](https://github.com/kevinjose-ma/ai-document-classification-summarization/tree/HEAD/Dockerfile): cloud config · Container and platform deployment definitions · `Dockerfile`

### External

Outside services and dependencies

- **Gemini**: external llm · External model provider for generation and classification

### Other components

- **Browser**: user client · User-facing entry point for the document workflow

## Connections

- Browser → App entry: opens · User loads the web application
- App entry → App routes: boots · App entry composes the route tree
- App routes → Auth state: uses · Routes and UI depend on auth state
- App routes → API client: calls · Frontend uses the shared API client
- App routes → Pages: renders · Pages provide the main user workflows
- Pages → UI components: composes · Pages are built from shared widgets
- API client → API app: requests · All backend communication goes through the API boundary
- API app → Auth API: routes to · FastAPI mounts authentication endpoints
- API app → Documents API: routes to · FastAPI mounts document endpoints
- API app → Sources API: routes to · FastAPI mounts ingestion-source endpoints
- Auth API → Domain models: validates · Authentication uses domain schemas and user records
- Documents API → Ingestion pipeline: orchestrates · Document operations drive ingestion and retrieval workflows
- Documents API → Retriever: queries · Q&A requests search indexed content
- Documents API → Summarizer: summarizes · Document views can request condensed outputs
- Sources API → Source model: manages · Ingestion sources are persisted as source records
- Ingestion pipeline → Extractor: extracts · Pipeline begins by extracting raw content
- Ingestion pipeline → Parser: parses · Pipeline normalizes and chunks extracted text
- Ingestion pipeline → Storage: stores · Pipeline persists files and derived artifacts
- Ingestion pipeline → Retriever: indexes · Chunks are prepared for semantic retrieval
- Ingestion pipeline → Domain models: writes metadata · Document metadata and versioning state are persisted
- Extractor → Uploads: reads · Extraction reads uploaded files from disk
- Storage → Uploads: writes · Raw uploads and temporary artifacts are stored locally
- Retriever → FAISS indexes: searches · Semantic lookup runs against FAISS artifacts
- Retriever → MongoDB: loads metadata · Retrieval is paired with persisted document metadata
- LLM service → Gemini: invokes · Generation and classification use the external model provider
- LLM service → Prompts: formats · LLM calls are built from shared prompt templates
- Summarizer → LLM service: uses · Summaries are generated through the LLM wrapper
- API app → Config: configures · Application startup is driven by shared settings
- API app → Logger: logs · Backend startup and requests use centralized logging
- Documents API → MongoDB: reads/writes · Document metadata is persisted in the database
- Maintenance → FAISS indexes: rebuilds · Operational scripts regenerate vector indexes
- Maintenance → Uploads: scans · Rebuild and cleanup operate over stored artifacts
- Deployment → API app: deploys · Container and platform config package the backend application
