# adityasohner/repoir architecture

> Architecture diagram of the GitHub repository adityasohner/repoir, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/adityasohner/repoir
- Repository: https://github.com/adityasohner/repoir

## Overview

1. **Repository shape**
   - This is a **full-stack monorepo** with a Python backend at the root and a separate React frontend in `RepoIR Frontend/`.
   - The backend is the core product: it handles ingestion, indexing, search, auth, storage, and AI orchestration.
   - The frontend is a TypeScript/Vite SPA that presents the search dashboard, upload flow, and account/settings screens.

2. **Backend entrypoint and service boundary**
   - The backend appears to start from `app_server.py` with `main.py` as an additional bootstrap/entry module.
   - The backend is organized as a single API service with internal subsystems rather than multiple microservices.
   - The frontend talks to this service through `src/lib/api.ts`, so the main runtime boundary is **browser SPA ↔ FastAPI backend**.

3. **Ingestion pipeline**
   - The ingestion flow is split into distinct stages under `app/ai/pipeline/`:
     - extraction from source data,
     - normalization,
     - chunking,
     - embedding generation,
     - indexing/storage.
   - This pipeline is designed for multiple source types: documents, images, URLs, and raw text snippets.

4. **Source-specific extraction layer**
   - `app/ai/extractors/` contains specialized extractors for documents, images, URLs, OCR, and vision description.
   - Images are not just stored; they are turned into searchable semantic text via OCR and/or vision captioning.
   - This is what enables concept search over photos and scanned content, not just literal filename matching.

5. **Text preparation and semantic indexing**
   - `app/ai/normalizers/` cleans and formats content by file type before chunking.
   - `app/ai/chunker/text_chunker.py` splits content into overlapping semantic chunks.
   - `app/ai/embeddings/text_embedder.py` converts chunks into vector embeddings, which are then used for semantic retrieval.

6. **Search architecture**
   - Search is two-stage, as described in the README and reflected by `app/ai/pipeline/search_pipeline.py`.
   - Stage 1 combines **keyword retrieval** and **vector retrieval**:
     - SQLite FTS5 for lexical/BM25-style lookup,
     - FAISS for vector similarity search,
     - fusion via Reciprocal Rank Fusion.
   - Stage 2 applies an LLM-based relevance filter/re-ranker using SambaNova-hosted Llama 3.1 8B.

7. **Storage subsystem**
   - `app/storage/` is the persistence boundary and includes:
     - `db_store.py` for metadata/index records,
     - `faiss_store.py` for vector index persistence,
     - `object_store.py` for file/object handling,
     - `gdrive_store.py` for the Google Drive-backed vault,
     - `encryption_manager.py` for protecting stored content,
     - `id_generator.py` for internal identity management.
   - The architecture separates **original file storage**, **metadata/index storage**, and **vector storage**.

8. **Cloud vault and external integrations**
   - A key external dependency is **Google Drive**, used as the “RepoIR_Vault” for original files.
   - OAuth-based access is handled in `app/auth/`, and the README indicates user password/JWT-based session management as well.
   - Another external service is **SambaNova API**, which powers LLM reranking rather than local model hosting.

9. **Auth and security boundaries**
   - `app/auth/manager.py` and `app/auth/validator.py` define the backend authentication layer.
   - `app/config.py` centralizes environment-driven configuration, including API keys and secrets.
   - The design emphasizes privacy by keeping chunking, metadata indexing, and retrieval logic inside the private deployment boundary, with only necessary integrations leaving the system.

10. **Frontend structure**
    - The frontend is a Vite/React/TypeScript app with Tailwind CSS and a shadcn-style component library under `src/components/ui/`.
    - `src/pages/` contains the main app surfaces: landing, login/signup, dashboard, profile, settings, password recovery, and OAuth callback.
    - `src/components/` contains dashboard-specific building blocks such as file grid/cards, preview modal, upload zone, sidebar, top nav, stats cards, search/command palette, and layout wrappers.

11. **Frontend state and backend interaction**
    - `src/contexts/AuthContext.tsx` and `src/contexts/AppContext.tsx` hold shared auth/app state across the SPA.
    - `src/lib/api.ts` is the main client-side integration point to the backend.
    - This suggests the UI is mostly a thin presentation layer over backend-managed data, search, and session state.

12. **Dashboard and analytics flow**
    - The dashboard is not just a file browser; it includes analytics and activity visibility.
    - Components like `StatsCards`, `FileGrid`, `ImmersiveSearch`, and `FilePreviewModal` imply a user flow of upload → indexing → search → preview → manage.
    - The README’s “real-time analytics” maps to dashboard views for storage usage, category breakdowns, and recent activity.

13. **Deployment and operational tooling**
    - The repo is containerized with a root `Dockerfile`, and `.github/workflows/ci.yml` plus `security.yml` indicate automated CI and security checks.
    - The frontend has a `public/_redirects` file, which usually supports SPA routing on static hosts.
    - Frontend testing uses Vitest; the backend uses Python dependency management via `requirements.txt`.

14. **How to draw the architecture**
    - A good diagram would place the **React SPA** on one side, the **FastAPI backend** in the center, and **Google Drive / SambaNova** as external services on the edge.
    - Inside the backend, show the flow from **ingestion/extraction** into **normalization/chunking/embeddings**, then into **SQLite metadata + FAISS vectors + encrypted vault storage**.
    - Search should be shown as a separate path: **query → hybrid retrieval → rerank → ranked results back to UI**.

Main entry points: [`app/ai/pipeline/ingestion_pipeline.py`](https://github.com/adityasohner/repoir/blob/main/app/ai/pipeline/ingestion_pipeline.py) (Ingestion pipeline), [`app_server.py`](https://github.com/adityasohner/repoir/blob/main/app_server.py) (API server), [`app/auth/manager.py`](https://github.com/adityasohner/repoir/blob/main/app/auth/manager.py) (Auth manager), [`app/ai/pipeline/search_pipeline.py`](https://github.com/adityasohner/repoir/blob/main/app/ai/pipeline/search_pipeline.py) (Search pipeline)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontend["Frontend SPA"]
  node_fe_entry(("SPA entry<br/>react entry<br/>[main.tsx]"))
  node_fe_pages["Pages<br/>route views<br/>[DashboardPage.tsx]"]
  node_fe_components["Dashboard UI<br/>ui components"]
  node_fe_state(("App state<br/>react context<br/>[AppContext.tsx]"))
  node_fe_auth(("Auth state<br/>react context<br/>[AuthContext.tsx]"))
  node_fe_api["API client<br/>backend client<br/>[api.ts]"]
end

subgraph group_backend["Backend Service"]
  node_be_entry(("API server<br/>python entrypoint<br/>[app_server.py]"))
  node_be_bootstrap(("Bootstrap<br/>startup module<br/>[main.py]"))
  node_be_config["Runtime config<br/>env config<br/>[config.py]"]
  node_be_auth_mgr["Auth manager<br/>auth service<br/>[manager.py]"]
  node_be_auth_val["Auth validator<br/>security checks<br/>[validator.py]"]
  node_be_ingest{{"Ingestion pipeline<br/>content pipeline"}}
  node_be_extract["Source extractors<br/>[base.py]"]
  node_be_normalize["Normalize content<br/>text prep<br/>[common.py]"]
  node_be_chunk["Chunk text<br/>chunker<br/>[text_chunker.py]"]
  node_be_embed["Create vectors<br/>embedder<br/>[text_embedder.py]"]
  node_be_search{{"Search pipeline<br/>hybrid retrieval<br/>[search_pipeline.py]"}}
end

subgraph group_storage["Storage Layer"]
  node_st_meta[("Metadata DB<br/>sqlite store<br/>[db_store.py]")]
  node_st_vector[("Vector index<br/>faiss store<br/>[faiss_store.py]")]
  node_st_files[("Object store<br/>file store<br/>[object_store.py]")]
  node_st_vault[("Vault store<br/>gdrive store<br/>[gdrive_store.py]")]
  node_st_crypto["Encryption<br/>security layer"]
  node_st_ids(("ID generator<br/>identity service<br/>[id_generator.py]"))
end

subgraph group_external["External Services"]
  node_ext_drive[("Google Drive<br/>external vault")]
  node_ext_llm(("SambaNova LLM<br/>external reranker"))
end

node_fe_entry -->|"routes"| node_fe_pages
node_fe_pages -->|"composes"| node_fe_components
node_fe_components -->|"reads state"| node_fe_state
node_fe_components -->|"reads auth"| node_fe_auth
node_fe_state -->|"calls"| node_fe_api
node_fe_auth -->|"auth calls"| node_fe_api
node_be_entry -->|"starts"| node_be_bootstrap
node_be_entry -->|"uses"| node_be_auth_mgr
node_be_entry -->|"ingests"| node_be_ingest
node_be_entry -->|"searches"| node_be_search
node_be_entry -->|"loads"| node_be_config
node_be_auth_mgr -->|"validates"| node_be_auth_val
node_be_auth_mgr -->|"reads secrets"| node_be_config
node_be_ingest -->|"extracts"| node_be_extract
node_be_ingest -->|"normalizes"| node_be_normalize
node_be_ingest -->|"chunks"| node_be_chunk
node_be_ingest -->|"embeds"| node_be_embed
node_be_ingest -->|"records"| node_st_meta
node_be_ingest -->|"indexes"| node_st_vector
node_be_ingest -->|"stores"| node_st_files
node_be_ingest -->|"uploads"| node_st_vault
node_be_ingest -->|"encrypts"| node_st_crypto
node_be_ingest -->|"assigns ids"| node_st_ids
node_be_search -->|"FTS5 lookup"| node_st_meta
node_be_search -->|"vector lookup"| node_st_vector
node_be_search -->|"reranks"| node_ext_llm
node_st_vault -->|"backed by"| node_ext_drive
node_fe_api -->|"requests"| node_be_entry
node_be_auth_mgr -->|"session data"| node_fe_auth

click node_fe_entry "https://github.com/adityasohner/repoir/blob/main/RepoIR Frontend/src/main.tsx"
click node_fe_pages "https://github.com/adityasohner/repoir/blob/main/RepoIR Frontend/src/pages/DashboardPage.tsx"
click node_fe_components "https://github.com/adityasohner/repoir/blob/main/RepoIR Frontend/src/components/DashboardLayout.tsx"
click node_fe_state "https://github.com/adityasohner/repoir/blob/main/RepoIR Frontend/src/contexts/AppContext.tsx"
click node_fe_auth "https://github.com/adityasohner/repoir/blob/main/RepoIR Frontend/src/contexts/AuthContext.tsx"
click node_fe_api "https://github.com/adityasohner/repoir/blob/main/RepoIR Frontend/src/lib/api.ts"
click node_be_entry "https://github.com/adityasohner/repoir/blob/main/app_server.py"
click node_be_bootstrap "https://github.com/adityasohner/repoir/blob/main/main.py"
click node_be_config "https://github.com/adityasohner/repoir/blob/main/app/config.py"
click node_be_auth_mgr "https://github.com/adityasohner/repoir/blob/main/app/auth/manager.py"
click node_be_auth_val "https://github.com/adityasohner/repoir/blob/main/app/auth/validator.py"
click node_be_ingest "https://github.com/adityasohner/repoir/blob/main/app/ai/pipeline/ingestion_pipeline.py"
click node_be_extract "https://github.com/adityasohner/repoir/blob/main/app/ai/extractors/base.py"
click node_be_normalize "https://github.com/adityasohner/repoir/blob/main/app/ai/normalizers/common.py"
click node_be_chunk "https://github.com/adityasohner/repoir/blob/main/app/ai/chunker/text_chunker.py"
click node_be_embed "https://github.com/adityasohner/repoir/blob/main/app/ai/embeddings/text_embedder.py"
click node_be_search "https://github.com/adityasohner/repoir/blob/main/app/ai/pipeline/search_pipeline.py"
click node_st_meta "https://github.com/adityasohner/repoir/blob/main/app/storage/db_store.py"
click node_st_vector "https://github.com/adityasohner/repoir/blob/main/app/storage/faiss_store.py"
click node_st_files "https://github.com/adityasohner/repoir/blob/main/app/storage/object_store.py"
click node_st_vault "https://github.com/adityasohner/repoir/blob/main/app/storage/gdrive_store.py"
click node_st_crypto "https://github.com/adityasohner/repoir/blob/main/app/storage/encryption_manager.py"
click node_st_ids "https://github.com/adityasohner/repoir/blob/main/app/storage/id_generator.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_fe_entry,node_fe_pages,node_fe_components,node_fe_state,node_fe_auth,node_fe_api toneBlue
class node_be_entry,node_be_bootstrap,node_be_config,node_be_auth_mgr,node_be_auth_val,node_be_ingest,node_be_extract,node_be_normalize,node_be_chunk,node_be_embed,node_be_search toneAmber
class node_st_meta,node_st_vector,node_st_files,node_st_vault,node_st_crypto,node_st_ids toneMint
class node_ext_drive,node_ext_llm toneRose
```

## Components

### Frontend SPA

React/Vite client for search, uploads, and account management

- [SPA entry](https://github.com/adityasohner/repoir/blob/HEAD/RepoIR%20Frontend/src/main.tsx): react entry · Bootstraps the browser application and routing · `RepoIR Frontend/src/main.tsx`
- [Pages](https://github.com/adityasohner/repoir/blob/HEAD/RepoIR%20Frontend/src/pages/DashboardPage.tsx): route views · Landing, auth, dashboard, profile, settings, and callback screens · `RepoIR Frontend/src/pages/DashboardPage.tsx`
- [Dashboard UI](https://github.com/adityasohner/repoir/blob/HEAD/RepoIR%20Frontend/src/components/DashboardLayout.tsx): ui components · Layout, search, upload, preview, stats, and navigation building blocks · `RepoIR Frontend/src/components/DashboardLayout.tsx`
- [App state](https://github.com/adityasohner/repoir/blob/HEAD/RepoIR%20Frontend/src/contexts/AppContext.tsx): react context · Shared application state across dashboard flows · `RepoIR Frontend/src/contexts/AppContext.tsx`
- [Auth state](https://github.com/adityasohner/repoir/blob/HEAD/RepoIR%20Frontend/src/contexts/AuthContext.tsx): react context · Session and identity state for the SPA · `RepoIR Frontend/src/contexts/AuthContext.tsx`
- [API client](https://github.com/adityasohner/repoir/blob/HEAD/RepoIR%20Frontend/src/lib/api.ts): backend client · Main client-side integration point to the backend service · `RepoIR Frontend/src/lib/api.ts`

### Backend Service

Python API service with auth, ingestion, search, and storage subsystems

- [API server](https://github.com/adityasohner/repoir/blob/main/app_server.py): python entrypoint · Primary backend process boundary and request entry · `app_server.py`
- [Bootstrap](https://github.com/adityasohner/repoir/blob/main/main.py): startup module · Additional backend launch and wiring entry · `main.py`
- [Runtime config](https://github.com/adityasohner/repoir/blob/main/app/config.py): env config · Central environment-driven configuration for secrets and integrations · `app/config.py`
- [Auth manager](https://github.com/adityasohner/repoir/blob/main/app/auth/manager.py): auth service · Session, password, and OAuth-oriented authentication logic · `app/auth/manager.py`
- [Auth validator](https://github.com/adityasohner/repoir/blob/main/app/auth/validator.py): security checks · Token and credential validation boundary · `app/auth/validator.py`
- [Ingestion pipeline](https://github.com/adityasohner/repoir/blob/main/app/ai/pipeline/ingestion_pipeline.py): content pipeline · End-to-end pipeline from source input to indexed content · `app/ai/pipeline/ingestion_pipeline.py`
- [Source extractors](https://github.com/adityasohner/repoir/blob/main/app/ai/extractors/base.py): extractors · Document, image, URL, OCR, and vision extraction for searchable text · `app/ai/extractors/base.py`
- [Normalize content](https://github.com/adityasohner/repoir/blob/main/app/ai/normalizers/common.py): text prep · File-type-specific cleanup and formatting before chunking · `app/ai/normalizers/common.py`
- [Chunk text](https://github.com/adityasohner/repoir/blob/main/app/ai/chunker/text_chunker.py): chunker · Overlapping semantic chunking of normalized content · `app/ai/chunker/text_chunker.py`
- [Create vectors](https://github.com/adityasohner/repoir/blob/main/app/ai/embeddings/text_embedder.py): embedder · Turns chunks into semantic embeddings for retrieval · `app/ai/embeddings/text_embedder.py`
- [Search pipeline](https://github.com/adityasohner/repoir/blob/main/app/ai/pipeline/search_pipeline.py): hybrid retrieval · Two-stage search with lexical plus vector retrieval and reranking · `app/ai/pipeline/search_pipeline.py`

### Storage Layer

Persistence and vault boundaries for metadata, vectors, files, and encryption

- [Metadata DB](https://github.com/adityasohner/repoir/blob/main/app/storage/db_store.py): sqlite store · Metadata and index records, including lexical search support · `app/storage/db_store.py`
- [Vector index](https://github.com/adityasohner/repoir/blob/main/app/storage/faiss_store.py): faiss store · Persistent vector similarity index · `app/storage/faiss_store.py`
- [Object store](https://github.com/adityasohner/repoir/blob/main/app/storage/object_store.py): file store · File and object handling boundary for persisted content · `app/storage/object_store.py`
- [Vault store](https://github.com/adityasohner/repoir/blob/main/app/storage/gdrive_store.py): gdrive store · Google Drive-backed vault for original files · `app/storage/gdrive_store.py`
- [Encryption](https://github.com/adityasohner/repoir/blob/main/app/storage/encryption_manager.py): security layer · Protects stored content at rest · `app/storage/encryption_manager.py`
- [ID generator](https://github.com/adityasohner/repoir/blob/main/app/storage/id_generator.py): identity service · Internal identity generation for stored records · `app/storage/id_generator.py`

### External Services

Third-party services used for vault storage and LLM reranking

- **Google Drive**: external vault · External vault used for original file storage
- **SambaNova LLM**: external reranker · Hosted Llama 3.1 model used for relevance filtering and reranking

## Connections

- SPA entry → Pages: routes · Application entry renders route-level screens
- Pages → Dashboard UI: composes · Pages are built from dashboard and account components
- Dashboard UI → App state: reads state · UI components consume shared application state
- Dashboard UI → Auth state: reads auth · UI components depend on session state
- App state → API client: calls · App-level actions are fulfilled through backend APIs
- Auth state → API client: auth calls · Login, signup, recovery, and session flows use the API client
- API server → Bootstrap: starts · Primary server process is initialized by bootstrap wiring
- API server → Auth manager: uses · Backend request handling depends on authentication services
- API server → Ingestion pipeline: ingests · Upload and indexing requests enter the ingestion pipeline
- API server → Search pipeline: searches · Query requests are handled by the search pipeline
- API server → Runtime config: loads · Runtime settings and secrets are injected from config
- Auth manager → Auth validator: validates · Authentication manager delegates trust checks to validators
- Auth manager → Runtime config: reads secrets · Auth depends on configured credentials and tokens
- Ingestion pipeline → Source extractors: extracts · Source-specific extraction produces canonical text
- Ingestion pipeline → Normalize content: normalizes · Extracted content is cleaned by file type
- Ingestion pipeline → Chunk text: chunks · Normalized content is split into overlapping semantic chunks
- Ingestion pipeline → Create vectors: embeds · Chunks are converted into vector representations
- Ingestion pipeline → Metadata DB: records · Ingestion writes metadata and index records
- Ingestion pipeline → Vector index: indexes · Embeddings are persisted in the vector index
- Ingestion pipeline → Object store: stores · Processed file objects are retained in the object layer
- Ingestion pipeline → Vault store: uploads · Original files are sent to the Google Drive vault
- Ingestion pipeline → Encryption: encrypts · Stored content is protected before persistence
- Ingestion pipeline → ID generator: assigns ids · Stable identifiers are generated for stored assets
- Search pipeline → Metadata DB: FTS5 lookup · Lexical retrieval uses SQLite full-text search
- Search pipeline → Vector index: vector lookup · Semantic retrieval uses FAISS similarity search
- Search pipeline → SambaNova LLM: reranks · Candidate results are filtered and reranked by the hosted LLM
- Vault store → Google Drive: backed by · Vault storage is implemented on top of Google Drive
- API client → API server: requests · Browser SPA communicates with the backend service
- Auth manager → Auth state: session data · Authentication responses feed SPA session state
