# udamale/rag_pipeline architecture

> Architecture diagram of the GitHub repository udamale/rag_pipeline, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/udamale/rag_pipeline
- Repository: https://github.com/udamale/rag_pipeline

## Overview

This is a banking-focused Retrieval-Augmented Generation service: operators ingest local or S3 documents, and users submit questions to retrieve grounded answers with citations. The graph separates document loading, chunking and indexing from query embedding, search, context assembly and generation. The sampled source confirms the API and core RAG calls; Terraform paths support the deployment components shown, but their detailed wiring and the static UI’s client behavior were not sampled.

Main entry points: [`app/api/main.py`](https://github.com/udamale/rag_pipeline/blob/main/app/api/main.py) (FastAPI service), [`app/ingestion/indexer.py`](https://github.com/udamale/rag_pipeline/blob/main/app/ingestion/indexer.py) (Event ingestion), [`app/ingestion/loader.py`](https://github.com/udamale/rag_pipeline/blob/main/app/ingestion/loader.py) (Document loader), [`app/retrieval/search.py`](https://github.com/udamale/rag_pipeline/blob/main/app/retrieval/search.py) (Vector and hybrid search)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["API surface"]
  node_api["FastAPI service<br/>[main.py]"]
  node_webui["Web UI<br/>[index.html]"]
end

subgraph group_ingestion["Document ingestion"]
  node_loader["Document loader<br/>[loader.py]"]
  node_chunker["Chunking<br/>[chunker.py]"]
  node_indexer["Index management<br/>[indexer.py]"]
  node_lambdaingest["Event ingestion<br/>[indexer.py]"]
  node_embedder["Text embeddings<br/>[embeddings.py]"]
end

subgraph group_retrieval["RAG retrieval"]
  node_search["Vector and hybrid search<br/>[search.py]"]
  node_rag["Answer orchestration<br/>[rag.py]"]
end

subgraph group_platform["AWS platform"]
  node_s3[("S3 documents<br/>[s3.tf]")]
  node_opensearch[("OpenSearch index<br/>[opensearch.tf]")]
  node_bedrockembed{{"Bedrock embedding model"}}
  node_bedrockllm{{"Bedrock LLM"}}
  node_alb{{"Load balancer<br/>[alb.tf]"}}
  node_ecs["ECS service<br/>[ecs.tf]"]
  node_lambda{{"AWS Lambda<br/>[lambda.tf]"}}
end

node_user(("Questioner"))
node_operator(("Ingest operator"))
node_localdocs["Local documents"]

node_operator -->|"submits ingest"| node_api
node_user -->|"asks question"| node_api
node_api -->|"serves UI"| node_webui
node_api -->|"loads source"| node_loader
node_localdocs -->|"reads files"| node_loader
node_s3 -->|"reads objects"| node_loader
node_api -->|"chunks documents"| node_chunker
node_api -->|"indexes chunks"| node_indexer
node_indexer -->|"embeds chunks"| node_embedder
node_embedder -->|"invokes model"| node_bedrockembed
node_indexer -->|"creates and writes"| node_opensearch
node_s3 -.->|"object event"| node_lambda
node_lambda -.->|"invokes handler"| node_lambdaingest
node_lambdaingest -.->|"loads objects"| node_loader
node_lambdaingest -.->|"chunks documents"| node_chunker
node_lambdaingest -.->|"indexes chunks"| node_indexer
node_api -->|"answers question"| node_rag
node_rag -->|"retrieves passages"| node_search
node_search -->|"embeds query"| node_embedder
node_search -->|"searches index"| node_opensearch
node_rag -->|"generates answer"| node_bedrockllm
node_alb -->|"routes traffic"| node_ecs

click node_s3 "https://github.com/udamale/rag_pipeline/blob/main/terraform/s3.tf"
click node_api "https://github.com/udamale/rag_pipeline/blob/main/app/api/main.py"
click node_webui "https://github.com/udamale/rag_pipeline/blob/main/app/api/static/index.html"
click node_loader "https://github.com/udamale/rag_pipeline/blob/main/app/ingestion/loader.py"
click node_chunker "https://github.com/udamale/rag_pipeline/blob/main/app/ingestion/chunker.py"
click node_indexer "https://github.com/udamale/rag_pipeline/blob/main/app/ingestion/indexer.py"
click node_lambdaingest "https://github.com/udamale/rag_pipeline/blob/main/app/ingestion/indexer.py"
click node_embedder "https://github.com/udamale/rag_pipeline/blob/main/app/ingestion/embeddings.py"
click node_search "https://github.com/udamale/rag_pipeline/blob/main/app/retrieval/search.py"
click node_rag "https://github.com/udamale/rag_pipeline/blob/main/app/retrieval/rag.py"
click node_opensearch "https://github.com/udamale/rag_pipeline/blob/main/terraform/opensearch.tf"
click node_alb "https://github.com/udamale/rag_pipeline/blob/main/terraform/alb.tf"
click node_ecs "https://github.com/udamale/rag_pipeline/blob/main/terraform/ecs.tf"
click node_lambda "https://github.com/udamale/rag_pipeline/blob/main/terraform/lambda.tf"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_api,node_webui toneBlue
class node_loader,node_chunker,node_indexer,node_lambdaingest,node_embedder toneAmber
class node_search,node_rag toneMint
class node_s3,node_opensearch,node_bedrockembed,node_bedrockllm,node_alb,node_ecs,node_lambda toneRose
class node_user,node_operator,node_localdocs toneIndigo
```

## Components

### API surface

- [FastAPI service](https://github.com/udamale/rag_pipeline/blob/main/app/api/main.py): `app/api/main.py`
- [Web UI](https://github.com/udamale/rag_pipeline/blob/main/app/api/static/index.html): `app/api/static/index.html`

### Document ingestion

- [Document loader](https://github.com/udamale/rag_pipeline/blob/main/app/ingestion/loader.py): `app/ingestion/loader.py`
- [Chunking](https://github.com/udamale/rag_pipeline/blob/main/app/ingestion/chunker.py): `app/ingestion/chunker.py`
- [Index management](https://github.com/udamale/rag_pipeline/blob/main/app/ingestion/indexer.py): `app/ingestion/indexer.py`
- [Event ingestion](https://github.com/udamale/rag_pipeline/blob/main/app/ingestion/indexer.py): `app/ingestion/indexer.py`
- [Text embeddings](https://github.com/udamale/rag_pipeline/blob/main/app/ingestion/embeddings.py): `app/ingestion/embeddings.py`

### RAG retrieval

- [Vector and hybrid search](https://github.com/udamale/rag_pipeline/blob/main/app/retrieval/search.py): `app/retrieval/search.py`
- [Answer orchestration](https://github.com/udamale/rag_pipeline/blob/main/app/retrieval/rag.py): `app/retrieval/rag.py`

### AWS platform

- [S3 documents](https://github.com/udamale/rag_pipeline/blob/main/terraform/s3.tf): `terraform/s3.tf`
- [OpenSearch index](https://github.com/udamale/rag_pipeline/blob/main/terraform/opensearch.tf): `terraform/opensearch.tf`
- **Bedrock embedding model**
- **Bedrock LLM**
- [Load balancer](https://github.com/udamale/rag_pipeline/blob/main/terraform/alb.tf): `terraform/alb.tf`
- [ECS service](https://github.com/udamale/rag_pipeline/blob/main/terraform/ecs.tf): `terraform/ecs.tf`
- [AWS Lambda](https://github.com/udamale/rag_pipeline/blob/main/terraform/lambda.tf): `terraform/lambda.tf`

### Other components

- **Questioner**
- **Ingest operator**
- **Local documents**

## Connections

- Ingest operator → FastAPI service: submits ingest
- Questioner → FastAPI service: asks question
- FastAPI service → Web UI: serves UI
- FastAPI service → Document loader: loads source
- Local documents → Document loader: reads files
- S3 documents → Document loader: reads objects
- FastAPI service → Chunking: chunks documents
- FastAPI service → Index management: indexes chunks
- Index management → Text embeddings: embeds chunks
- Text embeddings → Bedrock embedding model: invokes model
- Index management → OpenSearch index: creates and writes
- S3 documents → AWS Lambda: object event
- AWS Lambda → Event ingestion: invokes handler
- Event ingestion → Document loader: loads objects
- Event ingestion → Chunking: chunks documents
- Event ingestion → Index management: indexes chunks
- FastAPI service → Answer orchestration: answers question
- Answer orchestration → Vector and hybrid search: retrieves passages
- Vector and hybrid search → Text embeddings: embeds query
- Vector and hybrid search → OpenSearch index: searches index
- Answer orchestration → Bedrock LLM: generates answer
- Load balancer → ECS service: routes traffic