How udamale/rag_pipeline works
This is a banking-focused Retrieval-Augmented Generation service: operators ingest local or S3 documents, and users submit questions to retrieve grounded answers with citations. The graph separates document loading, chunking and indexing from query embedding, search, context assembly and generation. The sampled source confirms the API and core RAG calls; Terraform paths support the deployment components shown, but their detailed wiring and the static UI’s client behavior were not sampled.
Main entry points: main.py FastAPI service · indexer.py Event ingestion · loader.py Document loader · search.py Vector and hybrid search
Components 19
API surface
- FastAPI service
app/api/main.py - Web UI
app/api/static/index.html
Document ingestion
- Document loader
app/ingestion/loader.py - Chunking
app/ingestion/chunker.py - Index management
app/ingestion/indexer.py - Event ingestion
app/ingestion/indexer.py - Text embeddings
app/ingestion/embeddings.py
RAG retrieval
- Vector and hybrid search
app/retrieval/search.py - Answer orchestration
app/retrieval/rag.py
AWS platform
- S3 documents
terraform/s3.tf - OpenSearch index
terraform/opensearch.tf - Bedrock embedding model
- Bedrock LLM
- Load balancer
terraform/alb.tf - ECS service
terraform/ecs.tf - AWS Lambda
terraform/lambda.tf
Other components
- Questioner
- Ingest operator
- Local documents
Connections 22
- Ingest operator → FastAPI service: submits ingest
- Questioner → FastAPI service: asks question
- FastAPI service → Web UI: serves UI
- FastAPI service → Document loader: loads source
- Local documents → Document loader: reads files
- S3 documents → Document loader: reads objects
- FastAPI service → Chunking: chunks documents
- FastAPI service → Index management: indexes chunks
- Index management → Text embeddings: embeds chunks
- Text embeddings → Bedrock embedding model: invokes model
- Index management → OpenSearch index: creates and writes
- S3 documents → AWS Lambda: object event
- AWS Lambda → Event ingestion: invokes handler
- Event ingestion → Document loader: loads objects
- Event ingestion → Chunking: chunks documents
- Event ingestion → Index management: indexes chunks
- FastAPI service → Answer orchestration: answers question
- Answer orchestration → Vector and hybrid search: retrieves passages
- Vector and hybrid search → Text embeddings: embeds query
- Vector and hybrid search → OpenSearch index: searches index
- Answer orchestration → Bedrock LLM: generates answer
- Load balancer → ECS service: routes traffic