How vietnh1009/rag-pipeline-visualizer works
RAG-pipeline-visualizer is a Streamlit application for configuring and inspecting a two-stage RAG workflow. Users configure offline document indexing, then query the resulting index and inspect the online answer-generation steps. The README documents the stage sequence and available strategies; sampled source confirms key orchestration and dispatch points. Some strategy implementations and UI wiring were not sampled, so those relationships are shown only where documented or directly evidenced.
Main entry points: generation_pipeline.py Generation pipeline · pipeline_runners.py Stage runners · indexing.py Indexing page · generation.py Generation page
Components 22
Streamlit interface
- App entry
app.py - Indexing page
ui/pages/indexing.py - Generation page
ui/pages/generation.py - Index settings
ui/settings - Result views
ui/results
Offline indexing
- Indexing pipeline
pipeline/indexing_pipeline.py - Stage runners
core/pipeline_runners.py - PDF loader
loader/pdf_loader.py - Chunking strategies
chunking - Embedding
embedding - Vector database
vector_db
Online generation
- Generation pipeline
pipeline/generation_pipeline.py - Query transforms
pre_retrieval - Retrieval strategies
retrieval - Rerank and filter
post_retrieval/pipeline.py - Prompt builders
prompt - LLM generation
generation
Shared services
- Pipeline cache
pipeline_cache.py - Shared LLM calls
utils/llm.py - Environment helpers
utils/env.py
Other components
- User
- Answer
Connections 26
- User → App entry: opens
README.md - App entry → Indexing page: routes to
app.py - App entry → Generation page: routes to
app.py - User → Index settings: configures
README.md - Index settings → Indexing page: configures
ui/pages/indexing.py - Indexing page → Indexing pipeline: runs
ui/pages/indexing.py - Indexing pipeline → Stage runners: delegates to
pipeline/indexing_pipeline.py - Stage runners → PDF loader: loads documents
core/pipeline_runners.py - Stage runners → Chunking strategies: chunks documents
core/pipeline_runners.py - Stage runners → Embedding: embeds chunks
core/pipeline_runners.py - Stage runners → Vector database: stores vectors
core/pipeline_runners.py - Indexing pipeline → Pipeline cache: caches steps
pipeline/indexing_pipeline.py - Indexing page → Result views: renders results
ui/pages/indexing.py - User → Generation page: submits query
README.md - Generation page → Generation pipeline: runs
README.md - Generation pipeline → Query transforms: transforms query
pipeline/generation_pipeline.py - Generation pipeline → Retrieval strategies: retrieves documents
pipeline/generation_pipeline.py - Retrieval strategies → Vector database: searches index
README.md - Generation pipeline → Rerank and filter: processes results
pipeline/generation_pipeline.py - Rerank and filter → Shared LLM calls: uses LLM
post_retrieval/pipeline.py - Generation pipeline → Prompt builders: builds prompt
pipeline/generation_pipeline.py - Generation pipeline → LLM generation: generates answer
pipeline/generation_pipeline.py - LLM generation → Answer: returns
README.md - Generation page → Result views: shows results
ui/results/vdb_results.py - PDF loader → Shared LLM calls: uses LLM
- Environment helpers → Stage runners: supports setup
core/pipeline_runners.py