How abhavbhanot/bhagwan-gita-summarization-and-verse-retrieval-rag- works
Component Breakdown
a. Data Sources (Top Layer)
• Two main branches: Bhagavad Gita and Patanjali Yoga Sutras
• Sub-components for different data types (verses, concepts, word meanings)
b. Data Ingestion Layer
• Data normalization flow
• Data cleaning and preprocessing steps
• Demonstrate schema unification process
c. Processing Pipeline (Core)
• TF-IDF Vectorization component
• Retrieval Engine with cosine similarity
• T5 Model Integration
• Query Processing Unit
d. Generation Layer
• T5-base model component
• Summary generation pipeline
• Context integration module
e. Output Layer
• JSON response formatter
• Performance metrics collector
• User interface connector
Specific Elements
a. Input Processing:
• User Query → Query Processor → Vector Space
b. Retrieval Flow:
• Vector Space → Cosine Similarity → Ranked Results
c. Generation Pipeline:
• Retrieved Verses → T5 Model → Generated Summary
d. Evaluation System:
• Output → Metrics Collector → Performance Dashboard
Technical Details to Highlight
• T5-base model architecture
• Where TF-IDF vectorization occurs
• Points where cosine similarity is calculated
• Evaluation metric weightings
Performance Monitoring
• Monitoring points for the 8 evaluation criteria
• Cost efficiency metrics
• Error handling pathways
Integration Points
• Where future improvements can be integrated
• Extensibility points for additional languages
• Where new sacred texts can be added
Key Visualization Guidelines:
• Use standard UML notation where appropriate
• Implement a clear visual hierarchy
• System boundaries
• External dependencies
• Critical paths
• Failure recovery points