How vedhavyav/legalrag works
LegalRAG ingests Indian Kanoon judgments, normalizes them, and saves raw and processed documents; its README describes chunking, indexing, retrieval, and grounded local-LLM answering as the intended user workflow. The sampled runtime shows hybrid retrieval feeding answer generation, and retrieval evaluation comparing BM25, semantic, and hybrid methods. The excerpts do not establish how processed files enter the database or how chunks are populated, so those ingestion-to-index links are omitted. The README also describes answer-quality evaluation, but no such implementation is shown.
Main entry points: index.py Chunk loader and index · pipeline.py Ingestion pipeline · search.py BM25 search · hybrid_search.py Hybrid search
Components 22
Legal Ingestion
- Ingestion entry
app/main.py - Kanoon client
app/ingestion/indiankanoon.py - Ingestion pipeline
app/ingestion/pipeline.py - Document parser
app/ingestion/parser.py - Raw and processed files
app/ingestion/pipeline.py
Corpus Storage
- Chunking
app/chunking/chunker.py - Chunk persistence
app/db/chunks.py - Document repository
app/db/repository.py - Database connection
app/db/connection.py - Legal database
app/db/schema.sql
Retrieval
- BM25 ranking
app/retrieval/bm25.py - Tokenizer
app/retrieval/tokenizer.py - Chunk loader and index
app/retrieval/index.py - BM25 search
app/retrieval/search.py - Embedding index
app/retrieval/embeddings.py - Semantic search
app/retrieval/semantic_search.py - Hybrid search
app/retrieval/hybrid_search.py
Answering
- Question entry
app/ask.py - Answer generator
app/generation/answer_generator.py
Evaluation
- Retrieval evaluation
app/evaluation/evaluate_retrieval.py
Other components
- Legal user
- Ollama runtime
Connections 27
- Ingestion entry → Kanoon client: creates
app/main.py - Ingestion entry → Ingestion pipeline: calls
app/main.py - Ingestion pipeline → Kanoon client: fetches documents
app/ingestion/pipeline.py - Ingestion pipeline → Document parser: normalizes
app/ingestion/pipeline.py - Ingestion pipeline → Raw and processed files: writes files
app/ingestion/pipeline.py - Chunk persistence → Chunking: uses
app/db/chunks.py - Document repository → Database connection: opens connection
app/db/repository.py - Chunk persistence → Database connection: opens connection
app/db/chunks.py - Chunk loader and index → Database connection: loads chunks
app/retrieval/index.py - Chunk loader and index → Legal database: reads chunks
app/retrieval/index.py - Document repository → Legal database: upserts documents
app/db/repository.py - Chunk loader and index → BM25 ranking: builds index
app/retrieval/index.py - Chunk loader and index → Tokenizer: tokenizes corpus
app/retrieval/index.py - BM25 ranking → Tokenizer: tokenizes text
app/retrieval/bm25.py - BM25 search → Chunk loader and index: loads chunks
app/retrieval/search.py - BM25 search → Tokenizer: checks query
app/retrieval/search.py - Embedding index → Chunk loader and index: loads chunks
app/retrieval/embeddings.py - Semantic search → Embedding index: reads vectors
app/retrieval/semantic_search.py - Hybrid search → BM25 search: retrieves candidates
app/retrieval/hybrid_search.py - Hybrid search → Semantic search: retrieves candidates
app/retrieval/hybrid_search.py - Answer generator → Hybrid search: retrieves passages
app/generation/answer_generator.py - Answer generator → Ollama runtime: requests completion
app/generation/answer_generator.py - Retrieval evaluation → BM25 search: evaluates retrieval
app/evaluation/evaluate_retrieval.py - Retrieval evaluation → Semantic search: evaluates retrieval
app/evaluation/evaluate_retrieval.py - Retrieval evaluation → Hybrid search: evaluates retrieval
app/evaluation/evaluate_retrieval.py - Legal user → Question entry: asks question
- Question entry → Answer generator: dispatches question
app/ask.py