How shreya-1831/mil-evid works
MIL-EVID is an evidence-ingestion and retrieval-augmented analysis prototype: source records are mapped, preprocessed, chunked, and stored; analysis code retrieves and reranks evidence, builds a guarded context, and generates a structured analysis. The sampled application exposes a health route, but does not show an API route invoking the analysis service. README-described perspectives and dynamic evidence are broader or planned capabilities; their wiring is not established by the sampled source.
Main entry points: analysis_service.py Analysis service · pipeline.py Preprocessing pipeline · evidence_ingestion_service.py Ingestion service · hybrid_retriever.py Hybrid retriever
Components 26
Evidence ingestion
- Static runners
backend/app/modules/ingestion/static_ingestion_runner.py - CSV ingestion
backend/app/modules/ingestion/static_ingestion.py - Source mappers
backend/app/modules/ingestion - Ingestion service
backend/app/services/evidence_ingestion_service.py - Preprocessing pipeline
backend/app/modules/preprocessing/pipeline.py - Cleaning
backend/app/modules/preprocessing/cleaner.py - Normalization
backend/app/modules/preprocessing/normalizer.py - Deduplication
backend/app/modules/preprocessing/deduplicator.py - Chunking
backend/app/modules/preprocessing/chunker.py
Evidence storage
- Chunk text store
backend/app/repositories/chunk_file_store.py - Metadata repository
backend/app/repositories/evidence_repository.py - PostgreSQL metadata
backend/app/database/models.py
Hybrid retrieval
- Hybrid retriever
backend/app/modules/retrieval/hybrid_retriever.py - BM25 search
backend/app/modules/retrieval/bm25_index.py - FAISS search
backend/app/modules/retrieval/faiss_index.py - Rank fusion
backend/app/modules/retrieval/rrf.py - Cross-encoder reranker
backend/app/modules/retrieval/reranker.py
Analysis application
- Analysis service
backend/app/services/analysis_service.py - Context builder
backend/app/modules/analysis/context_builder.py - Evidence guard
backend/app/modules/analysis/evidence_guard.py - Evidence analyzer
backend/app/modules/analysis/analyzer.py - Ollama client
backend/app/modules/analysis/llm_client.py - FastAPI app
backend/app/main.py - Health route
backend/app/api/routes/health.py
Other components
- Ingestion operator
- Analyst
Connections 25
- Ingestion operator → Static runners: starts ingestion
- Static runners → CSV ingestion: runs CSV
- Static runners → Source mappers: selects mapper
- CSV ingestion → Source mappers: maps records
- CSV ingestion → Ingestion service: sends batches
- Ingestion service → Preprocessing pipeline: processes documents
- Preprocessing pipeline → Cleaning: cleans
- Preprocessing pipeline → Normalization: normalizes
- Preprocessing pipeline → Deduplication: deduplicates
- Preprocessing pipeline → Chunking: chunks
- Ingestion service → Chunk text store: appends chunks
- Ingestion service → Metadata repository: persists metadata
- Metadata repository → PostgreSQL metadata: writes metadata
- Analyst → Analysis service: submits query
- Analysis service → Hybrid retriever: searches evidence
- Hybrid retriever → BM25 search: lexical search
- Hybrid retriever → FAISS search: dense search
- Hybrid retriever → Rank fusion: fuses ranks
- Analysis service → Chunk text store: reads chunk text
- Analysis service → Cross-encoder reranker: reranks candidates
- Analysis service → Context builder: builds context
- Analysis service → Evidence guard: filters evidence
- Analysis service → Evidence analyzer: analyzes context
- FastAPI app → Health route: registers route
- Evidence analyzer → Ollama client: generates analysis