How mustafadngl/regulatory-rag works
Regulatory RAG ingests EU regulatory documents into a cited, searchable corpus, then answers questions using retrieved extracts and an OpenAI-compatible model. Its separate golden-set evaluator measures retrieval, citations, and refusals and applies a quality gate. The main paths are visible in the sampled API, ingestion, RAG, and evaluation code. README capabilities also inform the corpus-source and evaluation branches; unsampled subsystem details are not inferred. Although the README diagram mentions Redis, no sampled runtime code shows it in use.
Main entry points: rag.py RAG service · main.py App setup · ingest.py Ingestion command · run.py Evaluation runner
Components 22
Question API
- Ask endpoint
app/api/ask.py - Health endpoint
app/api/health.py - App setup
app/main.py
Corpus ingestion
- Ingestion command
app/corpus/ingest.py - EU Cellar
- Regulation parser
app/corpus/eurlex.py - Structure chunker
app/corpus/chunking.py - Article and chunk models
app/corpus/models.py
Retrieval and answering
- RAG service
app/rag.py - Embedding client
app/corpus/embeddings.py - Chunk store
app/corpus/store.py - Chat client
app/generation.py - Text normalization
app/text.py
Quality evaluation
- Golden questions
evaluation/golden_set.yaml - Evaluation runner
app/evaluation/run.py - Quality metrics
app/evaluation/metrics.py - Evaluation report
evaluation/report.json
Shared infrastructure
- Provider HTTP client
app/upstream.py - Rate limiter
app/rate_limit.py
Other components
- Questioner
- PostgreSQL and pgvector
- Model provider
Connections 28
- Questioner → Ask endpoint: submits question
README.md - Ask endpoint → RAG service: calls answer
app/api/ask.py - App setup → Ask endpoint: registers route
app/main.py - App setup → Health endpoint: registers route
app/main.py - App setup → RAG service: builds service
app/main.py - App setup → Chunk store: constructs store
app/main.py - App setup → Embedding client: constructs client
app/main.py - App setup → Chat client: constructs client
app/main.py - RAG service → Embedding client: embeds query
app/rag.py - RAG service → Chunk store: searches chunks
app/rag.py - RAG service → Chat client: generates answer
app/rag.py - RAG service → Text normalization: normalizes answer
app/rag.py - Chunk store → PostgreSQL and pgvector: reads and writes
app/corpus/store.py - Ingestion command → EU Cellar: fetches regulation
README.md - Ingestion command → Regulation parser: parses document
app/corpus/ingest.py - Regulation parser → Article and chunk models: creates articles
app/corpus/eurlex.py - Ingestion command → Structure chunker: chunks articles
app/corpus/ingest.py - Structure chunker → Article and chunk models: creates chunks
app/corpus/chunking.py - Ingestion command → Embedding client: embeds passages
app/corpus/ingest.py - Ingestion command → Chunk store: stores chunks
app/corpus/ingest.py - Embedding client → Provider HTTP client: posts embeddings
app/corpus/embeddings.py - Chat client → Provider HTTP client: posts completions
app/generation.py - Provider HTTP client → Rate limiter: paces requests
app/upstream.py - Provider HTTP client → Model provider: sends HTTP requests
app/upstream.py - Evaluation runner → Golden questions: loads questions
app/evaluation/run.py - Evaluation runner → RAG service: evaluates answers
app/evaluation/run.py - Evaluation runner → Quality metrics: scores outcomes
app/evaluation/run.py - Evaluation runner → Evaluation report: writes report
app/evaluation/run.py