# mishanya-maestro/electionrag architecture

> Architecture diagram of the GitHub repository mishanya-maestro/electionrag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mishanya-maestro/electionrag
- Repository: https://github.com/mishanya-maestro/electionrag

## Overview

1. **What this repo is**
   - This is a Russian-language legal-document RAG system for Belarusian нормативные правовые акты.
   - The core purpose is to answer questions from a fixed corpus of laws using retrieval over indexed text plus an LLM.

2. **Primary data source**
   - The canonical input corpus lives in `data/*.txt` and `data/*_raw.txt`.
   - Each file corresponds to a specific legal act listed in `README.md` such as `KONrb.txt`, `IZKrb.txt`, `SMIrb.txt`, etc.
   - The raw/clean split suggests a preprocessing boundary between source text normalization and downstream indexing.

3. **Ingestion and text preparation**
   - `src/parser.py`, `src/clean.py`, `src/ingest.py`, and `src/generate_fulltext.py` form the document-prep pipeline.
   - Their role is likely to parse legal text structure, clean formatting, split into searchable units, and materialize derived artifacts.
   - `src/check_sequence.py` and `src/validate_golden.py` suggest integrity checks over article ordering and a validation pass against expected outputs.

4. **Two indexing backends**
   - The repository persists retrieval data in `chroma_db/` and `chroma_db_new/`, which are Chroma vector stores.
   - It also contains `fulltext.db`, indicating a separate full-text search layer, likely SQLite-based.
   - This implies a hybrid retrieval architecture: vector similarity for semantic search and keyword/full-text search for exact legal references.

5. **Embedding layer**
   - `src/embeddings.py` is the boundary where text chunks are converted into vectors.
   - Those embeddings feed the Chroma collections stored on disk, enabling semantic retrieval across the legal corpus.

6. **Retrieval pipeline**
   - `src/retrieval.py` appears to orchestrate candidate selection from the indexed corpus.
   - `src/fts_index.py` likely integrates the full-text index, while `src/reranker.py` refines initial hits into a better-ranked context set.
   - The retrieval stage is probably the main decision point that combines semantic and lexical evidence before generation.

7. **Answer generation**
   - `src/generator.py` is the generation boundary, taking retrieved context and producing the final answer.
   - The README’s “Обычный” mode indicates a standard RAG flow: retrieval first, then LLM response grounded in the selected legal passages.
   - The “Сравнение” mode implies the generator can also call the same LLM without retrieved context, for side-by-side comparison.

8. **Web application layer**
   - `src/flask_app.py` exposes the system as a Flask web UI.
   - Frontend assets live in `src/static/` and `src/templates/`, with `index.html` and `source.html` defining the main views.
   - `src/static/app.js` and `styles.css` provide client-side interaction and presentation, but the architecture remains server-rendered Flask with static assets.

9. **Operational workflow**
   - The normal flow is: raw legal texts in `data/` → preprocessing/cleaning → full-text and vector indexing → retrieval/reranking → LLM generation → UI display.
   - The stored databases mean the app can run locally without reprocessing the corpus on every launch.

10. **Evaluation and regression testing**
   - `data/golden_eval.json` is an evaluation fixture for expected retrieval or answer behavior.
   - `src/eval_retrieval.py` and `src/test.py` indicate offline quality checks for retrieval behavior, likely used to compare index versions or tuning changes.
   - This matters architecturally because the repo is not just an app; it also includes a measurement loop for retrieval quality.

11. **Configuration and environment**
   - `src/config.py` centralizes runtime settings.
   - The presence of `.env` suggests external configuration for model/API access or local paths.
   - This separates deployment/runtime concerns from the indexing and app code.

12. **Key architectural boundaries to draw**
   - Boundary 1: source documents vs derived search artifacts.
   - Boundary 2: indexing layer vs online query-serving layer.
   - Boundary 3: retrieval/reranking vs generation.
   - Boundary 4: web UI vs backend search/LLM pipeline.
   - Boundary 5: offline evaluation scripts vs production request handling.

13. **What is persistent vs ephemeral**
   - Persistent: legal text files, Chroma stores, `fulltext.db`, and golden evaluation data.
   - Ephemeral: Flask request handling, retrieval contexts, reranked candidate lists, and generated answers.
   - The architecture is therefore data-driven and locally stateful rather than stateless cloud-native.

14. **Overall system shape**
   - This repo is best understood as a local RAG application over a curated legal corpus, with dual retrieval backends and a Flask front end.
   - The most important diagram elements are the corpus, preprocessing/indexing pipeline, Chroma + FTS stores, retrieval/reranking, LLM generation, and the comparison-oriented UI.

Main entry points: [`src/retrieval.py`](https://github.com/mishanya-maestro/electionrag/blob/master/src/retrieval.py) (Retriever), [`src/ingest.py`](https://github.com/mishanya-maestro/electionrag/blob/master/src/ingest.py) (Ingest), [`src/embeddings.py`](https://github.com/mishanya-maestro/electionrag/blob/master/src/embeddings.py) (Embeddings), [`src/flask_app.py`](https://github.com/mishanya-maestro/electionrag/blob/master/src/flask_app.py) (Flask app)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_corpus["Corpus & artifacts"]
  node_data_raw[("Raw laws<br/>legal corpus")]
  node_data_clean[("Clean laws<br/>normalized corpus")]
  node_chroma_db[("Chroma store<br/>vector index")]
  node_chroma_db_new[("New Chroma<br/>vector index")]
  node_fulltext_db[("FTS db<br/>full-text index<br/>[fulltext.db]")]
end

subgraph group_prep["Document prep"]
  node_parser["Parser<br/>[parser.py]"]
  node_clean["Cleaner<br/>normalizer<br/>[clean.py]"]
  node_ingest["Ingest<br/>pipeline<br/>[ingest.py]"]
  node_generate_fulltext["Build FTS<br/>index builder"]
  node_embeddings["Embeddings<br/>vectorization<br/>[embeddings.py]"]
  node_check_sequence["Seq check<br/>integrity check<br/>[check_sequence.py]"]
  node_database["Storage<br/>persistence<br/>[database.py]"]
end

subgraph group_serve["Query serving"]
  node_fts_index["FTS search<br/>lexical retrieval<br/>[fts_index.py]"]
  node_retrieval["Retriever<br/>hybrid retrieval<br/>[retrieval.py]"]
  node_reranker["Reranker<br/>ranking<br/>[reranker.py]"]
  node_generator["Generator<br/>rag generation<br/>[generator.py]"]
  node_flask_app["Flask app<br/>web server<br/>[flask_app.py]"]
  node_cli["CLI<br/>entrypoint<br/>[cli.py]"]
  node_templates["Templates<br/>server views"]
  node_static["Static UI<br/>frontend assets"]
end

subgraph group_eval["Evaluation"]
  node_golden_eval["Golden data<br/>eval fixture<br/>[golden_eval.json]"]
  node_eval_retrieval["Eval retrieval<br/>offline eval<br/>[eval_retrieval.py]"]
  node_validate_golden["Validate golden<br/>regression check<br/>[validate_golden.py]"]
  node_test["Test harness<br/>offline test<br/>[test.py]"]
end

node_data_raw -->|"source text"| node_parser
node_parser -->|"structured text"| node_clean
node_clean -->|"normalized docs"| node_ingest
node_ingest -->|"chunks"| node_embeddings
node_ingest -->|"documents"| node_generate_fulltext
node_embeddings -->|"vectors"| node_chroma_db
node_embeddings -->|"vectors"| node_chroma_db_new
node_generate_fulltext -->|"index data"| node_fulltext_db
node_check_sequence -.->|"integrity"| node_ingest
node_database -->|"storage access"| node_chroma_db
node_database -->|"storage access"| node_fulltext_db
node_fts_index -->|"query"| node_fulltext_db
node_retrieval -->|"semantic search"| node_chroma_db
node_retrieval -->|"lexical search"| node_fts_index
node_retrieval -->|"candidates"| node_reranker
node_reranker -->|"context"| node_generator
node_generator -->|"answer"| node_flask_app
node_flask_app -->|"render"| node_templates
node_flask_app -->|"assets"| node_static
node_cli -.->|"ops"| node_ingest
node_eval_retrieval -.->|"measure"| node_retrieval
node_validate_golden -.->|"compare"| node_golden_eval
node_test -.->|"check"| node_generator
node_test -.->|"check"| node_retrieval

click node_data_raw "https://github.com/mishanya-maestro/electionrag/tree/master/data"
click node_data_clean "https://github.com/mishanya-maestro/electionrag/tree/master/data"
click node_chroma_db "https://github.com/mishanya-maestro/electionrag/tree/master/chroma_db"
click node_chroma_db_new "https://github.com/mishanya-maestro/electionrag/tree/master/chroma_db_new"
click node_fulltext_db "https://github.com/mishanya-maestro/electionrag/blob/master/fulltext.db"
click node_parser "https://github.com/mishanya-maestro/electionrag/blob/master/src/parser.py"
click node_clean "https://github.com/mishanya-maestro/electionrag/blob/master/src/clean.py"
click node_ingest "https://github.com/mishanya-maestro/electionrag/blob/master/src/ingest.py"
click node_generate_fulltext "https://github.com/mishanya-maestro/electionrag/blob/master/src/generate_fulltext.py"
click node_embeddings "https://github.com/mishanya-maestro/electionrag/blob/master/src/embeddings.py"
click node_check_sequence "https://github.com/mishanya-maestro/electionrag/blob/master/src/check_sequence.py"
click node_database "https://github.com/mishanya-maestro/electionrag/blob/master/src/database.py"
click node_fts_index "https://github.com/mishanya-maestro/electionrag/blob/master/src/fts_index.py"
click node_retrieval "https://github.com/mishanya-maestro/electionrag/blob/master/src/retrieval.py"
click node_reranker "https://github.com/mishanya-maestro/electionrag/blob/master/src/reranker.py"
click node_generator "https://github.com/mishanya-maestro/electionrag/blob/master/src/generator.py"
click node_flask_app "https://github.com/mishanya-maestro/electionrag/blob/master/src/flask_app.py"
click node_cli "https://github.com/mishanya-maestro/electionrag/blob/master/src/cli.py"
click node_templates "https://github.com/mishanya-maestro/electionrag/tree/master/src/templates"
click node_static "https://github.com/mishanya-maestro/electionrag/tree/master/src/static"
click node_golden_eval "https://github.com/mishanya-maestro/electionrag/blob/master/data/golden_eval.json"
click node_eval_retrieval "https://github.com/mishanya-maestro/electionrag/blob/master/src/eval_retrieval.py"
click node_validate_golden "https://github.com/mishanya-maestro/electionrag/blob/master/src/validate_golden.py"
click node_test "https://github.com/mishanya-maestro/electionrag/blob/master/src/test.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_data_raw,node_data_clean,node_chroma_db,node_chroma_db_new,node_fulltext_db toneBlue
class node_parser,node_clean,node_ingest,node_generate_fulltext,node_embeddings,node_check_sequence,node_database toneAmber
class node_fts_index,node_retrieval,node_reranker,node_generator,node_flask_app,node_cli,node_templates,node_static toneMint
class node_golden_eval,node_eval_retrieval,node_validate_golden,node_test toneRose
```

## Components

### Corpus & artifacts

Canonical legal texts and persisted derived search data.

- [Raw laws](https://github.com/mishanya-maestro/electionrag/tree/master/data): legal corpus · Source legal-act text files before normalization. · `data`
- [Clean laws](https://github.com/mishanya-maestro/electionrag/tree/master/data): normalized corpus · Processed legal-act texts used for downstream indexing. · `data`
- [Chroma store](https://github.com/mishanya-maestro/electionrag/tree/master/chroma_db): vector index · Persisted semantic retrieval backend. · `chroma_db`
- [New Chroma](https://github.com/mishanya-maestro/electionrag/tree/master/chroma_db_new): vector index · Alternative or rebuilt vector store for retrieval experiments. · `chroma_db_new`
- [FTS db](https://github.com/mishanya-maestro/electionrag/blob/master/fulltext.db): full-text index · Persistent keyword/full-text retrieval layer. · `fulltext.db`

### Document prep

Parsing, cleaning, validation, and index building.

- [Parser](https://github.com/mishanya-maestro/electionrag/blob/master/src/parser.py): text parser · Parses legal documents into structured units. · `src/parser.py`
- [Cleaner](https://github.com/mishanya-maestro/electionrag/blob/master/src/clean.py): normalizer · Cleans formatting and prepares text for indexing. · `src/clean.py`
- [Ingest](https://github.com/mishanya-maestro/electionrag/blob/master/src/ingest.py): pipeline · Drives document preparation and artifact creation. · `src/ingest.py`
- [Build FTS](https://github.com/mishanya-maestro/electionrag/blob/master/src/generate_fulltext.py): index builder · Creates or refreshes the full-text search database. · `src/generate_fulltext.py`
- [Embeddings](https://github.com/mishanya-maestro/electionrag/blob/master/src/embeddings.py): vectorization · Converts text chunks into vectors for Chroma. · `src/embeddings.py`
- [Seq check](https://github.com/mishanya-maestro/electionrag/blob/master/src/check_sequence.py): integrity check · Validates article and section ordering. · `src/check_sequence.py`
- [Storage](https://github.com/mishanya-maestro/electionrag/blob/master/src/database.py): persistence · Shared database access boundary for on-disk indexes. · `src/database.py`

### Query serving

Retrieval, ranking, generation, and application layer.

- [FTS search](https://github.com/mishanya-maestro/electionrag/blob/master/src/fts_index.py): lexical retrieval · Queries the full-text index for exact legal matches. · `src/fts_index.py`
- [Retriever](https://github.com/mishanya-maestro/electionrag/blob/master/src/retrieval.py): hybrid retrieval · Orchestrates candidate selection from vector and lexical sources. · `src/retrieval.py`
- [Reranker](https://github.com/mishanya-maestro/electionrag/blob/master/src/reranker.py): ranking · Reorders retrieved passages into a better context set. · `src/reranker.py`
- [Generator](https://github.com/mishanya-maestro/electionrag/blob/master/src/generator.py): rag generation · Produces grounded answers from retrieved context. · `src/generator.py`
- [Flask app](https://github.com/mishanya-maestro/electionrag/blob/master/src/flask_app.py): web server · HTTP layer that serves the RAG experience. · `src/flask_app.py`
- [CLI](https://github.com/mishanya-maestro/electionrag/blob/master/src/cli.py): entrypoint · Command-line entry for local operations. · `src/cli.py`
- [Templates](https://github.com/mishanya-maestro/electionrag/tree/master/src/templates): server views · Rendered HTML views for the main pages. · `src/templates`
- [Static UI](https://github.com/mishanya-maestro/electionrag/tree/master/src/static): frontend assets · Client-side scripts and styles for the web UI. · `src/static`

### Evaluation

Offline checks and regression fixtures.

- [Golden data](https://github.com/mishanya-maestro/electionrag/blob/master/data/golden_eval.json): eval fixture · Expected outputs used for regression checking. · `data/golden_eval.json`
- [Eval retrieval](https://github.com/mishanya-maestro/electionrag/blob/master/src/eval_retrieval.py): offline eval · Measures retrieval quality against golden cases. · `src/eval_retrieval.py`
- [Validate golden](https://github.com/mishanya-maestro/electionrag/blob/master/src/validate_golden.py): regression check · Validates outputs against golden expectations. · `src/validate_golden.py`
- [Test harness](https://github.com/mishanya-maestro/electionrag/blob/master/src/test.py): offline test · Additional repository-level quality checks. · `src/test.py`

## Connections

- Raw laws → Parser: source text · Raw legal acts are parsed into structured content.
- Parser → Cleaner: structured text · Parsed documents are normalized for indexing.
- Cleaner → Ingest: normalized docs · Cleaned text is prepared for ingestion.
- Ingest → Embeddings: chunks · Prepared chunks are embedded for semantic search.
- Ingest → Build FTS: documents · Prepared documents are materialized into full-text artifacts.
- Embeddings → Chroma store: vectors · Embeddings are persisted to the vector store.
- Embeddings → New Chroma: vectors · Alternative vector index is also populated.
- Build FTS → FTS db: index data · Full-text search structures are written to SQLite.
- Seq check → Ingest: integrity · Sequence validation guards the preparation pipeline.
- Storage → Chroma store: storage access · Shared persistence layer reads and writes the vector store.
- Storage → FTS db: storage access · Shared persistence layer accesses the full-text database.
- FTS search → FTS db: query · Lexical search runs against the full-text index.
- Retriever → Chroma store: semantic search · Retriever pulls candidates from the vector store.
- Retriever → FTS search: lexical search · Retriever combines keyword evidence from FTS.
- Retriever → Reranker: candidates · Initial hits are refined by the reranker.
- Reranker → Generator: context · Ranked passages are passed into answer generation.
- Generator → Flask app: answer · Generated responses are returned through the web app.
- Flask app → Templates: render · Server-side views are rendered for the UI.
- Flask app → Static UI: assets · Frontend scripts and styles support interaction.
- CLI → Ingest: ops · CLI entrypoint can run indexing and maintenance tasks.
- Eval retrieval → Retriever: measure · Retrieval quality is assessed offline.
- Validate golden → Golden data: compare · Golden fixture drives regression validation.
- Test harness → Generator: check · Repository tests exercise the generation path.
- Test harness → Retriever: check · Repository tests also exercise retrieval behavior.