# sajidghoto/misaq-ai architecture

> Architecture diagram of the GitHub repository sajidghoto/misaq-ai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sajidghoto/misaq-ai
- Repository: https://github.com/sajidghoto/misaq-ai

## Overview

Misaq AI is a research prototype for statute-grounded contract-clause analysis. Its principal workflow parses and processes a frozen statute corpus, chunks and indexes it, retrieves relevant provisions for clause analysis, then extracts and verifies citations against the corpus. Experiment and evaluation components support reproducible retrieval and generation studies; contract ingestion is a separate documented capability. The main workflow follows the README; some internal links and ingestion wiring are not shown in the excerpts, so those relationships are omitted or marked unverified.

Main entry points: [`src/misaq/generation/pipeline.py`](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/generation/pipeline.py) (Clause generation), [`src/misaq/domain/statute.py`](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/domain/statute.py) (Statute corpus), [`src/misaq/parsing/pipeline.py`](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/parsing/pipeline.py) (PDF parsing), [`src/misaq/processing/pipeline.py`](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/processing/pipeline.py) (Statute processing)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_corpus["Corpus preparation"]
  node_statutes["Statute corpus<br/>[statute.py]"]
  node_parsing["PDF parsing<br/>[pipeline.py]"]
  node_processing["Statute processing<br/>[pipeline.py]"]
  node_chunks["Section-aware chunks<br/>[chunker.py]"]
end

subgraph group_retrieval["Retrieval system"]
  node_embedding["Text embeddings"]
  node_index[("FAISS index<br/>[index.py]")]
  node_retriever["Dense retrieval<br/>[retriever.py]"]
end

subgraph group_analysis["Clause analysis"]
  node_generation["Clause generation<br/>[pipeline.py]"]
  node_backends["Generation backends<br/>[base.py]"]
  node_prompts["Analysis prompts<br/>[prompts.py]"]
  node_cache[("Replay cache<br/>[cache.py]")]
  node_citation_extract["Citation extraction<br/>[extract.py]"]
  node_citation_verify["Citation verification<br/>[verify.py]"]
  node_registry[("Corpus registry<br/>[registry.py]")]
end

subgraph group_ingestion["Contract ingestion"]
  node_contract_reader["Contract readers"]
  node_structure["Document structure<br/>[builder.py]"]
  node_ingestion_eval["Ingestion evaluation<br/>[ingestion_eval.py]"]
end

subgraph group_research["Research evaluation"]
  node_retrieval_eval["Retrieval metrics<br/>[metrics.py]"]
  node_experiments["Experiment runner<br/>[runner.py]"]
  node_domain["Domain contracts"]
end

node_researcher(("Researcher"))

node_researcher -->|"starts parse"| node_parsing
node_parsing -->|"produces records"| node_statutes
node_statutes -->|"processes corpus"| node_processing
node_processing -->|"creates chunks"| node_chunks
node_chunks -->|"embeds text"| node_embedding
node_embedding -->|"builds vectors"| node_index
node_index -->|"searches index"| node_retriever
node_retriever -->|"supplies context"| node_generation
node_generation -->|"requests generation"| node_backends
node_generation -->|"assembles prompts"| node_prompts
node_generation -->|"reads or writes"| node_cache
node_generation -->|"extracts citations"| node_citation_extract
node_generation -->|"verifies citations"| node_citation_verify
node_citation_verify -->|"checks corpus"| node_registry
node_registry -->|"loads processed acts"| node_statutes
node_contract_reader -->|"uses structure"| node_structure
node_researcher -->|"runs studies"| node_experiments
node_experiments -->|"computes retrieval metrics"| node_retrieval_eval
node_researcher -.->|"ingests contracts"| node_contract_reader
node_researcher -.->|"evaluates annotations"| node_ingestion_eval

click node_statutes "https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/domain/statute.py"
click node_parsing "https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/parsing/pipeline.py"
click node_processing "https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/processing/pipeline.py"
click node_chunks "https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/chunking/chunker.py"
click node_embedding "https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/embedding/sentence_transformer.py"
click node_index "https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/indexing/index.py"
click node_retriever "https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/retrieval/retriever.py"
click node_generation "https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/generation/pipeline.py"
click node_backends "https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/generation/backends/base.py"
click node_prompts "https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/generation/prompts.py"
click node_cache "https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/generation/cache.py"
click node_citation_extract "https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/verification/extract.py"
click node_citation_verify "https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/verification/verify.py"
click node_registry "https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/verification/registry.py"
click node_contract_reader "https://github.com/sajidghoto/misaq-ai/tree/main/src/misaq/ingestion/readers"
click node_structure "https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/ingestion/structure/builder.py"
click node_ingestion_eval "https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/evaluation/ingestion_eval.py"
click node_retrieval_eval "https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/retrieval/metrics.py"
click node_experiments "https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/experiments/runner.py"
click node_domain "https://github.com/sajidghoto/misaq-ai/tree/main/src/misaq/domain"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_statutes,node_parsing,node_processing,node_chunks toneBlue
class node_embedding,node_index,node_retriever toneAmber
class node_generation,node_backends,node_prompts,node_cache,node_citation_extract,node_citation_verify,node_registry toneMint
class node_contract_reader,node_structure,node_ingestion_eval toneRose
class node_retrieval_eval,node_experiments,node_domain,node_researcher toneIndigo
```

## Components

### Corpus preparation

- [Statute corpus](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/domain/statute.py): `src/misaq/domain/statute.py`
- [PDF parsing](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/parsing/pipeline.py): `src/misaq/parsing/pipeline.py`
- [Statute processing](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/processing/pipeline.py): `src/misaq/processing/pipeline.py`
- [Section-aware chunks](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/chunking/chunker.py): `src/misaq/chunking/chunker.py`

### Retrieval system

- [Text embeddings](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/embedding/sentence_transformer.py): `src/misaq/embedding/sentence_transformer.py`
- [FAISS index](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/indexing/index.py): `src/misaq/indexing/index.py`
- [Dense retrieval](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/retrieval/retriever.py): `src/misaq/retrieval/retriever.py`

### Clause analysis

- [Clause generation](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/generation/pipeline.py): `src/misaq/generation/pipeline.py`
- [Generation backends](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/generation/backends/base.py): `src/misaq/generation/backends/base.py`
- [Analysis prompts](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/generation/prompts.py): `src/misaq/generation/prompts.py`
- [Replay cache](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/generation/cache.py): `src/misaq/generation/cache.py`
- [Citation extraction](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/verification/extract.py): `src/misaq/verification/extract.py`
- [Citation verification](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/verification/verify.py): `src/misaq/verification/verify.py`
- [Corpus registry](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/verification/registry.py): `src/misaq/verification/registry.py`

### Contract ingestion

- [Contract readers](https://github.com/sajidghoto/misaq-ai/tree/main/src/misaq/ingestion/readers): `src/misaq/ingestion/readers`
- [Document structure](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/ingestion/structure/builder.py): `src/misaq/ingestion/structure/builder.py`
- [Ingestion evaluation](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/evaluation/ingestion_eval.py): `src/misaq/evaluation/ingestion_eval.py`

### Research evaluation

- [Retrieval metrics](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/retrieval/metrics.py): `src/misaq/retrieval/metrics.py`
- [Experiment runner](https://github.com/sajidghoto/misaq-ai/blob/main/src/misaq/experiments/runner.py): `src/misaq/experiments/runner.py`
- [Domain contracts](https://github.com/sajidghoto/misaq-ai/tree/main/src/misaq/domain): `src/misaq/domain`

### Other components

- **Researcher**

## Connections

- Researcher → PDF parsing: starts parse (evidence: `README.md`)
- PDF parsing → Statute corpus: produces records (evidence: `README.md`)
- Statute corpus → Statute processing: processes corpus (evidence: `README.md`)
- Statute processing → Section-aware chunks: creates chunks (evidence: `README.md`)
- Section-aware chunks → Text embeddings: embeds text (evidence: `README.md`)
- Text embeddings → FAISS index: builds vectors (evidence: `README.md`)
- FAISS index → Dense retrieval: searches index (evidence: `src/misaq/retrieval/retriever.py`)
- Dense retrieval → Clause generation: supplies context (evidence: `src/misaq/generation/pipeline.py`)
- Clause generation → Generation backends: requests generation (evidence: `src/misaq/generation/pipeline.py`)
- Clause generation → Analysis prompts: assembles prompts (evidence: `src/misaq/generation/pipeline.py`)
- Clause generation → Replay cache: reads or writes (evidence: `src/misaq/generation/pipeline.py`)
- Clause generation → Citation extraction: extracts citations (evidence: `src/misaq/generation/pipeline.py`)
- Clause generation → Citation verification: verifies citations (evidence: `src/misaq/generation/pipeline.py`)
- Citation verification → Corpus registry: checks corpus (evidence: `src/misaq/verification/verify.py`)
- Corpus registry → Statute corpus: loads processed acts (evidence: `src/misaq/verification/registry.py`)
- Contract readers → Document structure: uses structure
- Researcher → Experiment runner: runs studies (evidence: `README.md`)
- Experiment runner → Retrieval metrics: computes retrieval metrics (evidence: `src/misaq/experiments/runner.py`)
- Researcher → Contract readers: ingests contracts
- Researcher → Ingestion evaluation: evaluates annotations
