# lakshayg2005/nlp_project architecture

> Architecture diagram of the GitHub repository lakshayg2005/nlp_project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lakshayg2005/nlp_project
- Repository: https://github.com/lakshayg2005/nlp_project

## Overview

This is an e-commerce copy compliance auditor. Product text enters through the API, is queued for asynchronous auditing, and passes through heuristic screening, risk classification, and phrase-level span extraction before results and flags are persisted for dashboard review. Human decisions are stored as feedback for classifier retraining. The graph also shows the shared risk taxonomy and text/span utilities. Pipeline and model-stage calls are supported by excerpts; queue and dashboard behavior are documented in the README, while unsampled implementation details are not assumed.

Main entry points: [`auditor/pipeline.py`](https://github.com/lakshayg2005/nlp_project/blob/main/auditor/pipeline.py) (Audit Orchestrator), [`auditor/api.py`](https://github.com/lakshayg2005/nlp_project/blob/main/auditor/api.py) (Audit API), [`auditor/extractor.py`](https://github.com/lakshayg2005/nlp_project/blob/main/auditor/extractor.py) (Span Extractor), [`auditor/db.py`](https://github.com/lakshayg2005/nlp_project/blob/main/auditor/db.py) (Audit Database)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Intake and Review"]
  node_api["Audit API<br/>[api.py]"]
  node_queue["Audit Queue"]
  node_worker["Batch Worker<br/>[api.py]"]
  node_dashboard["Review Dashboard<br/>[app.py]"]
end

subgraph group_analysis["Audit Analysis"]
  node_pipeline["Audit Orchestrator<br/>[pipeline.py]"]
  node_heuristics["Heuristic Screen<br/>[heuristics.py]"]
  node_embeddings["Semantic Similarity<br/>[embeddings.py]"]
  node_classifier["Risk Classifier<br/>[classifier.py]"]
  node_extractor["Span Extractor<br/>[extractor.py]"]
end

subgraph group_shared["Shared Domain Logic"]
  node_lexicon["Risk Phrase Lexicon<br/>[lexicon.py]"]
  node_taxonomy["Risk Taxonomy<br/>[taxonomy.py]"]
  node_textutils["Text Utilities<br/>[text_utils.py]"]
  node_spanutils["Span Markup<br/>[spans.py]"]
  node_metrics["Span Metrics<br/>[metrics.py]"]
end

subgraph group_feedback["Persistence and Feedback"]
  node_database[("Audit Database<br/>[db.py]")]
  node_training["SetFit Retraining<br/>[train_setfit.py]"]
end

node_seller(("Marketplace User"))
node_reviewer(("Human Reviewer"))

node_seller -->|"submits copy"| node_api
node_api -->|"queues products"| node_queue
node_queue -->|"dispatches batches"| node_worker
node_worker -->|"audits text"| node_pipeline
node_api -->|"audits synchronously"| node_pipeline
node_api -->|"stores and reads"| node_database
node_pipeline -->|"screens text"| node_heuristics
node_pipeline -->|"scores suspicious text"| node_classifier
node_pipeline -->|"extracts flagged spans"| node_extractor
node_heuristics -->|"scores semantic similarity"| node_embeddings
node_heuristics -->|"checks patterns"| node_lexicon
node_embeddings -->|"uses phrase bank"| node_lexicon
node_embeddings -->|"splits sentences"| node_textutils
node_classifier -->|"splits sentences"| node_textutils
node_classifier -->|"uses category labels"| node_taxonomy
node_extractor -->|"uses regex fallback"| node_lexicon
node_extractor -->|"maps span tags"| node_taxonomy
node_extractor -->|"merges spans"| node_textutils
node_dashboard -->|"views audit records"| node_database
node_reviewer -->|"reviews flags"| node_dashboard
node_dashboard -->|"records decisions"| node_database
node_database -->|"provides feedback data"| node_training
node_spanutils -.->|"supports span evaluation"| node_metrics

click node_api "https://github.com/lakshayg2005/nlp_project/blob/main/auditor/api.py"
click node_worker "https://github.com/lakshayg2005/nlp_project/blob/main/auditor/api.py"
click node_dashboard "https://github.com/lakshayg2005/nlp_project/blob/main/dashboard/app.py"
click node_pipeline "https://github.com/lakshayg2005/nlp_project/blob/main/auditor/pipeline.py"
click node_heuristics "https://github.com/lakshayg2005/nlp_project/blob/main/auditor/heuristics.py"
click node_embeddings "https://github.com/lakshayg2005/nlp_project/blob/main/auditor/embeddings.py"
click node_classifier "https://github.com/lakshayg2005/nlp_project/blob/main/auditor/classifier.py"
click node_extractor "https://github.com/lakshayg2005/nlp_project/blob/main/auditor/extractor.py"
click node_lexicon "https://github.com/lakshayg2005/nlp_project/blob/main/auditor/lexicon.py"
click node_taxonomy "https://github.com/lakshayg2005/nlp_project/blob/main/auditor/taxonomy.py"
click node_textutils "https://github.com/lakshayg2005/nlp_project/blob/main/auditor/text_utils.py"
click node_spanutils "https://github.com/lakshayg2005/nlp_project/blob/main/auditor/spans.py"
click node_metrics "https://github.com/lakshayg2005/nlp_project/blob/main/auditor/metrics.py"
click node_database "https://github.com/lakshayg2005/nlp_project/blob/main/auditor/db.py"
click node_training "https://github.com/lakshayg2005/nlp_project/blob/main/scripts/train_setfit.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_api,node_queue,node_worker,node_dashboard,node_seller,node_reviewer toneBlue
class node_pipeline,node_heuristics,node_embeddings,node_classifier,node_extractor toneAmber
class node_lexicon,node_taxonomy,node_textutils,node_spanutils,node_metrics toneMint
class node_database,node_training toneRose
```

## Components

### Intake and Review

- [Audit API](https://github.com/lakshayg2005/nlp_project/blob/main/auditor/api.py): `auditor/api.py`
- **Audit Queue**
- [Batch Worker](https://github.com/lakshayg2005/nlp_project/blob/main/auditor/api.py): `auditor/api.py`
- [Review Dashboard](https://github.com/lakshayg2005/nlp_project/blob/main/dashboard/app.py): `dashboard/app.py`

### Audit Analysis

- [Audit Orchestrator](https://github.com/lakshayg2005/nlp_project/blob/main/auditor/pipeline.py): `auditor/pipeline.py`
- [Heuristic Screen](https://github.com/lakshayg2005/nlp_project/blob/main/auditor/heuristics.py): `auditor/heuristics.py`
- [Semantic Similarity](https://github.com/lakshayg2005/nlp_project/blob/main/auditor/embeddings.py): `auditor/embeddings.py`
- [Risk Classifier](https://github.com/lakshayg2005/nlp_project/blob/main/auditor/classifier.py): `auditor/classifier.py`
- [Span Extractor](https://github.com/lakshayg2005/nlp_project/blob/main/auditor/extractor.py): `auditor/extractor.py`

### Shared Domain Logic

- [Risk Phrase Lexicon](https://github.com/lakshayg2005/nlp_project/blob/main/auditor/lexicon.py): `auditor/lexicon.py`
- [Risk Taxonomy](https://github.com/lakshayg2005/nlp_project/blob/main/auditor/taxonomy.py): `auditor/taxonomy.py`
- [Text Utilities](https://github.com/lakshayg2005/nlp_project/blob/main/auditor/text_utils.py): `auditor/text_utils.py`
- [Span Markup](https://github.com/lakshayg2005/nlp_project/blob/main/auditor/spans.py): `auditor/spans.py`
- [Span Metrics](https://github.com/lakshayg2005/nlp_project/blob/main/auditor/metrics.py): `auditor/metrics.py`

### Persistence and Feedback

- [Audit Database](https://github.com/lakshayg2005/nlp_project/blob/main/auditor/db.py): `auditor/db.py`
- [SetFit Retraining](https://github.com/lakshayg2005/nlp_project/blob/main/scripts/train_setfit.py): `scripts/train_setfit.py`

### Other components

- **Marketplace User**
- **Human Reviewer**

## Connections

- Marketplace User → Audit API: submits copy (evidence: `README.md`)
- Audit API → Audit Queue: queues products (evidence: `README.md`)
- Audit Queue → Batch Worker: dispatches batches (evidence: `README.md`)
- Batch Worker → Audit Orchestrator: audits text (evidence: `README.md`)
- Audit API → Audit Orchestrator: audits synchronously (evidence: `auditor/api.py`)
- Audit API → Audit Database: stores and reads (evidence: `auditor/api.py`)
- Audit Orchestrator → Heuristic Screen: screens text (evidence: `auditor/pipeline.py`)
- Audit Orchestrator → Risk Classifier: scores suspicious text (evidence: `auditor/pipeline.py`)
- Audit Orchestrator → Span Extractor: extracts flagged spans (evidence: `auditor/pipeline.py`)
- Heuristic Screen → Semantic Similarity: scores semantic similarity (evidence: `auditor/heuristics.py`)
- Heuristic Screen → Risk Phrase Lexicon: checks patterns (evidence: `auditor/heuristics.py`)
- Semantic Similarity → Risk Phrase Lexicon: uses phrase bank (evidence: `auditor/embeddings.py`)
- Semantic Similarity → Text Utilities: splits sentences (evidence: `auditor/embeddings.py`)
- Risk Classifier → Text Utilities: splits sentences (evidence: `auditor/classifier.py`)
- Risk Classifier → Risk Taxonomy: uses category labels (evidence: `auditor/classifier.py`)
- Span Extractor → Risk Phrase Lexicon: uses regex fallback (evidence: `auditor/extractor.py`)
- Span Extractor → Risk Taxonomy: maps span tags (evidence: `auditor/extractor.py`)
- Span Extractor → Text Utilities: merges spans (evidence: `auditor/extractor.py`)
- Review Dashboard → Audit Database: views audit records (evidence: `README.md`)
- Human Reviewer → Review Dashboard: reviews flags (evidence: `README.md`)
- Review Dashboard → Audit Database: records decisions (evidence: `README.md`)
- Audit Database → SetFit Retraining: provides feedback data (evidence: `README.md`)
- Span Markup → Span Metrics: supports span evaluation