How lakshayg2005/nlp_project works
This is an e-commerce copy compliance auditor. Product text enters through the API, is queued for asynchronous auditing, and passes through heuristic screening, risk classification, and phrase-level span extraction before results and flags are persisted for dashboard review. Human decisions are stored as feedback for classifier retraining. The graph also shows the shared risk taxonomy and text/span utilities. Pipeline and model-stage calls are supported by excerpts; queue and dashboard behavior are documented in the README, while unsampled implementation details are not assumed.
Main entry points: pipeline.py Audit Orchestrator · api.py Audit API · extractor.py Span Extractor · db.py Audit Database
Components 18
Intake and Review
- Audit API
auditor/api.py - Audit Queue
- Batch Worker
auditor/api.py - Review Dashboard
dashboard/app.py
Audit Analysis
- Audit Orchestrator
auditor/pipeline.py - Heuristic Screen
auditor/heuristics.py - Semantic Similarity
auditor/embeddings.py - Risk Classifier
auditor/classifier.py - Span Extractor
auditor/extractor.py
Shared Domain Logic
- Risk Phrase Lexicon
auditor/lexicon.py - Risk Taxonomy
auditor/taxonomy.py - Text Utilities
auditor/text_utils.py - Span Markup
auditor/spans.py - Span Metrics
auditor/metrics.py
Persistence and Feedback
- Audit Database
auditor/db.py - SetFit Retraining
scripts/train_setfit.py
Other components
- Marketplace User
- Human Reviewer
Connections 23
- Marketplace User → Audit API: submits copy
README.md - Audit API → Audit Queue: queues products
README.md - Audit Queue → Batch Worker: dispatches batches
README.md - Batch Worker → Audit Orchestrator: audits text
README.md - Audit API → Audit Orchestrator: audits synchronously
auditor/api.py - Audit API → Audit Database: stores and reads
auditor/api.py - Audit Orchestrator → Heuristic Screen: screens text
auditor/pipeline.py - Audit Orchestrator → Risk Classifier: scores suspicious text
auditor/pipeline.py - Audit Orchestrator → Span Extractor: extracts flagged spans
auditor/pipeline.py - Heuristic Screen → Semantic Similarity: scores semantic similarity
auditor/heuristics.py - Heuristic Screen → Risk Phrase Lexicon: checks patterns
auditor/heuristics.py - Semantic Similarity → Risk Phrase Lexicon: uses phrase bank
auditor/embeddings.py - Semantic Similarity → Text Utilities: splits sentences
auditor/embeddings.py - Risk Classifier → Text Utilities: splits sentences
auditor/classifier.py - Risk Classifier → Risk Taxonomy: uses category labels
auditor/classifier.py - Span Extractor → Risk Phrase Lexicon: uses regex fallback
auditor/extractor.py - Span Extractor → Risk Taxonomy: maps span tags
auditor/extractor.py - Span Extractor → Text Utilities: merges spans
auditor/extractor.py - Review Dashboard → Audit Database: views audit records
README.md - Human Reviewer → Review Dashboard: reviews flags
README.md - Review Dashboard → Audit Database: records decisions
README.md - Audit Database → SetFit Retraining: provides feedback data
README.md - Span Markup → Span Metrics: supports span evaluation