How millionchau/qualityservice works
DevRadar Quality Service accepts article text, cleans and protects content, normalizes Vietnamese/teencode and typos, segments words, classifies IT relevance and toxicity, evaluates quality rules, and returns a score with statistics and issues. The API also supports relabeling and listing stored results. The graph reflects the sampled pipeline and route wiring; documented capabilities such as fuzzy-search fallback are included, while unsampled internal details are represented only at their supported directory boundary.
Main entry points: pipeline.py Analysis pipeline · analyze.py Quality routes · quality_service.py Quality service · mongo_client.py MongoDB client
Components 23
API and service
- FastAPI app
app/main.py - Quality routes
app/api/routes/analyze.py - Quality service
app/services/quality_service.py - Quality schemas
app/schemas/quality.py
Text processing
- Analysis pipeline
app/pipeline/pipeline.py - Content cleaner
app/pipeline/cleaner.py - Text normalizer
app/pipeline/normalizer.py - Term dictionary
app/dictionary/aho_corasick.py - IT keywords
app/dictionaries/it_keywords.json - Toxic keywords
app/dictionaries/toxic_keywords.json - Vietnamese tokenizer
app/pipeline/tokenizer.py
Classification
- Classifier pipeline
app/pipeline/classifier.py - IT classifier
app/models/it_classifier.py - Toxic classifier
app/models/toxic_classifier.py - Saved models
app/models/saved
Quality assessment
- Rule analysis
app/pipeline/quality_analyzer.py - Quality rules
app/rules - Quality scorer
app/pipeline/scorer.py
Persistence and search
- MongoDB client
app/clients/mongo_client.py - MongoDB Atlas
- Elasticsearch client
app/clients/elasticsearch_client.py - Elasticsearch
Other components
- API caller
Connections 26
- API caller → Quality routes: submits article
- Quality routes → Quality service: analyzes article
- Quality service → Analysis pipeline: processes text
- Analysis pipeline → Content cleaner: cleans and protects
- Analysis pipeline → Text normalizer: normalizes text
- Text normalizer → Term dictionary: replaces terms
- Analysis pipeline → Vietnamese tokenizer: segments text
- Analysis pipeline → Classifier pipeline: classifies text
- Classifier pipeline → IT classifier: predicts IT relevance
- Classifier pipeline → Toxic classifier: predicts toxicity
- Analysis pipeline → Rule analysis: evaluates quality
- Rule analysis → Quality rules: applies rules
- Analysis pipeline → Quality scorer: calculates score
- Analysis pipeline → Quality schemas: returns result
- Quality service → MongoDB client: saves result
- Quality routes → MongoDB client: updates labels or lists
- MongoDB client → MongoDB Atlas: reads and writes
- Text normalizer → Elasticsearch client: fuzzy-search fallback
- Elasticsearch client → Elasticsearch: searches terms
- IT classifier → IT keywords: loads keywords
- Toxic classifier → Toxic keywords: loads keywords
- IT classifier → Saved models: loads classifier artifacts
- Toxic classifier → Saved models: loads classifier artifacts
- FastAPI app → Quality routes: registers routers
- FastAPI app → Quality service: initializes service
- FastAPI app → MongoDB client: connects on startup