How vinod1204/novelty_evaluator works
Novelty Evaluator accepts categorized submissions through a web UI, API, or CLI, then checks relevance and category-matched novelty against a learned baseline. The evaluator can update that baseline for highly novel submissions; the API schedules persistence and provides batch ranking and separate reasoning generation. The README also describes an LLM sentiment override and critique-and-revise reasoning loop. The sampled code supports the UI, API, evaluator, models, and baseline flow; some advertised evaluator internals and batch details are documented but not fully sampled.
Main entry points: evaluator.py Novelty Engine · api.py HTTP API · App.tsx Web Form · models.py Submission Model
Components 19
Evaluation Service
- HTTP API
src/api.py - Batch Ranking
src/api.py - Reasoning Route
src/api.py - Submission Model
src/models.py - Novelty Engine
src/evaluator.py - Relevance Gate
src/evaluator.py - Category Matching
src/evaluator.py - Novelty Scoring
src/evaluator.py
Baseline Learning
- Baseline Data
data/baseline_submissions.json - Embedding Cache
data/embeddings_cache.npz - Adaptive Learning
src/evaluator.py - Baseline Persistence
src/evaluator.py
AI Analysis
- Stance Override
src/evaluator.py - Reasoning Review Loop
src/evaluator.py
Other components
- Submitter
- Google GenAI
- Batch Client
Connections 27
- Submitter → Web Form: enters submission
frontend/src/App.tsx - Web Form → HTTP API: posts evaluation
frontend/src/App.tsx - Web Form → Reasoning Route: requests reasoning
frontend/src/App.tsx - Submitter → CLI: enters submission
cli.py - CLI → Novelty Engine: evaluates submission
cli.py - HTTP API → Submission Model: validates input
src/api.py - HTTP API → Novelty Engine: evaluates submission
src/api.py - Batch Client → Batch Ranking: sends batch
README.md - Batch Ranking → Submission Model: validates each item
src/api.py - Batch Ranking → Novelty Engine: evaluates each item
src/api.py - Novelty Engine → Relevance Gate: checks topic
src/evaluator.py - Novelty Engine → Category Matching: filters baseline
src/evaluator.py - Novelty Engine → Novelty Scoring: scores similarity
src/evaluator.py - Novelty Engine → Submission Model: embeds text
src/evaluator.py - Novelty Engine → Baseline Data: loads baseline
src/evaluator.py - Novelty Engine → Embedding Cache: loads cache
src/evaluator.py - Novelty Scoring → Adaptive Learning: learns high scores
src/evaluator.py - Adaptive Learning → Baseline Persistence: updates baseline
src/evaluator.py - Baseline Persistence → Baseline Data: writes baseline
src/evaluator.py - Baseline Persistence → Embedding Cache: writes cache
src/evaluator.py - HTTP API → Baseline Persistence: schedules save
src/api.py - Novelty Engine → Google GenAI: requests embeddings
README.md - Novelty Scoring → Stance Override: checks opposing stance
README.md - Stance Override → Google GenAI: checks sentiment
README.md - Reasoning Route → Novelty Engine: generates rationale
src/api.py - Novelty Engine → Reasoning Review Loop: drafts and reviews
README.md - Reasoning Review Loop → Google GenAI: requests reasoning
README.md