How somaskandan931/sathyabama-intelligrade works
This is a full-stack AI evaluation platform for grading handwritten subjective answers:
• Frontend: a Streamlit teacher dashboard (frontend/dashboard.py) for single/batch grading + analytics.
• Backend: a FastAPI service (backend/api.py) exposing endpoints for upload/OCR/evaluation/results/rubrics/stats.
• Core “AI Evaluation Engine”: orchestrated in backend/evaluator.py, combining:
• Image preprocessing (backend/preprocessor.py)
• OCR (backend/ocr_module.py, TrOCR primary, Tesseract fallback)
• Text processing (backend/text_processor.py)
• Similarity scoring (backend/similarity.py, Sentence-BERT)
• LLM grading (backend/llm_evaluator.py, Gemini API)
• Rubric matching (backend/rubric_matcher.py, DeBERTa NLI)
• Score aggregation + feedback generation (backend/evaluator.py)
• Persistence: SQLAlchemy ORM (backend/database.py) for Students/Questions/Submissions/Results/Rubrics; SQLite dev, PostgreSQL prod.
• Deployment: Docker Compose with separate Dockerfiles for backend/frontend, plus DB (docker-compose.yml, Dockerfile.backend, Dockerfile.frontend).
• Supporting tooling: dataset collection (datasets/collect_dataset.py), TrOCR training (models/train_trocr.py), prompts (prompts/evaluation_prompts.py), tests (tests/test_all.py).
Online grading runtime (request/response, data flow, services)
2) Offline tooling/training (dataset + model fine-tuning) that feeds artifacts into runtime
User-facing / entry points
• Teacher (browser) → Streamlit Dashboard (frontend/dashboard.py)
• Potential external clients (future LMS integration per README) → FastAPI (backend/api.py)
Backend service boundary
• API Layer: backend/api.py (routes: /upload, /ocr/{id}, /evaluate, /result/{id}, /rubric, /stats, /health)
• Evaluation Orchestrator: backend/evaluator.py
• Pipeline stages:
• Preprocessor: backend/preprocessor.py
• OCR engine: backend/ocr_module.py
• Text processing: backend/text_processor.py
• Similarity: backend/similarity.py
• Rubric matcher: backend/rubric_matcher.py
• LLM evaluator: backend/llm_evaluator.py
• Metrics/Analytics: backend/metrics.py (ties to /stats)
• Persistence Adapter: backend/database.py
• Shared utilities: backend/text_processor.py, backend/similarity.py, etc.
Data stores + external services
• Database (PostgreSQL in prod; SQLite in dev) — from .env and README.
• File storage for uploads (UPLOAD_DIR=./uploads) — model it as a local volume/object store.
Offline / non-runtime subsystems
• Dataset toolchain: datasets/collect_dataset.py
• Model training: models/train_trocr.py (outputs a fine-tuned TrOCR model used by OCR module)
• Prompt templates: prompts/evaluation_prompts.py (input to LLM evaluator)
Teacher Dashboard → triggers either
• /ocr/{id} (OCR-only path), OR
• /evaluate (full pipeline path)
Evaluator calls
• Preprocessor (OpenCV pipeline) → outputs cleaned image
• OCR Module (TrOCR primary, Tesseract fallback) → outputs extracted text + confidence
• Text Processor (spaCy/NLTK) → normalized text
• Parallel/branch:
• Similarity (SBERT cosine similarity vs teacher answer)
• Rubric Matcher (DeBERTa NLI vs rubric criteria)
• LLM Evaluator (Gemini) using Prompt Templates
Persist
• Results table + feedback JSON in DB
External boundary calls
• Evaluator → Gemini API (explicitly outside your trust boundary; label “external SaaS”)
• OCR engine may load local model artifacts; show TrOCR model files as an internal dependency (from models/ output).
Architectural patterns/principles to reflect
Explicitly because they’re “the story” of the system:
• API-based architecture: Streamlit is not directly running ML; it’s a client of FastAPI.
• Pipeline orchestration pattern: evaluator.py composes multiple stages; show it as a coordinator.
• Hybrid scoring: deterministic similarity + rubric coverage + LLM judgment.
• Separation of concerns:
• Preprocessing vs OCR vs NLP vs scoring vs persistence
• Environment-config driven behavior: .env selects OCR_ENGINE, weights, database URL.
• Name (e.g., “OCR Module”)
• Repo anchor (file path, e.g., backend/ocr_module.py)
• Tech (e.g., “TrOCR/Tesseract”, “SBERT”, “Gemini”, “SQLAlchemy”)
• Key I/O (e.g., “Input: image; Output: text+confidence”)
• protocol/type: HTTP/JSON, SQL, File I/O, External HTTPS
• payload: “answer image”, “rubric JSON”, “feedback JSON”, “scores”
• Streamlit Dashboard (frontend/dashboard.py)
• FastAPI API (backend/api.py)
• Evaluator Orchestrator (backend/evaluator.py)
• Preprocessing (backend/preprocessor.py)
• OCR (backend/ocr_module.py) with TrOCR + Tesseract fallback
• Text Processing (backend/text_processor.py)
• Similarity (backend/similarity.py, SBERT)
• Rubric Matcher (backend/rubric_matcher.py, DeBERTa NLI)
• LLM Evaluator (backend/llm_evaluator.py, Gemini) + prompts (prompts/evaluation_prompts.py)
• Metrics (backend/metrics.py) powering /stats
• Database layer (backend/database.py, SQLAlchemy) + DB (SQLite/Postgres)
• Upload storage (UPLOAD_DIR)
• Docker Compose boundary + separate backend/frontend containers
• Offline: dataset collection (datasets/collect_dataset.py) + TrOCR training (models/train_trocr.py) → model artifacts used by OCR