How muditbaid/skill-based-expert-routing-system-for-multilabel-hate-speech-detection works
1. Repository purpose
- This repo packages a deployment-oriented harmful-speech detection system that routes each input text through a skill-inference stage, a symbolic expert router, and a set of task-specific Llama 3.1 LoRA adapters.
- It is split into three clear areas: model/routing logic, serving backend, and static presentation assets.
2. Core runtime pipeline
- The main inference flow is: raw text in → skill extraction → router selection of experts → adapter-based generation/prediction → multilabel harm labels out.
- The canonical single-request entrypoint is symbolic-moe/predict_post.py, which the backend adapter uses for live inference.
3. Symbolic skill inference layer
- symbolic-moe/skill_inference.py infers ontology-level skills from user text using the Llama 3.1 base model plus a controlled skill vocabulary in skills.txt.
- symbolic-moe/skill_parsing.py constrains and normalizes predicted skills so downstream routing works on a stable symbolic representation.
- This stage is the first major boundary: it converts unstructured text into structured routing features.
4. Expert routing subsystem
- symbolic-moe/route_and_predict_nb.py contains the Bernoulli Naive Bayes router that selects the top-2 experts per example from the inferred skills.
- The router is trained from skill-annotated pools such as profile_pool_skills.jsonl, validation_pool_skills.jsonl, and test_pool_skills.jsonl.
- This is the decision layer that maps a skill profile to one or more specialist models.
5. Expert execution and adapter switching
- The system uses a shared Llama 3.1 base model with separate LoRA expert adapters stored under saves/llama31-8b/.../qlora.
- symbolic-moe/model_utils.py is the boundary that keeps the base model resident and swaps adapters instead of reloading full checkpoints for every task.
- The four experts correspond to the task directories in saves/llama31-8b/: dynahate, jigsaw_threat, kaggle_cyberbullying, and tweeteval_offensive.
6. Prediction aggregation
- symbolic-moe/predict_post.py ties together skill inference, routing, expert execution, and label filtering into a single request-response flow.
- It returns the final multilabel output with confidences after keeping only the labels relevant to the target task.
- This file is effectively the runtime composition layer for the whole system.
7. Offline profile and dataset artifacts
- symbolic-moe/build_profiles.py and profiles.json support constructing and storing learned expert profiles.
- The repo also retains the pools used for training and validation of routing: profile_pool.jsonl, validation_pool.jsonl, and test_pool.jsonl, plus their skill-annotated counterparts.
- These assets are the offline data foundation for the symbolic router and expert-selection logic.
8. Evaluation and observability
- symbolic-moe/evaluate_outputs.py scores model outputs and records evaluation traces.
- symbolic-moe/langsmith_utils.py and symbolic-moe/langsmith_workflows.py integrate with LangSmith for trace capture, dataset upload, backfilling, review queues, and hard-case analysis.
- LangSmith is not part of the live request path; it is an evaluation/inspection layer for debugging routing and model behavior.
9. Serving backend
- web app/backend/ is a FastAPI service that exposes the detection pipeline to clients.
- The primary API surface is POST /api/detect, with readiness/liveness endpoints for deployment health checks.
- web app/backend/app/services/ contains the service abstraction layer, including local pipeline execution, remote inference support, and adapter selection/factory logic.
10. Backend deployment modes
- The backend is designed to run either locally in-process or against a remote/Colab GPU workflow.
- web app/COLAB_BACKEND.md and web app/SERML_Backend.ipynb describe the Colab-oriented serving path, while web app/backend/app/services/remote_inference.py suggests a separate remote execution boundary.
- predict_post_adapter.py bridges the backend API contract to the symbolic inference pipeline.
11. Backend application structure
- web app/backend/app/main.py is the service entrypoint, with app/api/routes_detect.py defining request routes and app/schemas.py defining request/response shapes.
- app/config.py, app/logging_config.py, and app/audit.py provide operational concerns such as configuration, logging, and auditability.
- web app/backend/storage/ and web app/backend/logs/ are runtime persistence areas for local state and logs.
12. Presentation and static site
- docs/index.html and docs/assets/ power the GitHub Pages project page, which is a separate presentation surface from the model backend.
- project.html appears to be an additional static project artifact, while .nojekyll enables direct Pages hosting behavior.
- This part of the repo is purely informational/demo-facing and does not participate in inference.
13. Model artifacts and saved outputs
- The saves/ tree contains the trained LoRA adapter bundles and tokenizer/config metadata for each specialist task.
- Files such as adapter_model.safetensors, adapter_config.json, trainer_state.json, train_results.json, and eval_results.json document both training and evaluation outcomes for each expert.
- These artifacts are the deployable model layer consumed by the runtime, separate from the symbolic router code.
14. Architecture boundary summary
- The cleanest diagram is: client/UI → FastAPI backend → canonical prediction flow in symbolic-moe → shared Llama base model plus task adapters → response.
- Orthogonal to that, offline training/evaluation pipelines generate the router artifacts, skill vocab, expert profiles, and adapter checkpoints.
- LangSmith and the static docs site sit beside the runtime as observability and presentation subsystems, not as inference dependencies.
Main entry points: predict_post.py Predict flow · routes_detect.py Detect API · skill_inference.py Skill inference · route_and_predict_nb.py NB router
Components 18
Runtime pipeline
Live symbolic routing and prediction flow used for inference.
- Predict flow · Canonical request-response inference flow for a single example.
symbolic-moe/predict_post.py - Skill inference · Converts raw text into ontology-level skills.
symbolic-moe/skill_inference.py - Skill parsing · Constrains and normalizes predicted skills for routing.
symbolic-moe/skill_parsing.py - NB router · Bernoulli Naive Bayes selector that chooses top experts.
symbolic-moe/route_and_predict_nb.py - Model utils · Keeps the base model resident and swaps LoRA adapters.
symbolic-moe/model_utils.py
Serving backend
FastAPI service that exposes the detection pipeline to clients.
- Detect API · Primary POST endpoint for inference requests.
web app/backend/app/api/routes_detect.py - Backend main · Application startup and API wiring.
web app/backend/app/main.py - API schemas · Request and response shapes for the service contract.
web app/backend/app/schemas.py - Pipeline factory · Chooses local or remote execution path.
web app/backend/app/services/pipeline_factory.py - Predict adapter · Adapts backend requests to the symbolic prediction pipeline.
web app/backend/app/services/predict_post_adapter.py - Remote inference · Separate remote or Colab-oriented execution path.
web app/backend/app/services/remote_inference.py
Offline artifacts
Training, routing, and evaluation assets that support deployment.
- Evaluation · Scores outputs and records evaluation traces.
symbolic-moe/evaluate_outputs.py - LangSmith · Trace capture, dataset upload, review queues, and analysis workflows.
symbolic-moe/langsmith_workflows.py - Profiles · Learned expert profiles and skill pools for routing.
symbolic-moe/profiles.json - Skills vocab · Controlled skill vocabulary used by skill inference.
symbolic-moe/skills.txt - LoRA experts · Task-specific adapter bundles for the specialist models.
saves/llama31-8b
Presentation
Static project pages and demo-facing assets.
- Docs site · GitHub Pages-style project page and assets.
docs/index.html
Other components
- Client / UI · Any external caller sending text for detection.
Connections 18
- Client / UI → Detect API: POST /api/detect · Client sends text detection request.
- Detect API → API schemas: validates · Request and response payloads are shaped by schemas.
- Backend main → Detect API: mounts · Service entrypoint registers detection routes.
- Detect API → Pipeline factory: selects pipeline · API delegates execution path selection.
- Pipeline factory → Predict adapter: local path · Chooses in-process adapter-backed inference.
- Pipeline factory → Remote inference: remote path · Optionally routes to remote or Colab execution.
- Predict adapter → Predict flow: calls · Backend adapter bridges into canonical symbolic prediction flow.
- Predict flow → Skill inference: extracts skills · Raw text is converted into structured skills.
- Skill inference → Skill parsing: normalizes · Predicted skills are constrained to the stable ontology.
- Skill parsing → NB router: routes on · Normalized skills feed expert selection.
- NB router → Model utils: selects expert · Router chooses which adapter to activate for prediction.
- Model utils → LoRA experts: loads swaps · Shared base model switches among task-specific LoRA experts.
- Predict flow → LoRA experts: infers with · Final predictions are produced via the shared base model plus adapters.
- NB router → Profiles: trained from · Router is built from skill-annotated profile pools.
- Skill inference → Skills vocab: constrained by · Skill extraction uses the controlled vocabulary.
- Evaluation → Predict flow: scores outputs · Evaluation consumes runtime predictions and traces.
- LangSmith → Evaluation: supports · Observability layer captures traces and review workflows for evaluation.
- Docs site → Client / UI: informational · Static site provides project-facing presentation but not inference.