omarankh/evalora

How omarankh/evalora works

Evalora is a rubric-conditioned evaluator: users submit JSONL examples, which are validated, formatted, split, and tokenized before LoRA fine-tuning and evaluation. The UI can run the end-to-end pipeline, try model inference, and upload a model to Hugging Face. The graph emphasizes relationships visible in excerpts, the source index, or README; unsampled implementation details are represented only at supported subsystem paths.

Main entry points: preprocessing_service.py Preprocessing service · pipeline.py Pipeline routes · main.py FastAPI application · api.ts API client

Components 29

User experience

Pipeline form ui/src/components/PipelineForm.tsx
API client ui/src/lib/api.ts
Results display ui/src/components/ResultsDisplay.tsx
Try model ui/src/components/TryModel.tsx
Model upload ui/src/components/HuggingFaceUpload.tsx

API and orchestration

FastAPI application src/api/main.py
Pipeline routes src/api/routes/pipeline.py
Dataset routes src/api/routes/dataset.py
Evaluation routes src/api/routes/evaluation.py
Upload route src/api/routes/upload_model.py
Pipeline orchestration src/services/pipeline_service.py

Dataset preparation

JSONL validation src/services/dataset_upload_service.py
Preprocessing service src/services/preprocessing_service.py
Prompt formatter src/data/formatter.py
Dataset splitting src/data/splitter.py
Record preprocessing src/data/preprocessor.py
Tokenizer service src/services/tokenizer_service.py

Training and evaluation

Training service src/services/training_service.py
LoRA trainer src/training/trainer.py
Training data src/training/data.py
Model setup src/training/modeling.py
Training configuration src/training/config.py
Evaluation service src/services/evaluation.py
Evaluation metrics src/evaluation/evaluators/evaluator.py
Inference predictor src/inference/predictor.py
Prediction parser src/evaluation/parsers/prediction_parser.py
Hugging Face uploader src/services/hf_uploader.py

Other components

Evaluator user
Hugging Face

Connections 34

Evaluator user → Pipeline form: submits dataset ui/src/components/PipelineForm.tsx
Pipeline form → API client: starts full run ui/src/components/PipelineForm.tsx
API client → Pipeline routes: posts full run ui/src/lib/api.ts
API client → Evaluation routes: requests inference ui/src/lib/api.ts
API client → Upload route: requests upload ui/src/lib/api.ts
FastAPI application → Pipeline routes: registers route src/api/main.py
FastAPI application → Dataset routes: registers route src/api/main.py
FastAPI application → Evaluation routes: registers route src/api/main.py
FastAPI application → Upload route: registers route src/api/main.py
Pipeline routes → Pipeline orchestration: runs pipeline src/api/routes/pipeline.py
Pipeline routes → Training service: starts training src/api/routes/pipeline.py
Pipeline routes → Evaluation service: compares models src/api/routes/pipeline.py
Dataset routes → JSONL validation: validates upload src/api/routes/dataset.py
Dataset routes → Preprocessing service: preprocesses data src/api/routes/dataset.py
Evaluation routes → Evaluation service: requests evaluation src/api/routes/evaluation.py
Upload route → Hugging Face uploader: uploads model src/api/routes/upload_model.py
Pipeline orchestration → JSONL validation: parses and saves src/services/pipeline_service.py
Pipeline orchestration → Preprocessing service: formats and splits src/services/pipeline_service.py
Preprocessing service → Record preprocessing: preprocesses records src/services/preprocessing_service.py
Preprocessing service → Prompt formatter: formats prompts src/services/preprocessing_service.py
Preprocessing service → Dataset splitting: creates splits src/services/preprocessing_service.py
Preprocessing service → Tokenizer service: tokenizes splits src/services/preprocessing_service.py
Training service → LoRA trainer: runs training src/services/training_service.py
LoRA trainer → Training configuration: uses configuration src/training/trainer.py
LoRA trainer → Training data: loads training data src/training/trainer.py
LoRA trainer → Model setup: sets up model src/training/trainer.py
Evaluation service → Evaluation metrics: computes metrics src/services/evaluation.py
Evaluation service → Inference predictor: runs inference src/services/evaluation.py
Inference predictor → Prompt formatter: formats input src/inference/predictor.py
Inference predictor → Prediction parser: parses output src/inference/predictor.py
Hugging Face uploader → Hugging Face: publishes model
Results display → Evaluator user: shows results
Try model → Evaluator user: accepts prompt
Model upload → Evaluator user: collects credentials

Drawn by GitDiagram on November 12, 2026 from the repository's file tree, README and source files. Read as Markdown ·