# omarankh/evalora architecture

> Architecture diagram of the GitHub repository omarankh/evalora, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/omarankh/evalora
- Repository: https://github.com/omarankh/evalora

## Overview

Evalora is a rubric-conditioned evaluator: users submit JSONL examples, which are validated, formatted, split, and tokenized before LoRA fine-tuning and evaluation. The UI can run the end-to-end pipeline, try model inference, and upload a model to Hugging Face. The graph emphasizes relationships visible in excerpts, the source index, or README; unsampled implementation details are represented only at supported subsystem paths.

Main entry points: [`src/services/preprocessing_service.py`](https://github.com/omarankh/evalora/blob/main/src/services/preprocessing_service.py) (Preprocessing service), [`src/api/routes/pipeline.py`](https://github.com/omarankh/evalora/blob/main/src/api/routes/pipeline.py) (Pipeline routes), [`src/api/main.py`](https://github.com/omarankh/evalora/blob/main/src/api/main.py) (FastAPI application), [`ui/src/lib/api.ts`](https://github.com/omarankh/evalora/blob/main/ui/src/lib/api.ts) (API client)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_experience["User experience"]
  node_pipeline_ui["Pipeline form<br/>[PipelineForm.tsx]"]
  node_api_client["API client<br/>[api.ts]"]
  node_results_ui["Results display<br/>[ResultsDisplay.tsx]"]
  node_try_ui["Try model<br/>[TryModel.tsx]"]
  node_upload_ui["Model upload"]
end

subgraph group_api["API and orchestration"]
  node_app["FastAPI application<br/>[main.py]"]
  node_pipeline_route["Pipeline routes<br/>[pipeline.py]"]
  node_dataset_route["Dataset routes<br/>[dataset.py]"]
  node_evaluation_route["Evaluation routes<br/>[evaluation.py]"]
  node_upload_route["Upload route<br/>[upload_model.py]"]
  node_pipeline_service["Pipeline orchestration"]
end

subgraph group_data["Dataset preparation"]
  node_upload_service["JSONL validation"]
  node_preprocessing["Preprocessing service"]
  node_formatter["Prompt formatter<br/>[formatter.py]"]
  node_splitter["Dataset splitting<br/>[splitter.py]"]
  node_preprocessor["Record preprocessing<br/>[preprocessor.py]"]
  node_tokenizer["Tokenizer service"]
end

subgraph group_models["Training and evaluation"]
  node_training_service["Training service"]
  node_trainer["LoRA trainer<br/>[trainer.py]"]
  node_training_data["Training data<br/>[data.py]"]
  node_modeling["Model setup<br/>[modeling.py]"]
  node_train_config["Training configuration<br/>[config.py]"]
  node_evaluation_service["Evaluation service<br/>[evaluation.py]"]
  node_evaluator["Evaluation metrics<br/>[evaluator.py]"]
  node_predictor["Inference predictor<br/>[predictor.py]"]
  node_prediction_parser["Prediction parser"]
  node_hf_uploader["Hugging Face uploader<br/>[hf_uploader.py]"]
end

node_user(("Evaluator user"))
node_huggingface{{"Hugging Face"}}

node_user -->|"submits dataset"| node_pipeline_ui
node_pipeline_ui -->|"starts full run"| node_api_client
node_api_client -->|"posts full run"| node_pipeline_route
node_api_client -->|"requests inference"| node_evaluation_route
node_api_client -->|"requests upload"| node_upload_route
node_app -->|"registers route"| node_pipeline_route
node_app -->|"registers route"| node_dataset_route
node_app -->|"registers route"| node_evaluation_route
node_app -->|"registers route"| node_upload_route
node_pipeline_route -->|"runs pipeline"| node_pipeline_service
node_pipeline_route -->|"starts training"| node_training_service
node_pipeline_route -->|"compares models"| node_evaluation_service
node_dataset_route -->|"validates upload"| node_upload_service
node_dataset_route -->|"preprocesses data"| node_preprocessing
node_evaluation_route -->|"requests evaluation"| node_evaluation_service
node_upload_route -->|"uploads model"| node_hf_uploader
node_pipeline_service -->|"parses and saves"| node_upload_service
node_pipeline_service -->|"formats and splits"| node_preprocessing
node_preprocessing -->|"preprocesses records"| node_preprocessor
node_preprocessing -->|"formats prompts"| node_formatter
node_preprocessing -->|"creates splits"| node_splitter
node_preprocessing -->|"tokenizes splits"| node_tokenizer
node_training_service -->|"runs training"| node_trainer
node_trainer -->|"uses configuration"| node_train_config
node_trainer -->|"loads training data"| node_training_data
node_trainer -->|"sets up model"| node_modeling
node_evaluation_service -->|"computes metrics"| node_evaluator
node_evaluation_service -->|"runs inference"| node_predictor
node_predictor -->|"formats input"| node_formatter
node_predictor -->|"parses output"| node_prediction_parser
node_hf_uploader -.->|"publishes model"| node_huggingface
node_results_ui -.->|"shows results"| node_user
node_try_ui -.->|"accepts prompt"| node_user
node_upload_ui -.->|"collects credentials"| node_user

click node_pipeline_ui "https://github.com/omarankh/evalora/blob/main/ui/src/components/PipelineForm.tsx"
click node_api_client "https://github.com/omarankh/evalora/blob/main/ui/src/lib/api.ts"
click node_results_ui "https://github.com/omarankh/evalora/blob/main/ui/src/components/ResultsDisplay.tsx"
click node_try_ui "https://github.com/omarankh/evalora/blob/main/ui/src/components/TryModel.tsx"
click node_upload_ui "https://github.com/omarankh/evalora/blob/main/ui/src/components/HuggingFaceUpload.tsx"
click node_app "https://github.com/omarankh/evalora/blob/main/src/api/main.py"
click node_pipeline_route "https://github.com/omarankh/evalora/blob/main/src/api/routes/pipeline.py"
click node_dataset_route "https://github.com/omarankh/evalora/blob/main/src/api/routes/dataset.py"
click node_evaluation_route "https://github.com/omarankh/evalora/blob/main/src/api/routes/evaluation.py"
click node_upload_route "https://github.com/omarankh/evalora/blob/main/src/api/routes/upload_model.py"
click node_pipeline_service "https://github.com/omarankh/evalora/blob/main/src/services/pipeline_service.py"
click node_upload_service "https://github.com/omarankh/evalora/blob/main/src/services/dataset_upload_service.py"
click node_preprocessing "https://github.com/omarankh/evalora/blob/main/src/services/preprocessing_service.py"
click node_formatter "https://github.com/omarankh/evalora/blob/main/src/data/formatter.py"
click node_splitter "https://github.com/omarankh/evalora/blob/main/src/data/splitter.py"
click node_preprocessor "https://github.com/omarankh/evalora/blob/main/src/data/preprocessor.py"
click node_tokenizer "https://github.com/omarankh/evalora/blob/main/src/services/tokenizer_service.py"
click node_training_service "https://github.com/omarankh/evalora/blob/main/src/services/training_service.py"
click node_trainer "https://github.com/omarankh/evalora/blob/main/src/training/trainer.py"
click node_training_data "https://github.com/omarankh/evalora/blob/main/src/training/data.py"
click node_modeling "https://github.com/omarankh/evalora/blob/main/src/training/modeling.py"
click node_train_config "https://github.com/omarankh/evalora/blob/main/src/training/config.py"
click node_evaluation_service "https://github.com/omarankh/evalora/blob/main/src/services/evaluation.py"
click node_evaluator "https://github.com/omarankh/evalora/blob/main/src/evaluation/evaluators/evaluator.py"
click node_predictor "https://github.com/omarankh/evalora/blob/main/src/inference/predictor.py"
click node_prediction_parser "https://github.com/omarankh/evalora/blob/main/src/evaluation/parsers/prediction_parser.py"
click node_hf_uploader "https://github.com/omarankh/evalora/blob/main/src/services/hf_uploader.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pipeline_ui,node_api_client,node_results_ui,node_try_ui,node_upload_ui,node_user toneBlue
class node_app,node_pipeline_route,node_dataset_route,node_evaluation_route,node_upload_route,node_pipeline_service toneAmber
class node_upload_service,node_preprocessing,node_formatter,node_splitter,node_preprocessor,node_tokenizer toneMint
class node_training_service,node_trainer,node_training_data,node_modeling,node_train_config,node_evaluation_service,node_evaluator,node_predictor,node_prediction_parser,node_hf_uploader toneRose
class node_huggingface toneIndigo
```

## Components

### User experience

- [Pipeline form](https://github.com/omarankh/evalora/blob/main/ui/src/components/PipelineForm.tsx): `ui/src/components/PipelineForm.tsx`
- [API client](https://github.com/omarankh/evalora/blob/main/ui/src/lib/api.ts): `ui/src/lib/api.ts`
- [Results display](https://github.com/omarankh/evalora/blob/main/ui/src/components/ResultsDisplay.tsx): `ui/src/components/ResultsDisplay.tsx`
- [Try model](https://github.com/omarankh/evalora/blob/main/ui/src/components/TryModel.tsx): `ui/src/components/TryModel.tsx`
- [Model upload](https://github.com/omarankh/evalora/blob/main/ui/src/components/HuggingFaceUpload.tsx): `ui/src/components/HuggingFaceUpload.tsx`

### API and orchestration

- [FastAPI application](https://github.com/omarankh/evalora/blob/main/src/api/main.py): `src/api/main.py`
- [Pipeline routes](https://github.com/omarankh/evalora/blob/main/src/api/routes/pipeline.py): `src/api/routes/pipeline.py`
- [Dataset routes](https://github.com/omarankh/evalora/blob/main/src/api/routes/dataset.py): `src/api/routes/dataset.py`
- [Evaluation routes](https://github.com/omarankh/evalora/blob/main/src/api/routes/evaluation.py): `src/api/routes/evaluation.py`
- [Upload route](https://github.com/omarankh/evalora/blob/main/src/api/routes/upload_model.py): `src/api/routes/upload_model.py`
- [Pipeline orchestration](https://github.com/omarankh/evalora/blob/main/src/services/pipeline_service.py): `src/services/pipeline_service.py`

### Dataset preparation

- [JSONL validation](https://github.com/omarankh/evalora/blob/main/src/services/dataset_upload_service.py): `src/services/dataset_upload_service.py`
- [Preprocessing service](https://github.com/omarankh/evalora/blob/main/src/services/preprocessing_service.py): `src/services/preprocessing_service.py`
- [Prompt formatter](https://github.com/omarankh/evalora/blob/main/src/data/formatter.py): `src/data/formatter.py`
- [Dataset splitting](https://github.com/omarankh/evalora/blob/main/src/data/splitter.py): `src/data/splitter.py`
- [Record preprocessing](https://github.com/omarankh/evalora/blob/main/src/data/preprocessor.py): `src/data/preprocessor.py`
- [Tokenizer service](https://github.com/omarankh/evalora/blob/main/src/services/tokenizer_service.py): `src/services/tokenizer_service.py`

### Training and evaluation

- [Training service](https://github.com/omarankh/evalora/blob/main/src/services/training_service.py): `src/services/training_service.py`
- [LoRA trainer](https://github.com/omarankh/evalora/blob/main/src/training/trainer.py): `src/training/trainer.py`
- [Training data](https://github.com/omarankh/evalora/blob/main/src/training/data.py): `src/training/data.py`
- [Model setup](https://github.com/omarankh/evalora/blob/main/src/training/modeling.py): `src/training/modeling.py`
- [Training configuration](https://github.com/omarankh/evalora/blob/main/src/training/config.py): `src/training/config.py`
- [Evaluation service](https://github.com/omarankh/evalora/blob/main/src/services/evaluation.py): `src/services/evaluation.py`
- [Evaluation metrics](https://github.com/omarankh/evalora/blob/main/src/evaluation/evaluators/evaluator.py): `src/evaluation/evaluators/evaluator.py`
- [Inference predictor](https://github.com/omarankh/evalora/blob/main/src/inference/predictor.py): `src/inference/predictor.py`
- [Prediction parser](https://github.com/omarankh/evalora/blob/main/src/evaluation/parsers/prediction_parser.py): `src/evaluation/parsers/prediction_parser.py`
- [Hugging Face uploader](https://github.com/omarankh/evalora/blob/main/src/services/hf_uploader.py): `src/services/hf_uploader.py`

### Other components

- **Evaluator user**
- **Hugging Face**

## Connections

- Evaluator user → Pipeline form: submits dataset (evidence: `ui/src/components/PipelineForm.tsx`)
- Pipeline form → API client: starts full run (evidence: `ui/src/components/PipelineForm.tsx`)
- API client → Pipeline routes: posts full run (evidence: `ui/src/lib/api.ts`)
- API client → Evaluation routes: requests inference (evidence: `ui/src/lib/api.ts`)
- API client → Upload route: requests upload (evidence: `ui/src/lib/api.ts`)
- FastAPI application → Pipeline routes: registers route (evidence: `src/api/main.py`)
- FastAPI application → Dataset routes: registers route (evidence: `src/api/main.py`)
- FastAPI application → Evaluation routes: registers route (evidence: `src/api/main.py`)
- FastAPI application → Upload route: registers route (evidence: `src/api/main.py`)
- Pipeline routes → Pipeline orchestration: runs pipeline (evidence: `src/api/routes/pipeline.py`)
- Pipeline routes → Training service: starts training (evidence: `src/api/routes/pipeline.py`)
- Pipeline routes → Evaluation service: compares models (evidence: `src/api/routes/pipeline.py`)
- Dataset routes → JSONL validation: validates upload (evidence: `src/api/routes/dataset.py`)
- Dataset routes → Preprocessing service: preprocesses data (evidence: `src/api/routes/dataset.py`)
- Evaluation routes → Evaluation service: requests evaluation (evidence: `src/api/routes/evaluation.py`)
- Upload route → Hugging Face uploader: uploads model (evidence: `src/api/routes/upload_model.py`)
- Pipeline orchestration → JSONL validation: parses and saves (evidence: `src/services/pipeline_service.py`)
- Pipeline orchestration → Preprocessing service: formats and splits (evidence: `src/services/pipeline_service.py`)
- Preprocessing service → Record preprocessing: preprocesses records (evidence: `src/services/preprocessing_service.py`)
- Preprocessing service → Prompt formatter: formats prompts (evidence: `src/services/preprocessing_service.py`)
- Preprocessing service → Dataset splitting: creates splits (evidence: `src/services/preprocessing_service.py`)
- Preprocessing service → Tokenizer service: tokenizes splits (evidence: `src/services/preprocessing_service.py`)
- Training service → LoRA trainer: runs training (evidence: `src/services/training_service.py`)
- LoRA trainer → Training configuration: uses configuration (evidence: `src/training/trainer.py`)
- LoRA trainer → Training data: loads training data (evidence: `src/training/trainer.py`)
- LoRA trainer → Model setup: sets up model (evidence: `src/training/trainer.py`)
- Evaluation service → Evaluation metrics: computes metrics (evidence: `src/services/evaluation.py`)
- Evaluation service → Inference predictor: runs inference (evidence: `src/services/evaluation.py`)
- Inference predictor → Prompt formatter: formats input (evidence: `src/inference/predictor.py`)
- Inference predictor → Prediction parser: parses output (evidence: `src/inference/predictor.py`)
- Hugging Face uploader → Hugging Face: publishes model
- Results display → Evaluator user: shows results
- Try model → Evaluator user: accepts prompt
- Model upload → Evaluator user: collects credentials