# aitullalimon/creative-writing-llm-benchmark architecture

> Architecture diagram of the GitHub repository aitullalimon/creative-writing-llm-benchmark, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aitullalimon/creative-writing-llm-benchmark
- Repository: https://github.com/aitullalimon/creative-writing-llm-benchmark

## Overview

Creative-Writing-LLM-Benchmark is a Next.js application for comparing creative-writing models, scoring their outputs with an LLM judge, and reviewing saved results. Its main workflow is prompt/model selection, benchmark execution, model generation and judging, then local run-history storage and leaderboard analysis. The README also documents LiteLLM routing and theme-coherence analytics. The benchmark implementation and several supporting modules were not sampled, so their internal call wiring is not asserted; the graph follows the documented workflow and visible page/API interactions.

Main entry points: [`app/leaderboard/page.tsx`](https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/app/leaderboard/page.tsx) (Leaderboard page), [`app/page.tsx`](https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/app/page.tsx) (Benchmark setup/results), [`app/api/benchmark/route.ts`](https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/app/api/benchmark/route.ts) (Run benchmark), [`app/api/config-models/route.ts`](https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/app/api/config-models/route.ts) (Configured models)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ui["Benchmark UI"]
  node_home["Benchmark setup/results<br/>[page.tsx]"]
  node_leaderboard["Leaderboard page<br/>[page.tsx]"]
  node_chartcard["Leaderboard charts"]
  node_coherencechart["Coherence history chart"]
end

subgraph group_api["API routes"]
  node_benchmarkapi["Run benchmark<br/>[route.ts]"]
  node_configapi["Configured models<br/>[route.ts]"]
  node_metadataapi["Model metadata<br/>[route.ts]"]
  node_chatapi["Chat proxy<br/>[route.ts]"]
end

subgraph group_eval["Evaluation"]
  node_bench["Benchmark logic<br/>[bench.ts]"]
  node_judge["LLM judge scoring<br/>[judge.ts]"]
  node_models["Benchmark types<br/>[models.ts]"]
end

subgraph group_results["History and analytics"]
  node_browserstore[("Saved run history")]
  node_browserstorage[("Run storage helper<br/>[storage.ts]")]
  node_stats["Model statistics"]
end

subgraph group_integrations["Model integrations"]
  node_config["LiteLLM configuration"]
  node_litellm{{"LiteLLM routing proxy"}}
  node_providers{{"LLM providers"}}
  node_openai{{"OpenAI chat API"}}
end

node_user(("Benchmark user"))

node_user -->|"sets prompt"| node_home
node_home -->|"loads model list"| node_configapi
node_configapi -->|"reads model config"| node_config
node_home -->|"submits benchmark"| node_benchmarkapi
node_benchmarkapi -->|"runs evaluation"| node_bench
node_bench -->|"requests scoring"| node_judge
node_benchmarkapi -.->|"routes model calls"| node_litellm
node_litellm -.->|"dispatches requests"| node_providers
node_benchmarkapi -->|"returns results"| node_home
node_home -->|"saves run"| node_browserstore
node_home -->|"loads recent runs"| node_browserstore
node_user -->|"opens analytics"| node_leaderboard
node_leaderboard -->|"loads history"| node_browserstore
node_leaderboard -->|"aggregates runs"| node_stats
node_stats -->|"returns model stats"| node_leaderboard
node_leaderboard -->|"loads model config"| node_configapi
node_leaderboard -->|"renders rankings"| node_chartcard
node_leaderboard -->|"renders run trends"| node_coherencechart
node_home -->|"stores runs"| node_browserstorage
node_leaderboard -->|"reads runs"| node_browserstorage
node_leaderboard -.->|"requests metadata"| node_metadataapi
node_chatapi -->|"sends chat request"| node_openai

click node_home "https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/app/page.tsx"
click node_leaderboard "https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/app/leaderboard/page.tsx"
click node_chartcard "https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/components/LeaderboardChartCard.tsx"
click node_coherencechart "https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/components/ThemeCoherenceLineChart.tsx"
click node_benchmarkapi "https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/app/api/benchmark/route.ts"
click node_configapi "https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/app/api/config-models/route.ts"
click node_metadataapi "https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/app/api/model-metadata/route.ts"
click node_chatapi "https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/app/api/litellm-chat/route.ts"
click node_bench "https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/lib/bench.ts"
click node_judge "https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/lib/judge.ts"
click node_models "https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/lib/models.ts"
click node_browserstore "https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/lib/leaderboardStore.ts"
click node_browserstorage "https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/lib/storage.ts"
click node_stats "https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/lib/leaderboardStore.ts"
click node_config "https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/litellm-config.yaml"
click node_litellm "https://github.com/aitullalimon/creative-writing-llm-benchmark/tree/main/litellm"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_home,node_leaderboard,node_chartcard,node_coherencechart,node_user toneBlue
class node_benchmarkapi,node_configapi,node_metadataapi,node_chatapi toneAmber
class node_bench,node_judge,node_models toneMint
class node_browserstore,node_browserstorage,node_stats toneRose
class node_config,node_litellm,node_providers,node_openai toneIndigo
```

## Components

### Benchmark UI

- [Benchmark setup/results](https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/app/page.tsx): `app/page.tsx`
- [Leaderboard page](https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/app/leaderboard/page.tsx): `app/leaderboard/page.tsx`
- [Leaderboard charts](https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/components/LeaderboardChartCard.tsx): `components/LeaderboardChartCard.tsx`
- [Coherence history chart](https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/components/ThemeCoherenceLineChart.tsx): `components/ThemeCoherenceLineChart.tsx`

### API routes

- [Run benchmark](https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/app/api/benchmark/route.ts): `app/api/benchmark/route.ts`
- [Configured models](https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/app/api/config-models/route.ts): `app/api/config-models/route.ts`
- [Model metadata](https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/app/api/model-metadata/route.ts): `app/api/model-metadata/route.ts`
- [Chat proxy](https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/app/api/litellm-chat/route.ts): `app/api/litellm-chat/route.ts`

### Evaluation

- [Benchmark logic](https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/lib/bench.ts): `lib/bench.ts`
- [LLM judge scoring](https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/lib/judge.ts): `lib/judge.ts`
- [Benchmark types](https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/lib/models.ts): `lib/models.ts`

### History and analytics

- [Saved run history](https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/lib/leaderboardStore.ts): `lib/leaderboardStore.ts`
- [Run storage helper](https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/lib/storage.ts): `lib/storage.ts`
- [Model statistics](https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/lib/leaderboardStore.ts): `lib/leaderboardStore.ts`

### Model integrations

- [LiteLLM configuration](https://github.com/aitullalimon/creative-writing-llm-benchmark/blob/main/litellm-config.yaml): `litellm-config.yaml`
- [LiteLLM routing proxy](https://github.com/aitullalimon/creative-writing-llm-benchmark/tree/main/litellm): `litellm`
- **LLM providers**
- **OpenAI chat API**

### Other components

- **Benchmark user**

## Connections

- Benchmark user → Benchmark setup/results: sets prompt
- Benchmark setup/results → Configured models: loads model list
- Configured models → LiteLLM configuration: reads model config
- Benchmark setup/results → Run benchmark: submits benchmark
- Run benchmark → Benchmark logic: runs evaluation
- Benchmark logic → LLM judge scoring: requests scoring
- Run benchmark → LiteLLM routing proxy: routes model calls
- LiteLLM routing proxy → LLM providers: dispatches requests
- Run benchmark → Benchmark setup/results: returns results
- Benchmark setup/results → Saved run history: saves run
- Benchmark setup/results → Saved run history: loads recent runs
- Benchmark user → Leaderboard page: opens analytics
- Leaderboard page → Saved run history: loads history
- Leaderboard page → Model statistics: aggregates runs
- Model statistics → Leaderboard page: returns model stats
- Leaderboard page → Configured models: loads model config
- Leaderboard page → Leaderboard charts: renders rankings
- Leaderboard page → Coherence history chart: renders run trends
- Benchmark setup/results → Run storage helper: stores runs
- Leaderboard page → Run storage helper: reads runs
- Leaderboard page → Model metadata: requests metadata
- Chat proxy → OpenAI chat API: sends chat request