# lilakhiz/tokenlean architecture

> Architecture diagram of the GitHub repository lilakhiz/tokenlean, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lilakhiz/tokenlean
- Repository: https://github.com/lilakhiz/tokenlean

## Overview

TokenLean is an LLM token-optimisation proxy: applications send compatible requests through it, the proxy applies configured optimisation and safety stages, then routes requests to an LLM provider and reports usage and savings. Its principal runtime is the proxy middleware pipeline; separate document and fine-tuning pipelines support knowledge preparation. The README documents the main capabilities and provider breadth; some subsystem links below reflect those documented roles because their source was not sampled.

Main entry points: [`src/proxy/middleware/pipeline.py`](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/pipeline.py) (Middleware pipeline), [`src/proxy/main.py`](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/main.py) (Proxy API), [`src/proxy/middleware/g18_observability.py`](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g18_observability.py) (Observability), [`src/proxy/providers/__init__.py`](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/providers/__init__.py) (Provider adapters)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Proxy runtime"]
  node_proxy["Proxy API<br/>[main.py]"]
  node_pipeline["Middleware pipeline<br/>[pipeline.py]"]
  node_context["Tenant context<br/>[context.py]"]
  node_config["Configuration<br/>[config_loader.py]"]
  node_keys["API key management<br/>[api_key_manager.py]"]
  node_providers["Provider adapters<br/>[__init__.py]"]
end

subgraph group_optimise["Request optimisation"]
  node_compression["Prompt compression<br/>[g01_compression.py]"]
  node_templates["Prompt templates"]
  node_cache["Semantic cache<br/>[g05_cache.py]"]
  node_routing["Model routing<br/>[g06_routing.py]"]
  node_tools["Tool loading"]
  node_output["Output formatting"]
  node_tool_output["Tool output reduction<br/>[g14_tool_output.py]"]
  node_server_compute["Server-side compute"]
end

subgraph group_knowledge["Knowledge pipelines"]
  node_document_trigger["Document trigger"]
  node_retrieval["Retrieval<br/>[g07_retrieval.py]"]
  node_doc_pipeline["Document processing<br/>[pipeline.py]"]
  node_finetune["Fine-tuning pipeline<br/>[pipeline.py]"]
end

subgraph group_operations["Safety and operations"]
  node_guardrails["Guardrails<br/>[g30_guardrails.py]"]
  node_pii["PII redaction"]
  node_billing["Usage metering<br/>[metering.py]"]
  node_observability["Observability"]
  node_savings["Savings calculation<br/>[calculator.py]"]
end

node_application(("Client application"))
node_upstream(("LLM providers"))

node_application -->|"sends request"| node_proxy
node_proxy -->|"validates keys"| node_keys
node_proxy -.->|"passes request"| node_pipeline
node_pipeline -->|"resolves tenant"| node_context
node_pipeline -->|"loads settings"| node_config
node_pipeline -->|"optimises prompt"| node_compression
node_pipeline -->|"applies templates"| node_templates
node_pipeline -->|"checks cache"| node_cache
node_pipeline -->|"routes model"| node_routing
node_pipeline -->|"adds context"| node_retrieval
node_pipeline -->|"prunes tools"| node_tools
node_pipeline -->|"formats output"| node_output
node_pipeline -->|"reduces tool output"| node_tool_output
node_pipeline -->|"offloads compute"| node_server_compute
node_pipeline -->|"checks safety"| node_guardrails
node_pipeline -->|"redacts PII"| node_pii
node_pipeline -->|"dispatches request"| node_providers
node_providers -->|"calls provider"| node_upstream
node_pipeline -->|"records usage"| node_observability
node_observability -->|"estimates savings"| node_savings
node_observability -.->|"meters usage"| node_billing
node_document_trigger -->|"triggers processing"| node_doc_pipeline
node_doc_pipeline -.->|"feeds fine-tuning"| node_finetune

click node_proxy "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/main.py"
click node_pipeline "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/pipeline.py"
click node_context "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/tenancy/context.py"
click node_config "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/config_loader.py"
click node_keys "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/auth/api_key_manager.py"
click node_providers "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/providers/__init__.py"
click node_compression "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g01_compression.py"
click node_templates "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g02_template_registry.py"
click node_document_trigger "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g03_doc_pipeline.py"
click node_cache "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g05_cache.py"
click node_routing "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g06_routing.py"
click node_retrieval "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g07_retrieval.py"
click node_tools "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g08_tool_loading.py"
click node_output "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g11_output_format.py"
click node_tool_output "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g14_tool_output.py"
click node_server_compute "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g15_server_compute.py"
click node_guardrails "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g30_guardrails.py"
click node_pii "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g29_pii_redaction.py"
click node_billing "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/billing/metering.py"
click node_observability "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g18_observability.py"
click node_savings "https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/savings/calculator.py"
click node_doc_pipeline "https://github.com/lilakhiz/tokenlean/blob/main/src/doc-pipeline/pipeline.py"
click node_finetune "https://github.com/lilakhiz/tokenlean/blob/main/src/finetune-pipeline/pipeline.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_proxy,node_pipeline,node_context,node_config,node_keys,node_providers,node_application toneBlue
class node_compression,node_templates,node_cache,node_routing,node_tools,node_output,node_tool_output,node_server_compute toneAmber
class node_document_trigger,node_retrieval,node_doc_pipeline,node_finetune toneMint
class node_guardrails,node_pii,node_billing,node_observability,node_savings toneRose
class node_upstream toneIndigo
```

## Components

### Proxy runtime

- [Proxy API](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/main.py): `src/proxy/main.py`
- [Middleware pipeline](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/pipeline.py): `src/proxy/middleware/pipeline.py`
- [Tenant context](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/tenancy/context.py): `src/proxy/tenancy/context.py`
- [Configuration](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/config_loader.py): `src/proxy/config_loader.py`
- [API key management](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/auth/api_key_manager.py): `src/proxy/auth/api_key_manager.py`
- [Provider adapters](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/providers/__init__.py): `src/proxy/providers/__init__.py`

### Request optimisation

- [Prompt compression](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g01_compression.py): `src/proxy/middleware/g01_compression.py`
- [Prompt templates](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g02_template_registry.py): `src/proxy/middleware/g02_template_registry.py`
- [Semantic cache](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g05_cache.py): `src/proxy/middleware/g05_cache.py`
- [Model routing](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g06_routing.py): `src/proxy/middleware/g06_routing.py`
- [Tool loading](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g08_tool_loading.py): `src/proxy/middleware/g08_tool_loading.py`
- [Output formatting](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g11_output_format.py): `src/proxy/middleware/g11_output_format.py`
- [Tool output reduction](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g14_tool_output.py): `src/proxy/middleware/g14_tool_output.py`
- [Server-side compute](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g15_server_compute.py): `src/proxy/middleware/g15_server_compute.py`

### Knowledge pipelines

- [Document trigger](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g03_doc_pipeline.py): `src/proxy/middleware/g03_doc_pipeline.py`
- [Retrieval](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g07_retrieval.py): `src/proxy/middleware/g07_retrieval.py`
- [Document processing](https://github.com/lilakhiz/tokenlean/blob/main/src/doc-pipeline/pipeline.py): `src/doc-pipeline/pipeline.py`
- [Fine-tuning pipeline](https://github.com/lilakhiz/tokenlean/blob/main/src/finetune-pipeline/pipeline.py): `src/finetune-pipeline/pipeline.py`

### Safety and operations

- [Guardrails](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g30_guardrails.py): `src/proxy/middleware/g30_guardrails.py`
- [PII redaction](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g29_pii_redaction.py): `src/proxy/middleware/g29_pii_redaction.py`
- [Usage metering](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/billing/metering.py): `src/proxy/billing/metering.py`
- [Observability](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/middleware/g18_observability.py): `src/proxy/middleware/g18_observability.py`
- [Savings calculation](https://github.com/lilakhiz/tokenlean/blob/main/src/proxy/savings/calculator.py): `src/proxy/savings/calculator.py`

### Other components

- **Client application**
- **LLM providers**

## Connections

- Client application → Proxy API: sends request
- Proxy API → API key management: validates keys (evidence: `src/proxy/main.py`)
- Proxy API → Middleware pipeline: passes request (evidence: `src/proxy/main.py`)
- Middleware pipeline → Tenant context: resolves tenant (evidence: `src/proxy/middleware/pipeline.py`)
- Middleware pipeline → Configuration: loads settings (evidence: `src/proxy/middleware/pipeline.py`)
- Middleware pipeline → Prompt compression: optimises prompt (evidence: `src/proxy/middleware/pipeline.py`)
- Middleware pipeline → Prompt templates: applies templates (evidence: `src/proxy/middleware/pipeline.py`)
- Middleware pipeline → Semantic cache: checks cache (evidence: `src/proxy/middleware/pipeline.py`)
- Middleware pipeline → Model routing: routes model (evidence: `src/proxy/middleware/pipeline.py`)
- Middleware pipeline → Retrieval: adds context (evidence: `src/proxy/middleware/pipeline.py`)
- Middleware pipeline → Tool loading: prunes tools (evidence: `src/proxy/middleware/pipeline.py`)
- Middleware pipeline → Output formatting: formats output (evidence: `src/proxy/middleware/pipeline.py`)
- Middleware pipeline → Tool output reduction: reduces tool output (evidence: `src/proxy/middleware/pipeline.py`)
- Middleware pipeline → Server-side compute: offloads compute (evidence: `src/proxy/middleware/pipeline.py`)
- Middleware pipeline → Guardrails: checks safety (evidence: `src/proxy/middleware/pipeline.py`)
- Middleware pipeline → PII redaction: redacts PII (evidence: `src/proxy/middleware/pipeline.py`)
- Middleware pipeline → Provider adapters: dispatches request (evidence: `src/proxy/middleware/pipeline.py`)
- Provider adapters → LLM providers: calls provider
- Middleware pipeline → Observability: records usage (evidence: `src/proxy/middleware/pipeline.py`)
- Observability → Savings calculation: estimates savings (evidence: `src/proxy/middleware/g18_observability.py`)
- Observability → Usage metering: meters usage
- Document trigger → Document processing: triggers processing (evidence: `src/proxy/middleware/g03_doc_pipeline.py`)
- Document processing → Fine-tuning pipeline: feeds fine-tuning
