# microsoft/llmlingua architecture

> Architecture diagram of the GitHub repository microsoft/llmlingua, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/microsoft/llmlingua
- Repository: https://github.com/microsoft/llmlingua

## Overview

LLMLingua is a Python library that compresses prompts before they are sent to an LLM. Its principal workflow accepts prompt context and task text, uses a compact model and token-level analysis to select or preserve important content, and returns a shorter prompt. The README also documents a LongLLMLingua mode for long-context and retrieval scenarios and a faster LLMLingua-2 mode. The sampled implementation excerpt is incomplete, so internal method-level wiring is shown only where supported; experiment and training scripts are not part of the runtime map.

Main entry points: [`llmlingua/prompt_compressor.py`](https://github.com/microsoft/llmlingua/blob/main/llmlingua/prompt_compressor.py) (PromptCompressor), [`llmlingua/utils.py`](https://github.com/microsoft/llmlingua/blob/main/llmlingua/utils.py) (Token utilities)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_compression["Prompt compression"]
  node_compressor["PromptCompressor"]
  node_token_analysis["Token analysis"]
  node_context_ranking["Context ranking"]
  node_long_mode["LongLLMLingua mode"]
  node_lingua2_mode["LLMLingua-2 mode"]
end

subgraph group_models["Model support"]
  node_utilities["Token utilities<br/>[utils.py]"]
  node_compact_model["Compact model"]
  node_tokenizer["Tokenizer"]
  node_embedding_api["Embedding API"]
end

subgraph group_consumers["Integration"]
  node_llm["Downstream LLM"]
end

node_app(("Application"))

node_app -->|"submits prompt"| node_compressor
node_compressor -->|"tokenizes text"| node_tokenizer
node_compressor -->|"loads scores"| node_compact_model
node_compressor -->|"analyzes tokens"| node_token_analysis
node_token_analysis -->|"uses utilities"| node_utilities
node_compressor -->|"ranks context"| node_context_ranking
node_compressor -->|"selects mode"| node_long_mode
node_compressor -->|"selects mode"| node_lingua2_mode
node_compressor -.->|"requests embeddings"| node_embedding_api
node_compressor -->|"returns compressed prompt"| node_app
node_app -->|"sends prompt"| node_llm

click node_compressor "https://github.com/microsoft/llmlingua/blob/main/llmlingua/prompt_compressor.py"
click node_token_analysis "https://github.com/microsoft/llmlingua/blob/main/llmlingua/prompt_compressor.py"
click node_context_ranking "https://github.com/microsoft/llmlingua/blob/main/llmlingua/prompt_compressor.py"
click node_long_mode "https://github.com/microsoft/llmlingua/blob/main/llmlingua/prompt_compressor.py"
click node_lingua2_mode "https://github.com/microsoft/llmlingua/blob/main/llmlingua/prompt_compressor.py"
click node_utilities "https://github.com/microsoft/llmlingua/blob/main/llmlingua/utils.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_compressor,node_token_analysis,node_context_ranking,node_long_mode,node_lingua2_mode toneBlue
class node_utilities,node_compact_model,node_tokenizer,node_embedding_api toneAmber
class node_llm toneMint
class node_app toneIndigo
```

## Components

### Prompt compression

- [PromptCompressor](https://github.com/microsoft/llmlingua/blob/main/llmlingua/prompt_compressor.py): `llmlingua/prompt_compressor.py`
- [Token analysis](https://github.com/microsoft/llmlingua/blob/main/llmlingua/prompt_compressor.py): `llmlingua/prompt_compressor.py`
- [Context ranking](https://github.com/microsoft/llmlingua/blob/main/llmlingua/prompt_compressor.py): `llmlingua/prompt_compressor.py`
- [LongLLMLingua mode](https://github.com/microsoft/llmlingua/blob/main/llmlingua/prompt_compressor.py): `llmlingua/prompt_compressor.py`
- [LLMLingua-2 mode](https://github.com/microsoft/llmlingua/blob/main/llmlingua/prompt_compressor.py): `llmlingua/prompt_compressor.py`

### Model support

- [Token utilities](https://github.com/microsoft/llmlingua/blob/main/llmlingua/utils.py): `llmlingua/utils.py`
- **Compact model**
- **Tokenizer**
- **Embedding API**

### Integration

- **Downstream LLM**

### Other components

- **Application**

## Connections

- Application → PromptCompressor: submits prompt
- PromptCompressor → Tokenizer: tokenizes text
- PromptCompressor → Compact model: loads scores
- PromptCompressor → Token analysis: analyzes tokens
- Token analysis → Token utilities: uses utilities
- PromptCompressor → Context ranking: ranks context
- PromptCompressor → LongLLMLingua mode: selects mode
- PromptCompressor → LLMLingua-2 mode: selects mode
- PromptCompressor → Embedding API: requests embeddings
- PromptCompressor → Application: returns compressed prompt
- Application → Downstream LLM: sends prompt