How shikijs/shiki works
Shiki is a TextMate-grammar-based syntax highlighter. Its main workflow is to accept code and language/theme configuration, match grammar patterns with a regex engine, then produce highlighted output for consumers. The workspace also provides editor and Markdown adapters, token transformers, and framework-specific rendering products. The source index supports several internal links, but many package implementations were not sampled; uncertain connections are therefore omitted or marked dashed.
Main entry points: core.ts Shiki API · code-to-hast.ts Highlight conversion · engine-compile.ts JavaScript regex · scanner.ts Regex scanner
Components 22
Highlighting runtime
- Shiki API
packages/shiki/src/core.ts - Highlight conversion
packages/core/src/highlight/code-to-hast.ts - HTML output
packages/core/src/highlight/code-to-html.ts - Token output
packages/core/src/highlight/code-to-tokens.ts
Grammar and engines
- TextMate runtime
packages/primitive/src/textmate/grammar-state.ts - Primitive tokenizer
packages/primitive/src/highlight/code-to-tokens-base.ts - JavaScript regex
packages/engine-javascript/src/engine-compile.ts - Regex scanner
packages/engine-javascript/src/scanner.ts - Oniguruma engine
packages/engine-oniguruma/src/oniguruma/index.ts - Languages and themes
packages/shiki/src/langs.ts - String adapter
packages/engine-javascript/src/string.ts
Consumer integrations
- Terminal CLI
packages/cli/src/cli.ts - ANSI conversion
packages/cli/src/code-to-ansi.ts - Monaco adapter
packages/monaco/src/index.ts - Rehype adapter
packages/rehype/src/core.ts - Markdown-it adapter
packages/markdown-it/src/core.ts
Highlighting extensions
- Token transformers
packages/transformers/src/shared/notation-transformer.ts - Bracket coloring
packages/colorized-brackets/src/colorizeBracketTokens.ts - Rich Twoslash renderer
packages/twoslash/src/renderer-rich.ts
Framework rendering
- Magic Move
packages/magic-move/src/renderer.ts - Streaming highlighter
packages/stream/src/stream.ts
Other components
- Application
Connections 19
- Application → Shiki API: configures and highlights
- Shiki API → Highlight conversion: uses
- Highlight conversion → HTML output: renders HTML
- Highlight conversion → Token output: produces tokens
- Primitive tokenizer → TextMate runtime: uses grammar state
packages/primitive/src/highlight/code-to-tokens-base.ts - JavaScript regex → Regex scanner: constructs scanner
packages/engine-javascript/src/engine-compile.ts - Regex scanner → String adapter: uses string adapter
packages/engine-javascript/src/scanner.ts - Shiki API → JavaScript regex: re-exports engine
- Shiki API → Oniguruma engine: re-exports engine
- Shiki API → Languages and themes: uses bundled assets
- Terminal CLI → ANSI conversion: formats terminal output
packages/cli/src/cli.ts - Monaco adapter → Shiki API: integrates highlighter
- Rehype adapter → Highlight conversion: adapts code blocks
- Markdown-it adapter → Highlight conversion: adapts Markdown
- Token transformers → Highlight conversion: transforms output
- Bracket coloring → Token output: colors tokens
- Rich Twoslash renderer → Shiki API: highlights popup code
- Magic Move → Shiki API: animates highlighted code
- Streaming highlighter → Shiki API: streams highlighting