How emacs-tree-sitter/tree-sitter-langs works
This project is the “Tree-sitter Language Bundle for Emacs” – a bundle that provides pre-compiled grammar binaries and accompanying query files (such as highlighting patterns) for multiple programming languages to be used by the Emacs tree-sitter package. The goal of the system is to offer an interim distribution mechanism that Emacs major modes can use until full native support for tree-sitter is integrated.
Project Type & Purpose
• That this is not a traditional full-stack web app or a command-line tool but rather a language bundle system. It aggregates multiple language grammars (stored as submodules in “repos/”) alongside language query files (in “queries/”) and provides scripts to build and test these components.
• Its primary use case is supporting Emacs integration – encouraging language-specific functionality for syntax highlighting, code folding, and other text-processing tasks via tree-sitter.
Key Components
• Emacs Integration Layer:
• Emacs tree-sitter package and potential major modes that depend on the precompiled grammars and query definitions.
• Build & Compilation System:
• The “script” directory (with components like “compile” and “test”) drives the process of generating the environment – compiling language grammars from source using NodeJS and the tree-sitter CLI.
• The “bin” directory (even if minimal initially) represents any binary artifacts.
• Language Grammar Sources:
• The “repos/” directory is central: each subdirectory (e.g., repos/rust, repos/python, etc.) is a git submodule that contains the source grammar for a particular programming language.
• Query Definitions:
• The “queries/” directory holds highlight and additional query files. Each language has its dedicated subdirectory with at least a “highlights.scm” file (and occasionally mode-specific overrides).
• CI/CD and Configuration:
• The “.github/” directory and related CI workflow files (like bump-version.yml, release.yml, etc.) indicate the automated testing and release processes.
• Other config files (.gitattributes, .gitignore, .gitmodules, etc.) support version control and submodule management.
• External Dependencies:
• The project depends on tools like “eask” (an Emacs package tool), NodeJS (for generating grammar code), and the tree-sitter CLI.
Relationships and Interactions
• The Build System (scripts in “script/”) acts as the orchestrator. It:
• Pulls in grammar source code from the “repos/” submodules.
• Uses external tools (NodeJS, tree-sitter CLI) to compile grammar binaries.
• Packages these binaries into a distributable bundle along with the “queries/” files.
• CI/CD pipelines (from “.github/workflows/”) trigger these build and test sequences, ensuring that new grammar definitions or query updates do not break the bundle.
• The final output (pre-compiled grammars and queries) is consumed by the Emacs ecosystem via the tree-sitter integration.
Architectural Patterns & Technologies
• Modular Design:
• Each language is encapsulated as a module (submodule in “repos/”, corresponding query in “queries/”) allowing independent updates and extension.
• Build and Dependency Management:
• The use of “eask”, NodeJS, and tree-sitter CLI as dependencies emphasizes that the system is a build toolchain that automates the generation of binaries.
• CI/CD Integration:
• Automated workflows facilitate testing, building, and releasing, which is represented in the CI/CD pipelines.