# github-linguist/linguist architecture

> Architecture diagram of the GitHub repository github-linguist/linguist, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/github-linguist/linguist
- Repository: https://github.com/github-linguist/linguist

## Overview

##### Main components
- Core Ruby Library: Represent the “lib/linguist” directory that contains the primary Ruby classes (e.g. linguist.rb, classifier.rb, heuristics.rb, repository.rb) which encapsulate the language detection algorithms.
- Native Extensions: Include the “ext/linguist” folder where components in C (such as lexers and tokenizers) are built. These extend the core Ruby code for performance.
- Command-Line Interface: Include the “bin/github-linguist” and “bin/git-linguist” scripts that provide CLI usage (and reference to the executable behavior described in the README).
- Tests: Show the “test” directory that holds various tests (for blob, classifier, file blob, heuristics, repository, samples, etc.) to ensure the system behaves as expected.
- Documentation and Samples: Represent the “docs”, “samples”, and “readme” that provide usage, guidelines, and example language files; these help users and contributors understand and extend Linguist.
- External Dependencies: Illustrate integration with external libraries such as charlock_holmes (for character encoding) and rugged (for Git repository access), including their underlying native dependencies (like ICU, pkg-config, libcurl, OpenSSL, etc).

##### Interactions and relationships
- The Ruby entry-point (CLI) uses the core library classes to load a Git repository (via Rugged), process file blobs, and then classify languages using heuristic rules (contained in “lib/linguist/heuristics.rb” and “lib/linguist/classifier.rb”).
- The tools (in “tools/grammars”) can be represented as a separate component that both consumes and produces grammar data for the library. They might interact with the core via configuration files (grammar definitions, overrides, etc.).
- External libraries (rugged and charlock_holmes) are injected into the system at runtime, so represent them as dependency nodes interfacing with the core Ruby application.

##### Architectural patterns/design principles
- Modular Design: The project is clearly separated into core library code, native extensions, CLI tools, and testing – each in its own directory.
- Dependency Injection & Extensibility: The README emphasizes how Linguist can be overridden and its behavior changed. This points out an extensible design where heuristics and language definitions are configurable.
- Command Line as an Adapter: The CLI scripts act as adapters between a user’s command environment and the underlying programmatic API.

##### Relevant technologies to highlight
- Ruby and its gem ecosystem (as the main language)
- Native C extensions (in “ext”)
- External dependencies (ICU, zlib, libcurl, OpenSSL, etc.) which can be represented as peripheral services utilized by charlock_holmes and rugged.
- Docker and Continuous Integration components in .devcontainer, Dockerfiles, and GitHub workflows that support development and testing.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interface and CLI Layer
    subgraph "User Interface & CLI Layer"
        U("User/CLI/Environment (Files&Git Input)"):::ui
        subgraph "CLI Tools"
            CLI1("GitHub CLI"):::cli
            CLI2("Git CLI"):::cli
        end
    end

    %% Core Linguist Engine
    subgraph "Core Linguist Engine"
        Core("Core Ruby Library"):::ruby
        Blob("Blob Processing"):::ruby
        Class("Classification (classifier.rb)"):::ruby
        Heur("Heuristics (heuristics.rb)"):::ruby
        Repo("Repository Handling (repository.rb)"):::ruby
        Linguist("Main Module (linguist.rb)"):::ruby
    end

    %% Native Extensions
    subgraph "Native Extensions"
        Native("Native Extensions"):::native
    end

    %% Tools & Grammar Compiler
    subgraph "Tools & Grammar Compiler"
        Tools("Grammar Tools"):::tools
    end

    %% External Dependencies
    subgraph "External Dependencies"
        Rugged("Rugged (Git)"):::external
        Charlock("Charlock Holmes"):::external
        ICU("ICU"):::external
        Zlib("zlib"):::external
        Libcurl("libcurl"):::external
        OpenSSL("OpenSSL"):::external
    end

    %% Supporting Components
    subgraph "Supporting Components"
        Tests("Tests"):::tests
        CI("CI/Containers"):::ci
    end

    %% Documentation & Samples
    subgraph "Documentation & Samples"
        Docs("Documentation"):::docs
        Samples("Samples"):::docs
        Readme("README"):::docs
    end

    %% Connections
    U -->|"uses"| CLI1
    U -->|"uses"| CLI2

    CLI1 -->|"invokes"| Core
    CLI2 -->|"invokes"| Core

    Core -->|"processes"| Blob
    Core -->|"classifies"| Class
    Core -->|"applies"| Heur
    Core -->|"wraps"| Repo
    Core -->|"includes"| Linguist

    Core -->|"calls-native"| Native

    Tools -->|"configures"| Core

    Rugged -->|"integrated"| Core
    Charlock -->|"integrated"| Core

    Tests -->|"tests"| Core
    Tests -->|"tests"| Native

    CI -->|"builds-tests"| Tests
    CI -->|"deploys"| Docs

    Docs -->|"guides"| Core

    %% Interdependencies among External Dependencies
    Rugged ---|"depends"| Charlock
    Charlock ---|"uses"| ICU
    Charlock ---|"uses"| Zlib
    Charlock ---|"uses"| Libcurl
    Charlock ---|"uses"| OpenSSL

    %% Click Events
    click Core "https://github.com/github-linguist/linguist/tree/main/lib/linguist"
    click Linguist "https://github.com/github-linguist/linguist/blob/main/lib/linguist.rb"
    click Native "https://github.com/github-linguist/linguist/tree/main/ext/linguist"
    click CLI1 "https://github.com/github-linguist/linguist/tree/main/bin/github-linguist"
    click CLI2 "https://github.com/github-linguist/linguist/tree/main/bin/git-linguist"
    click Tools "https://github.com/github-linguist/linguist/tree/main/tools/grammars"
    click Tests "https://github.com/github-linguist/linguist/tree/main/test"
    click Docs "https://github.com/github-linguist/linguist/tree/main/docs"
    click Samples "https://github.com/github-linguist/linguist/tree/main/samples"
    click Readme "https://github.com/github-linguist/linguist/blob/main/README.md"

    %% Styles
    classDef ui fill:#f0f8ff,stroke:#333,stroke-width:2px;
    classDef cli fill:#ffa500,stroke:#333,stroke-width:2px;
    classDef ruby fill:#87CEFA,stroke:#333,stroke-width:2px;
    classDef native fill:#90ee90,stroke:#333,stroke-width:2px;
    classDef tools fill:#dda0dd,stroke:#333,stroke-width:2px;
    classDef external fill:#d3d3d3,stroke:#333,stroke-width:2px;
    classDef tests fill:#ffdead,stroke:#333,stroke-width:2px;
    classDef docs fill:#e6e6fa,stroke:#333,stroke-width:2px;
    classDef ci fill:#add8e6,stroke:#333,stroke-width:2px;
```
