# davidyz/vectorcode architecture

> Architecture diagram of the GitHub repository davidyz/vectorcode, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/davidyz/vectorcode
- Repository: https://github.com/davidyz/vectorcode

## Overview

##### Project type and purpose
- VectorCode is an open-source developer tool comprising two deliverables:
a) A Python-based command-line application (“vectorcode” CLI) for indexing, embedding, and querying code repositories using a vector database (ChromaDB).
b) A Neovim plugin, written in Lua, which embeds the CLI functionality into the Neovim editor via job runners and LSP-like interactions.
- It is not a web app; rather it’s a full-featured CLI + editor-integration tool focused on code retrieval, chunking, and prompt augmentation for LLMs.

##### File structure
- Top level:
- src/vectorcode/… : Python package implementing the CLI, subcommands, LSP entry point, MCP integration, chunking, caching, reranking, etc.
- lua/vectorcode/… & plugin/vectorcode.lua : Neovim plugin code – extension points for CodeCompanion, CopilotChat, lualine, jobrunner, caching, configuration.
- docs/, README.md : User docs and CLI/Neovim usage guides.
- tests/… : Comprehensive unit tests for Python subcommands and core logic.
- pyproject.toml, Makefile, GitHub workflows, pre-commit config – build/test pipelines and packaging.

##### README for additional insights
- Core functionality: index a codebase, vectorise chunks, run queries with reranking, integrate with LLM prompt-building.
- Backend: Chromadb for vector storage, SentenceTransformer for embeddings, optional py-tree-sitter for smarter chunk splitting.
- Integration: Model Context Protocol (MCP), CodeCompanion.nvim, and native Neovim UI via Lua.

a. Main components:
1) CLI Application (Python)
– Entry point: `vectorcode/main.py` (commands: init, vectorise, query, drop, etc.)
– LSP mode: `lsp_main.py` for interactive sessions.
– MCP client: `mcp_main.py` to speak Model Context Protocol.
– Internal modules: chunking, cacher (default, LSP), jobrunner (cmd, lsp), subcommand handlers, rerankers.
2) Vector Database (ChromaDB)
– Persists document embeddings, metadata.
3) Embedding Engine
– SentenceTransformer (or pluggable alternative).
4) Neovim Plugin (Lua)
– Core plugin: `plugin/vectorcode.lua`
– Integrations: codecompanion, CopilotChat, lualine.
– Cacher layer to skip redundant retrieval.
– Jobrunner invoking Python CLI or LSP endpoint.
5) External Services
– CodeCompanion.nvim (UI for queries), Copilot Chat, Lualine statusline.

b. Relationships & data flows:
- User invokes CLI → CLI loads config → splits files into chunks → calls Embedding Engine → stores vectors in ChromaDB → returns status.
- User issues a query via CLI or Neovim → CLI/MCP/LSP server retrieves nearest-neighbor chunks from ChromaDB → optional reranking → returns JSON docs.
- Neovim plugin starts jobrunner → spawns CLI subcommand or establishes LSP session → receives results → displays via CodeCompanion or native UI.
- Plugin caching layer checks last indexed state to skip redundant requests.

c. Architectural patterns & design principles:
- Layered architecture (Presentation: CLI/UI; Application: subcommands & LSP; Domain: chunking/reranking; Infrastructure: ChromaDB, embedding, file I/O).
- Plugin-based extensibility (Lua integrations, Python subcommands, reranker strategy pattern).
- Asynchronous job runners for non-blocking editor integration.
- Clear separation of concerns: indexing vs. querying vs. LSP/MCP interface.

d. Key technologies & libraries:
- Python 3, pyproject.toml packaging, pytest for tests.
- ChromaDB vector store.
- SentenceTransformer (HuggingFace).
- py-tree-sitter for semantic chunking.
- Neovim Lua API, codecompanion.nvim integration.
- Model Context Protocol client.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Python CLI Application Layer
    subgraph "Python CLI Application" 
        direction TB
        CLI["main.py"]:::python
        LSP["lsp_main.py"]:::python
        MCP["mcp_main.py"]:::python
        Chunk["chunking.py"]:::python
        Common["common.py"]:::python
        CLIUtil["cli_utils.py"]:::python
        subgraph "Subcommands"
            direction TB
            Init["init.py"]:::python
            Vec["vectorise.py"]:::python
            subgraph "query"
                direction TB
                QueryFolder["query/"]:::python
                subgraph "reranker"
                    RerankFolder["reranker/"]:::python
                end
            end
        end
    end

    %% Embedding and Database
    Emb["Embedding Engine\n(SentenceTransformer)"]:::embed
    DB[(ChromaDB)]:::db

    %% Neovim Plugin Layer
    subgraph "Neovim Plugin" 
        direction TB
        NVim["plugin/vectorcode.lua"]:::lua
        Core["lua/vectorcode/init.lua"]:::lua
        subgraph "Cacher"
            direction TB
            DefC["default.lua"]:::lua
            LspC["lsp.lua"]:::lua
        end
        subgraph "Jobrunner"
            direction TB
            CmdJR["cmd.lua"]:::lua
            LspJR["lsp.lua"]:::lua
        end
        subgraph "Integrations"
            direction TB
            CCInt["codecompanion/"]:::lua
            Copilot["copilotchat.lua"]:::lua
            Lualine["lualine.lua"]:::lua
        end
        Types["types.lua"]:::lua
        Utils["utils.lua"]:::lua
    end

    %% External Services
    CC["CodeCompanion.nvim"]:::external
    CP["Copilot Chat"]:::external
    LL["Lualine"]:::external

    %% Data Flows
    CLI -->|"chunks files"| Chunk
    Chunk -->|"embeds via"| Emb
    Emb -->|"stores vectors"| DB
    Vec --> Emb
    Init --> DB
    QueryFolder --> DB
    DB -->|"nearest-neighbors"| RerankFolder
    RerankFolder -->|"results JSON"| CLI
    CLI -->|"output"| CLIUtil

    %% Neovim Plugin Flows
    NVim --> Core
    Core --> DefC
    Core --> LspC
    Core --> Types
    Core --> Utils
    Core --> CmdJR
    Core --> LspJR
    CmdJR --> CLI
    LspJR --> LSP
    LSP --> DB
    CmdJR --> DB
    NVim --> CCInt
    NVim --> Copilot
    NVim --> Lualine
    CCInt --> CC
    Copilot --> CP
    Lualine --> LL

    %% Click events for Python components
    click CLI "https://github.com/davidyz/vectorcode/blob/main/src/vectorcode/main.py"
    click LSP "https://github.com/davidyz/vectorcode/blob/main/src/vectorcode/lsp_main.py"
    click MCP "https://github.com/davidyz/vectorcode/blob/main/src/vectorcode/mcp_main.py"
    click Init "https://github.com/davidyz/vectorcode/blob/main/src/vectorcode/subcommands/init.py"
    click Vec "https://github.com/davidyz/vectorcode/blob/main/src/vectorcode/subcommands/vectorise.py"
    click QueryFolder "https://github.com/davidyz/vectorcode/tree/main/src/vectorcode/subcommands/query/"
    click RerankFolder "https://github.com/davidyz/vectorcode/tree/main/src/vectorcode/subcommands/query/reranker/"
    click Chunk "https://github.com/davidyz/vectorcode/blob/main/src/vectorcode/chunking.py"
    click Common "https://github.com/davidyz/vectorcode/blob/main/src/vectorcode/common.py"
    click CLIUtil "https://github.com/davidyz/vectorcode/blob/main/src/vectorcode/cli_utils.py"

    %% Click events for Lua components
    click NVim "https://github.com/davidyz/vectorcode/blob/main/plugin/vectorcode.lua"
    click Core "https://github.com/davidyz/vectorcode/blob/main/lua/vectorcode/init.lua"
    click DefC "https://github.com/davidyz/vectorcode/blob/main/lua/vectorcode/cacher/default.lua"
    click LspC "https://github.com/davidyz/vectorcode/blob/main/lua/vectorcode/cacher/lsp.lua"
    click CmdJR "https://github.com/davidyz/vectorcode/blob/main/lua/vectorcode/jobrunner/cmd.lua"
    click LspJR "https://github.com/davidyz/vectorcode/blob/main/lua/vectorcode/jobrunner/lsp.lua"
    click CCInt "https://github.com/davidyz/vectorcode/blob/main/lua/codecompanion/_extensions/vectorcode/init.lua"
    click Copilot "https://github.com/davidyz/vectorcode/blob/main/lua/vectorcode/integrations/copilotchat.lua"
    click Lualine "https://github.com/davidyz/vectorcode/blob/main/lua/vectorcode/integrations/lualine.lua"
    click Types "https://github.com/davidyz/vectorcode/blob/main/lua/vectorcode/types.lua"
    click Utils "https://github.com/davidyz/vectorcode/blob/main/lua/vectorcode/utils.lua"

    %% Styles
    classDef python fill:#D6EAF8,stroke:#1B4F72,color:#1B2631
    classDef embed fill:#FAD7A0,stroke:#B9770E,color:#4E342E
    classDef db fill:#ABEBC6,stroke:#196F3D,color:#145A32
    classDef lua fill:#FCF3CF,stroke:#9A7D0A,color:#4D2600
    classDef external fill:#D5D8DC,stroke:#566573,color:#1B2631
```
