# sturdy-dev/semantic-code-search architecture

> Architecture diagram of the GitHub repository sturdy-dev/semantic-code-search, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sturdy-dev/semantic-code-search
- Repository: https://github.com/sturdy-dev/semantic-code-search

## Overview

##### Project Type and Purpose
- This is an open-source, command-line application written in Python, named `semantic-code-search` (invoked as `sem`).
- It indexes a local Git repo’s function/method definitions by generating and caching vector embeddings, then performs a nearest-neighbor search on those embeddings when a user issues a natural-language query.

##### File Structure Analysis
- Top-level:
- setup.py / setup.cfg / requirements.txt – packaging and dependencies
- docs/ – README assets and example screenshots
- src/semantic_code_search/ – core modules:
- cli.py – entry point; parses flags and dispatches commands
- embed.py – scans code, extracts functions, generates embeddings, writes `.embeddings` cache
- query.py – embeds the query text and invokes similarity search
- cluster.py – optional clustering of similar code embeddings
- prompt.py – formatting of results and interactive UI logic

##### Key Components
a. Command-Line Interface (CLI)
- Accepts user commands and flags (`sem 'query'`, `sem --embed`, etc.)
- Dispatches to embed, query, or cluster workflows.
b. Git Repository / Local File System
- Contains source files (.py, etc.)
- Holds the cache file `.embeddings` at repo root.
c. Code Extraction & Embedding Generation
- A pipeline: parse source files → extract functions/methods → batch-embed via a Sentence-Transformer model
d. Embeddings Cache (.embeddings)
- Serialized vector store used for subsequent queries.
e. Transformer Model
- Locally downloaded on first run (~500 MB).
- Provides embedding API (via HuggingFace / sentence-transformers).
f. Nearest-Neighbor Search & Clustering
- Query embeddings vs. cached embeddings → cosine similarity ranking
- Clustering module optionally groups similar code snippets.
g. Results Renderer & Editor Launcher
- Formats top-N matches with score, file path, line, snippet.

##### Relationships and Data Flows
- CLI → (on embed) → Code Extraction → Transformer Model → write to `.embeddings`
- CLI → (on query) → Transformer Model → load `.embeddings` → Similarity Search → Results Renderer → Editor
- CLI → (on cluster) → load `.embeddings` → Clustering Algorithm → print clusters
- All data flows are local (no network calls after model download).

##### Architectural Patterns & Tech Stack
- Modular Python package with single-process command dispatcher
- Embedding generation follows a producer-consumer pattern (code → embeddings → cache)
- Model inference uses the sentence-transformers library
- Nearest-neighbor search over dense vectors (e.g., via Faiss or brute-force in-memory)
- Optional clustering based on distance thresholds

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CLI Entry
    CLI["CLI (sem)"]:::terminal
    click CLI "https://github.com/sturdy-dev/semantic-code-search/blob/main/src/semantic_code_search/cli.py"

    %% Preparation Phase
    subgraph "Preparation Phase"
        direction TB
        Extract["Code Extraction\n& Embedding Generation"]:::compute
        click Extract "https://github.com/sturdy-dev/semantic-code-search/blob/main/src/semantic_code_search/embed.py"
        Model["Transformer Model\n(sentence-transformers)"]:::external
        Cache[(.embeddings)]:::artifact
        click Cache "https://github.com/sturdy-dev/semantic-code-search/blob/main/.embeddings"
    end

    %% Query Phase
    subgraph "Query Phase"
        direction TB
        Query["Natural-Language Query\nHandling & Search"]:::compute
        click Query "https://github.com/sturdy-dev/semantic-code-search/blob/main/src/semantic_code_search/query.py"
        Search["Nearest-Neighbor\nSimilarity Search"]:::compute
        Renderer["Results Renderer\n& Interactive UI"]:::compute
        click Renderer "https://github.com/sturdy-dev/semantic-code-search/blob/main/src/semantic_code_search/prompt.py"
        Editor["Editor Launcher\n(VSCode/vim)"]:::external
    end

    %% Clustering Phase
    subgraph "Clustering"
        direction TB
        Cluster["Optional Embedding\nClustering"]:::compute
        click Cluster "https://github.com/sturdy-dev/semantic-code-search/blob/main/src/semantic_code_search/cluster.py"
    end

    %% Packaging & Documentation
    subgraph "Utilities"
        class Config config
        Config["Packaging &\nDependencies"]:::config
        click Config "https://github.com/sturdy-dev/semantic-code-search/blob/main/setup.py"
        Docs["Documentation & Examples"]:::doc
        click Docs "https://github.com/sturdy-dev/semantic-code-search/tree/main/docs/"
    end

    %% Data Flows
    CLI -->|"--embed"| Extract
    Extract -->|"vectors"| Model
    Model -->|"embeddings"| Cache

    CLI -->|"query"| Query
    Query -->|"query vector"| Model
    Model -->|"loads embeddings"| Cache
    Cache -->|"vectors"| Search
    Search -->|"top-N matches"| Renderer
    Renderer -->|"open snippet"| Editor

    CLI -->|"--cluster"| Cache
    Cache -->|"embeddings"| Cluster
    Cluster -->|"clusters"| Renderer

    %% Styling
    classDef compute fill:#AED6F1,stroke:#1B4F72,color:#1B2631;
    classDef artifact fill:#D5DBDB,stroke:#646464,color:#1B2631;
    classDef external fill:#A9DFBF,stroke:#196F3D,color:#1B2631;
    classDef terminal fill:#F9E79F,stroke:#B9770E,color:#1B2631,stroke-dasharray: 5 5;
    classDef config fill:#FADBD8,stroke:#922B21,color:#1B2631;
    classDef doc fill:#EBDEF0,stroke:#6C3483,color:#1B2631;
```
