# zamalali/deepgit architecture

> Architecture diagram of the GitHub repository zamalali/deepgit, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zamalali/deepgit
- Repository: https://github.com/zamalali/deepgit

## Overview

##### Project type and purpose
- DeepGit is a full-stack, agentic Python application whose core is an orchestration layer (agent.py + langgraph.json) that invokes a series of expert “tools” to perform multi-stage semantic search, ranking, and analysis of GitHub repositories.
- The user interacts via a lightweight web UI (app.py), issues a natural-language query, and receives a ranked list of repos enriched with metadata, similarity scores, and hardware-compatibility badges.

##### File structure
- Top level:
- app.py – the web-server/entrypoint for user queries
- agent.py – the Langgraph-based orchestrator agent
- langgraph.json – workflow definition for orchestrator
- tools/ – a folder of individual modules, one per step in the pipeline
- tests/ – unit tests validating each tool
- docs/ – architecture, Docker, testing guides
- Dockerfile & .github/workflows – containerization and CI/CD
- Each tools/*.py module encapsulates a single responsibility, suggesting a micro-pipeline architecture.

##### README for additional insights
- The “How It Works” section enumerates seven sequential steps (query expansion, hardware spec detection, dense retrieval, re-ranking, filtering, community/quality analysis, multi-factor ranking, and presentation).
- It calls out specific technologies: LLMs for query expansion and hardware hint extraction, ColBERT v2 for embedding retrieval, MiniLM for cross-encoder re-ranking, GitHub API for metadata, and Python environment checks for hardware filtering.

a. Main components:
- Frontend/UI (app.py + static assets)
- Orchestrator Agent (agent.py + langgraph runtime)
- Tool modules (each tools/*.py)
- External services: LLM provider (OpenAI or similar), embedding/ColBERT service, GitHub API
- Optional data stores: in-memory context or cache (not persisted)
b. Relationships & interactions:
1. User → Frontend (HTTP request)
2. Frontend → Orchestrator Agent (invoke langgraph workflow)
3. Agent → Tool modules in sequence (via Langgraph, defined in langgraph.json)
4. Each tool → External service as needed (LLM calls, embedding API, GitHub API)
5. Results flow back upstream to the agent → formatted by output_presentation → returned to Frontend → User
c. Architectural patterns & principles:
- Agent-based orchestration (Langgraph)
- Single-responsibility modules (tools folder)
- Pipeline pattern (linear, stepwise data transformations)
- Hybrid retrieval (vector + lexical + metadata)
d. Key technologies/frameworks:
- Python 3.11+, Flask/FastAPI or similar for app.py
- Langgraph/langsmith for orchestration
- OpenAI / other LLM API (query expansion, reasoning)
- Hugging Face’s ColBERT v2 & MiniLM for retrieval & re-ranking
- GitHub REST API
- Docker & GitHub Actions for CI/CD

##### IMPORTANT – Modules making LLM calls
- convert_query.py (LLM-driven query expansion)
- chat.py (generic LLM interactions if used)
- decision.py & decision_maker.py (LLM for reasoning/hardware-aware filtering logic)
- mcp_adapter.py (if it adapts between tools and LLM responses)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Layers
    subgraph "UI Layer"
        UI["Frontend/UI (app.py)"]:::internal
    end

    subgraph "Orchestrator Layer"
        Agent["Orchestrator Agent (agent.py)"]:::internal
        Workflow["Workflow Definition (langgraph.json)"]:::internal
    end

    subgraph "Tool Modules"
        Convert["Query Expander (convert_query.py)"]:::llm
        Chat["Generic LLM Interface (chat.py)"]:::llm
        Decision["Decision Maker (decision_maker.py / decision.py)"]:::llm
        MCP["LLM Response Adapter (mcp_adapter.py)"]:::llm
        ParseHW["Hardware Spec Parser (parse_hardware.py)"]:::internal
        Dense["Dense Retrieval (dense_retrieval.py)"]:::internal
        Rerank["Cross-Encoder Re-Ranking (cross_encoder_reranking.py)"]:::internal
        Filter["Filtering (filtering.py)"]:::internal
        Activity["Community/Quality Analysis (activity_analysis.py)"]:::internal
        DepAnalysis["Dependency Analysis (dependency_analysis.py)"]:::internal
        Merge["Merge/Community Metrics Analysis (merge_analysis.py)"]:::internal
        Rank["Ranking & Aggregation (ranking.py)"]:::internal
        Output["Output Presentation (output_presentation.py)"]:::internal
        GitHub["GitHub Metadata Fetcher (github.py)"]:::internal
        Eval["Evaluation Helpers (evaluation.py)"]:::internal
    end

    subgraph "External Services"
        LLM["LLM Provider"]:::external
        Embedding["Embedding Service (ColBERT v2)"]:::external
        GitHubAPI["GitHub REST API"]:::external
    end

    subgraph "CI/CD & Deployment"
        Docker["Dockerfile"]:::ci
        CI["GitHub Actions (.github/workflows/docker-image.yml)"]:::ci
    end

    %% Data Flows
    UI -->|"HTTP Request"| Agent
    Agent -->|"invoke workflow"| Workflow

    Workflow --> Convert
    Convert -->|"via Adapter"| MCP
    MCP -->|"API Call"| LLM

    Workflow --> Chat
    Chat -->|"via Adapter"| MCP

    Workflow --> Decision
    Decision -->|"via Adapter"| MCP

    Workflow --> ParseHW

    Workflow --> Dense
    Dense -->|"retrieve embeddings"| Embedding

    Workflow --> Rerank
    Rerank -->|"re-rank embeddings"| Embedding

    Workflow --> Filter

    Workflow --> Activity
    Activity -->|"fetch metadata"| GitHubAPI

    Workflow --> DepAnalysis
    DepAnalysis -->|"fetch metadata"| GitHubAPI

    Workflow --> Merge
    Merge -->|"fetch metrics"| GitHubAPI

    Workflow --> Rank

    Workflow --> Output
    Output --> UI

    %% CI/CD flows
    Docker --> CI

    %% Click Events
    click UI "https://github.com/zamalali/deepgit/blob/main/app.py"
    click Agent "https://github.com/zamalali/deepgit/blob/main/agent.py"
    click Workflow "https://github.com/zamalali/deepgit/blob/main/langgraph.json"
    click Convert "https://github.com/zamalali/deepgit/blob/main/tools/convert_query.py"
    click Chat "https://github.com/zamalali/deepgit/blob/main/tools/chat.py"
    click Decision "https://github.com/zamalali/deepgit/blob/main/tools/decision_maker.py"
    click MCP "https://github.com/zamalali/deepgit/blob/main/tools/mcp_adapter.py"
    click ParseHW "https://github.com/zamalali/deepgit/blob/main/tools/parse_hardware.py"
    click Dense "https://github.com/zamalali/deepgit/blob/main/tools/dense_retrieval.py"
    click Rerank "https://github.com/zamalali/deepgit/blob/main/tools/cross_encoder_reranking.py"
    click Filter "https://github.com/zamalali/deepgit/blob/main/tools/filtering.py"
    click Activity "https://github.com/zamalali/deepgit/blob/main/tools/activity_analysis.py"
    click DepAnalysis "https://github.com/zamalali/deepgit/blob/main/tools/dependency_analysis.py"
    click Merge "https://github.com/zamalali/deepgit/blob/main/tools/merge_analysis.py"
    click Rank "https://github.com/zamalali/deepgit/blob/main/tools/ranking.py"
    click Output "https://github.com/zamalali/deepgit/blob/main/tools/output_presentation.py"
    click GitHub "https://github.com/zamalali/deepgit/blob/main/tools/github.py"
    click Eval "https://github.com/zamalali/deepgit/blob/main/tools/evaluation.py"
    click Docker "https://github.com/zamalali/deepgit/tree/main/Dockerfile"
    click CI "https://github.com/zamalali/deepgit/blob/main/.github/workflows/docker-image.yml"

    %% Styles
    classDef internal fill:#B3D1FF,stroke:#0366d6,stroke-width:1px;
    classDef external fill:#C1E1C1,stroke:#228B22,stroke-width:1px;
    classDef ci fill:#FFD1A9,stroke:#FF8C00,stroke-width:1px;
    classDef llm fill:#F7D6E0,stroke:#D33682,stroke-width:1px;
```
