How zamalali/deepgit works
Project type and purpose
• DeepGit is a full-stack, agentic Python application whose core is an orchestration layer (agent.py + langgraph.json) that invokes a series of expert “tools” to perform multi-stage semantic search, ranking, and analysis of GitHub repositories.
• The user interacts via a lightweight web UI (app.py), issues a natural-language query, and receives a ranked list of repos enriched with metadata, similarity scores, and hardware-compatibility badges.
File structure
• Top level:
• app.py – the web-server/entrypoint for user queries
• agent.py – the Langgraph-based orchestrator agent
• langgraph.json – workflow definition for orchestrator
• tools/ – a folder of individual modules, one per step in the pipeline
• tests/ – unit tests validating each tool
• docs/ – architecture, Docker, testing guides
• Dockerfile & .github/workflows – containerization and CI/CD
• Each tools/*.py module encapsulates a single responsibility, suggesting a micro-pipeline architecture.
README for additional insights
• The “How It Works” section enumerates seven sequential steps (query expansion, hardware spec detection, dense retrieval, re-ranking, filtering, community/quality analysis, multi-factor ranking, and presentation).
• It calls out specific technologies: LLMs for query expansion and hardware hint extraction, ColBERT v2 for embedding retrieval, MiniLM for cross-encoder re-ranking, GitHub API for metadata, and Python environment checks for hardware filtering.
a. Main components:
• Frontend/UI (app.py + static assets)
• Orchestrator Agent (agent.py + langgraph runtime)
• Tool modules (each tools/*.py)
• External services: LLM provider (OpenAI or similar), embedding/ColBERT service, GitHub API
• Optional data stores: in-memory context or cache (not persisted)
b. Relationships & interactions:
1. User → Frontend (HTTP request)
2. Frontend → Orchestrator Agent (invoke langgraph workflow)
3. Agent → Tool modules in sequence (via Langgraph, defined in langgraph.json)
4. Each tool → External service as needed (LLM calls, embedding API, GitHub API)
5. Results flow back upstream to the agent → formatted by output_presentation → returned to Frontend → User
c. Architectural patterns & principles:
• Agent-based orchestration (Langgraph)
• Single-responsibility modules (tools folder)
• Pipeline pattern (linear, stepwise data transformations)
• Hybrid retrieval (vector + lexical + metadata)
d. Key technologies/frameworks:
• Python 3.11+, Flask/FastAPI or similar for app.py
• Langgraph/langsmith for orchestration
• OpenAI / other LLM API (query expansion, reasoning)
• Hugging Face’s ColBERT v2 & MiniLM for retrieval & re-ranking
• GitHub REST API
• Docker & GitHub Actions for CI/CD
IMPORTANT – Modules making LLM calls
• convert_query.py (LLM-driven query expansion)
• chat.py (generic LLM interactions if used)
• decision.py & decision_maker.py (LLM for reasoning/hardware-aware filtering logic)
• mcp_adapter.py (if it adapts between tools and LLM responses)