How fsoft-ai4code/repohyper works
Main Components
a. Data Preparation/Acquisition:
• Cloning and downloading repositories (using the Repobench dataset).
• Call graph generation from repositories (see scripts/data/download_repos.py and generate_call_graph scripts).
b. Graph Construction & Embedding:
• Construction of Code Property Graphs (repository-level semantic graphs) from the source code.
• Generation of embeddings for nodes and creation of adjacency matrices (scripts/data/repo_to_embeddings.py, src/repo_graph/repo_to_graph.py).
c. Search and Expand Stage:
• Implementation of the KNN search and expansion algorithm on the semantic graph to retrieve and refine context (see scripts/data/matching_repobench_graphs.py and parts of src/repo_graph/search_policy).
d. GNN Link Predictor Training:
• A dedicated training module for the Graph Neural Network (GNN) using various architectures (GraphSage, GCN variants) as seen in src/models/ and training scripts (train_gnn.py, train_gnn_he.py, scripts/trainer/train.py).
• Utilization of configuration via ds_config.json and training orchestration through Slurm scripts in src/train.
e. Evaluation & Integration with LLM:
• Evaluation pipeline that ranks or feeds refined contexts into a Code LLM for next line prediction (scripts/evaluate_llm.py).
• This component verifies the performance of both the graph-based retrieval and the Code LLM integration.
Relationships and Data Flow
• Flow to Graph Construction where raw source code is parsed to build Code Property Graphs.
• Feed the refined graphs into the GNN training pipeline for the link prediction task.
• Finally, channel the output from the GNN (retrieved and reranked contexts) into the Code LLM model for code completion.
Architectural Patterns & Technologies
• Modular Design: Separate directories for data processing (src/data), evaluation (src/eval), models (src/models), and repository graph construction (src/repo_graph) indicate a modular structure.
• Two-Stage Pipeline: Distinct separation between the retrieval/refinement stage (graph-based techniques) and the completion stage (LLM-based code prediction).
• Distributed/Parallel Processing: Utilization of multi-process calls (e.g., generating call graphs using 60 processes) and deepspeed for GNN model training.
• Use of Graph Neural Networks (GNNs): Different variants (GraphSage, GCN, etc.) are implemented, showing the importance of graph-based learning.
• Integration with modern Code Language Models: The system is designed to work with models such as GPT-3.5.