How qcri/llmxcpg works
Project Type and Purpose
• This is a research-oriented vulnerability‐detection pipeline that ties together Code Property Graph (CPG) analysis (via Joern) with two fine‐tuned LLMs.
• It is neither a traditional web app nor a pure library; it is a full end‐to‐end data‐processing and ML inference/training framework.
File‐Structure Overview
• baselines/: reference implementations for comparative evaluation (GNN, VulBERTa, VulSim, dx2021 code2vec).
• data/: JSONL files and test/train splits for vulnerability detection datasets.
• inference/: scripts to run LLMxCPG-Q (query generator) and LLMxCPG-D (detector) on unseen code.
• prompts/: system‐prompt templates for the two LLM phases.
• queries/:
• Components (c_parser, joern_manager, slice builder, enhancer)
• Dockerfile / docker‐compose for spinning up a Joern/CPG‐query service
• Orchestration scripts that generate and dispatch CPGQL queries
• training/: fine‐tuning code for LLMxCPG-Q and LLMxCPG-D (LoRA + Unsloth)
• pipeline.sh: end‐to‐end orchestration (CPG generation → query gen → slice extraction → detection)
• requirements.txt, example.env: environment and dependency definitions
Architecture Synopsis from README
• Two‐phase approach:
a. Slice Construction: LLMxCPG-Q generates targeted CPGQL queries
b. Vulnerability Detection: LLMxCPG-D classifies the extracted slice
• External dependency: Joern for CPG generation and query execution
• LLMs hosted on Hugging Face; fine‐tuned via Unsloth + LoRA
Key Components
a. Input Layer
• Raw code repositories (on local disk or Git URLs)
• Dataset files under data/ for training/inference
b. CPG Generation Service
• Joern (Docker container)
• Generates in-memory/on-disk code property graphs
c. Query Generation Service (LLMxCPG-Q)
• Python script (training: training/llmxcpg_query_finetune.py; inference: inference/detect_inference.py or analogous)
• Reads CPG metadata and outputs CPGQL queries
d. Query Executor & Slice Extractor
• queries/ components (joern_manager, c_parser, slice builder)
• Executes CPGQL, returns minimal code slice
e. Vulnerability Detection Service (LLMxCPG-D)
• Python script (training: training/llmxcpg_detect_finetune.py; inference: inference/detect_inference.py)
• Classifies extracted slice as vulnerable or safe
f. Orchestration Layer
• pipeline.sh or docker-compose.yml
• Manages service startup, data flow, and parameter injection (via example.env)
g. Outputs & Baseline Evaluation
• Classification results (JSONL or text)
• Side‐by‐side baseline results under baselines/ for performance comparison
Relationships & Data Flow
• Raw Code → Joern CPG Service
→ CPG Graph Data
→ LLMxCPG-Q (reads CPG metadata, emits CPGQL)
→ Query Executor (executes CPGQL against Joern, returns slice)
→ LLMxCPG-D (classifies slice)
→ Results (aggregated vulnerability labels)
Architectural Patterns & Principles
• Modular Microservices: each major function runs as a separate Docker/Python service
• Pipelines as Code: pipeline.sh, docker-compose.yml encode end-to-end flow
• Separation of Concerns:
• CPG generation vs. query logic vs. detection logic vs. baseline comparators
• Training vs. inference code paths
• Reproducibility: environment pinned via requirements.txt and example.env
Technologies & Frameworks
• Python 3.8+, Docker & Docker Compose
• Joern (v4.x) for CPG generation & query
• Hugging Face Transformers + Unsloth for LoRA fine‐tuning
• JSONL, shell scripts for orchestration