# eshe0922/reposvul architecture

> Architecture diagram of the GitHub repository eshe0922/reposvul, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/eshe0922/reposvul
- Repository: https://github.com/eshe0922/reposvul

## Overview

Based on our analysis of the provided file tree and README, this project—ReposVul—is a data collection framework for repository-level vulnerability detection and analysis. It integrates multiple modules that (a) crawl raw data from different sources, (b) untangle vulnerability-fixing patches using both LLMs and static analysis tools, (c) extract multi-granularity dependency information from codebases (e.g., inter-procedural call relationships), and (d) apply trace-based filtering to ensure that only up-to-date patches are included in the final dataset.

##### Main Components
- Raw Data Crawling Module:
- This part is responsible for crawling vulnerability entries and patches.
- Within the “Raw_Data_Crawling” directory, note the presence of subdirectories for different platforms (android, chrome, github). Each has its own run scripts or merge logic.

- Vulnerability Untangling Module:
- Found under the “Vulnerability_Untangling_Module” folder.
- It has two subcomponents:
▪ LLM-based analysis (under the “llm” subdirectory, including a CSV file and evaluation script).
▪ Static Analysis (under the “static” subdirectory with various scripts for tools like cppcheck, flawfinder, rats, and semgrep).

- Multi-granularity Dependency Extraction Module:
- Located in “Multi-granularity_Dependency_Extraction_Module”.
- The Python files (with naming patterns “process_*” and “parse_getout_nearfunc_*”) indicate that this module handles dependency parsing/extraction for different programming languages (C, C++, Python, and their “plus” variants).

- Trace-Based Filtering Module:
- Found in “Trace-based_Filtering_Module”.
- Contains subdirectories (android, chrome, github) with a “window.py” script in each, suggesting that commit trace information (e.g., file path trace or commit history) is used to filter outdated patches.

##### Relationships and Interactions
- Data Flow:
- The process likely begins with the Raw Data Crawling Module collecting raw patches and vulnerability data.
- This raw data then feeds into the Vulnerability Untangling Module where both static analysis and LLM decisions help filter and label vulnerability-related changes.
- The Multi-granularity Dependency Extraction Module then analyzes the patched code to extract relationships at various granularity levels (repository, file, function, and line levels).
- Finally, the Trace-based Filtering Module examines commit history and file path traces to weed out outdated patches.

- Interactions:
- There are clearly defined “internal” data exchanges between submodules (e.g., vulnerability metadata produced by the untangling module is used by the dependency extraction module).

##### Architectural Patterns and Key Technologies
- The project adopts a pipeline architecture where each module represents a processing stage.
- The design is modular and language-agnostic since modules (especially in dependency extraction) support multiple programming languages.
- Integration of external tools (static analyzers like cppcheck, flawfinder, etc.) and Large Language Models for contextual decisions is a central architectural design choice.

- Multi-granularity Dependency Extraction Module (indicate distinct processes for each language and mark “process” and “parse” functions)
- Trace-Based Filtering Module (each platform’s filtering process, e.g., android window, chrome window, github window)
- Rectangles for processing modules,
- Diamonds for decision or filtering stages,

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Raw Data Crawling Module
    subgraph "Raw Data Crawling Module"
        AD["Android Source"]:::craw
        CC["Chrome Source"]:::craw
        GH["Github Source"]:::craw
    end

    %% Aggregated Raw Data
    RD["Collected Raw Data"]:::craw

    %% Vulnerability Untangling Module
    subgraph "Vulnerability Untangling Module"
        subgraph "LLM-based Analysis"
            LLM["LLM Analysis"]:::untg
        end
        subgraph "Static Analysis"
            STAT["Static Analysis"]:::untg
        end
        VM["Vulnerability Metadata"]:::untg
    end

    %% Multi-granularity Dependency Extraction Module
    subgraph "Multi-granularity Dependency Extraction Module"
        DE["Dependency Extraction Engine"]:::extr
    end

    %% Trace-Based Filtering Module
    subgraph "Trace-Based Filtering Module"
        T_A["Android Trace Filter"]:::filt
        T_C["Chrome Trace Filter"]:::filt
        T_G["Github Trace Filter"]:::filt
    end

    %% Data Flow Connections
    AD -->|"raw_data"| RD
    CC -->|"raw_data"| RD
    GH -->|"raw_data"| RD

    RD -->|"sends_raw"| LLM
    RD -->|"sends_raw"| STAT

    LLM -->|"produces_meta"| VM
    STAT -->|"produces_meta"| VM

    VM -->|"feeds_data"| DE

    DE --->|"filters_to"| T_A
    DE --->|"filters_to"| T_C
    DE --->|"filters_to"| T_G

    %% Click Events
    click AD "https://github.com/eshe0922/reposvul/tree/main/Raw_Data_Crawling/android"
    click CC "https://github.com/eshe0922/reposvul/tree/main/Raw_Data_Crawling/chrome"
    click GH "https://github.com/eshe0922/reposvul/tree/main/Raw_Data_Crawling/github"
    click LLM "https://github.com/eshe0922/reposvul/tree/main/Vulnerability_Untangling_Module/llm"
    click STAT "https://github.com/eshe0922/reposvul/tree/main/Vulnerability_Untangling_Module/static"
    click DE "https://github.com/eshe0922/reposvul/tree/main/Multi-granularity_Dependency_Extraction_Module"
    click T_A "https://github.com/eshe0922/reposvul/blob/main/Trace-based_Filtering_Module/android/window.py"
    click T_C "https://github.com/eshe0922/reposvul/blob/main/Trace-based_Filtering_Module/chrome/window.py"
    click T_G "https://github.com/eshe0922/reposvul/blob/main/Trace-based_Filtering_Module/github/window.py"

    %% Styles
    classDef craw fill:#f9c,stroke:#333,stroke-width:2px;
    classDef untg fill:#bbf,stroke:#333,stroke-width:2px;
    classDef extr fill:#afa,stroke:#333,stroke-width:2px;
    classDef filt fill:#ffa,stroke:#333,stroke-width:2px;
```
