Skip to content

R2Vul

1. Tool Information

2. Authors and Contact

  • Authors: Martin Weyssow, Chengran Yang, Junkai Chen, Ratnadira Widyasari, Ting Zhang, Huihui Huang, Huu Hung Nguyen, Yan Naing Tun, Tan Bui, Yikun Li, Ang Han Wei, Frank Liauw, Eng Lieh Ouh, Lwin Khin Shar, David Lo

3. Overview

This tool automatically detects vulnerabilities in source code, using RL and structured reasoning. It takes input as a function and returns a binary prediction if the input is vulnerable or not.

Further details can be found in R2Vul paper

4. Installation

First, clone the R2Vul GitHub Repo then navigate into the repo.

We provide a Dockerfile inside the repo to setup a docker image to run our code. The image is based on nvidia/cuda:12.4.0 for Ubuntu. Depending on your machine, you can look for an appropriate base image that runs cuda 12.4.0 on dockerhub.

a. Build the docker image 

docker build -t r2vul-image .
This builds the docker image and ensures Python 3 is properly installed.

b. Create the docker container

Next, you can instantiate a new docker container based on the image we just created. 

docker run -it --name r2vul -d -v R2Vul:/r2vul --gpus all r2vul-image
Note that if you plan to run inference with a local TGI model, add --network=r2vul-inference-net to have both containers on the same network.

You can then start the container and attach to it: 

docker start r2vul
docker exec r2vul -it bash
cd r2vul # go to the source code directory

c. Setup the virtual environment

Create a new virtual environment and install the required Python libraries. The requirements.txt file is located under the repo. 

python -m venv venv
pip install -r requirements.txt
source venv/bin/activate # activate the venv
Note that if you do not wish to use Docker, you can simply rely on the Python venv, but we cannot guarantee that everything will run smoothly.

5. Usage

We provide bash scripts to run specific experiments. You simply have to run them and change some variables depending on what you want to run.

a. Main

Inference
  • MSIVD: /scripts/main/run_inference_msivd.sh
  • VulLLM: /scripts/main/run_inference_vulllm.sh
  • CLS: /scripts/main/run_inference_cls.sh
  • Commercial LLMs: /scripts/main/run_inference_oai.sh
  • CoT: /scripts/main/run_inference_tgi.sh (run tgi_serve.sh first)
  • SFT and R2Vul: /scripts/main/run_inference_tgi.sh (run tgi_serve_r2vul.sh first)
Fine-Tuning
  • CodeBERT (CLS): /scripts/main/run_training_cls_codebert.sh
  • Qwen2.5-Coder-Instruct (CLS): /scripts/main/run_training_cls_qwen.sh
  • SFT: /scripts/main/run_training_sft.sh
  • R2Vul: /scripts/main/run_training_r2vul.sh

b. External Test Set - CLS: /scripts/external_test_set/run_inference_cls.sh - SFT: /scripts/external_test_set/run_inference_sft.sh (run tgi_serve_r2vul.sh first) - R2Vul: /scripts/external_test_set/run_inference_r2vul.sh (run tgi_serve_r2vul.sh first)

c. Data Ablation

Fine-Tuning
  • SFT: /scripts/data_ablation/run_training_sft.sh
  • R2Vul: /scripts/data_ablation/run_training_r2vul.sh

For inference, run tgi_serve_r2vul.sh with a specific checkpoint, then run inference using run/main/run_inference_tgi.sh.

d. Class Imbalance - SFT: /scripts/class_imbalance/run_inference_sft.sh - R2Vul: /scripts/class_imbalance/run_inference_r2vul.sh

e. Model Calibration - R2Vul: /scripts/calibration/run_inference_r2vul.sh

Further details of setting up environments, datasets and running the tool can be found in the the GitHub repo's README [https://github.com/martin-wey/titan-r2vul/blob/main/README.md]

6. Input and Output Format

  • Input format: An "arrorw" file with 3 columns: "function", "vulnerable" (optional) and "lang"
  • Output format: A jsonl file with fields: "function", "vulnerable" and "explanation" ```