LLOR (LLVM OpenMP Repairer) is our tool that can repair data race errors in parallel programs written in C/C++ and Fortran using the OpenMP API. It uses LLOV [1] as the verification oracle.
-
Install the Git client to download the source code
-
Install build-essential, which is a meta-package that includes the tools that are essential to build C/C++ programs. Install cmake and ninja for building the tools from source. Install the .NET SDK to compile the C# code
sudo apt update
sudo apt install git
sudo apt install build-essential
sudo apt install cmake
sudo apt install ninja-build
sudo apt install dotnet-sdk-6.0
sudo apt install libopenmpi-dev- Download the source code of LLOV from the cs17resch01003/llov repository and compile it
export BUILD_ROOT=/path/to/build
# download the source code
cd ${BUILD_ROOT}
git clone https://github.com/cs17resch01003/llov.git
# generate the build files
mkdir ${BUILD_ROOT}/llov/build
cd ${BUILD_ROOT}/llov/build
cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD=X86 -DLLVM_ENABLE_PROJECTS="clang;openmp;polly" ../llvm
# start the build (recommended value of N is the number of cores)
cmake --build . -jN
sudo cmake --build . --target install- Download the source code of classic-flang from the classic-flang repository and flang from the flang repository
# download the source code
cd ${BUILD_ROOT}
git clone -b release_12x https://github.com/flang-compiler/classic-flang-llvm-project.git
git clone -b legacy https://github.com/flang-compiler/flang.git- Compile the source code of classic-flang
# generate the build files
mkdir ${BUILD_ROOT}/classic-flang-llvm-project/build
cd ${BUILD_ROOT}/classic-flang-llvm-project/build
cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=/usr/bin/gcc -DCMAKE_CXX_COMPILER=/usr/bin/g++ -DCMAKE_INSTALL_PREFIX=${BUILD_ROOT}/flang/install -DLLVM_TARGETS_TO_BUILD=X86 -DLLVM_ENABLE_CLASSIC_FLANG=ON -DLLVM_ENABLE_PROJECTS="clang;openmp" ../llvm
# start the build (recommended value of N is the number of cores)
cmake --build . -jN
cmake --build . --target install- Compile the source code of the Fortran runtime math library from the flang repository
# generate the build files
mkdir ${BUILD_ROOT}/flang/runtime/libpgmath/build
cd ${BUILD_ROOT}/flang/runtime/libpgmath/build
cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=${BUILD_ROOT}/flang/install/bin/clang -DCMAKE_CXX_COMPILER=${BUILD_ROOT}/flang/install/bin/clang++ -DCMAKE_INSTALL_PREFIX=${BUILD_ROOT}/flang/install ..
# start the build (recommended value of N is the number of cores)
cmake --build . -jN
cmake --build . --target install- Compile the source code of flang
# generate the build files
mkdir ${BUILD_ROOT}/flang/build
cd ${BUILD_ROOT}/flang/build
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=${BUILD_ROOT}/flang/install/bin/clang -DCMAKE_CXX_COMPILER=${BUILD_ROOT}/flang/install/bin/clang++ -DCMAKE_Fortran_COMPILER=${BUILD_ROOT}/flang/install/bin/flang -DCMAKE_Fortran_COMPILER_ID=Flang -DLLVM_CONFIG=${BUILD_ROOT}/flang/install/bin/llvm-config -DCMAKE_INSTALL_PREFIX=${BUILD_ROOT}/flang/install -DLLVM_TARGETS_TO_BUILD=X86 -DFLANG_LLVM_EXTENSIONS=ON ..
# start the build (recommended value of N is the number of cores)
cmake --build . -jN
cmake --build . --target install- Add the install directory of flang to the PATH environment variable. Set the other environment variables for building the large benchmarks. To make this permanent, include these lines in ~/.bashrc
export PATH=$PATH:${BUILD_ROOT}/flang/install/bin
export OPENMP_INCLUDE_PATH=${BUILD_ROOT}/llov/build/projects/openmp/runtime/src
export OMPI_CC=${BUILD_ROOT}/llov/build/bin/clang
export OMPI_CXX=${BUILD_ROOT}/llov/build/bin/clang++- Download the source code of LLOR from the llor repository and compile it
# download the source code
cd ${BUILD_ROOT}
git clone https://github.com/cs17resch01003/llor.git
# compile the code
cd ${BUILD_ROOT}/llor
dotnet build --configuration Release ./src/Repair/Repair.csproj
dotnet build --configuration Release ./src/Transform/Transform.csproj
dotnet build --configuration Release ./src/TestRunner/TestRunner.csproj
# create symbolic links
sudo ln -s ${BUILD_ROOT}/llor/src/Repair/bin/Release/net6.0/Repair /usr/bin/llor
sudo ln -s ${BUILD_ROOT}/llor/src/Transform/bin/Release/net6.0/Transform /usr/bin/llor_transform
sudo ln -s ${BUILD_ROOT}/llor/src/TestRunner/bin/Release/net6.0/TestRunner /usr/bin/llor_testrunner
sudo ln -s ${BUILD_ROOT}/llor/scripts/llov.sh /usr/bin/llov
sudo ln -s ${BUILD_ROOT}/llor/scripts/llov_transform.sh /usr/bin/llov_transform
sudo ln -s ${BUILD_ROOT}/llor/scripts/llov_compile.sh /usr/bin/llov_compile
# test the build
llor_testrunner --folder ${BUILD_ROOT}/llor/testsuiteLLOR has a simple execution syntax which uses the symbolic links created in the previous step. Below is a C program that has a data race. This program is located at ${BUILD_ROOT}/llor/testsuite/baseline/B01_simple_race.c
#include "omp.h"
#define NUM_THREADS 10
int main()
{
int data[NUM_THREADS+1];
omp_set_num_threads(NUM_THREADS);
for (int i = 0; i < NUM_THREADS+1; i++)
data[i] = i;
#pragma omp parallel
{
int id = omp_get_thread_num();
int temp = data[id+1];
data[id] = temp;
}
}To repair this program, run the following command.
cd ${BUILD_ROOT}/llor
llor --file ./testsuite/baseline/B01_simple_race.cThis command repairs the program and prints the number of changes that are required to correct this program. The changes that are needed are written to a summary file at the same location where the program is.
cat ./testsuite/baseline/B01_simple_race.summary
# prints the line number where the barrier needs to be inserted to fix the data raceThe changes in the summary file specify the file and line where a barrier needs to be added or removed. The fixed program is also available in its LLVM Intermediate Representation (IR) at the same location with the extension .inst.ll
The repaired program will look like below.
#include "omp.h"
#define NUM_THREADS 10
int main()
{
int data[NUM_THREADS+1];
omp_set_num_threads(NUM_THREADS);
for (int i = 0; i < NUM_THREADS+1; i++)
data[i] = i;
#pragma omp parallel
{
int id = omp_get_thread_num();
int temp = data[id+1];
#pragma omp barrier
data[id] = temp;
}
}[1] Utpal Bora, Santanu Das, Pankaj Kukreja, Saurabh Joshi, Ramakrishna Upadrasta, and Sanjay Rajopadhye. LLOV: A Fast Static Data-Race Checker for OpenMP Programs. In ACM Transactions on Architecture and Code Optimization 17(4): 35:1-35:26.