Skip to content

Latest commit

 

History

History

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 
 
 

README.md

Building the experimental LLVM compiler for static reflection

The simdjson has experimental support for static reflection, when the compiler supports it. We focus on the latest p2996 paper that is targeting C++26.

Current status

There are 2 versions of compiler that aim to support the C++26 reflection paper (p2996).

  1. Clang-p2996 llvm branch that open-source and available in the Compiler Explorer.
  2. EDG reflection branch that is only publicly available in the Compiler Explorer.

For now, we will resort to #1, since it is open-source.

We are excited to hear that this reflection proposal seems to be on-track for C++26. As per Herb Sutter.

Instructions for building simdjson for static reflection (using docker)

We are assuming that you are running Linux or macOS. We recommend that Windows users rely on WSL.

Follow the following two steps:

  1. Clone the project
git clone https://github.com/simdjson/simdjson.git
cd simdjson

As of March 2025, support for static reflection is available only in the json_builder_init branch, so you should switch:

git checkout -B json_builder_init
  1. Make sure that you have docker installed and running on your system. Most Linux distributions support docker though some (like RedHat) have the equivalent (Podman). Users of Apple systems may want to consider OrbStack. You do not need to familiar with docker, you just need to make sure that you are have it running.

  2. While inside the simdjson repository, run the following bash script:

bash ./p2996/run_docker.sh bash

This will enter a bash shell with access to the repo directory. Note that this will take some time when running it for the first time, since the specific container image has to be built.

This step builds and executes a docker container defined by our Dockerfile which provides the necessary environment.

Importantly, we build the experimental LLVM compiler based on the current state of the p2996 branch of the https://github.com/bloomberg/clang-p2996.git repository. It is possible that the build could fail. Furthermore, you may need to refresh the build from time to time. We provide a script which allows you to delete the docker image you have built (bash ./p2996/remove_docker.sh) so you can start anew.

  1. While inside the docker shell, configure the build system with cmake:
CXX=clang++ cmake -B buildreflect -D SIMDJSON_STATIC_REFLECTION=ON -DSIMDJSON_DEVELOPER_MODE=ON

This only needs to be done once. To build the Rust code, add -D SIMDJSON_USE_RUST=ON. Note that you should have Rust on your system as a prerequisite for this option to be meaningful.

  1. Build the code...
cmake --build buildreflect --target benchmark_serialization_citm_catalog benchmark_serialization_twitter benchmark_parsing_twitter  benchmark_parsing_citm

This is sufficient if you only mean to run the benchmarks (and skip the tests).

  1. Run the tests... (optional)
cmake --build buildreflect
ctest --test-dir buildreflect --output-on-failure
  1. Run the benchmarks.
./buildreflect/benchmark/static_reflect/citm_catalog_benchmark/benchmark_serialization_citm_catalog
./buildreflect/benchmark/static_reflect/citm_catalog_benchmark/benchmark_parsing_citm
./buildreflect/benchmark/static_reflect/twitter_benchmark/benchmark_serialization_twitter
./buildreflect/benchmark/static_reflect/twitter_benchmark/benchmark_parsing_twitter

These benchmarks should print performance counters if run in privileged mode (e.g., run under sudo). If you run the code inside a docker container, under Linux, it should have access to the performance counters if they are enabled on the host. Note that they are usually disabled in a cloud setting unless you have so-called metal access.

You can modify the source code with your favorite editor and run again steps 5 (Build the code) and 6 (Run the tests) and 7 (Run the benchmark). Importantly, you should remain in the docker shell.

You can create a new docker shell at any time by running step 3 (bash script).

Using static reflection in your own projects

You can import simdjson in your own CMake project. You can configure your project like so:

# Fetch simdjson from GitHub
# Replace GIT_TAG by the commit you require.
include(FetchContent)
FetchContent_Declare(
    simdjson
    GIT_REPOSITORY https://github.com/simdjson/simdjson.git
    GIT_TAG 015daad6a95a4f67c08ed5980d24b57be221c38f
    CMAKE_ARGS -DSIMDJSON_STATIC_REFLECTION=ON
)
FetchContent_MakeAvailable(simdjson)

##########
# You may also use CPM.
# https://github.com/cpm-cmake/CPM.cmake
# CPMAddPackage(
#    NAME simdjson
#    GITHUB_REPOSITORY simdjson/simdjson
#    GIT_TAG 015daad6a95a4f67c08ed5980d24b57be221c38f
#    OPTIONS "SIMDJSON_STATIC_REFLECTION ON"
# )
target_link_libraries(webservice PRIVATE simdjson::simdjson)

Replace the GIT_TAG by the appropriate value.

Once C++26 support will be officially available in mainstream compilers, we will simplify these instructions and it will no longer be needed to specify SIMDJSON_STATIC_REFLECTION.