[DO NOT MERGE] Experiment with using cuda::memory_resource interface#840
[DO NOT MERGE] Experiment with using cuda::memory_resource interface#840harrism wants to merge 16 commits intorapidsai:branch-21.12from
Conversation
… template classes.
…d_memory_resource is a different type from cuda_memory_resource, etc.
|
With cudf updated to For example to use your custom fork of libcudacxx you would write ( for both rmm and cudf ): function(find_and_configure_libcudacxx)
include(${rapids-cmake-dir}/cpm/libcudacxx.cmake)
include(${rapids-cmake-dir}/cpm/package_override.cmake)
file(WRITE ${CMAKE_BINARY_DIR}/libcudacxx.json [=[
{
"packages" : {
"libcudacxx" : {
"version" : "1.5.0",
"git_url" : "https://github.com/mzient/libcudacxx.git",
"git_tag" : "memres_view"
}
}
}]=])
rapids_cpm_package_override(${CMAKE_BINARY_DIR}/libcudacxx.json)
rapids_cpm_libcudacxx(BUILD_EXPORT_SET rmm-exports
INSTALL_EXPORT_SET rmm-exports)
set(LIBCUDACXX_INCLUDE_DIR "${libcudacxx_SOURCE_DIR}/include" PARENT_SCOPE)
endfunction()
find_and_configure_libcudacxx()
|
|
Thanks @robertmaynard ! #883 is the more relevant PR for this. Mentioning it here so this shows up on that PR. |
|
This PR has been labeled |
|
This PR has been labeled |
This is an experiment, opening as a PR for discussion.
Currently only
stream_ordered_memory_resource<cuda::memory_kind::device>is used as a basis. So far I've converted onlycuda_memory_resourceandmanaged_memory_resource. Obviously this is not ideal because these should really have different memory kinds. And really we should be able to use a single class with different kinds for the implementation of these two. But it's a start, and it works with our current test infrastructure.Adding other memory kinds will result in multiple independent base classes which will require refactoring how we write MR tests so that they are not value parameterized on the MR. This is a pain because value parameterization lets us create simple factory functions for instantiating the MR, which is valuable since some MRs are multi-layered (e.g. hybrid_MR) so a simple call to the ctor is not sufficient.
Suggested roadmap to integrating
cuda::memory_resource(from @jrhemstad )device_memory_resourceclassdevice_memory_resourceinherit fromcuda::stream_ordered_memory_resource<Kind>, where in instance ofcuda_memory_resourceKindisdeviceand forpool_memory_resourceKindis a template parameter dependent on the upstreamdevice_memory_resource*tocuda::resource_view<Props...>whereProps...is a variadic type packdevice_buffertobuffer<Kind>libcudf(and other libs) interfaces that takedevice_memory_resource*toresource_view<device_accessible>Comments (from @harrism):
This is obviously incredibly invasive to RAPIDS and other dependent libraries. We need to plan how to manage that disruption and how to break up the above into separate PRs. The final step should be doable independently, however we would need to keep
device_memory_resourceuntil that is complete.