C++17 has introduced a new tool for memory management: std::pmr::memory_resource. This talk discusses it, but from a new perspective: primarily heterogeneous systems with non-uniform accesses to memory.
CUDA Thrust is a C++ parallel programming library built around the concepts and interfaces of the C++ standard library. When faced with the need for a composable interface for memory allocation in Thrust, we've reached to std::pmr - but std::pmr is inherently based around raw pointers, embedded deeply into signatures of virtual functions; this means it's not a great fit for a library that enables the use of GPUs for accelerated computation, which brings a need to handle different memory spaces in a type-safe way. Additionally, because accesses to memory are not uniform, the std::pmr model of pool resources doesn't quite work for CUDA and similar ecosystems. Thus came thrust::mr, which is a slight variation on std::pmr.
There have been many talks about std::pmr in general; this is not quite one of them, but rather a recap of how we ended up with the design that we've shipped as a part of Thrust, explaining the challenges we've encountered and the solutions we've settled on. I'll explain where we are similar to std::pmr, where we differ, and what are my musings on how the standard should evolve the idea of memory resources to properly support heterogeneous systems - but also why I think that some of the ideas we've ended up with are also applicable to homogeneous, uniform systems.
