tf::cudaFlow class

#include <src/taskflow/cuda/cuda_flow.hpp>

tf::cudaFlow::cudaFlow()

constructs a standalone cudaFlow

A standalone cudaFlow does not go through any taskflow and can be run by the caller thread using explicit offload methods (e.g., tf::cudaFlow::offload).

void tf::cudaFlow::dump_native_graph(std::ostream& os) const

dumps the native CUDA graph into a DOT format through an output stream

The native CUDA graph may be different from the upper-level cudaFlow graph when flow capture is involved.

cudaTask tf::cudaFlow::noop()

creates a no-operation task

An empty node performs no operation during execution, but can be used for transitive ordering. For example, a phased execution graph with 2 groups of n nodes with a barrier between them can be represented using an empty node and 2*n dependency edges, rather than no empty node and n^2 dependency edges.

template<typename C>

cudaTask tf::cudaFlow::host(C&& callable)

creates a host task that runs a callable on the host

A host task can only execute CPU-specific functions and cannot do any CUDA calls (e.g., cudaMalloc).

template<typename F, typename... ArgsT>

cudaTask tf::cudaFlow::kernel(dim3 g, dim3 b, size_t s, F&& f, ArgsT && ... args)

creates a kernel task

template<typename F, typename... ArgsT>

cudaTask tf::cudaFlow::kernel_on(int d, dim3 g, dim3 b, size_t s, F&& f, ArgsT && ... args)

creates a kernel task on a specific GPU

cudaTask tf::cudaFlow::memset(void* dst, int v, size_t count)

creates a memset task that fills untyped data with a byte value

A memset task fills the first count bytes of device memory area pointed by dst with the byte value v.

cudaTask tf::cudaFlow::memcpy(void* tgt, const void* src, size_t bytes)

creates a memcpy task that copies untyped data in bytes

A memcpy task transfers bytes of data from a source location to a target location. Direction can be arbitrary among CPUs and GPUs.

template<typename T, std::enable_if_t<is_pod_v<T> && (sizeof(T)==1||sizeof(T)==2||sizeof(T)==4), void>* = nullptr>

cudaTask tf::cudaFlow::zero(T* dst, size_t count)

creates a memset task that sets a typed memory block to zero

A zero task zeroes the first count elements of type T in a device memory area pointed by dst.

template<typename T, std::enable_if_t<is_pod_v<T> && (sizeof(T)==1||sizeof(T)==2||sizeof(T)==4), void>* = nullptr>

cudaTask tf::cudaFlow::fill(T* dst, T value, size_t count)

creates a memset task that fills a typed memory block with a value

A fill task fills the first count elements of type T with value in a device memory area pointed by dst. The value to fill is interpreted in type T rather than byte.

template<typename T, std::enable_if_t<!std::is_same_v<T, void>, void>* = nullptr>

cudaTask tf::cudaFlow::copy(T* tgt, const T* src, size_t num)

creates a memcopy task that copies typed data

A copy task transfers num*sizeof(T) bytes of data from a source location to a target location. Direction can be arbitrary among CPUs and GPUs.

template<typename P>

void tf::cudaFlow::offload_until(P&& predicate)

offloads the cudaFlow onto a GPU and repeatedly runs it until the predicate becomes true

Immediately offloads the present cudaFlow onto a GPU and repeatedly runs it until the predicate returns true.

An offloaded cudaFlow forces the underlying graph to be instantiated. After the instantiation, you should not modify the graph topology but update node parameters.

By default, if users do not offload the cudaFlow, the executor will offload it once.

void tf::cudaFlow::offload_n(size_t N)

offloads the cudaFlow and executes it by the given times

template<typename C>

void tf::cudaFlow::update_host(cudaTask task, C&& callable)

updates parameters of a host task created from tf::cudaFlow::host

The method updates the parameters of a host callable associated with the given task.

template<typename... ArgsT>

void tf::cudaFlow::update_kernel(cudaTask task, dim3 g, dim3 b, size_t shm, ArgsT && ... args)

updates parameters of a kernel task created from tf::cudaFlow::kernel

The method updates the parameters of a kernel associated with the given task. We do not allow you to change the kernel function.

template<typename T, std::enable_if_t<!std::is_same_v<T, void>, void>* = nullptr>

void tf::cudaFlow::update_copy(cudaTask task, T* tgt, const T* src, size_t num)

updates parameters of a memcpy task to form a copy task

The method updates the parameters of a copy task. The source/destination memory may have different address values but must be allocated from the same contexts as the original source/destination memory.

void tf::cudaFlow::update_memcpy(cudaTask task, void* tgt, const void* src, size_t bytes)

updates parameters of a memcpy task

The method updates the parameters of a memcpy task. The source/destination memory may have different address values but must be allocated from the same contexts as the original source/destination memory.

void tf::cudaFlow::update_memset(cudaTask task, void* dst, int ch, size_t count)

updates parameters of a memset task

The method updates the parameters of a memset task. The source/destination memory may have different address values but must be allocated from the same contexts as the original source/destination memory.

template<typename T, std::enable_if_t<is_pod_v<T> && (sizeof(T)==1||sizeof(T)==2||sizeof(T)==4), void>* = nullptr>

void tf::cudaFlow::update_fill(cudaTask task, T* dst, T value, size_t count)

updates parameters of a memset task to form a fill task

The method updates the parameters of a copy task. The given arguments and type must comply with the rules of tf::cudaFlow::fill. The source/destination memory may have different address values but must be allocated from the same contexts as the original source/destination memory.

template<typename T, std::enable_if_t<is_pod_v<T> && (sizeof(T)==1||sizeof(T)==2||sizeof(T)==4), void>* = nullptr>

void tf::cudaFlow::update_zero(cudaTask task, T* dst, size_t count)

updates parameters of a memset task to form a zero task

The method updates the parameters of a copy task. The given arguments and type must comply with the rules of tf::cudaFlow::zero. The source/destination memory may have different address values but must be allocated from the same contexts as the original source/destination memory.

template<typename C>

cudaTask tf::cudaFlow::single_task(C&& callable)

runs a callable with only a single kernel thread

template<typename I, typename C>

cudaTask tf::cudaFlow::for_each(I first, I last, C&& callable)

applies a callable to each dereferenced element of the data array

This method is equivalent to the parallel execution of the following loop on a GPU:

for(auto itr = first; itr != last; itr++) {
  callable(*itr);
}

template<typename I, typename C>

cudaTask tf::cudaFlow::for_each_index(I first, I last, I step, C&& callable)

applies a callable to each index in the range with the step size

This method is equivalent to the parallel execution of the following loop on a GPU:

// step is positive [first, last)
for(auto i=first; i<last; i+=step) {
  callable(i);
}

// step is negative [first, last)
for(auto i=first; i>last; i+=step) {
  callable(i);
}

template<typename I, typename C, typename... S>

cudaTask tf::cudaFlow::transform(I first, I last, C&& callable, S... srcs)

applies a callable to a source range and stores the result in a target range

This method is equivalent to the parallel execution of the following loop on a GPU:

while (first != last) {
  *first++ = callable(*src1++, *src2++, *src3++, ...);
}

template<typename I, typename T, typename C>

cudaTask tf::cudaFlow::reduce(I first, I last, T* result, C&& op)

performs parallel reduction over a range of items

This method is equivalent to the parallel execution of the following loop on a GPU:

while (first != last) {
  *result = op(*result, *first++);
}

template<typename I, typename T, typename C>

cudaTask tf::cudaFlow::uninitialized_reduce(I first, I last, T* result, C&& op)

similar to tf::cudaFlow::reduce but does not assume any initial value to reduce

This method is equivalent to the parallel execution of the following loop on a GPU:

*result = *first++;  // no initial values partitipcate in the loop
while (first != last) {
  *result = op(*result, *first++);
}

template<typename C>

cudaTask tf::cudaFlow::capture(C&& callable)

constructs a subflow graph through tf::cudaFlowCapturer

A captured subflow forms a sub-graph to the cudaFlow and can be used to capture custom (or third-party) kernels that cannot be directly constructed from the cudaFlow.

Example usage:

taskflow.emplace([&](tf::cudaFlow& cf){
  
  tf::cudaTask my_kernel = cf.kernel(my_arguments);
  
  // create a flow capturer to capture custom kernels
  tf::cudaTask my_subflow = cf.capture([&](tf::cudaFlowCapturer& capturer){
    capturer.on([&](cudaStream_t stream){
      invoke_custom_kernel_with_stream(stream, custom_arguments);
    }); 
  });

  my_kernel.precede(my_subflow);
});