No valid engine configs for slice with NCHW

[swimvtec]

Describe the bug
First setting up a slice operation is very hard as it cannot stand on its own and needs an additional no-op like an addition with 0.f. Second, it is not possible to set up a slice operation with NCHW layout tensors. This always ends in "no valid engine configs".

Expected behavior
Slice operations can be set up as single operation graphs and work with NCHW layout tensors.

System Environment (please complete the following information):

• cudnn_frontend version: 1.12
• cudnn_backend version: 9.11
• GPU arch: RTX A2000
• cuda runtime version: 12.8
• cuda driver version: 570.172.08
• host compiler: clang19
• OS: ubuntu22.04

API logs

be.log
fe.log

To Reproduce

struct Shape
{
  int64_t batch_size;
  int64_t depth;
  int64_t height;
  int64_t width;
};

static void SetTensorShape(
    std::shared_ptr<cudnn_frontend::graph::Tensor_attributes>& tensor,
    Shape const&                                               shape)
{
  tensor->set_dim({shape.batch_size, shape.depth, shape.height, shape.width})
      .set_stride({shape.height * shape.width * shape.depth,
                   shape.height * shape.width, shape.width, 1});
}

static std::shared_ptr<cudnn_frontend::graph::Tensor_attributes>
TensorFromShape(cudnn_frontend::graph::Graph* graph, Shape const& shape,
                std::string const& name)
{
  namespace fe = cudnn_frontend;

  std::shared_ptr<fe::graph::Tensor_attributes> tensor =
      graph->tensor(fe::graph::Tensor_attributes().set_name(name));

  SetTensorShape(tensor, shape);

  return tensor;
}

TEST(Slice, FailsWithNCHW)
{
  ... code to setup cudnn_handle ...

  namespace fe                = cudnn_frontend;
  auto              fwd_graph = std::make_unique<fe::graph::Graph>();
  fe::graph::Graph* graph     = fwd_graph.get();

  graph->set_io_data_type(fe::DataType_t::FLOAT)
      .set_compute_data_type(fe::DataType_t::FLOAT)
      .set_intermediate_data_type(fe::DataType_t::FLOAT);

  Shape input_shape{2, 5, 2, 5};

  auto tensor_x = TensorFromShape(graph, input_shape, "input");

  auto slice_attribs =
      fe::graph::Slice_attributes().set_name("slice").set_slices(
          {{0, 2}, {2, 4}, {0, 2}, {0, 2}});

  auto tensor_slice = graph->slice(tensor_x, slice_attribs);

  Shape output_shape{2, 2, 2, 2};
  SetTensorShape(tensor_slice, output_shape);

  auto tensor_y =
      graph->pointwise(tensor_slice, graph->tensor(0.0f),
                       fe::graph::Pointwise_attributes()
                           .set_mode(fe::PointwiseMode_t::ADD)
                           .set_compute_data_type(fe::DataType_t::FLOAT));

  SetTensorShape(tensor_y, output_shape);
  tensor_y->set_output(true);

  if (auto err = graph->validate(); err.is_bad())
  {
    GTEST_FAIL();
  }
  if (auto err = graph->build_operation_graph(cudnn_handle); err.is_bad())
  {
    GTEST_FAIL();
  }

  if (auto err = graph->create_execution_plans(
          {fe::HeurMode_t::B, fe::HeurMode_t::FALLBACK});
      err.is_bad())
  {
    printf("%s\n", err.get_message().c_str());
    GTEST_FAIL();
  }
}

[vedaanta]

Pasting the error from frontend logs:

[cudnn_frontend] ERROR: No valid engine configs returned from heuristics.
{"engineId":1,"smVersion":860,"knobChoices":{"CUDNN_KNOB_TYPE_SPLIT_K_SLC":-1,"CUDNN_KNOB_TYPE_KERNEL_CFG":-1}}
Warning: CUDNN_STATUS_NOT_SUPPORTED_LAYOUT; Reason: !(tensor_desc->isNSpatialC_ordered_fullyPacked() || tensor_desc->isCNSpatial_ordered_fullyPacked() || tensor_desc->isCSpatialN_ordered_fullyPacked())
Warning: CUDNN_STATUS_NOT_SUPPORTED_LAYOUT; Reason: check_cutlass_iterator_layout_support(port)
Warning: CUDNN_STATUS_NOT_SUPPORTED_LAYOUT; Reason: check_pointwise_support_fort(node)
Warning: CUDNN_STATUS_NOT_SUPPORTED_LAYOUT; Reason: check_node_support_fort(node_ptr)
Warning: CUDNN_STATUS_NOT_SUPPORTED_LAYOUT; Reason: check_for_support()
Warning: CUDNN_STATUS_NOT_SUPPORTED; Reason: (CUDNN_STATUS_NOT_INITIALIZED == status) || (CUDNN_STATUS_NOT_SUPPORTED == ((status) / 1000 * 1000))
Warning: CUDNN_STATUS_NOT_SUPPORTED; Reason: ptr->isSupported()
Warning: CUDNN_STATUS_NOT_SUPPORTED; Reason: finalize_internal()

{"engineId":0,"smVersion":860,"knobChoices":{"CUDNN_KNOB_TYPE_SPLIT_K_SLC":-1,"CUDNN_KNOB_TYPE_KERNEL_CFG":-1}}
Warning: CUDNN_STATUS_NOT_SUPPORTED_ARCH_MISMATCH; Reason: fortBackend.is_cutlass_sm7x() && ((this->getDeviceProp()->deviceVer < 700) || (this->getDeviceProp()->deviceVer > 800))
Warning: CUDNN_STATUS_NOT_SUPPORTED_ARCH_MISMATCH; Reason: init_kernelgen_backend()
Warning: CUDNN_STATUS_NOT_SUPPORTED; Reason: (CUDNN_STATUS_NOT_INITIALIZED == status) || (CUDNN_STATUS_NOT_SUPPORTED == ((status) / 1000 * 1000))
Warning: CUDNN_STATUS_NOT_SUPPORTED; Reason: ptr->isSupported()
Warning: CUDNN_STATUS_NOT_SUPPORTED; Reason: finalize_internal()

{"engineId":2,"smVersion":860,"knobChoices":{"CUDNN_KNOB_TYPE_SPLIT_K_SLC":-1,"CUDNN_KNOB_TYPE_KERNEL_CFG":-1}}
Warning: CUDNN_STATUS_NOT_SUPPORTED_GRAPH_PATTERN; Reason: is_pointwise_fusion && node->opType != CUDNN_BACKEND_OPERATION_RESAMPLE_FWD_DESCRIPTOR
Warning: CUDNN_STATUS_NOT_SUPPORTED_GRAPH_PATTERN; Reason: check_pointwise_support_fort(node)
Warning: CUDNN_STATUS_NOT_SUPPORTED_GRAPH_PATTERN; Reason: check_node_support_fort(node_ptr)
Warning: CUDNN_STATUS_NOT_SUPPORTED_GRAPH_PATTERN; Reason: check_for_support()
Warning: CUDNN_STATUS_NOT_SUPPORTED; Reason: (CUDNN_STATUS_NOT_INITIALIZED == status) || (CUDNN_STATUS_NOT_SUPPORTED == ((status) / 1000 * 1000))
Warning: CUDNN_STATUS_NOT_SUPPORTED; Reason: ptr->isSupported()
Warning: CUDNN_STATUS_NOT_SUPPORTED; Reason: finalize_internal()

[vedaanta]

So seems like with the new graph API, there is no support for the pure pointwise fusions with:
Dim [ 2,2,2,2 ] Str [ 8,4,2,1 ]
This is the NCSpatial layout and the most widely supported one is NSpatialC.

[vedaanta]

First setting up a slice operation is very hard as it cannot stand on its own
Slice operations can be set up as single operation graphs and work with NCHW layout tensors.

A slice is only meant to manipulate strides in current cudnn graph API.
It is expected to act on tensors that either are inputs or outputs, that is, have some device memory associated with them. Slice cannot act on virtual tensors.

So, given the usage cudnn knows of today, slice is not needed to stand on its own in a graph. Can you please describe your graph in more detail on how you want to use the slice operation?