Wmma support for gemm_bias_add_reduce

[EnricoDeg]

Proposed changes

Summary:

• Change EpilogueReduceCShuffle to support bias + add operations before reduction (multiple Ds)
• Add wmma device struct for gemm_bias_add_reduce
• Add instances (xdl parity)
• Add tests

Checklist

Please put an x into the boxes that apply. You can also fill these out after creating the PR. If you're not sure, please don't hesitate to ask.

• I have added tests relevant to the introduced functionality, and the unit tests are passing locally
• I have added the test to REGRESSION_TESTS list defined at the top of CMakeLists.txt in tests/CMakeLists.txt, IF the test takes more than 30 seconds to run.
• I have added inline documentation which enables the maintainers with understanding the motivation
• I have removed the stale documentation which is no longer relevant after this pull request
• (If this change is user-facing) I have added release notes which provide the end users with a brief summary of the improvement from this pull request
• I have run clang-format on all changed files
• Any dependent changes have been merged

Discussion

If this is a relatively large or complex change, feel free to start a discussion by explaining why you chose the solution you did and what alternatives you considered

[krithalith]

Can we not just use the instance factory get_device_gemm_add_add_mean_squaremean_instances() here instead of manually using the add_device_xxx_instances() functions?

[krithalith]

In the profiler impl we have:

std::size_t num_byte = sizeof(ADataType) * M * K + sizeof(BDataType) * K * N +
                       sizeof(CDataType) * M * N + sizeof(BiasDataType) * M * N +
                       sizeof(D0DataType) * M * N + sizeof(ReduceDataType) * M +
                       sizeof(ReduceDataType) * M;

But I thought the Bias was a simple 1D vector of size N?

[wj-laskowski]

nit: typo

[wj-laskowski]

nit: this is int value initialization