[BUG]onnx convert failed("KeyError: 'max_sequence_len'")

[ahqzy]

hugectr2onnx_bst_test.zip
when do onnx inference(HugeCTR/test/onnx_converter_test/hugectr2onnx_bst_test.py), it failed as below:

# python ./hugectr2onnx_bst_test.py
[HUGECTR2ONNX][INFO]: Converting Data layer to ONNX
[HUGECTR2ONNX][INFO]: Converting DistributedSlotSparseEmbeddingHash layer to ONNX
[HUGECTR2ONNX][INFO]: Converting DistributedSlotSparseEmbeddingHash layer to ONNX
[HUGECTR2ONNX][INFO]: Converting DistributedSlotSparseEmbeddingHash layer to ONNX
[HUGECTR2ONNX][INFO]: Converting DistributedSlotSparseEmbeddingHash layer to ONNX
[HUGECTR2ONNX][INFO]: Converting DistributedSlotSparseEmbeddingHash layer to ONNX
[HUGECTR2ONNX][INFO]: Converting Slice layer to ONNX
Traceback (most recent call last):
  File "/nvidia_merlin/HugeCTR/test/onnx_converter_test/./bst.py", line 91, in <module>
    hugectr2onnx_bst_test(
  File "/nvidia_merlin/HugeCTR/test/onnx_converter_test/./bst.py", line 63, in hugectr2onnx_bst_test
    hugectr2onnx.converter.convert(onnx_model_path, graph_config, dense_model, True, sparse_models)
  File "/usr/local/lib/python3.10/dist-packages/hugectr2onnx-0.0.0-py3.10.egg/hugectr2onnx/converter.py", line 44, in convert
  File "/usr/local/lib/python3.10/dist-packages/hugectr2onnx-0.0.0-py3.10.egg/hugectr2onnx/hugectr_loader.py", line 320, in load_layer
KeyError: 'max_sequence_len'

How to solve this?

(attach is python code)

[KingsleyLiu-NV]

Hi @ahqzy , thanks for reporting this. There is an API mismatch in hugectr2onnx converter for SequenceMask and MultiHeadAttention layers in the latest versions, and we will try to fix it.

You can use nvcr.io/nvidia/merlin-hugectr:23.06 to do onnx conversion correctly for BST model:

docker run --runtime=nvidia --rm -it --net=host --cap-add SYS_NICE  -u root -v $(pwd):/hugectr -w /hugectr nvcr.io/nvidia/merlin/merlin-hugectr:23.06

python3 test/onnx_converter_test/train_scripts/bst_avg_pooling.py

python3 test/onnx_converter_test/hugectr2onnx_bst_test.py