I tried to reproduce the code of this paper and found that the original description presents an unfeasible process.

[BUG423]

I tried to reproduce the code of this paper and found that the original description presents an unfeasible process. The Encoder adopts convolutional downsampling, which leads to changes in the channel and temporal dimensions, especially the reduction of the temporal dimension. However, none of the operations in the Decoder part can modify the temporal dimension. This makes it impossible for the multi-head attention part of the Decoder to match the dimensions when performing multi-head attention calculations using the output of the Encoder and the output of the masked attention.
I also attempted to add padding during the downsampling in the Encoder to ensure that the size remains unchanged, but this already goes against the concept of downsampling. At the same time, when the convolution kernel and stride do not match, it will still cause changes in the temporal dimension.
I can only stop here without going any further.

[Aljen-511]

Hello, I've met the similar problem when tried to reproduce this paper recently. Have you figured it out?

[BUG423]

Hello, I have received your email. Thank you very much! Best regards,

[Aljen-511]

Hi, I just began to reproduce this work few days ago.
If we adopt a 1d convolution layer with kernel size of 1 (which referred as 1X1 convolution in the original paper) in the encoder, the length of sequence may not reduce with the assumption that the tensors fed to encoder are in shape (batch_size, channels, sequence_length).
The assumption holds when the output tensor shape of the embedding layers is (batch_size, channels, sequence_length).
So far, I haven't confirm the shape of the output tensor from the embedding layer. If you have any ideas, maybe we could have a discussion?