Wrong implementation of Spiking_Self_Attention

[frostylight]

It seems that the last lif (proj_lif) in Spiking_Self_Attention get input with shape $[T\times B, C, N]$ rather than $[T, B, C, N]$.
The reshape operators should be executed after proj_bn and before proj_lif.

The Accuracy of trained HST-10-384 is 78.732% when batch_size is set to $32$ (default in imagenet/test.py).
When fixing this problem by setting batch_size to 1, the performance drops to 74.202% (-4.530%).
When fixing this problem by replacing it with following code, the performance drops to 74.196% (-4.536%).

x = self.proj_lif(self.proj_bn(self.proj_conv(x)).reshape(T, B, C, W, H))

QKFormer/cifar10-dvs/model.py

Lines 143 to 146 in 43f0adf

	x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
	x = self.attn_lif(x)
	x = x.flatten(0, 1)
	x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T, B, C, W, H)

QKFormer/cifar10/model.py

Lines 115 to 118 in 43f0adf

	x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
	x = self.attn_lif(x)
	x = x.flatten(0,1)
	x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)

QKFormer/cifar100/model.py

Lines 115 to 118 in 43f0adf

	x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
	x = self.attn_lif(x)
	x = x.flatten(0,1)
	x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)

QKFormer/dvs128-gesture/model.py

Lines 143 to 146 in 43f0adf

	x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
	x = self.attn_lif(x)
	x = x.flatten(0, 1)
	x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T, B, C, W, H)

QKFormer/imagenet/qkformer.py

Lines 155 to 158 in 43f0adf

	x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
	x = self.attn_lif(x)
	x = x.flatten(0,1)
	x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)

[frostylight]

For HST-10-768 ($224\times 224$), 84.248% -> 78.698% (-5.55%).

[xiaochenzhao-svg]

对于 HST-10-768 （ 224 \倍 $ 224 ），84.248% -> 78.698% （-5.55%）。

Hello,I would like to ask if the maximum mem value when reproducing the paper is the same as the one mentioned in the author's log. For example, QKFormer_16M.txt indicates that the maximum mem value is 17867, but the value I reproduced is 21469. I am very confused about this.

[bic-L]

对于 HST-10-768 （ 224 \倍 $ 224 ），84.248% -> 78.698% （-5.55%）。

Hello,I would like to ask if the maximum mem value when reproducing the paper is the same as the one mentioned in the author's log. For example, QKFormer_16M.txt indicates that the maximum mem value is 17867, but the value I reproduced is 21469. I am very confused about this.

In my experience, this is related to the GPU model. For example, the same model will require different GPU memory on A30 and H20

[xiaochenzhao-svg]

对于 HST-10-768 （ 224 \倍 $ 224 ），84.248% -> 78.698% （-5.55%）。

Hello,I would like to ask if the maximum mem value when reproducing the paper is the same as the one mentioned in the author's log. For example, QKFormer_16M.txt indicates that the maximum mem value is 17867, but the value I reproduced is 21469. I am very confused about this.

In my experience, this is related to the GPU model. For example, the same model will require different GPU memory on A30 and H20

Thank you for your response. I also have the same question regarding the issue raised by the original poster. Do you have any insightful thoughts on this phenomenon?

[bic-L]

对于 HST-10-768 （ 224 \倍 $ 224 ），84.248% -> 78.698% （-5.55%）。

Hello,I would like to ask if the maximum mem value when reproducing the paper is the same as the one mentioned in the author's log. For example, QKFormer_16M.txt indicates that the maximum mem value is 17867, but the value I reproduced is 21469. I am very confused about this.

In my experience, this is related to the GPU model. For example, the same model will require different GPU memory on A30 and H20

Thank you for your response. I also have the same question regarding the issue raised by the original poster. Do you have any insightful thoughts on this phenomenon?

I personally agree with the original post, yet it is undeniable that the hierarchical design of QKFormer clearly brings performance gain. I have reproduced QKFormer with the membrane potential shortcut after correcting this problem and provided the detailed analysis regarding the memory usage and energy consumption (results can be found in https://arxiv.org/pdf/2505.18608). QKFormer is still better than the previous baseline.

[xiaochenzhao-svg]

For HST-10-768 (224\times $224), 84.248% -> 78.698% (-5.55%).

Hello,I would like to ask if the maximum mem value when reproducing the paper is the same as the one mentioned in the author's log. For example, QKFormer_16M.txt indicates that the maximum mem value is 17867, but the value I reproduced is 21469. I am very confused about this.

In my experience, this is related to the GPU model. For example, the same model will require different GPU memory on A30 and H20

Thank you for your response. I also have the same question regarding the issue raised by the original poster. Do you have any insightful thoughts on this phenomenon?

I personally agree with the original post, yet it is undeniable that the hierarchical design of QKFormer clearly brings performance gain. I have reproduced QKFormer with the membrane potential shortcut after correcting this problem and provided the detailed analysis regarding the memory usage and energy consumption (results can be found in https://arxiv.org/pdf/2505.18608). QKFormer is still better than the previous baseline.

Thank you for your reply. I’ve read your paper, and it's truly an inspiring piece of work that I should definitely study carefully.

[xiaochenzhao-svg]

It seems that the last lif (proj_lif) in Spiking_Self_Attention get input with shape [ T × B , C , N ] rather than [ T , B , C , N ] . The reshape operators should be executed after proj_bn and before proj_lif.

The Accuracy of trained HST-10-384 is 78.732% when batch_size is set to 32 (default in imagenet/test.py). When fixing this problem by setting batch_size to 1, the performance drops to 74.202% (-4.530%). When fixing this problem by replacing it with following code, the performance drops to 74.196% (-4.536%).

x = self.proj_lif(self.proj_bn(self.proj_conv(x)).reshape(T, B, C, W, H))
QKFormer/cifar10-dvs/model.py

Lines 143 to 146 in 43f0adf

x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0, 1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T, B, C, W, H)
QKFormer/cifar10/model.py

Lines 115 to 118 in 43f0adf

x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)
QKFormer/cifar100/model.py

Lines 115 to 118 in 43f0adf

x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)
QKFormer/dvs128-gesture/model.py

Lines 143 to 146 in 43f0adf

x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0, 1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T, B, C, W, H)
QKFormer/imagenet/qkformer.py

Lines 155 to 158 in 43f0adf

x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)

Did you keep the complete log files when reproducing the experiments?

[frostylight]

It seems that the last lif (proj_lif) in Spiking_Self_Attention get input with shape [ T × B , C , N ] rather than [ T , B , C , N ] . The reshape operators should be executed after proj_bn and before proj_lif.
The Accuracy of trained HST-10-384 is 78.732% when batch_size is set to 32 (default in imagenet/test.py). When fixing this problem by setting batch_size to 1, the performance drops to 74.202% (-4.530%). When fixing this problem by replacing it with following code, the performance drops to 74.196% (-4.536%).
x = self.proj_lif(self.proj_bn(self.proj_conv(x)).reshape(T, B, C, W, H))
QKFormer/cifar10-dvs/model.py
Lines 143 to 146 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0, 1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T, B, C, W, H)
QKFormer/cifar10/model.py
Lines 115 to 118 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)
QKFormer/cifar100/model.py
Lines 115 to 118 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)
QKFormer/dvs128-gesture/model.py
Lines 143 to 146 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0, 1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T, B, C, W, H)
QKFormer/imagenet/qkformer.py
Lines 155 to 158 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)

Did you keep the complete log files when reproducing the experiments?

In fact, I didn't reproduce it on ImageNet. I simply downloaded the trained models and test them using the given test.py.
The whole test didn't took too long and the test.py doesn't write logs into file like train.py , so unfortunately I don't have the log files.

[xiaochenzhao-svg]

It seems that the last lif (proj_lif) in Spiking_Self_Attention get input with shape [ T × B , C , N ] rather than [ T , B , C , N ] . The reshape operators should be executed after proj_bn and before proj_lif.
The Accuracy of trained HST-10-384 is 78.732% when batch_size is set to 32 (default in imagenet/test.py). When fixing this problem by setting batch_size to 1, the performance drops to 74.202% (-4.530%). When fixing this problem by replacing it with following code, the performance drops to 74.196% (-4.536%).
x = self.proj_lif(self.proj_bn(self.proj_conv(x)).reshape(T, B, C, W, H))
QKFormer/cifar10-dvs/model.py
Lines 143 to 146 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0, 1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T, B, C, W, H)
QKFormer/cifar10/model.py
Lines 115 to 118 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)
QKFormer/cifar100/model.py
Lines 115 to 118 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)
QKFormer/dvs128-gesture/model.py
Lines 143 to 146 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0, 1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T, B, C, W, H)
QKFormer/imagenet/qkformer.py
Lines 155 to 158 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)

Did you keep the complete log files when reproducing the experiments?

In fact, I didn't reproduce it on ImageNet. I simply downloaded the trained models and test them using the given test.py. The whole test didn't took too long and the test.py doesn't write logs into file like train.py , so unfortunately I don't have the log files.

Okay, thank you for your reply.

[Rangs02245]

It seems that the last lif (proj_lif) in Spiking_Self_Attention get input with shape [ T × B , C , N ] rather than [ T , B , C , N ] . The reshape operators should be executed after proj_bn and before proj_lif.
The Accuracy of trained HST-10-384 is 78.732% when batch_size is set to 32 (default in imagenet/test.py). When fixing this problem by setting batch_size to 1, the performance drops to 74.202% (-4.530%). When fixing this problem by replacing it with following code, the performance drops to 74.196% (-4.536%).
x = self.proj_lif(self.proj_bn(self.proj_conv(x)).reshape(T, B, C, W, H))
QKFormer/cifar10-dvs/model.py
Lines 143 to 146 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0, 1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T, B, C, W, H)
QKFormer/cifar10/model.py
Lines 115 to 118 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)
QKFormer/cifar100/model.py
Lines 115 to 118 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)
QKFormer/dvs128-gesture/model.py
Lines 143 to 146 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0, 1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T, B, C, W, H)
QKFormer/imagenet/qkformer.py
Lines 155 to 158 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)

Did you keep the complete log files when reproducing the experiments?

In fact, I didn't reproduce it on ImageNet. I simply downloaded the trained models and test them using the given test.py. The whole test didn't took too long and the test.py doesn't write logs into file like train.py , so unfortunately I don't have the log files.

Hi frostylight, would you mind sharing the pretrained checkpoints, since they are no longer available? Many thanks!

[bic-L]

It seems that the last lif (proj_lif) in Spiking_Self_Attention get input with shape [ T × B , C , N ] rather than [ T , B , C , N ] . The reshape operators should be executed after proj_bn and before proj_lif.
The Accuracy of trained HST-10-384 is 78.732% when batch_size is set to 32 (default in imagenet/test.py). When fixing this problem by setting batch_size to 1, the performance drops to 74.202% (-4.530%). When fixing this problem by replacing it with following code, the performance drops to 74.196% (-4.536%).
x = self.proj_lif(self.proj_bn(self.proj_conv(x)).reshape(T, B, C, W, H))
QKFormer/cifar10-dvs/model.py
Lines 143 to 146 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0, 1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T, B, C, W, H)
QKFormer/cifar10/model.py
Lines 115 to 118 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)
QKFormer/cifar100/model.py
Lines 115 to 118 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)
QKFormer/dvs128-gesture/model.py
Lines 143 to 146 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0, 1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T, B, C, W, H)
QKFormer/imagenet/qkformer.py
Lines 155 to 158 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)

Did you keep the complete log files when reproducing the experiments?

In fact, I didn't reproduce it on ImageNet. I simply downloaded the trained models and test them using the given test.py. The whole test didn't took too long and the test.py doesn't write logs into file like train.py , so unfortunately I don't have the log files.

Hi frostylight, would you mind sharing the pretrained checkpoints, since they are no longer available? Many thanks!

Maybe I still have a copy for 10-384 and 10-768, not sure though ...

[Rangs02245]

It seems that the last lif (proj_lif) in Spiking_Self_Attention get input with shape [ T × B , C , N ] rather than [ T , B , C , N ] . The reshape operators should be executed after proj_bn and before proj_lif.
The Accuracy of trained HST-10-384 is 78.732% when batch_size is set to 32 (default in imagenet/test.py). When fixing this problem by setting batch_size to 1, the performance drops to 74.202% (-4.530%). When fixing this problem by replacing it with following code, the performance drops to 74.196% (-4.536%).
x = self.proj_lif(self.proj_bn(self.proj_conv(x)).reshape(T, B, C, W, H))
QKFormer/cifar10-dvs/model.py
Lines 143 to 146 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0, 1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T, B, C, W, H)
QKFormer/cifar10/model.py
Lines 115 to 118 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)
QKFormer/cifar100/model.py
Lines 115 to 118 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)
QKFormer/dvs128-gesture/model.py
Lines 143 to 146 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0, 1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T, B, C, W, H)
QKFormer/imagenet/qkformer.py
Lines 155 to 158 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)

Did you keep the complete log files when reproducing the experiments?

In fact, I didn't reproduce it on ImageNet. I simply downloaded the trained models and test them using the given test.py. The whole test didn't took too long and the test.py doesn't write logs into file like train.py , so unfortunately I don't have the log files.

Hi frostylight, would you mind sharing the pretrained checkpoints, since they are no longer available? Many thanks!

Maybe I still have a copy for 10-384 and 10-768, not sure though ...

Appreciate it if you would find out and share them!!!

[JasonKitty]

It seems that the last lif (proj_lif) in Spiking_Self_Attention get input with shape [ T × B , C , N ] rather than [ T , B , C , N ] . The reshape operators should be executed after proj_bn and before proj_lif.
The Accuracy of trained HST-10-384 is 78.732% when batch_size is set to 32 (default in imagenet/test.py). When fixing this problem by setting batch_size to 1, the performance drops to 74.202% (-4.530%). When fixing this problem by replacing it with following code, the performance drops to 74.196% (-4.536%).
x = self.proj_lif(self.proj_bn(self.proj_conv(x)).reshape(T, B, C, W, H))
QKFormer/cifar10-dvs/model.py
Lines 143 to 146 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0, 1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T, B, C, W, H)
QKFormer/cifar10/model.py
Lines 115 to 118 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)
QKFormer/cifar100/model.py
Lines 115 to 118 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)
QKFormer/dvs128-gesture/model.py
Lines 143 to 146 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0, 1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T, B, C, W, H)
QKFormer/imagenet/qkformer.py
Lines 155 to 158 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)

Did you keep the complete log files when reproducing the experiments?

In fact, I didn't reproduce it on ImageNet. I simply downloaded the trained models and test them using the given test.py. The whole test didn't took too long and the test.py doesn't write logs into file like train.py , so unfortunately I don't have the log files.

Hi! Could you please share the original pretrained weights? Because the download link is already expired. Thank you very much!

[bic-L]

It seems that the last lif (proj_lif) in Spiking_Self_Attention get input with shape [ T × B , C , N ] rather than [ T , B , C , N ] . The reshape operators should be executed after proj_bn and before proj_lif.
The Accuracy of trained HST-10-384 is 78.732% when batch_size is set to 32 (default in imagenet/test.py). When fixing this problem by setting batch_size to 1, the performance drops to 74.202% (-4.530%). When fixing this problem by replacing it with following code, the performance drops to 74.196% (-4.536%).
x = self.proj_lif(self.proj_bn(self.proj_conv(x)).reshape(T, B, C, W, H))
QKFormer/cifar10-dvs/model.py
Lines 143 to 146 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0, 1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T, B, C, W, H)
QKFormer/cifar10/model.py
Lines 115 to 118 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)
QKFormer/cifar100/model.py
Lines 115 to 118 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)
QKFormer/dvs128-gesture/model.py
Lines 143 to 146 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0, 1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T, B, C, W, H)
QKFormer/imagenet/qkformer.py
Lines 155 to 158 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)

Did you keep the complete log files when reproducing the experiments?

In fact, I didn't reproduce it on ImageNet. I simply downloaded the trained models and test them using the given test.py. The whole test didn't took too long and the test.py doesn't write logs into file like train.py , so unfortunately I don't have the log files.

Hi! Could you please share the original pretrained weights? Because the download link is already expired. Thank you very much!

I don't know if this will help, but you can find the checkpoints for MS-QKFormer via the following link: https://github.com/bic-L/MaxFormer/releases/tag/checkpoints.

[JasonKitty]

It seems that the last lif (proj_lif) in Spiking_Self_Attention get input with shape [ T × B , C , N ] rather than [ T , B , C , N ] . The reshape operators should be executed after proj_bn and before proj_lif.
The Accuracy of trained HST-10-384 is 78.732% when batch_size is set to 32 (default in imagenet/test.py). When fixing this problem by setting batch_size to 1, the performance drops to 74.202% (-4.530%). When fixing this problem by replacing it with following code, the performance drops to 74.196% (-4.536%).
x = self.proj_lif(self.proj_bn(self.proj_conv(x)).reshape(T, B, C, W, H))
QKFormer/cifar10-dvs/model.py
Lines 143 to 146 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0, 1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T, B, C, W, H)
QKFormer/cifar10/model.py
Lines 115 to 118 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)
QKFormer/cifar100/model.py
Lines 115 to 118 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)
QKFormer/dvs128-gesture/model.py
Lines 143 to 146 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0, 1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T, B, C, W, H)
QKFormer/imagenet/qkformer.py
Lines 155 to 158 in 43f0adf
x = x.transpose(3, 4).reshape(T, B, C, N).contiguous()
x = self.attn_lif(x)
x = x.flatten(0,1)
x = self.proj_lif(self.proj_bn(self.proj_conv(x))).reshape(T,B,C,W,H)

Did you keep the complete log files when reproducing the experiments?

In fact, I didn't reproduce it on ImageNet. I simply downloaded the trained models and test them using the given test.py. The whole test didn't took too long and the test.py doesn't write logs into file like train.py , so unfortunately I don't have the log files.

Hi! Could you please share the original pretrained weights? Because the download link is already expired. Thank you very much!

I don't know if this will help, but you can find the checkpoints for MS-QKFormer via the following link: https://github.com/bic-L/MaxFormer/releases/tag/checkpoints.

Thank you!