after applying speedup_model, model inference got error

[Jervint]

Describe the issue:
origin:
conv1(in_channels=32,out_channels=64)
conv2(in_channels=64,out_channels=32)
sparsity=0.5
after compress && speedup:
conv1(in_channels=32,out_channels=32)
conv2(in_channels=41,out_channels=16)

the error maybe cased by infer_mask.update_indirect_sprarity when some module has channel/group dependency

Environment:

• NNI version:
• Training service (local|remote|pai|aml|etc):
• Client OS:
• Server OS (for remote mode only):
• Python version:
• PyTorch/TensorFlow version:
• Is conda/virtualenv/venv used?:
• Is running in Docker?:

Configuration:

• Experiment config (remember to remove secrets!):
• Search space:

Log message:

• nnimanager.log:
• dispatcher.log:
• nnictl stdout and stderr:

How to reproduce it?:

[J-shang]

hello @Jervint , could you show your model structure? we will try to find out how it occurred.

[Jervint]

hello @Jervint , could you show your model structure? we will try to find out how it occurred.

https://github.com/microsoft/nni/blob/v2.4/examples/model_compress/models/cifar10/resnet.py resnet18

[J-shang]

hello @Jervint , could you show your model structure? we will try to find out how it occurred.

https://github.com/microsoft/nni/blob/v2.4/examples/model_compress/models/cifar10/resnet.py resnet18

We tried resnet18 with L1NormPruner, and it seems to work well. Could you supply more information about how you meet this issue?

[luigiluz]

Is there a way to force that the output and of the input of the next layer remain the same? (To allow matrix multiplication)

I'm currently facing the same issue when trying to prune a model and after pruning + speed up the output and next input dimensions are different what brings up an error not allowing matrix multiplication.

RuntimeError: mat1 and mat2 shapes cannot be multiplied (64x20097 and 20086x38)

[J-shang]

Hello @luigiluz , could you give us an example to reproduce your issue? It seems a bug in speedup, I think it may be caused by some special op. Speedup should ensure that output and next layer input match the dimensions.