MatMul op is not supported

[JosePereiraUA]

I'm trying to load an .ONNX file, and get the error on the title.

Also:

Closest candidates are:
  load_node!(::Ghost.Tape, ::ONNX.OpConfig{:ONNX, :Gemm}, ::Vector{Ghost.Variable}, ::Dict{Symbol, Any}) at ~/.julia/packages/ONNX/TEIEp/src/load.jl:67
  load_node!(::Ghost.Tape, ::ONNX.OpConfig{:ONNX, :Conv}, ::Vector{Ghost.Variable}, ::Dict{Symbol, Any}) at ~/.julia/packages/ONNX/TEIEp/src/load.jl:86
  load_node!(::Ghost.Tape, ::ONNX.OpConfig{:ONNX, :MaxPool}, ::Vector{Ghost.Variable}, ::Dict{Symbol, Any}) at ~/.julia/packages/ONNX/TEIEp/src/load.jl:92

If I understand correctly, there's not a load_node! dispatch for the :MatMul operation?
How can I fix this? Thank you.

[ToucheSir]

How can I fix this? Thank you.

Put in a PR adding the op 😉

[JosePereiraUA]

I'm 100% a noob here.

I've added something like:

function load_node!(tape::Tape, ::OpConfig{:ONNX, :MatMul}, args::VarVec, attrs::AttrDict)
    return push_call!(tape, mul, args...)
end

The mul function (mul(xs...) = .*(xs...)) should work?

[ToucheSir]

I'm not exactly sure what would be equivalent, but if it can match the examples in https://numpy.org/doc/stable/reference/generated/numpy.matmul.html then you're probably on the right track.

[JosePereiraUA]

So, according to those examples, Julia does the same thing:

julia> a = [1 0; 0 1]
2×2 Matrix{Int64}:
 1  0
 0  1

julia> b = [4 1; 2 2]
2×2 Matrix{Int64}:
 4  1
 2  2

julia> a * b
2×2 Matrix{Int64}:
 4  1
 2  2

This means the implementation should be:

function load_node!(tape::Tape, ::OpConfig{:ONNX, :MatMul}, args::VarVec, attrs::AttrDict)
    return push_call!(tape, *, args...)
end

Right? I don't know is this makes sense.

---

Also, I'm trying to load the following .onnx file.
model_C.zip

It first asks me:

julia> ONNX.load("model_C.onnx")
ERROR: AssertionError: Neither initializer, nor argument is provided for input dense_1_input

I understand that I have to input some dummy data? I try to but get a bunch of dimension missmatch errors. I understand that another option is to run ONNX.load("model_C.onnx", exec = false). Can you explain this part a little better, please?

[ToucheSir]

That's just one example. Vector-vector and >2D arrays don't work in *. One or more of LinearAlgebra.dot or NNlib.batched_mul might be required for the others.

Can you explain this part a little better, please?

I don't understand the API terribly well myself, but have a look at how this is done in the tests.

[JosePereiraUA]

That makes sense. I'm not entirelly sure how to implement this. Oh well.

[ToucheSir]

I didn't mean to discourage you! Perhaps as a start we could just consider the 2D case by importing :MatMul as *. Can you double check that's enough for the model you're working with?

[darsnack]

I think the correct thing to do would be to define matmul(x, y) similar to mul in this repo. You can define different methods of matmul for different types of x or y. When x and y are AbstractVectors, then it should call LinearAlgebra.dot. When x and y are AbstractMatrix, then it should call *. And when either x or y is a AbstractArray{T, 3}, it should call NNlib.batched_mul. All other combinations should error for now.

In terms of loading the operation

function load_node!(tape::Tape, ::OpConfig{:ONNX, :MatMul}, args::VarVec, attrs::AttrDict)
    return push_call!(tape, matmul, args...)
end

should work.

For running your model, you need to pass in some dummy data. This can be done by initializing an input that's the same size as your input data:

# dummy input that is 224 x 224 x 3 x 1 (the last dimension is 1 batch)
x = rand(Float32, 224, 224, 3, 1)
model = ONNX.load("model_C.onnx", x)

[JosePereiraUA]

My model is supposed to work on a 2D case, so I've implemented the * operation only. It solved that particular error. It seems I can't publish my changes as a PR (says I don't have permission).

For running your model, you need to pass in some dummy data. This can be done by initializing an input that's the same size as your input data:

# dummy input that is 224 x 224 x 3 x 1 (the last dimension is 1 batch)
x = rand(Float32, 224, 224, 3, 1)
model = ONNX.load("model_C.onnx", x)

Did you actually download the model and test these lines, or are these just an example? Because I get:

julia> model = ONNX.load("model_C.onnx", x)
ERROR: MethodError: no method matching *(::Array{Float32, 4}, ::Matrix{Float32})

It somehow has to do with the * implementation for the :MatMul.

[ToucheSir]

You'd need to fork the repo first, create a branch and then file a PR with that. Happy to help walk through the steps if you're interested.

I think Kyle's snippet was just an example. Judging from the MethodError, model_C actually needs the higher-dim case and not the matrix-matrix multiplication that * covers.

[JosePereiraUA]

I'm currently working on forking the thing.

I understand that, those dims were not in accordance to what I expected, so I also think it was an example. I've changed the * to LinearAlgebra.dot, still getting dimension missmatch errors which I can't pinpoint. Thank you for all the patience

[darsnack]

Did you actually download the model and test these lines, or are these just an example?

Yes, these are just an example. Do you know what size input your model expects? These lines will need to be changed to match what your model expects. Or if this is indeed the correct dimensionality for your model, then you appear to need the higher dim matmul like Brian mentioned.