Provide runtime parameters from host code at runtime

[etabeta1]

Hi all,

I'm trying to understand if there is a way to pass runtime parameters to the NPU after the kernel has been uploaded.
It seems to me, by reading the examples, that the only way to set the runtime parameters is to hardcode them into the MLIR module using inline_ops and that there is no way to "leave them blank" and fill them when the kernel is called.
I tried looking in the source code of GlobalBuffer, in the Runtime source code and also in the InlineOpRuntimeTask source code but I didn't find anything relevant.

The question is: is there a way to provide runtime parameters when calling the kernel instead of hardcoding them into the MLIR module and having to reprogram the NPU every time they change?

[hunhoffe]

@andrej would you like to weigh in on this?

[andrej]

This is an interesting question, sorry for not getting to this when it was fresh.

You can write to run-time parameters from the runtime sequence, like so. This does not require reloading the rest of the static design (xclbin/ELF) onto the NPU: Compile your design once with aiecc.py --aie-generate-xclbin --xclbin-name=my_xclbin.xclbin to get an .xclbin for the static part of the design. Then, re-compile with different run-time parameter values as many times as needed using aiecc.py --aie-generate-npu --npu-insts-name=insts_X.bin. In the CPU host code, you can pass different runtime sequences as the second argument to your kernel invocation.

If you need many different values for run-time parameters, instead of compiling many insts.bins, you could also patch the runtime sequence in-memory on the CPU before calling the kernel. We currently don't have any CPU runtime support features for this, so you'd have to look at the generated insts.bins to see where you'd have to patch the value. Easiest would be either to look at the compiler pass that generates the insts.bins or generate multiple insts.bins with different RTP values and diff them. We may in the future add a light runtime layer that allows you to do this more easily.

[etabeta1]

Thanks, I'll try this as soon as possible.