How to set the external mode ?

[surajkarki66]

Hi @LinusJungemann and @fpjentzsch

I hope this message finds you well. I am currently working on deploying various YOLO model versions on a ZCU104 FPGA using FINN+. Due to model sizes, I'm encountering BRAM constraints that prevent some models from fitting onto the FPGA.

I wanted to explore using external memory for weight storage, but I have concerns about potential AXI Lite interface limitations when implementing this approach. Could you please advise:

Is there a recommended method to configure external memory access for weights in FINN+ ?

Are there specific considerations or constraints regarding AXI Lite connections when using external memory mode (using lot of MVAU/VVAU nodes with external mem_mode) ?

I would greatly appreciate your insights on managing these resource constraints while maintaining reasonable performance.

Thank you in advance for your time and expertise.

Best regards,
Suraj Karki
Hochschule Bielefeld

[LinusJungemann]

Hi Suraj,
you can use the folding config to set the mem_mode to external. This is in my opinion the best option.

From the FINN+ side, there shouldn't really be a constraint on the usage of external memory, but there might be a constraint based on the FPGA that you use. I am only familiar with the Alveo type cards, therefore I do not know if the number of AXI Lite connections is somehow limited on your board, but I would probably not go above 1 connection per port on each of your HBM/DDR banks. One thing that you have to be careful about is the maximum AXI width. If I remember it correctly it is either 4k or 8k bits. You can run into problems if you have extremely unfolded layers, as unfolding also increases the width of your AXI streams.

I hope these information help. If you have any other questions, feel free to ask :)

Best regards,
Linus Jungemann