ISSUE regarding mem_mode = external

[DaZai3219]

Prerequisites

Please make sure to check off these prerequisites before submitting a bug report.

• [ Y] Test that the bug appears on the current version of the dev-branch. Make sure to include the commit hash of the commit you checked out.
• [ Y] Check that the issue hasn't already been reported, by checking the currently open issues.
• [ Y] If there are steps to reproduce the problem, make sure to write them down below.
• [ N] If relevant, please include the ONNX files, which were created directly before and/or after the bug.

Quick summary

after changing memory mode to External, driver generation is failing

Details

After setting "mem_mode": "external" in the generated folding config file. Bitfile is generated but the driver fails. This might be due to missing weights of the external nodes.
What is the correct way of using memory mode as external. How many external nodes can we set and do we have to enable runtime_writeable_weights. Is there any other way to tackle over utilization of BRAM.

Expected behavior

driver should be generated

Actual behavior

driver generation fails and produces the following error

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[1], line 4
      2 from qonnx.core.modelwrapper import ModelWrapper
      3 model = ModelWrapper("output_mobilenetv1-w4a4_KV260_SOM/intermediate_models/step_synthesize_bitfile.onnx")
----> 4 model = model.transform(MakePYNQDriver("zynq-iodma"))

File /home/xxx/finn/deps/qonnx/src/qonnx/core/modelwrapper.py:140, in ModelWrapper.transform(self, transformation, make_deepcopy, cleanup)
    138 model_was_changed = True
    139 while model_was_changed:
--> 140     (transformed_model, model_was_changed) = transformation.apply(transformed_model)
    141 if cleanup:
    142     transformed_model.cleanup()

File /home/xxx/finn/src/finn/transformation/fpgadataflow/make_pynq_driver.py:295, in MakePYNQDriver.apply(self, model)
    289         fcl_w = dataflow_model.get_initializer(node.input[1])
    290         w_filename = weights_dir + "/%d_%d_%s.dat" % (
    291             sdp_ind,
    292             rt_layer_ind,
    293             node.name,
    294         )
--> 295         node_inst.make_weight_file(fcl_w, "decoupled_runtime", w_filename)
    296         rt_layer_ind += 1
    297 elif node.op_type == "StreamingDataflowPartition":

File /home/xxx/finn/src/finn/custom_op/fpgadataflow/matrixvectoractivation.py:654, in MVAU.make_weight_file(self, weights, weight_file_mode, weight_file_name)
    641 """Produce a file containing given weights in appropriate format for this
    642 layer. This file can be used for either synthesis or run-time reconfig
    643 of weights.
   (...)
    651 
    652 """
    653 # convert weights into hlslib/rtllib-compatible format
--> 654 weight_tensor = self.get_hw_compatible_weight_tensor(weights)
    655 export_wdt = self.get_weight_datatype()
    656 # we have converted bipolar weights to binary for export,
    657 # so use it as such for weight generation

File /home/xxx/finn/src/finn/custom_op/fpgadataflow/matrixvectoractivation.py:616, in MVAU.get_hw_compatible_weight_tensor(self, orig_weight_matrix)
    614 simd = self.get_nodeattr("SIMD")
    615 wmem = self.calc_wmem()
--> 616 assert orig_weight_matrix.shape == (
    617     mw,
    618     mh,
    619 ), """Weights matrix doesn't
    620 have expected shape (mw, mh)"""
    621 assert mw % simd == 0, "Requirement MH divisable by SIMD is violated."
    622 assert mh % pe == 0, "Requirement MH divisable by PE is violated."

AttributeError: 'NoneType' object has no attribute 'shape'

Steps to Reproduce

Add what needs to be done to reproduce the bug. Add code examples where useful
and make sure to include the resulting ONNX files, and the commit hash you are working on.

1. Clone the FINN repository
2. Checkout the dev branch, with commit hash: [...]
3. Start the docker container with the command: [...]
4. Run transformation [...] on ONNX file [...] or run the dataflow builder with the following settings: [...]
5. After setting mem_mode = 'External' in the folding config file, like this:

"MVAU_rtl_13": {
"PE": 4,
"SIMD": 4,
"ram_style": "block",
"resType": "dsp",
"mem_mode": "external",
"runtime_writeable_weights": 1

[auphelia]

Hi @DaZai3219,

In general, what you have done looks fine. I will have a look to see what might be the issue here. One thing to be aware of is that the external mem_mode for the RTL MVAU is only experimental, I would recommend using the HLS MVAU instead if you like to use external mem_mode. If it's just about balancing resources, you could use mem_mode="internal_decoupled" and use the ram_style similar to how you have done it in your issue report, but set it to "distributed" or "ultra": https://github.com/Xilinx/finn/blob/dev/src/finn/custom_op/fpgadataflow/matrixvectoractivation.py#L103
You can use the resource estimation to get a feeling for what consequences which setting has. If you have an overuse of BRAM, maybe switching for FIFOs to URAM or having a look at the resource usage of the Thresholding layers might also be an option. Here are some more details: https://finn-dev.readthedocs.io/en/latest/faq.html and https://finn-dev.readthedocs.io/en/latest/reference/mem-modes.html#ram-style-attribute

[DaZai3219]

Hi @DaZai3219,

In general, what you have done looks fine. I will have a look to see what might be the issue here. One thing to be aware of is that the external mem_mode for the RTL MVAU is only experimental, I would recommend using the HLS MVAU instead if you like to use external mem_mode. If it's just about balancing resources, you could use mem_mode="internal_decoupled" and use the ram_style similar to how you have done it in your issue report, but set it to "distributed" or "ultra": https://github.com/Xilinx/finn/blob/dev/src/finn/custom_op/fpgadataflow/matrixvectoractivation.py#L103 You can use the resource estimation to get a feeling for what consequences which setting has. If you have an overuse of BRAM, maybe switching for FIFOs to URAM or having a look at the resource usage of the Thresholding layers might also be an option. Here are some more details: https://finn-dev.readthedocs.io/en/latest/faq.html and https://finn-dev.readthedocs.io/en/latest/reference/mem-modes.html#ram-style-attribute

Thankyou for the reply. I have tried with hls variant as well, and even if the pynq driver is generated, The model gets stuck at the execution on fpga, as in No output is generated and which gives a sense that some sort of bottleneck is there. i have tried with different boards, such as ZCU102, Pynq Z2 and KV260. As of now i have only been able to run tfc_2w2a with some nodes as external memory mode. I had a few doubts:

1. Is there any series of steps that i should follow specifically to set node as external, or just changing mem_mode in folding config is sufficient?
2. What is the limit on how many nodes can we set as external mem_mode? and is there any limit on the size?
3. Should the runtime_writeable_weights be enabled when setting the mem_mode to external?