check_axi_stream not detecting errros in 5.0.0.dev6

[obruendl]

I noticed that in 5.0.0.dev6 check_axi_stream() does not detect data mismatches. Testbenches that worked correctly and actually checked results in 5.0.0.dev6 seem to no more check results and always pass.

IMO this is very CRITICAL. If VUnit does fail due to a bug, this is unhandy but it does not lead to buggy designs passing verification - the problem is detected and a solution can be found. Here the situation is that VUnit does NOT FAIL even if bugs are present. Even worse - testbenches that used to work correctly in the past and in which developers may trust will not do their checks and always succeed. IMO this problem should be tackled better sooner than later.

I have created a reproduction setup here:
https://github.com/open-logic/open-logic/tree/debug/vunit-check-axi-stream-issue

The incorrectly working check_axi_stream() call is in /test/base/olo_base_arb_wrr/olo_base_arb_wrr_tb.vhd on line 138

I am working with ghdl 5.0.1 but I believe the simulator should not play a role

How to reproduce:

1. Checkout the branch mentioned above
2. Ensure vunit_hdl 5.0.0.dev6 is installed
3. Navigate to /sim
4. Execute python3 run.py *arb_wrr* --> This will succeed although it should fail
5. Change vunit_hdl installation to 4.7.0
6. Execute python3 run.py *arb_wrr* --> This will fail (correct behavior)

[LarsAsplund]

Hi @obruendl, after 4.7.0 we made a bug fix that takes tkeep into account when checking tdata against the expected value. Only bytes with a 1 in tkeep will be checked. For example, if tdata is 0x12345678, tkeep is 1001, and the expected data is 0x12000078, the check will still pass.

The tkeep input to the VC has the length of tdata divided by 8 and has a default value of all 1s. If not connected, that default value will cause all bytes in tdata to be checked.

In your case, tdata is less than 8 bits which means that the length of the tdata port becomes 5 / 8 = 0. Since tkeep has no bits it contains no 1s and no bytes are checked.

Note that tdata must be a multiple of 8 bits in length according to the AXI stream standard. I guess we could consider an arbitrary width by setting the width of tkeep to ceil(tdata'length / 8) but I fear that there may be other aspects of the standard that aren't that easily fixed.

[obruendl]

Hi @LarsAsplund

Thank you for the reply. I get the point.

Yes, the official standard is defining multiples of 8 - and for data-moving this makes perfect sense. Also for top-level connections (e.g. between some signal processing and a DMA) it makes sense.

However, in many applications arbitrary width data is natural at least internally. E.g. 18 bits is a common word size for DSP. And still the AXI-S handshaking often is used. So I'd clearly vote for supporting this case in the standard VCs.

I'd see the following options:

1. Rounding tkeep width up as you suggested. I don't see any problems - but I agree, we could overlook something.
2. Add a "wordwise" option to the VCs. In this mode they would check data wordwise.

My vote would be (1) but both would solve my problem.

IMO this aoplies to AXI4-Stream only. AXI4 I have never seen with arbitrary word-width.

If (2) is chosen, the VC should at least report an error in "bytewise mode" (default) when data width is not a multiple of 8. This would at least let users know there is a problem with their TB.

[LarsAsplund]

I think the way forward would be to generate an error for non-byte widths and have an option to override that. Making sure that the full test suite passes with a non-byte width would be a good first start. It would have caught your issue.

[LarsAsplund]

@obruendl I created a PR that you can test. Need some additional thinking to make sure that there are no unforeseen side-effects.