Arc A770 Slow speed

[galkinvv]

Discussed in #51

^{Originally posted by mia14112 December 15, 2024}
Hi, my Arc A770 is real slow in the test. In GPU-Z the Bandwith is 512gb/s and in the test it's only achieving ~105gb/s. The fans are not even starting to spin on the card.

My System is:
Ryzen 7700X
X670E Carbon
64GB DDR5
Asrock A770 8GB
Corsair HX1200

Hi, the card is only drawing ~144w (max is 252) and only ~4gb of memory are actually used.
CPU usage was around 2% during the run.
HWInfo is saying the Card is limited by PL4 (Powerlimit 4) which seems odd as PL4 should be 800W

Originally posted by @mia14112 in #51 (comment)

~2-3gb/s read and ~0.1-0.2gb/s write without memtest running

[galkinvv]

Thanks @mia14112 for reporting!

I'm summarizing here the discussion result:

The so low bandwidth usage is definitely an issue. And its quite starnge that much higher read bandwidth is reported by GPU-Z. Maybe the Arc GPU somehow do strange things regarding read caching - either is pulling to huge blocks that then are discarded before usage or discards small blocks to early before they are processed. Or uses exessive read operations during the writing stage for some reason.

The "only ~4GB tested " is unrelated problem. Some drivers do not expose more then 4Gb in a single block. This is not related to low bandwidth and appears on some AMD gpus too, not to severe problem for signal lines stability testing, so this may be treated as minor by now.

Unfortunately, I don't think that I can fix this before I perform some debugging on an Arc GPU.

[m1yur1]

The memory range tested is small when using Intel Arc.
Here is the error output when using the Intel Arc B570.

The parameters regarding defects in the memory range to be tested, and their values ​​for the B570, are as follows.
maxBufferSize := 4294967296
maxMemoryAllocationSize := 4294901760
maxStorageBufferRange := 1073741820

Also, I think it is best to use the atomic<> qualifier for those elements of IOBuf that require atomic access.

err.txt

[galkinvv]

@m1yur1 those messages are from validation layers, some notices about sctrict compatibility checks. Those are actually warnings.

Is testing started after getting those messages? What's the first several lines of output regarding speed and tested sizes?

[m1yur1]

memtest_vulkan-v0.5.0.exe 1> out.txt 2>&1

out.txt

[galkinvv]

      1 iteration. Passed  0.1277 seconds  written:    1.8GB 166.7GB/sec        checked:    3.5GB  29.9GB/sec

So, allocatiuon succeded only 3.5GB consequtive buffer. Thats seem to be some limitation of ARC vulkan API implementation. The warning messages are mostly abouth this.
Compared to Arc770 test, the written speed is greater, but the check speed is lower, not sure what's causing such difference.

Maybe in addition to the 4GB limit there is some speed related 2GB limit

Please create test18.cmd file contaiinig
memtest_vulkan-v0.5.0.exe 1 1800000000

And run it to test performance whilr testing 1.8GB. Which GB/s speeds it would show?

Statistically, running VRAM tests on higher performance is more important than on higher memory allocation, since most (not all) VRAM errors are appearing on high performance usage, not on high capacity usage.

However, to test more capacity you can launch several memtest_vualkn instances in parallel

[m1yur1]

memtest_vulkan-v0.5.0.exe 1 1800000000 1> out18.txt 2>&1

out18.txt

Unlike Radeon, Arc has less lane independence, so it may not work well with the memory access patterns of memtest_vulkan.

[galkinvv]

Unlike Radeon, Arc has less lane independence, so it may not work well with the memory access patterns of memtest_vulkan.

Interesting idea. To check taht every shader core has stable access to every memory IC memtest_vulkan divides all tested area into VRAM into 8KB blocks. Blocks are processed sequentially depending on shader invocation id, this is common and not a surprise to the hardware.

the 8KB constant is defined as let TEST_WINDOW_READ_ADDR_ROTATION_GRANULARITY: u32 = 0x2000u; in main.rs

But the acccess order within the 8KB block is a bit randomized, not sequential.

However, I'm not sure that 30GB/S is the real speed, since the first screenshot in the topic reports 100GB/s for both write and check. Maybe the driver started reporting that write is done actually before its really done, and then is finazling write when read is called.

I'd think how to implement more reliable estimation of write performance, maybe adding some CPU-observable side effects.

[m1yur1]

It appears that Intel Arc GPUs perform data compression and decompression during write and read operations when using Vulkan, and this seems to be causing a performance degradation.
I wouldn't have gained this insight without your program.

[galkinvv]

Intel Arc GPUs perform data compression and decompression during write and read operations when using Vulkan, and this seems to be causing a performance degradation.

Never heard of such technology for non-graphis-texture data, and it should be really hard implementable for random-acess data.
Is there any documentaion for this?

[m1yur1]

For insights on memory compression, see the results of using https://github.com/ProjectPhysX/OpenCL-Benchmark at https://www.reddit.com/r/OpenCL/comments/1ofrfl0/comment/nlbgnqa/

Also, when I wrote my own program to test this (using an Arc B570 GPU with a specified memory bandwidth of 380 GB/s),
I initially used OpenCL and achieved write speeds of about 330 GiB/s and read speeds of about 310 GiB/s.
When I tried using Vulkan, the performance was around 204–218 GiB/s for writes and 30 GiB/s for reads, which was the same as memtest_vulkan.
After modifying the write kernel to always write zeros, the performance was 433–456 GiB/s for writes and 248–256 GiB/s for reads.
I don't know why there's a difference between OpenCL and Vulkan.