Problematic HW Loops on ESP32-P4 (DSP-158)

[f4lc0n-asm]

Answers checklist.

• I have read the ESP-DSP documentation and the issue is not addressed there.
• I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
• I have updated my ESP-DSP branch (master or release) to the latest version and checked that the issue is present there.
• I have searched the issue tracker for a similar issue and not found a similar issue.

General issue report

Hello,

I just obtained a Rev. 1.0 ESP32-P4 DevKit. It seems that hw loops on P4 are not utilized by GCC for a reason - when the loop setup asm instruction is not aligned on a 4-byte boundary, the CPU sometimes goes haywire (PMP faults - load/store/instr_fetch). It looks like it can be cured by inserting the .balignw 4, 0x0001 directive above the loop setup instruction, which inserts a 16-bit c.nop instruction if req'd. The P4 hasn't overcome its teething problems so far :/

Anyone with more knowledge can elucidate this arcane CPU bug?

f4lc0n

[igrr]

Yes, I think there is an errata related to HW loops in the engineering samples of P4. It will be fixed in the mass production version.

It seems that hw loops on P4 are not utilized by GCC for a reason

It's much simpler than that, we just didn't implement hardware loops support based on this instruction extension in GCC yet.

@o-marshmallow do you recall what are the conditions under which the hardware loop option can be used in the existing silicon revisions?

[o-marshmallow]

Hello @f4lc0n-asm , hello @igrr ,

I am not sure which hardware revision (ECO) you are using, but to be safe, here are the constraints to respect:

• Make sure the start of the loop (first instruction) is aligned on 4
• Make sure you have at least six 32-bit instructions in the loop or twelve 16-bit instructions. If you have a mix of 16-bit and 32-bit instructions, make sure the total is more than 6*4=24 bytes
• Make sure the last instruction of the loop is not a coprocessor instruction. So it should not be an FPU instruction, nor an AI/PIE coprocessor instruction.
• Make sure to use the latest version of IDF, a bug was recently fix regardign context saving for the HWLP feature

[f4lc0n-asm]

@o-marshmallow, @igrr: Thank you for the elucidation! Btw I have ESP32-P4 Rev. 1.0 (as detected by the bootloader).

Can't help it, but I also have to ask about the fmadd.s instruction - it seems it has variable length in cycles (1-3) depending on the input multiplicands being a sharp zero or not - looks like a cool math shortcut :) …

[howard0su]

I compare the hand write assembly code for ESP32S3 with C++ code generated by GCC. Looks like it is almost same except a hardware loop instruction is used by hand write code. Any option to gcc can enable hardware loop for ESP32S3? If so, we can remove lots of hand write assembly code in this repo.

[igrr]

Any option to gcc can enable hardware loop for ESP32S3

It is possible to support hardware loops for ESP32-S3 in GCC, but just like with ESP32-P4, this was not done yet.

(If we do implement something like this, I think at this point it is more likely that this will be done in LLVM rather than GCC.)

[f4lc0n-asm]

@howard0su Hello, on ESP32-S3 if you really want to achieve the maximum speed, even subtle nuances like the order of fp32 instructions or loop/jump/branch target alignment within a natural 4-byte boundary do matter - see e.g. the new ASM FFT routines for ESP32-S3 in this repo. I thus wouldn't write off the hand-coded ASM so quickly. The best way is to compare the speed of hand coded ASM with GCC in CPU cycles. Please see e.g. the FFT benchmarks table.

[howard0su]

@howard0su Hello, on ESP32-S3 if you really want to achieve the maximum speed, even subtle nuances like the order of fp32 instructions or loop/jump/branch target alignment within a natural 4-byte boundary do matter - see e.g. the new ASM FFT routines for ESP32-S3 in this repo. I thus wouldn't write off the hand-coded ASM so quickly. The best way is to compare the speed of hand coded ASM with GCC in CPU cycles. Please see e.g. the FFT benchmarks table.

I agree with you that it will not be fast path. But we surely needs more work in compiler to optimize this in gcc/llvm for the things you mentioned. Creating more and more hand coded asm is so painful and hard to maintain. If you look at CMSIS-DSP for ARM, there is almost zero hand coded ASM code.

[f4lc0n-asm]

@howard0su I worked with ARM Cortex M GCC to create 128/64-bit IIRs and thanks to very efficient instruction set and of course the compiler itself, it sometimes produced optimal code or just 1 CPU cycle slower. Xtensa GCC is not there yet, but the progress cannot be stopped…