Fix more SyntaxWarnings in Python 3.13

[Ordoviz]

Following #209, this fixes two runtime warnings:

• SyntaxWarning: invalid escape sequence '\?', and
• SyntaxWarning: invalid escape sequence '\['

You can see that escaping ? was indeed intentional by considering that the arm instruction blr reg is 32-bit wide and follows this format. I replaced \? with \x3f because chr(0x3f) == '?'.

[nurmukhametov]

Unfortunately, this PR changes behaviour, as CI shows that fewer gadgets are found ending with blr reg:

>>> opcode = b"\x20\x00\x3f\xd6"
>>> [m.start() for m in re.finditer(b"[\x00\x20\x40\x60\x80\xa0\xc0\xe0]{1}[\x00-\x03]{1}\?\xd6", opcode)]
<stdin>:1: SyntaxWarning: invalid escape sequence '\?'
[0]
>>> [m.start() for m in re.finditer(b"[\x00\x20\x40\x60\x80\xa0\xc0\xe0]{1}[\x00-\x03]{1}\x3f\xd6", opcode)]
[]

What about using rb prefix with the unchanged value? It seems to work as expected and doesn't produce any warnings:

>>> [m.start() for m in re.finditer(rb"[\x00\x20\x40\x60\x80\xa0\xc0\xe0]{1}[\x00-\x03]{1}\?\xd6", opcode)]
[0]

[Ordoviz]

What about using rb prefix with the unchanged value?

Good idea. I added the rb prefix to all strings. This should avoid surprises like "[\x0d\x09\x05\x01\x1d\x19\x15\x11\x2d\x29\x25\x21\x3d\x39\x35\x31\x4d\x49\x45\x41\x5d\x59\x55\x51]" == "[\r\t\x05\x01\x1d\x19\x15\x11-)%!=951MIEA]YUQ]" where \x11-) is a regex range expression.

[Ordoviz]

The tests still fail, but now we're finding more gadgets!

[SweetVishnya]

The tests still fail, but now we're finding more gadgets!

Some newly found 'gadgets' are actually not real gadgets, e.g., andi sp, sp, -0x10, as they do not end with a jump/ret instruction. Can you take a look at test results and try to fix the regexes?

[Ordoviz]

I'm trying to figure out why ROPgadget.py --binary test-suite-binaries/elf-Linux-RISCV_32 --dump reports the following gadgets:

0x00010580 : andi sp, sp, -0x10 // 137101ff
0x00010582 : c.bnez a4, -0xe8 // 01ff

The first "gadget" does not end with a jump/ret instruction, so it should not be printed. The second one is fine.
Both are matched by the regex in line 341:

[rb"[\x0d\x09\x05\x01\x1d\x19\x15\x11\x2d\x29\x25\x21\x3d\x39\x35\x31\x4d\x49\x45\x41\x5d\x59\x55\x51][\xa0-\xff]{1}", 2, 1], # c.j | c.beqz | c.bnez

The problem is that this regex has gad_align == 1 so when i == 2, this will find the bad gadget at 0x00010580:

for i in range(self.__options.depth):
    start = ref - (i * gad_align)
    if (sec_vaddr + start) % gad_align == 0:  # always True for gad_align == 1

I don't know how to fix this: We would need to set gad_align = 4 to avoid the "bad" gadget but we cannot set it to a number bigger than 2 or else the second "good" gadget is not printed. Perhaps we need an equivalent of __passCleanX86 for riscv to filter out bad gadgets.

[nurmukhametov]

The first "gadget" does not end with a jump/ret instruction, so it should not be printed. The second one is fine. Both are matched by the regex in line 341:

[rb"[\x0d\x09\x05\x01\x1d\x19\x15\x11\x2d\x29\x25\x21\x3d\x39\x35\x31\x4d\x49\x45\x41\x5d\x59\x55\x51][\xa0-\xff]{1}", 2, 1], # c.j | c.beqz | c.bnez

The problem is that this regex has gad_align == 1 so when i == 2, this will find the bad gadget at 0x00010580:

It seems strange to me that gadgets are specified as [regexp, 2, 1] (with alignment 1) for 16-bit compressed instruction. I would expect that 16-bit compressed instructions must be aligned to 2-byte boundaries. See Section 1.5 in link for details.

[SweetVishnya]

@Ordoviz, can you try aligning with 2-byte boundaries as @nurmukhametov suggested?

[Ordoviz]

@Ordoviz, can you try aligning with 2-byte boundaries as @nurmukhametov suggested?

This does not help. The "bad gadget" is 2 bytes before the good gadget, so it will be found when i == 1.

[SweetVishnya]

@Ordoviz, can you try aligning with 2-byte boundaries as @nurmukhametov suggested?

This does not help. The "bad gadget" is 2 bytes before the good gadget, so it will be found when i == 1.

Then can you try adding pass clean as you proposed earllier?

[Ordoviz]

I finally found a good way to filter out the bad RISC-V gadgets. See the commit messages for details. This is ready to merge now.

[Ordoviz]

The CI fails because it uses Python 2 where rb"foobar" is a SyntaxError. Are we still interested in supporting Python 2?

[SweetVishnya]

Yeah, we're still willing to support python2 as far as possible

[Ordoviz]

It turns out that Python2 supports br"foobar" string literals, so fixing the CI was simply a matter of swapping b and r.

[SweetVishnya]

Thank you! Give me a week to review the changes.

[SweetVishnya]

Yeah, thank you very much for the fix, I'll merge it