Why is the list of gadgets missing a lot of entries

[OevreFlataeker]

I am relatively new to pwntools and doing the ROPemporium challenges at the moment.

My pwntools has version 4.0.1

Besides pwntools I am using ropper to get my gadgets to solve the challenge.
I wonder about a severe discrepancy in both tools:

>>> from pwn import *
>>> context.binary = './pivot32'
[*] '/tmp/7_pivot/pivot32'
    Arch:     i386-32-little
    RELRO:    Partial RELRO
    Stack:    No canary found
    NX:       NX enabled
    PIE:      No PIE (0x8048000)
    RPATH:    './'
>>> context
ContextType(arch = 'i386', binary = ELF('/tmp/7_pivot/pivot32'), bits = 32, endian = 'little', os = 'linux')
>>> r = ROP(context.binary)
[*] Loaded 11 cached gadgets for './pivot32'
>>> r.gadgets
{134514880L: Gadget(0x80488c0, ['pop eax', 'ret'], ['eax'], 0x8), 134513412L: Gadget(0x8048304, ['ret'], [], 0x4), 134514981L: Gadget(0x8048925, ['add esp, 0xc', 'pop ebx', 'pop esi', 'pop edi', 'pop ebp', 'ret'], ['ebx', 'esi', 'edi', 'ebp'], 0x20), 134514984L: Gadget(0x8048928, ['pop ebx', 'pop esi', 'pop edi', 'pop ebp', 'ret'], ['ebx', 'esi', 'edi', 'ebp'], 0x14), 134514985L: Gadget(0x8048929, ['pop esi', 'pop edi', 'pop ebp', 'ret'], ['esi', 'edi', 'ebp'], 0x10), 134514986L: Gadget(0x804892a, ['pop edi', 'pop ebp', 'ret'], ['edi', 'ebp'], 0xc), 134514987L: Gadget(0x804892b, ['pop ebp', 'ret'], ['ebp'], 0x8), 134514030L: Gadget(0x804856e, ['add esp, 8', 'pop ebx', 'ret'], ['ebx'], 0x10), 134514033L: Gadget(0x8048571, ['pop ebx', 'ret'], ['ebx'], 0x8), 134514344L: Gadget(0x80486a8, ['leave', 'ret'], ['ebp', 'esp'], 0x2540be403), 134514341L: Gadget(0x80486a5, ['add esp, 0x10', 'leave', 'ret'], ['ebp', 'esp'], 0x2540be413)}
>>>

pwntools always says "Loaded 11 cached gadgets"

Wheres ropper gives me:

ropper -f pivot32 | tail -1
[INFO] Load gadgets from cache
[LOAD] loading... 100%
[LOAD] removing double gadgets... 100%
123 gadgets found

-> Ropper finds 123 gadgets

Also the list in "gadgets" only contains pop elements but no other gadgets like mov, xchg, etc.
Do I need to rebuild the cache or configure that pwntools should look more generic or something?
These gadgets mentioned are explicitly built into the binary and should be found;

pwndbg> disassemble usefulGadgets
Dump of assembler code for function usefulGadgets:
   0x080488c0 <+0>:     pop    eax
   0x080488c1 <+1>:     ret
   0x080488c2 <+2>:     xchg   esp,eax
   0x080488c3 <+3>:     ret
   0x080488c4 <+4>:     mov    eax,DWORD PTR [eax]
   0x080488c6 <+6>:     ret
   0x080488c7 <+7>:     add    eax,ebx
   0x080488c9 <+9>:     ret
   0x080488ca <+10>:    xchg   ax,ax
   0x080488cc <+12>:    xchg   ax,ax
   0x080488ce <+14>:    xchg   ax,ax
End of assembler dump.
pwndbg>

Found this past issue. Not sure whether it might be related? #1369

[OevreFlataeker]

I think I just found the reason in rop.py

def __load(self):
        """Load all ROP gadgets for the selected ELF files"""
        #
        # We accept only instructions that look like these.
        #
        # - leave
        # - pop reg
        # - add $sp, value
        # - ret
        #
        # Currently, ROPgadget does not detect multi-byte "C2" ret.
        # https://github.com/JonathanSalwan/ROPgadget/issues/53
        #

And further down the code is also the explicit filtering.

But why is this? Can it be changed? Why would I not want to have a larger/more complete list here?

[zachriggle]

Ultimately, the ROP autogeneration is limited in scope and capability. We chose to make it simple and reliable, rather than exposing additional ROP gadgets of unknown reliability to the user through the standard pwnlib.rop library.

You're always free to add your own gadgets to the ROP.gadgets instance, or invoke them directly via ROP.raw. ROP.gadgets is specifically a list of how to load various registers and consume stack space -- it's not intended to be an exhaustive tool. I recommend using ropper and ROPgadget if you need more flexibility.

We had a Summer of Code project that was intended to use symbolic execution to determine additional ROP gadgets, but it never landed in the mainline branch due to lack of time.

If you can provide a Pull Request that adds the needed smarts to the ROP module, they'd be much appreciated -- though I expect it will be a large undertaking.

[Marshall-Hallenbeck]

@zachriggle Why not allow users to hook into the ROPgadget command ran and customize the gadgets returned, or at least allow the functionality to return everything and keep the default the same?

[zachriggle]

There's nothing stopping you from adding additional gadgets to `rop_instance.gadgets`, no need for hooks! Alternately you can subclass ROP and do whatever you want. *Zach Riggle*

…

On Thu, May 14, 2020 at 4:44 PM Marshall Hallenbeck < ***@***.***> wrote: @zachriggle <https://github.com/zachriggle> Why not allow users to hook into the ROPgadget command ran and customize the gadgets returned, or at least allow the functionality to return everything and keep the default the same? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1492 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA3IGEFKILYCML3AINI2MDRRRQ4HANCNFSM4MRIXTDQ> .

[Arusekk]

I do not feel we should implement SAT solving. There are better tools for that (see Angr).