Visual debugging and profiling.

[mateli]

1. A machine code debugger that allows for live viewing of memory, current CPU registers, disassembly etc. This would be similar to many applications that use x86 debugging instructions to step trough an exe file although by integrating it with an emulator it can be used to figure out what an program actually are doing.
2. A profiler that counts instruction executions to figure out where a program spends time.
3. A live patcher. A patch specifies a memory location or a function call to replace with a call to a target native dll/so file. This can be used both to speed up programs and also to modify their behavior. Potentially this can be done in a way that is invisible to the guest application by having the emulator just recognize where the program counter enters the code to replace.
4. For live patching it is useful to be able to replace parts of a function. For example the beginning of main could be replaced with native code that does something that is useful to do in native host-optimized code but then still run parts of the main function. It could also call some of the functions that are called from the original main functions while replacing others with target native code. This would allow an application to be gradually rewritten and translated to a modern programming language. Although it will only make sense to patch parts where features or performance can be improved.

[mateli]

Something like this for debugging: https://x64dbg.com/
Other inspirations (emulator based):
https://syncsim.sourceforge.net/index.php?show=screenshots
https://github.com/slatinsky/x86sim

[evmar]

If you click one of the samples linked from the home page, there is a "view in debugger" button at the top. Click 'step' a few times to see some asm.

I can't tell from this if you are sharing a general idea, or requesting something specific, could you clarify?

[mateli]

I will do a bit more thinking on the debugger part of this issue as I see that there are a primitive debugger already implemented. The first thing that comes to mind is that there are no restart button. Also neither register content nor memory can be edited. Saving the application state (think action replay) in a format that can be reloaded would also be useful both in the debugger and directly from the emulator.

As for profiling that doesn't seem to be implemented at all. All I can think of here is to count how often the program counter visits a memory location and run an instruction.

[mateli]

I am thinking that when there is editing/live patching support it would be useful to have a short way to call into API:s/DLL:s in a way that use a minimum amount of instructions. Rewriting machine code requires that the new code is shorter than the original code and if we live patch code this will be code intended to run inside retrowin32 so it doesn't need to work anywhere else.

One way to do this is to have an instruction that is illegal on x86-32 but in retrowin32 it represents making a call into a native API function. This could be followed by a pointer to a data structure representing both parameters and the return value of the function or something similar. Performance is the most important aspect when implementing this. Although instead of having a standardized way to send data to the native code it may be more useful for the native call to fetch the data it need, placing return data wherever it needs to be.

[evmar]

I have this! For DLLs provided by retrowin32, the implementation uses the syscall instruction as the signal to the emulator to take over control. https://github.com/evmar/retrowin32/blob/main/doc/shims.md describes this a bit, where retrowin32_syscall mentioned there is the syscall instruction.

The existing mechanism for DLL calls (where parameters are pushed onto the x86 stack) is about as efficient as you can make it, I think.

[mateli]

Would it be possible to run retrowin32 as a debugger to an IDE so that when we have the source code of the application we can debug it with both source code and assembly?