Unicorn 2 - looking for sponsors

[aquynh]

Since its born in 2015, Unicorn has been inspiring so many cool research & tools, that we believe it deserves serious upgrade. Based on Qemu 2.1.2, there is a lot to catch up with the latest version of Qemu (Qemu is at v4.2 at the moment). Unfortunately, the code base of Qemu changes so much since 2015, that we believe it is faster to rewrite Unicorn from scratch.

For this reason, next major update of Unicorn (lets call it Unicorn 2) requires huge effort & lots of time. So we call for the support from Unicorn users!

If you are using Unicorn for your work, and wish to sponsor the development of Unicorn 2, please contact us: http://www.unicorn-engine.org/contact/

[korniltsev]

are you going to keep in sync with upsream after rewrite?

[aquynh]

That is not easy to keep syncing with the upstream. This means we must trade off with some sacrifices in features. Keep posting here if you have any ideas.

[Waterman178]

Support for the next version!

[XVilka]

Related #1190
Regarding the keeping in sync - it might be more useful to create a series of semantic patches (with spatch tool) to generate the Unicorn on the fly. There is a mature tool to do this called Coccinelle, see their documentation for more details. There is a good introductory article on how to create semantic patches. Coccinelle used for the Linux kernel sources refactoring, so it's stable and featureful enough.
See the example of this semantic patch:

@@
expression lock, flags;
expression urb;
@@

 spin_lock_irqsave(lock, flags);
 <...
- usb_submit_urb(urb)
+ usb_submit_urb(urb, GFP_ATOMIC)
 ...>
 spin_unlock_irqrestore(lock, flags);

@@
expression urb;
@@

- usb_submit_urb(urb)
+ usb_submit_urb(urb, GFP_KERNEL)

[minexew]

@ XVilka Due to how Unicorn is created (ripping out the heart of QEMU and fixing it up to make it nicely embeddable), it unfortunately seems that any new branch of Unicorn is doomed to ultimately fall out of sync with QEMU upstream again.

[raphaelthegreat]

Is there any update on Unicorn 2?

[kzsnow]

My two cents: I did a bit of "Unicorn 2" prototyping with Qemu 4.2. The two biggests "lifts" I encountered, in terms of source code modifications, were compartmentalizing qemu globals (e.g. when multiple emu instances are used), and dealing with the memory implementation.

Like others, I'm cautiously optimistic there is a way to minimize Qemu source code mods to make it easier to keep up with development there, and greatly lessen the burden for updates.

A quick hack to get around the issue of qemu globals without source code mods is to load unique instances of the shared library for each "emu" instance. Of course the trade-off is increased memory consumption (however big the qemu lib is, for each instance). I've used Unicorn quite a bit, and in all my use cases (personally) this would be an acceptable trade-off.

Regarding memory, there are two options: softmmu or using the host system's mmu. To use the host system's mmu, I didn't need to modify qemu source code at all (basically just linux-user mode without syscall passthrough). Besides that, performance is 5x better than Unicorn/softmmu in some rudimentary tests. It would be really useful to be able to run "Unicorn 2" in this mode for the performance boost, when needed (i.e. fuzzing, etc.).

There are massive downsides to using the host mmu though: emulated code can no longer be securely sandboxed (not that I can tell, at least), and it restricts where regions can be mapped in the emulation (compatibility). Hence, ultimately we'd want the ability to use softmmu. This is where I ran into the biggest issue. The qemu "system vs. host" and "softmmu vs. hostmmu" #defines are not separated well, and source code mods started building up. At this point, the time investment outweighed the benefit.

So I'll put this out there for discussion. The way forward might be to start with pull requests on Qemu towards the goal of adding a "softmmu vs. hostmmu" option for the linux-user build of qemu. I vaguely recall that someone was attempting this years ago and Qemu devs seemed receptive, but it didn't materialize. If we had this option in Qemu, I'd wager that Unicorn-like functionality would be much easier to build out.