Catching async exceptions in worker thread means it can't be cancelled

We probably do not want to use `try` in the worker thread. If something goes wrong here it is very bad and the whole program should just crash I think. The `try` is around `wait` as well which makes little sense, the `wait` will never throw an exception. 