[Openstack cloud provider] Improvements on the floating ip logic

[armagankaratosun]

Hey all

I improved the floating ip logic by using asyncio since the create_vm function itself runs on the same event loop.

Cheers!

[jacobtomlinson]

We shouldn't be grabbing the loop here like this. Why are we changing from the standard call_async mechanism that Dask uses internally?

[armagankaratosun]

Well, you are right. Let me give you a recap on why I implemented this in the first place;

I wanted to use OpenStackScheduler/OpenStackWorker on their own (without spinning up a full OpenStackCluster), but I realized that if I use cluster.call_async without a cluster, Python quite naturally complains about “different event loop.” 😄

I completely understand that this workaround (grabbing the loop and running it directly in an executor) is a bit of a hack—please let me know if you’d prefer a different approach.

[jacobtomlinson]

We don't really support using ProcessInterface subclasses outside of a Cluster. What is your use case for doing this?

[armagankaratosun]

I have a use case where I have to provision a Dask cluster on two different OpenStack infrastructures (and they will communicate over the floating IPs). The trick here is that I only want to provision one scheduler on one of these clouds, and the workers should enroll themselves to it.

[jacobtomlinson]

I see. Instead of changing this logic for everyone I've opened #456 which means that you could subclass OpenStackInstance in your code and just override call_async with your own logic here. That way you have the flexibility to do what you need, but without changing this functionality for all users.