Distributed Federated Leaning

[mmRoshani]

I am currently working on Distributed Federated Learning (DFL) and am looking for a framework or library to achieve this goal. I found the NVIDIA Federated Learning Application Runtime Environment and am wondering if there are any plans to support DFL?

I am also interested in contributing to the efforts to establish support for DFL.

[mmRoshani]

@YuanTingHsieh
Hello, Would you please guide me?

[YuanTingHsieh]

Hello, I am not sure what does distributed federated learning mean?

From my understanding, federated learning is a type of distributed learning.
So if you want to try federated learning, you can start with our simple examples using JobAPI, then simulator, then check how to deploy to the real world (distributed), you can follow: https://nvidia.github.io/NVFlare/

[mmRoshani]

Yes, federated learning is a type of distributed learning. However, in the case of aggregation, there is typically a centralized server that performs this task. I prefer each client to conduct the aggregation with their neighbors, for example, using gossip protocols.
I am looking for a P2P implementation of FL in NVFlare.
Would you like to see the latest DFL papers?

[YuanTingHsieh]

Got it, yes, we do have P2P communication and we can do client side aggregation.
One example we have is this "swarm learning" example: https://github.com/NVIDIA/NVFlare/tree/main/examples/advanced/swarm_learning

I think the gossip learning can be achieved using similar way^
Sure, it would be great if you could share the latest papers in this direction.

[ZiyueXu77]

Please check this example for P2P capabilities
https://github.com/NVIDIA/NVFlare/tree/main/examples/advanced/distributed_optimization