P2P and Sharded DeepSeek-R1

[lonnietc]

Hi All,

I am new to LocalAI and have a great interest in P2P systems. On the AI side of things, I have looked at Petals (P2P AI Model) and have now just come across LocalAI which seems to also support P2P across many commodity based systems.

What I am seeking to do is to shard and replicate extremely large models like DeepSeek-R1 (and others) so that instead of inferencing across centralized server farms, it seems that P2P could offer the approach of utilizing existing CPU/GPU systems on the Internet. Of course, latency in the network is still problematic, but over time that will get faster and faster.

I am wondering if LocalAI can run Ollama LLM models since I know that DeepSeek-R1 is available on HuggingFace as well, so I thought that I might explore it a bit.

Any thoughts, ideas, or suggestions would be greatly appreciated.
Thanks and have a great day