Proposal: Integrate Arbitrum RPC fallback into livepeer-go

[FranckUltima]

In order to improve the quality and robustness of the network, I believe it could be useful to integrate an Arbitrum RPC fallback option directly into the livepeer-go code.

For example, we could enter a main RPC and 1 backup RPC, and if the main RPC no longer responds or provides erroneous data, the node could switch to the backup RPC, until the main RPC responds correctly again.

It would also be useful to have a metric available for prometheus/grafana so that we can integrate the RPC status into our monitoring.

This would also allow us to maintain our Arbitrum RPCs without stopping the availability of our Livepeer nodes.

[leszko]

@FranckUltima Thanks for posting. I think it's a duplicate of this: #1959

[FranckUltima]

Actually, sorry about that. I think my post was a good reminder for the new developers on the Livepeer project, since the old post was from two years ago. You can delete my request, since it's a duplicate. I don't think I can do it myself.

[JJassonn69]

@FranckUltima you can use proxyd right now to have a backup RPC, I have multiple RPC backends just incase the main RPC goes down, it automatically switches to the other RPC providers. But i would guess having it in the main livepeer itself would be more straight forward.

[eliteprox]

I think we should implement this within go-livepeer to help both Orchestrator and Broadcaster nodes stay up. We can allow for a comma-delimited list of RPC endpoints and then cycle through them if any blockWatcher error occurs instead of breaking.

I think the changes would be made to this go func:

go-livepeer/cmd/livepeer/starter/starter.go

Lines 1003 to 1018 in e9cbadb

	blockWatcherErr := make(chan error, 1)
	go func() {
	if err := blockWatcher.Watch(blockWatchCtx); err != nil {
	blockWatcherErr <- fmt.Errorf("block watcher error: %v", err)
	}
	}()
	go func() {
	var err error
	select {
	case err = <-timeWatcherErr:
	case err = <-blockWatcherErr:
	}
	watcherErr <- err
	}()