Detect errors when fetching remote domain or list and skip item

[Crazypedia]

when trying to pull from multiple sources too build up a consensus for my block list, any error when pulling a block list from a server that does not have a publicly listed blocked domain list or an incorrectly formatted entry in that list, the script fails and requires troubleshooting.

with a large list of peers i am checking with, having it fail on every one that isnt publicly listed makes generating a list very difficult with much trial and error.

How can we add error detection handling for important lists so it does not crash as often?

[jpwarren]

Hmm. Sounds like there are a few options here because there are multiple issues.

Can you provide a bit more detail on the kinds of errors that happen?

And a bit more about what behaviour you'd like to see from the tool.

As a general philosophy I prefer things to break hard, because that encourages systemic fixes that make underlying problems go away. Automating workarounds that mask the problems keeps things running, for now, but it tends to lead to bigger explosions when stuff eventually breaks.

Which is not to say that defensive programming and robustness are bad! They're good. But robustness comes from making sure we don't pile up lots of workarounds and hacks instead of addressing the underlying issues if we can.