Occasional DNS lookup failures when calling hologram agent

[copumpkin]

I'm not sure what would cause this, but I've seen this a few times on more than one computer:

$ hologram use somerole
[ERROR  ] 2016-02-22T09:04:09-05:00 dial tcp: lookup hologram.server.dns: no such host

If I try pinging hologram.server.dns (not the real domain) myself, everything else on my system seems to be able to resolve it just fine. Hologram will continue complaining about the "no such host" for a while and then after a few minutes, will start resolving it fine again.

It might worth noting that hologram.server.dns started as a cname for an Amazon ELB. After seeing this issue for a bit, I pointed my hologram agent directly at the ELB DNS, in case hologram didn't like cnames, but the issue persists.

Any suggestions? I don't see anything particularly interesting in the agent logs, but if someone has suggestions on how to debug this I'm happy to try things next time it happens.

[copumpkin]

I'd normally use ltrace here, but I think that's less useful with Go programs.

[aebie]

I have been having the same issue intermittently. Until today it generally cleared up in a short 10 minutes or so, but I have been getting the error for about 3 hours straight now.

[copumpkin]

I am on Mac OS when this happens, and https://golang.org/pkg/net/#pkg-overview suggests there might be some intricacies in ho Go does DNS on Mac OS.

[copumpkin]

Also potentially relevant: http://stackoverflow.com/questions/12952833/lookup-host-no-such-host-error-in-go and golang/go#3575.

[copumpkin]

I've noticed that I get the issue when I switch roles in rapid succession. Anyone else get this?