Explain me please how to work with icmp stats on the dashboard

[AndrewSav]

Hey, I'm slightly confused about this aspect of gatus as a new user:

With ping/icmp endpoint when ping drops, for reason unclear to me the response time is reported equal to time out. This seems to make the statistics unreliable, since it's start averaging the timeout value into the final figure that to me makes little sense. Am I doing it wrong? Is there a switch to exclude failed pings from statistics so we do not get incorrect skew?

Thanks you in advance!

[TwiN]

You are right that the ping timeouts skew the average response time, but this is intended, as it accurately reflects what you should expect from the service (though the timeout does prevent things from getting out of hand).

I understand people may have different perspectives on how response time should be measured, but as I see it, trying to only measure response time from successful checks is doctoring the real-life average response times.

That said, you can reduce the client timeout to lessen the impact of timeouts on the average response time.

[AndrewSav]

@TwiN, thank you, I understand that it's unlikely that I'll convince you, but those pings that timed out did not arrive at all they did not simply took longer. Since they did not arrive, the response time IMO is wrong. With the timeout 10s the time reported is 10s and the ping did not take 10s, it simply did not come back. It would be awesome if we had an option not to record response time that is wrong.

[TwiN]

I'm not against making the behavior at least configurable (endpoints.ui.ignore-failures-in-duration?), but the bottom line is, if you had an application on the same host as Gatus that also performed a ping, the expectation would be that the requests would time out as well.

It's also important to make the distinction that the duration/response time is from the client (Gatus)'s perspective, not the server's perspective, so regardless of whether the server received it or not, it does not change the fact that the duration of the request, from the client's perspective, is impacted by the inability of the infrastructure/networking configuration/environment/server to return a response to the client.

It is also not completely impossible that the timeout is simply too short, which then brings in the question, what if it actually takes 15 seconds to receive a response, but we're assuming that it's just some mysterious error that shouldn't count as part of the response time, and then this elevated response time, cut off by the configured timeout duration, is not reflected by the reported duration/response time?

Moreover, what about people monitoring nothing except the duration of an endpoint? If the duration was not set because the request timed out, this would mean that the moment an endpoint took too long to respond, the default behavior would be to never alert on it because the [DURATION] placeholder would resolve to 0.

I do want to clarify that I'm not saying all this because I think your use case is wrong, it's completely fine to have different use cases, but as a maintainer, I have to consider what users are using Gatus for, and given the flexibility that Gatus offers, sometimes I have to make these types of decisions, and continue respecting them even if other users disagree in the future, because nothing angers people more than breaking changes, especially if those breaking changes are what makes the difference between being alerted when your applications are degraded, or not being alerted at all when they rely on Gatus to monitor their business/home lab/applications.

Anyhow, long story short, I think your idea has merit as a configuration parameter, but the default behavior is not something I believe should be modified.

[AndrewSav]

Thank you. First of all, my approach is very pragmatic, I want to be able to do what I need to do, so I'm not bothered with the default behaviour is not changed. Moreover, since I'm not unfamiliar with software development lifecycle, I know that backward compatibility have many merits, so from that perspective one should not change the defaults unless there is a reason compelling enough, In this case it it is not obvious to me that such reason exists. (It might, or, most likely, it might not). I'm happy with your response.

Now, there are a few ways to solve the issue. My line of thoughts was this:

• STATUS will give you the failure condition, so if that's what you are using for failure then changing this option won't affect your alerting
• If you use RESPONSE_TIME for failure condition, it can be implemented that it's still honoured event if the option is on, it just is not included in stats displayed in the UI and/or metrics.
• I have not given much thought to non icmp scenarios. You choose to model different type of monitors as the moniker/schema on the endpoint uri, but in reality, different types require different parameters even now, and they also get different results (placeholders). You arguments are most compelling for HTTP(s) a web server ostensibly can take several minutes to return response, but not so much for ping. I do not think that it's likely that you will receive a ping to something close after 10 seconds if you did not receive it so far. I understand that we probably do not want this conversation is to morph to the discussion about different needs of different monitors though.
• Finally, this issue bothers me because it shows me that there is a ping latency in my network, where there actually is not. There are just some missing pings. It is also very easy to mis-interpret by an onlooker this way.

Thank you for the long and detailed responses, you do not have to respond to this one, unless you want to, I've already taken up enough of your time.

[TwiN]

No problem at all.

I'm completely open to this being a configurable feature, even if it just applies to icmp (though I do believe there is merit in making it work more broadly to all endpoint types).