False detection of failover by aws-advanced-jdbc-wrapper

[AmeyaBidwalkar]

Hello Folks ,

We have a case where aws jdbc advanced driver has detected a failover event in one of our service running on EKS connecting to postgres aurora db , but when we check the aws console and engine uptime for node it seems to be false alarm .This has happened now twice during a week .
We have checked internally with AWS Support they have confirmed with their internal tools, that the DB remained available during the periods when the Advanced JDBC Wrapper Driver detected the supposed failover events. No failover events were captured during these times and further requested us to create a issue here.

version details
Postgresql Aurora version ="15.8"
postgresql-aws-jdbc-wrapper = "2.5.6"
jvmVersion = "21"
kotlin = "2.1.20"

Attached you have the Logs and engineUptime snippet .Let us know in case some more details are required.

[sergiyvamz]

Hi @AmeyaBidwalkar

Thank you for reaching out with this issue. The AWS Advanced JDBC Driver supports failover on a DB cluster by reconnecting to other available DB nodes when necessary. Since the driver doesn't receive any failover status notifications from the DB cluster, reconnection is triggered exclusively by connection issues or network errors. When the driver detects any network-related exception, it initiates a failover procedure that culminates in establishing a connection to an alternative node. Network connectivity can be temporarily affected by various events that may cause increased latency or bandwidth degradation. Such occurrences might trigger
false positive failover processes in the driver. Applications should be designed to handle these situations appropriately for optimal performance and reliability.

To troubleshoot this issue effectively, we recommend examining exceptions that occurred prior to the failover procedure in the driver. These exceptions can provide valuable insights into the root cause of false failover events. Consider reviewing the following configurations to address false failover detection:

• Evaluate your socketTimeout settings for connections
• If you're utilizing the Host Monitoring Plugin (efm and efm2 plugins), assess the monitoring parameters and make necessary adjustments
• Determine whether TCP keepalive functionality can be implemented to maintain connection stability

See the following links for more details and guidance:
https://github.com/aws/aws-advanced-jdbc-wrapper/blob/main/docs/using-the-jdbc-driver/FailoverConfigurationGuide.md
https://github.com/aws/aws-advanced-jdbc-wrapper/blob/main/docs/using-the-jdbc-driver/using-plugins/UsingTheFailoverPlugin.md
https://github.com/aws/aws-advanced-jdbc-wrapper/blob/main/docs/using-the-jdbc-driver/using-plugins/UsingTheFailover2Plugin.md