Add Custom Engine Availability Probing Feature for Multi-User Sharelevel in Kyuubi

[Echan0808]

Currently, under Kyuubi’s multi-user Sharelevel (e.g., Group Sharelevel), it has been observed that the loading of a jar file by User A can potentially affect the availability of the entire Engine. This impact primarily occurs at the Spark layer, causing errors in subsequently submitted tasks.

From my understanding, the existing aliveprobe mechanism is mainly designed to detect whether an Engine is alive, but it lacks the ability to perform customized SQL probing. For example, similar to setConnectionTestQuery in HikariConfig, it would be beneficial for Kyuubi administrators to configure lightweight probe statements to determine if the Engine is fully operational.

My proposal is to implement periodic probing of all Engines under a specific namespace at the KyuubiServer level to ensure their availability. After reviewing the community resources, it seems this functionality is not yet available. I’d like to ask if the community has any plans in this direction or considers this a worthwhile feature to develop? Looking forward to your feedback and discussion!