ProxySQL pods restarting/crashing in MySQL_Session.cpp:3161: void MySQL_Session::handler___status_WAITING_CLIENT_DATA___STATE_SLEEP___MYSQL_COM_STMT_EXECUTE(PtrSize_t&): Assertion `0' failed error

[tritonph79]

We are trying to troubleshoot an issue with our proxySQL pods which suddenly started crashing in Kubernetes on Jul 6th, ~3PM Pacific with the error below. We're trying to isolate the connections which might be causing the issue but unable to find in the logs.

ProxySQL version: ProxySQL version 2.4.1-1-g1ea371d, codename Truls
OS: Debian 11 (bullseye)

proxysql proxysql: MySQL_Session.cpp:3161: void MySQL_Session::handler___status_WAITING_CLIENT_DATA___STATE_SLEEP___MYSQL_COM_STMT_EXECUTE(PtrSize_t&): Assertion `0' failed.                                                                       proxysql Error: signal 6:
proxysql proxysql(_Z13crash_handleri+0x29)[0x557712789689]                                                                                                                                                                                          proxysql /lib/x86_64-linux-gnu/libc.so.6(+0x3bd60)[0x7fa38c51cd60]
proxysql /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x141)[0x7fa38c51cce1]
proxysql /lib/x86_64-linux-gnu/libc.so.6(abort+0x123)[0x7fa38c506537]
proxysql /lib/x86_64-linux-gnu/libc.so.6(+0x2540f)[0x7fa38c50640f]
proxysql /lib/x86_64-linux-gnu/libc.so.6(+0x34662)[0x7fa38c515662]
proxysql proxysql(_ZN13MySQL_Session75handler___status_WAITING_CLIENT_DATA___STATE_SLEEP___MYSQL_COM_STMT_EXECUTEER10_PtrSize_t+0x651)[0x557712801f11]
proxysql proxysql(_ZN13MySQL_Session20get_pkts_from_clientERbR10_PtrSize_t+0x7b5)[0x55771280b4f5]
proxysql proxysql(_ZN13MySQL_Session7handlerEv+0xda)[0x55771280bf6a]
proxysql proxysql(_ZN12MySQL_Thread20process_all_sessionsEv+0x33c)[0x5577127ee71c]
proxysql proxysql(_ZN12MySQL_Thread3runEv+0x4ec)[0x5577127eeeac]
proxysql proxysql(_Z24mysql_worker_thread_funcPv+0x6c)[0x5577127818ac]
proxysql /lib/x86_64-linux-gnu/libpthread.so.0(+0x8ea7)[0x7fa38c9eaea7]
proxysql /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7fa38c5dedef]
proxysql  ---- proxysql(_Z13crash_handleri+0x29) [0x557712789689] : crash_handler(int)
proxysql  ---- proxysql(_ZN13MySQL_Session20get_pkts_from_clientERbR10_PtrSize_t+0x7b5) [0x55771280b4f5] : MySQL_Session::get_pkts_from_client(bool&, _PtrSize_t&)
proxysql  ---- proxysql(_ZN13MySQL_Session7handlerEv+0xda) [0x55771280bf6a] : MySQL_Session::handler()
proxysql  ---- proxysql(_ZN12MySQL_Thread20process_all_sessionsEv+0x33c) [0x5577127ee71c] : MySQL_Thread::process_all_sessions()
proxysql  ---- proxysql(_ZN12MySQL_Thread3runEv+0x4ec) [0x5577127eeeac] : MySQL_Thread::run()
proxysql  ---- proxysql(_Z24mysql_worker_thread_funcPv+0x6c) [0x5577127818ac] : mysql_worker_thread_func(void*)
proxysql To report a crashing bug visit: https://github.com/sysown/proxysql/issues
proxysql For support visit: https://proxysql.com/services/support/
proxysql /entrypoint.sh: line 10:     9 Aborted                 (core dumped) proxysql -f --idle-threads -D /var/lib/proxysql

Would appreciate any assistance regarding this issue. Please let me know if additional information is needed.

Thank you.

[mirostauder]

Logs are in /var/lib/proxysql by default, unless you start ProxySQL with -f which directs error log to stdout.
For docker images (proxysql started with -f) you can dump logs using docker logs as described here:
https://docs.docker.com/engine/reference/commandline/logs/
Please refer to the docs how to configure error logging
https://proxysql.com/documentation/Error-Log/