Non-UTF8 characters get mangled in statement_summary, slow_query, and logs

[madhavramesh]

Enhancement

Right now, non-UTF8 binary values get mangled in statement_summary, slow_query, and various logs. All non-UTF8 characters are replaced by ?. Here's a simple example:

CREATE TABLE test.t1 (c1 binary(16) NOT NULL PRIMARY KEY);
INSERT INTO test.t1 VALUES (0xd2e4a6b8c1f3e5d7a9b2c4d6e8f1a3b5);

Then, run

printf "SELECT * FROM test.t1 WHERE c1 = '\xd2\xe4\xa6\xb8\xc1\xf3\xe5\xd7\xa9\xb2\xc4\xd6\xe8\xf1\xa3\xb5';\n" \
 | mysql --binary-mode --host 127.0.0.1 --port 4000 -u root

When statement_summary is queried, the non-UTF8 values in the binary string will be replaced by ?, meaning the query can't be copied and rerun and the original text is lost.

SELECT QUERY_SAMPLE_TEXT FROM INFORMATION_SCHEMA.STATEMENTS_SUMMARY
WHERE QUERY_SAMPLE_TEXT LIKE '%t1%' AND QUERY_SAMPLE_TEXT LIKE '%SELECT%'
 ORDER BY LAST_SEEN DESC LIMIT 1\G
*************************** 1. row ***************************
QUERY_SAMPLE_TEXT: SELECT * FROM test.t1 WHERE c1 = '?䦸???ש???????'

[ti-chi-bot]

Welcome @madhavramesh! It looks like this is your first issue to pingcap/tidb 🎉