(8.4)PXC-5208: Force SST after inconsistency eviction - #2320
(8.4)PXC-5208: Force SST after inconsistency eviction#2320jaideepkarande wants to merge 1 commit into
Conversation
https://perconadev.atlassian.net/browse/PXC-5208 When a node is voted out of the cluster due to a data inconsistency, remove grastate.dat on shutdown so the node is forced to perform a full SST on the next start instead of IST, which would otherwise resurrect the inconsistent dataset via --wsrep-recover.
|
https://pxc.cd.percona.com/view/8.0%20parallel%20MTR/job/pxc-8.x-pipeline-parallel-mtr/853/console Failures are known , since galera got updates and related changes are present in 8.4.10 yet to be released. |
kamil-holubicki
left a comment
kamil-holubicki
left a comment
There was a problem hiding this comment.
Could it be possible to add a test to IST failure path?
| --connection node_2 | ||
| --disable_query_log | ||
|
|
||
| SET SESSION wsrep_on=OFF; |
There was a problem hiding this comment.
Multiple occurences: inconsistent formating between = and similar
| SET GLOBAL wsrep_provider_options='pc.weight=1'; | ||
| DROP TABLE t1; | ||
| --connection node_2 | ||
| CALL mtr.add_suppression('Replica SQL: Could not execute Delete_rows event on table test.t1; Can\'t find record in \'t1\', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event\'s source log FIRST, end_log_pos 0, Error_code: MY-001032'); |
There was a problem hiding this comment.
Let's use more general suppression like
CALL mtr.add_suppression("Could not execute Delete_rows event on table test.t1.*Error_code: 1032");
| DROP TABLE t1; | ||
| --connection node_2 | ||
| CALL mtr.add_suppression('Replica SQL: Could not execute Delete_rows event on table test.t1; Can\'t find record in \'t1\', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event\'s source log FIRST, end_log_pos 0, Error_code: MY-001032'); | ||
| CALL mtr.add_suppression('Delete_rows apply failed: 120, seqno'); |
There was a problem hiding this comment.
The same here. Let's do
CALL mtr.add_suppression('Delete_rows apply failed:')
| # voting and node_2 is evicted. | ||
| # | ||
| --connection node_2 | ||
| --let $dummy = `SELECT COUNT(*) FROM t1` |
There was a problem hiding this comment.
Why do we need this select?
| --connection node_2 | ||
| --disable_query_log | ||
|
|
||
| SET SESSION wsrep_on=OFF; |
There was a problem hiding this comment.
Why do we need wsrep_on=OFF here?
There was a problem hiding this comment.
Actually, I realized that this behavior should be under the configuration variable
From Jira ticket:
'Also, would be nice if we could configure behavior of whether the node should restarted after an inconsistency or if it should remain up-n-running for review (wsrep_restart_after_inconsistency or such?)'
I think it should be off by default (original behavior by default).
Probably it will be easier to go through wsrep_provider_options, as grastate.dat is Galera thing
2 participants
https://perconadev.atlassian.net/browse/PXC-5208
When a node is voted out of the cluster due to a data inconsistency, remove grastate.dat on shutdown so the node is forced to perform a full SST on the next start instead of IST, which would otherwise resurrect the inconsistent dataset via --wsrep-recover.