Skip to content

Commit b12da8d

Browse files
authored
fix(tee): correct previous fix for race condition in batch locking (#3358)
## What ❔ Commit a7dc0ed (PR #3342) was supposed to fix a race condition in batch locking by introducing SQL row-locking, but it [didn't work][2] as expected. ![Screenshot From 2024-12-04 11-32-32](https://github.com/user-attachments/assets/959ffc3c-593f-409a-87ab-68ec197040a0) Now we are switching back to coarser-grained table-level locking as [originally suggested][1] by Harald. The original fix was hard to test unless deployed to `stage` due to the undeterministic nature of the problem, so we needed to merge it to the `main` branch to properly test it. [1]: #3342 (comment) [2]: https://grafana.matterlabs.dev/goto/AhEd5FVNg?orgId=1 ## Why ❔ To fix the bug that only activates after running `zksync-tee-prover` on multiple instances. ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [x] Code has been formatted via `zkstack dev fmt` and `zkstack dev lint`.
1 parent dc2c476 commit b12da8d

File tree

2 files changed

+12
-8
lines changed

2 files changed

+12
-8
lines changed

core/lib/dal/.sqlx/query-8ead57cdda5909348f31f8c4d989f73e353da3bc6af7ecb81102c4194df631aa.json renamed to core/lib/dal/.sqlx/query-b6961d273f833f8babaf16f256822a6e92698fcfcf5d0a9252d84b75459b2664.json

Lines changed: 2 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

core/lib/dal/src/tee_proof_generation_dal.rs

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -66,10 +66,16 @@ impl TeeProofGenerationDal<'_, '_> {
6666
let min_batch_number = i64::from(min_batch_number.0);
6767
let mut transaction = self.storage.start_transaction().await?;
6868

69-
// Lock rows in the proof_generation_details table to prevent race conditions. The
70-
// tee_proof_generation_details table does not have corresponding entries yet if this is the
71-
// first time the query is invoked for a batch. Locking rows in proof_generation_details
72-
// ensures that two different TEE prover instances will not try to prove the same batch.
69+
// Lock the entire tee_proof_generation_details table in EXCLUSIVE mode to prevent race
70+
// conditions. Locking the table ensures that two different TEE prover instances will not
71+
// try to prove the same batch.
72+
sqlx::query("LOCK TABLE tee_proof_generation_details IN EXCLUSIVE MODE")
73+
.instrument("lock_batch_for_proving#lock_table")
74+
.execute(&mut transaction)
75+
.await?;
76+
77+
// The tee_proof_generation_details table does not have corresponding entries yet if this is
78+
// the first time the query is invoked for a batch.
7379
let batch_number = sqlx::query!(
7480
r#"
7581
SELECT
@@ -95,8 +101,6 @@ impl TeeProofGenerationDal<'_, '_> {
95101
)
96102
)
97103
LIMIT 1
98-
FOR UPDATE OF p
99-
SKIP LOCKED
100104
"#,
101105
tee_type.to_string(),
102106
TeeProofGenerationJobStatus::PickedByProver.to_string(),

0 commit comments

Comments
 (0)