Lock db_mtx around arc_release() in couple places

* Lock db_mtx around arc_release() in dbuf_release_bp()

While this function is called only in sync context, the same buffer
can be touched by dbuf_hold_impl() in open context, creating races.
All other accesses to arc_release() are already protected by db_mtx,
so just take it here too.

Signed-off-by: Alexander Motin <alexander.motin@TrueNAS.com>

* Lock db_mtx in sa_byteswap()

While SA code seems protected by sa_lock, there is a back door of
dmu_objset_userquota_get_ids(), that may hold and access the dbuf
without sa_lock, relying only on db_mtx. Taking db_mtx here should
protect both the arc_release() and the data for db_buf.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Ameer Hamza <ahamza@ixsystems.com>
Signed-off-by: Alexander Motin <alexander.motin@TrueNAS.com>
Closes #18146

Author: amotin

module/zfs/dbuf.c

@@ -2137,7 +2137,9 @@ dbuf_release_bp(dmu_buf_impl_t *db)
 	    list_link_active(&os->os_dsl_dataset->ds_synced_link));
 	ASSERT(db->db_parent == NULL || arc_released(db->db_parent->db_buf));
 
+	mutex_enter(&db->db_mtx);
 	(void) arc_release(db->db_buf, db);
+	mutex_exit(&db->db_mtx);
 }
 
 /*

module/zfs/sa.c

@@ -1250,10 +1250,15 @@ sa_byteswap(sa_handle_t *hdl, sa_buf_type_t buftype)
 
 	db = SA_GET_DB(hdl, buftype);
 
-	if (buftype == SA_SPILL) {
+	/*
+	 * Acquire db_mtx to protect against concurrent access to the buffer
+	 * by dmu_objset_userquota_get_ids() while we perform byte-swapping.
+	 * We also need it for doing arc_release()/arc_buf_freeze().
+	 */
+	mutex_enter(&db->db_mtx);
+
+	if (buftype == SA_SPILL)
 		arc_release(db->db_buf, NULL);
-		arc_buf_thaw(db->db_buf);
-	}
 
 	sa_hdr_phys->sa_magic = BSWAP_32(sa_hdr_phys->sa_magic);
 	sa_hdr_phys->sa_layout_info = BSWAP_16(sa_hdr_phys->sa_layout_info);
@@ -1273,7 +1278,9 @@ sa_byteswap(sa_handle_t *hdl, sa_buf_type_t buftype)
 	    sa_byteswap_cb, NULL, hdl);
 
 	if (buftype == SA_SPILL)
-		arc_buf_freeze(((dmu_buf_impl_t *)hdl->sa_spill)->db_buf);
+		arc_buf_freeze(db->db_buf);
+
+	mutex_exit(&db->db_mtx);
 }
 
 static int