CBG-5184 use cbgt one shot mode by torcolvin · Pull Request #8266 · couchbase/sync_gateway

torcolvin · 2026-05-12T21:01:09Z

CBG-5184 use cbgt one shot mode

This does not yet work in multi node mode (expected) and all tests are not enabled. I'm working on why some of the tests fail or flake. I do not believe there will be major changes from the failing tests, so I am putting this up for review as is. Most tests do pass in distributed resync mode (where distributed resync is only run on a single node).

Register high sequence numbers endSeqNos with cbgt via sourceParams. This marks when cbgt will stop processing documents.
when specifying a high sequence number to kv, it will always stream sequences to the end of the snapshot, which might be higher than the specified endSeqNo. Avoid checkpointing these sequence numbers in DCPCommon.updateSeq to avoid a ERANGE error when re-opening the feed. https://github.com/couchbase/cbgt/blob/6e34ff79a0e6e97d5fb5b548c217a8a9d279a94b/feed_dcp_gocbcore.go#L1034
Add ForceCheckpointWrite when the feed shuts down to force the checkpoints to be written. This is really only helpful for tests, but DCPCommon.setMetaData will only be called on a 1 minute interval, so the last checkpoints are not set. For a test that makes sure that some documents are processed, and then resumed from that point, this test fails without this call.
cbgt.EventHandler.OnUnregisterFeed is now used to discover when vBuckets are completed. This is tracked locally (for now, see TODO) and used to close dcpDoneChan to indicate when the DCP operations should finish. There are N feeds for N partitions used by cbgt. This function is called whenever a feed (set of vBuckets for a partition) completes. This is true on reaching the endSeqNos, or when unregistering a feed (normal shutdown, such as /db/resync?action=stop)

Small fixes:

Created an options struct for DCPDestOptions to avoid having a another nil-able optino.
cbgt changed a type from string to ConsistencyLevel, this is type alias.
Allow dbExpVarsStat to be nil, especially in tests. I do not have a place for this stat yet in distributed resync.

Pre-review checklist

Removed debug logging (fmt.Print, log.Print, ...)
Logging sensitive data? Make sure it's tagged (e.g. base.UD(docID), base.MD(dbName))
Updated relevant information in the API specifications (such as endpoint descriptions, schemas, ...) in docs/api

Integration Tests

https://jenkins.sgwdev.com/job/SyncGatewayIntegration/702/

Copilot

Pull request overview

This WIP/demo PR experiments with running Sync Gateway’s cbgt-backed DCP in “one-shot” mode by providing end sequence numbers and wiring cbgt feed shutdown events back into the resync background process so it can terminate when all vbuckets are complete.

Changes:

Switches cbgt to a fork via go.mod replace and updates go.sum accordingly.
Refactors DCP destination construction to use a DCPDestOptions struct and threads one-shot end-seq handling into cbgt feed params and DCP dest/common logic.
Adds an OnUnregisterFeed callback path in sharded DCP manager event handlers and tracks completion in resync; updates resync tests/logging.

Reviewed changes

Copilot reviewed 9 out of 10 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
go.sum	Removes upstream cbgt sums and adds sums for the forked cbgt module.
go.mod	Adds a replace directive to point `github.com/couchbase/cbgt` at `github.com/torcolvin/cbgt`.
db/import_listener.go	Updates DCP dest creation to use `base.DCPDestOptions`.
db/background_mgr_resync_dcp.go	Adds sharded one-shot end-seq support, feed-unregister callback completion, and new run-state fields.
db/background_mgr_resync_dcp_test.go	Adjusts resync test behavior and enables debug logging (with leftover commented code).
base/util.go	Introduces a generic `MutexMap` helper used for completion tracking.
base/dcp_sharded.go	Extends sharded DCP options and cbgt manager event handlers to support unregister-feed callbacks and end-seq feed params.
base/dcp_feed_type.go	Extends cbgt feed params to include stop-after settings and adds `GetHighSeqNos`.
base/dcp_dest.go	Introduces `DCPDestOptions` and updates dest creation/signatures accordingly.
base/dcp_common.go	Threads one-shot end-seq handling into DCP common processing.

+	// TODO distributed resync: this value has to be serialized and evaluated in a distributed context.
+	completedVBs        base.MutexMap[uint16, struct{}]
 	useXattrs           bool


 			if distributed && base.UnitTestUrlIsWalrus() {
 				t.Skip("Distribute resync not supported for rosmar")
 			}
+			//base.SetUpTestLogging(t, base.LevelDebug, base.KeyDCP, base.KeyCRUD)


Copilot

Pull request overview

Copilot reviewed 9 out of 10 changed files in this pull request and generated 5 comments.

adamcfraser

Generally looks fine, a few comments to look at.

adamcfraser · 2026-05-27T19:41:22Z

 	defer c.m.Unlock()

+	// Check the expected maximum sequence number when running a one shot feed. Do not checkpoint if the incoming
+	// sequence is greater than the expected maximum sequence number.


Can we do this check earlier (in dataUpdate) and also avoid callback processing for any sequences higher than endSeqNo?

adamcfraser · 2026-05-27T19:44:33Z

 type SGDest interface {
 	cbgt.Dest
 	cbgt.DestEx
+	ForceCheckpointWrite()


As discussed, PR doesn't currently have any callers of ForceCheckpointWrite. Review whether that's intentional.

adamcfraser · 2026-05-27T19:57:42Z

+// doneChan when all vBuckets have completed, which will allow the resync process to finish.
+func (r *ResyncManagerDCP) getUnregisterFeedFunc(ctx context.Context, totalVBuckets uint16) base.CbgtUnregisterFeedCallback {
+	return func(feed cbgt.Feed) {
+		for _, d := range feed.Dests() {


As discussed, don't think we have a use case where our feed will have multiple Dests, but to be defensive can make this code just look for at least one SGDest to use for the ForceCheckpoint call (and exit the loop when it finds it)

adamcfraser

Changes look good - one minor question on handling the error case in shouldProcessSequence

adamcfraser · 2026-05-28T17:57:20Z

+	// DCP will provide mutations that run to the end of the snapshot that contains the end sequence number.
+	endSeq, ok := c.endSeqNos[vBucketID]
+	if !ok {
+		AssertfCtx(c.loggingCtx, "Received DCP event for vbno %d which is not tracked by the expected endSeqNos %#+v. This means that endSeqNos was specified with the incorrect number of vBuckets", vBucketID, c.endSeqNos)


In the case where the endSeq isn't found, it looks like the code will skip the mutation (endSeq will be zero for line 237). Would it be more defensive to return true in that scenario?

Copilot AI review requested due to automatic review settings May 12, 2026 21:01

torcolvin marked this pull request as draft May 12, 2026 21:01

Copilot started reviewing on behalf of torcolvin May 12, 2026 21:01 View session

Copilot AI reviewed May 12, 2026

View reviewed changes

torcolvin commented May 12, 2026

View reviewed changes

Comment thread base/dcp_common.go Outdated

torcolvin self-assigned this May 21, 2026

CBG-5184 add support for cbgt stopping after a sequence

662f58c

torcolvin force-pushed the CBG-5184 branch from 336f730 to 662f58c Compare May 27, 2026 15:02

torcolvin requested a review from Copilot May 27, 2026 15:03

Copilot started reviewing on behalf of torcolvin May 27, 2026 15:03 View session

Copilot AI reviewed May 27, 2026

View reviewed changes

Comment thread db/background_mgr_resync_dcp.go Outdated

Comment thread db/background_mgr_resync_dcp.go Outdated

Comment thread base/dcp_common.go Outdated

Comment thread base/dcp_feed_type.go

Comment thread base/dcp_sharded.go

Fix warning message

e22a3ca

torcolvin force-pushed the CBG-5184 branch from 5f13320 to e22a3ca Compare May 27, 2026 15:22

add reset function

aabbddd

torcolvin changed the title ~~WIP: CBG-5184 use cbgt one shot mode~~ CBG-5184 use cbgt one shot mode May 27, 2026

Add TODO

7113953

torcolvin marked this pull request as ready for review May 27, 2026 15:46

torcolvin requested a review from adamcfraser May 27, 2026 15:46

torcolvin assigned adamcfraser and unassigned torcolvin May 27, 2026

adamcfraser requested changes May 27, 2026

View reviewed changes

torcolvin added 4 commits May 28, 2026 11:05

Merge remote-tracking branch 'origin/main' into CBG-5184

52c68f2

Enable tests

435b6be

fix checkpointing

4339186

update comment

abcd258

adamcfraser requested changes May 28, 2026

View reviewed changes

torcolvin added 2 commits May 28, 2026 14:24

Merge remote-tracking branch 'origin/main' into CBG-5184

7376c66

setup processing sequence

6116307

Conversation

torcolvin commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pre-review checklist

Integration Tests

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

adamcfraser left a comment

Choose a reason for hiding this comment

Uh oh!

adamcfraser May 27, 2026

Choose a reason for hiding this comment

Uh oh!

adamcfraser May 27, 2026

Choose a reason for hiding this comment

Uh oh!

adamcfraser May 27, 2026

Choose a reason for hiding this comment

Uh oh!

adamcfraser left a comment

Choose a reason for hiding this comment

Uh oh!

adamcfraser May 28, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

torcolvin commented May 12, 2026 •

edited

Loading