-
Notifications
You must be signed in to change notification settings - Fork 3.2k
enhance: Optimize shard serviceable mechanism #41937
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
enhance: Optimize shard serviceable mechanism #41937
Conversation
e91be31
to
cb3de2d
Compare
@weiliu1031 go-sdk check failed, comment |
@weiliu1031 E2e jenkins job failed, comment |
cb3de2d
to
17973bd
Compare
serviceable := checkDelegatorServiceable(ctx, dh, dmChannel.View) | ||
// trigger pull next target until shard leader is ready | ||
if !serviceable { | ||
dh.lastUpdateTs = 0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lastUpdateTs is not guarded by lock
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
each querynode has it's own dist_handler, and dist_handler pull data distribution in serial, the lock is unnecessary
zap.String("channel", view.Channel), | ||
zap.Error(err)) | ||
if status := view.Status; status != nil { | ||
if !status.GetServiceable() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no need to GetServiceable twice
17973bd
to
1f7d364
Compare
@weiliu1031 E2e jenkins job failed, comment |
- Merge leader view and channel management into ChannelDistManager, allowing a channel to have multiple delegators. - Improve shard leader switching to ensure a single replica only has one shard leader per channel. The shard leader handles all resource loading and query requests. - Refine the serviceable mechanism: after QC completes loading, sync the query view to the delegator. The delegator then determines its serviceable status based on the query view. - When a delegator encounters forwarding query or deletion failures, mark the corresponding segment as offline and transition it to an unserviceable state. Signed-off-by: Wei Liu <[email protected]>
1f7d364
to
82eac09
Compare
/lgtm |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: weiliu1031, xiaofan-luan The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
issue: milvus-io#42098 milvus-io#42404 related to: #milvus-io#42009 milvus-io#41937 Implement new method to handle partition removal from next target without directly modifying current target. Changes include: - Add RemovePartitionFromNextTarget method and deprecate RemovePartition - Update target_observer to use new method for ReleasePartition operations - Add unit tests and mock methods for new functionality This ensures that all changes to next target will propagates to delegator's query view. Signed-off-by: Wei Liu <[email protected]>
issue: milvus-io#42098 milvus-io#42404 related to: #milvus-io#42009 milvus-io#41937 Implement new method to handle partition removal from next target without directly modifying current target. Changes include: - Add RemovePartitionFromNextTarget method and deprecate RemovePartition - Update target_observer to use new method for ReleasePartition operations - Add unit tests and mock methods for new functionality This ensures that all changes to next target will propagates to delegator's query view. Signed-off-by: Wei Liu <[email protected]>
issue: milvus-io#42098 milvus-io#42404 related to: #milvus-io#42009 milvus-io#41937 Implement new method to handle partition removal from next target without directly modifying current target. Changes include: - Add RemovePartitionFromNextTarget method and deprecate RemovePartition - Update target_observer to use new method for ReleasePartition operations - Add unit tests and mock methods for new functionality This ensures that all changes to next target will propagates to delegator's query view. Signed-off-by: Wei Liu <[email protected]>
issue: #41690