Admin cutover improve the pod label watch function#881
Conversation
Code Review Agent Run #329c1bActionable Suggestions - 0Review Details
Bito Usage GuideCommands Type the following command in the pull request comment and save the comment.
Refer to the documentation for additional commands. Configuration This repository uses Documentation & Help |
Changelist by BitoThis pull request implements the following key changes.
|
There was a problem hiding this comment.
Code Review Agent Run #0bb55a
Actionable Suggestions - 1
-
v2v-helper/reporter/reporter.go - 1
- Deferred watch.Stop() won't execute per iteration · Line 255-255
Review Details
-
Files reviewed - 1 · Commit Range:
957c074..980a003- v2v-helper/reporter/reporter.go
-
Files skipped - 0
-
Tools
- Golangci-lint (Linter) - ✖︎ Failed
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful
Bito Usage Guide
Commands
Type the following command in the pull request comment and save the comment.
-
/review- Manually triggers a full AI review. -
/pause- Pauses automatic reviews on this pull request. -
/resume- Resumes automatic reviews. -
/resolve- Marks all Bito-posted review comments as resolved. -
/abort- Cancels all in-progress reviews.
Refer to the documentation for additional commands.
Configuration
This repository uses Default Agent You can customize the agent settings here or contact your Bito workspace admin at mithil@platform9.com.
Documentation & Help
| watch, err := r.Clientset.CoreV1().Pods(r.PodNamespace).Watch(ctx, metav1.ListOptions{ | ||
| FieldSelector: fmt.Sprintf("metadata.name=%s", r.PodName), | ||
| TimeoutSeconds: &timeoutSeconds, | ||
| }) | ||
| if err != nil { | ||
| fmt.Printf("Error: Failed to start watch for pod %s: %v\n", r.PodName, err) | ||
| time.Sleep(5 * time.Second) | ||
| continue | ||
| } |
There was a problem hiding this comment.
It will hog resources on the pod
There was a problem hiding this comment.
This only creates one watcher and the inner loop is a blocking for loop which does not exits until the channel gets closed, hence only one watch is present at a given particular time. the outer for loop is to ensure that if a channel gets closed due to some reason we create a new watch so we don't miss any update.
OmkarDeshpande7
left a comment
There was a problem hiding this comment.
Looks Good.
Please add more testing details. Possibly snippet/video that indicates adminCutOver was triggered after waiting for at-least 7-8 hours.
|
Added the logs and snippets in the issue and in the testing notes as well. |
|
Bito Automatic Review Failed - Technical Failure |
What this PR does / why we need it
This PR enhances pod label watch functionality with improved logging, error handling, and timeout mechanisms. It implements better handling for non-pod events, channel blockages, and adds a 24-hour timeout. The changes include graceful context cancellation handling and systematic retry mechanisms, significantly improving the reliability and observability of the pod cutover process.
Which issue(s) this PR fixes
(optional, in
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close the issue(s) when PR gets merged)fixes #877
Special notes for your reviewer
Testing done
As you can see in the screenshot the migration for centos6 was crearted at 09/09/2025, 01:28 and time elapsed for this migration to finsih was 9h 22m,
Logs of the migration:
2025/09/08 21:09:04 Admin initiated cutover detected, skipping changed blocks copy
Info: Received event for pod v2v-helper-centos-6-vj-test-7f595-4b125-595ts: type=MODIFIED
2025/09/08 21:09:04 Waiting for Admin Cutover conditions to be met
Info: Received event for pod v2v-helper-centos-6-vj-test-7f595-4b125-595ts: type=MODIFIED
2025/09/09 05:21:15 Label: yes
Info: Received event for pod v2v-helper-centos-6-vj-test-7f595-4b125-595ts: type=MODIFIED
Info: Label changed for pod v2v-helper-centos-6-vj-test-7f595-4b125-595ts: no -> yes
Info: Sent label yes for pod v2v-helper-centos-6-vj-test-7f595-4b125-595ts to channel
2025/09/09 05:21:15 Cutover conditions met
2025/09/09 05:21:15 Shutting down source VM and performing final copy
migration was waiting for cutover from 21:09:04 from 8th of sept to 05:21:15 9th sept which is more than 9hours and the admicutover was succesfully done. hence the fix we have verified it works.
please add testing details (logs, screenshots, etc.)
Summary by Bito
This PR enhances pod label watch functionality with improved logging, error handling, and timeout mechanisms. It implements better handling for non-pod events, channel blockages, and adds a 24-hour timeout. The changes include graceful context cancellation handling and systematic retry mechanisms, significantly improving the reliability and observability of the pod cutover process.