fix(examples/jobs): wait on job triggers instead of wall-clock time - #834
ihopenre-eng wants to merge 1 commit into
Conversation
The jobs example slept for a fixed 3 seconds after scheduling the job and then asserted that three triggers had already been printed. That leaves no margin for a sidecar that took longer to initialise, and none at all for a trigger that failed and had to be retried: the constant failure policy used a 30 second retry interval, which is longer than the whole example lives. A single transient error therefore drops "job 0/1/2 received" from the output and fails validate-examples. Wait on the triggers themselves, with a timeout, so the example proceeds as soon as the job has fired the expected number of times and the ordering of the trigger output relative to getjob is deterministic. Lower the retry interval so a failed trigger can actually be retried within the run, and raise the background step timeout in the README to absorb variable sidecar start-up time. Also make the trigger counter atomic, since it is written from the handler goroutine. Closes dapr#805 Signed-off-by: ihopenre-eng <247072151+ihopenre-eng@users.noreply.github.com>
|
The CI workflows on this PR are queued at Could a maintainer approve the workflow runs when convenient? Happy to rebase if anything has drifted in the meantime. |
| @@ -18,6 +19,14 @@ import ( | |||
|
|
|||
There was a problem hiding this comment.
Added expectedJobRuns constant; makes expected count explicit and reusable.
|
|
||
| var logger = log.New(os.Stdout, "", log.LstdFlags) | ||
|
|
||
| // expectedJobRuns is the number of triggers this example waits for before it |
There was a problem hiding this comment.
Added jobRuns channel with buffer size matching expected count; signals trigger receipt from handler goroutine.
| daprc.WithJobConstantFailurePolicy(), | ||
| daprc.WithJobConstantFailurePolicyMaxRetries(4), | ||
| daprc.WithJobConstantFailurePolicyInterval(time.Second*30), | ||
| // The retry interval has to stay well below the lifetime of this |
There was a problem hiding this comment.
Lowered retry interval from 30s to 1s; ensures retries can complete within example timeout.
| fmt.Println("schedulejob - success") | ||
|
|
||
| time.Sleep(3 * time.Second) | ||
| // Wait for the job to actually fire. Sleeping for a fixed duration instead |
There was a problem hiding this comment.
Replaced fixed sleep with waitForJobRuns using channel and timeout; more robust to sidecar init time and transient failures.
| @@ -97,14 +114,35 @@ func main() { | |||
| } | |||
There was a problem hiding this comment.
Added waitForJobRuns function; properly uses timer with defer Stop() and select for timeout handling.
| select { | ||
| case <-jobRuns: | ||
| case <-deadline.C: | ||
| return fmt.Errorf("timed out after %s waiting for %d job triggers, got %d", timeout, count, i) |
There was a problem hiding this comment.
Changed jobCount to atomic.Int64; correct for concurrent writes from handler goroutine. Usage Add(1)-1 prints current count before increment.
| } | ||
| fmt.Printf("job %d received:\n type: %v \n payload: %v\n", jobCount, job.JobType, jobPayload) | ||
| jobCount++ | ||
| fmt.Printf("job %d received:\n type: %v \n payload: %v\n", jobCount.Add(1)-1, job.JobType, jobPayload) |
There was a problem hiding this comment.
Non-blocking send to jobRuns in handler; prevents slow response from being treated as failure by Dapr sidecar.
|
|
||
| background: true | ||
| sleep: 30 | ||
| sleep: 45 |
There was a problem hiding this comment.
Raised background sleep from 30s to 45s; accommodates longer wait for trigger channel.
2 participants
Description
Fixes the flaky
validate-example (jobs)job reported in #805.The example sleeps for a fixed 3 seconds after scheduling the job and then relies on three triggers having already been printed. Two things make that unreliable:
No margin for a retry. The job is created with
WithJobConstantFailurePolicyInterval(time.Second*30). In the linked failing run the first trigger came backstatus code returned: 14(Unavailable), so the next attempt was scheduled 30 seconds later, long aftermainhad already deleted the job and exited. Thejob 0/1/2 receivedlines never appeared:No margin for a slow sidecar. That run logged
dapr initialized. Status: Running. Init Elapsed 20044ms, and the whole step is capped atsleep: 30. The 10 second intermission plus 3 second sleep leaves the triggers almost no room inside the window.Because
getjobis only ordered after the trigger output by that sleep,match_order: sequentialis also only satisfied by chance today.Changes
time.Sleep(3 * time.Second). The example continues as soon as the job has fired the expected number of times, and the trigger output is now guaranteed to precedegetjob.sleep: 30tosleep: 45to absorb variable sidecar start-up time.atomic.Int64; it is written from the handler goroutine.Expected stdout in the README is unchanged.
Issue reference
Closes #805
Checklist
go build ./jobs/...,go vet ./jobs/...,gofmtclean)