feat: Add individual task telemetry to dumper system #949
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
kaxuna
commented
Aug 27, 2025
kaxuna
commented
- Create TaskRunMetrics class for individual task timing and error tracking
- Enhance TasksRunner to capture start/end times and errors for each task
- Update MetadataDumper to pass telemetry processor to TasksRunner
- Create TaskRunMetrics class for individual task timing and error tracking - Enhance TasksRunner to capture start/end times and errors for each task - Add addTaskTelemetry method to TelemetryProcessor - Update MetadataDumper to pass telemetry processor to TasksRunner - Add comprehensive tests for new telemetry functionality - Maintain backward compatibility with existing telemetry system This enhancement provides granular visibility into individual task execution performance and error details, complementing the existing overall runtime metrics.
kaxuna
requested review from
shevek-google and
vladislav-sidorovich
as code owners
dumper/app/src/main/java/com/google/edwmigration/dumper/application/dumper/TasksRunner.java
Outdated
Show resolved
Hide resolved
dumper/app/src/main/java/com/google/edwmigration/dumper/application/dumper/TasksRunner.java
Outdated
Show resolved
Hide resolved
dumper/app/src/main/java/com/google/edwmigration/dumper/application/dumper/TasksRunner.java
Outdated
Show resolved
Hide resolved
|
|
||
| private void addTaskTelemetry( | ||
| String taskName, Instant startTime, Instant endTime, TaskState state) { | ||
| if (telemetryProcessor != null) { |
Member
There was a problem hiding this comment.
Let's add Precondition to constructor and remove this if.
dumper/app/src/main/java/com/google/edwmigration/dumper/application/dumper/TasksRunner.java
Outdated
Show resolved
Hide resolved
| */ | ||
| public class TaskRunMetrics implements TelemetryPayload { | ||
|
|
||
| @JsonProperty private String name; |
Member
There was a problem hiding this comment.
I thinks we need both, taskName and taskClass. Another option is to redesign somehow our hierarchy of connectors and tasks.
|
|
||
| @JsonProperty private String name; | ||
|
|
||
| @JsonProperty private EventType eventType = EventType.TASK_RUN_METRICS; |
Member
There was a problem hiding this comment.
It should be on level above, before payload.
The logic on the service to parse event should be something like this:
EventType type = event.getType();
if (type is A) {
//payload is audi, do audi specific
} else if (type is B) {
//payload is bwm, do bwm specific
}
.../src/main/java/com/google/edwmigration/dumper/application/dumper/metrics/TaskRunMetrics.java
Outdated
Show resolved
Hide resolved
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.