Issue #3003 : Add Output transform metrics transform - #8178
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Port Kettle Output steps metrics as a Hop-native Output transform metrics transform.
Fixes #3003
What
A local-engine generator that waits for listed sibling transform copies to finish, then emits one metrics row per copy (name, plugin id, line counters, rejected rows, duration). Incoming hops are not consumed and are treated as an error because they can deadlock the pipeline.
Why
Pipeline Logging / Execution Information cover project-wide monitoring, but they do not replace this in-pipeline transform for Kettle/PDI migrations.
How
plugins/transforms/transformsmetrics, plugin idTransformsMetrics(deprecated extra idStepsMetrics).@HopMetadataPropertywith Hoptransform*XML keys. The dialog uses groupedGuiCompositeWidgetstabs plus aTableViewextra group.BlockUntilTransformsFinishwait semantics, then emits metrics instead of passing rows through.StepsMetrics→TransformsMetricsandstep*tags →transform*(including the Kettlesteplineswrittentfieldtypo). Stored output field values are left unchanged.Tests
getFields,check()(TransformsMetricsMetaTest)KettleImportTransformsMetricsTest)0110-output-transform-metricswith a golden datasetDocs
User-manual page, sample pipeline, Transforms and Fields tab screenshots.
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
mvn clean install apache-rat:checkto make sure basic checks pass. A more thorough check will be performed on your pull request automatically.git rebase -i.addresses #123), if applicable.To make clear that you license your contribution under the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.