Skip to content

Make step param separator configurable to allow __ in step IDs and param names#201

Merged
akashdw merged 4 commits intomainfrom
ad/configurable-step-param-separator
Apr 6, 2026
Merged

Make step param separator configurable to allow __ in step IDs and param names#201
akashdw merged 4 commits intomainfrom
ad/configurable-step-param-separator

Conversation

@akashdw
Copy link
Copy Markdown
Collaborator

@akashdw akashdw commented Apr 1, 2026

Pull Request type

  • Bugfix
  • [ x] Feature
  • Refactoring (no functional changes, no api changes)
  • Build related changes (Please run ./gradlew build --write-locks to refresh dependencies)
  • Other (please describe):

NOTE: Please remember to run ./gradlew spotlessApply to fix any format violations.

Changes in this PR

The step parameter separator (default __) used in cross-step references like step1__param was previously hardcoded. As a result, any step ID or parameter name containing __ was rejected by validation.

This PR makes the separator configurable via maestro.param-evaluator.step-param-separator. Netflix OSS users keep the existing behavior (__ by default). Teams can now choose a different separator (e.g., ___), allowing __ to appear freely in step IDs and parameter names without triggering validation errors.

* @param conn the connection
* @throws SQLException sql exception
*/
protected void markTransactionSerializable(Connection conn) throws SQLException {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this deletion intentional?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes to fix the build check. markTransactionSerializable is added twice in this file.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, unrelated but this is to fix the broken build.


/** Convenience constructor using the default step param separator {@code __}. */
public ParamEvaluator(ExprEvaluator exprEvaluator, ObjectMapper objectMapper) {
this(exprEvaluator, objectMapper, "__");
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Would it make sense to keep this a private final and call it DEFAULT_STEP_PARAM_SEPARATOR? I always find it weird to pass in string literals for defaults.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively I think we could use both @AllArgsConstructor and @RequiredArgsConstructor from lombok and set private final String stepParamSeparator = "__";

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both annotations generate the same 2-arg constructor when stepParamSeparator has an initializer, Lombok's @AllArgsConstructor also skips initialized final fields.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems so dumb 🤦🏻

@@ -353,8 +358,8 @@ private Map<String, Parameter> getReferencedParams(
/** Extract step id and param name from the reference. It handles `__`, `___`, and `____`. */
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these javadocs accurate anymore?

@Getter
@Setter
public class ParamEvaluatorProperties implements StepParamSeparator {
private String stepParamSeparator = "__";
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If possible add DEFAULT_STEP_PARAM_SEPARATOR in constant and refer it here as well as in MaestroIdValidator instead of creating DEFAULT_STEP_PARAM_SEPARATOR variable there.

Copy link
Copy Markdown
Collaborator

@praneethy91 praneethy91 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks good, clean extraction of a hardcoded separator into a configurable property with sensible defaults.

int idx2 = refParam.lastIndexOf(STEP_PARAM_SEPARATOR);
int idx1 = refParam.indexOf(stepParamSeparator);
int idx2 = refParam.lastIndexOf(stepParamSeparator);
// ___ or ____ case
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The inline comment on the idx1 != idx2 branch still says // ___ or ____ case, which assumes __ as the separator. The Javadoc can be updated. Suggest updating to something like // separator overlap case to stay accurate regardless of configured separator.

private final StepActionProperties stepAction;

/** Returns the param evaluator properties, defaulting to {@code __} separator if not set. */
public ParamEvaluatorProperties getParamEvaluator() {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The custom getter that overrides Lombok's @Getter to return a default when null works, but the other properties (queue, sel, stepAction) don't do this. Consider whether initializing the field directly at initialization time or using @DefaultValue would be more consistent.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The asymmetry is intentional. queue, sel, and stepAction are pre-existing required fields, if absent the app should fail to start. paramEvaluator is a new optional field added for backward compatibility, so it needs a sensible default when not configured. I considered both alternatives:

  • @DefaultValue requires dropping @AllArgsConstructor and writing a manual constructor which seems more disruptive
  • Field initializer doesn't work since @AllArgsConstructor generates a constructor that overwrites it, so Spring Boot's constructor binding still passes null when the property is absent

Happy to switch to a manual constructor with @DefaultValue if you prefer consistency over brevity.

@akashdw akashdw merged commit f6ed55c into main Apr 6, 2026
1 check passed
@akashdw akashdw deleted the ad/configurable-step-param-separator branch April 6, 2026 20:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants