Skip to content

Conversation

@awelless
Copy link
Contributor

@awelless awelless commented Nov 13, 2025

Summary

NIFI-15180

So far the issues which failed Scheduler initialization are caused by misconfiguration and are not recoverable.
Like inability to access a Dynamodb/Kinesis endpoint due to invalid network rules, or an inability to act on a table/stream due to absent iam permissions.

If KCL scheduler fails to initialize, the processor's initialization fails as well.

The change was tested manually.

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Pull Request Tracking

  • Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-15180
  • Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-15180

Pull Request Formatting

  • Pull Request based on current revision of the main branch
  • Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

  • Build completed using ./mvnw clean install -P contrib-check
    • JDK 21
    • JDK 25

Licensing

  • New dependencies are compatible with the Apache License 2.0 according to the License Policy
  • New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

  • Documentation formatting appears as expected in rendered files

@awelless awelless force-pushed the NIFI-15180_Log_initialization_errors_in_ConsumeKinesis branch from b6d9fc7 to f2955c9 Compare November 14, 2025 15:27

final ProcessException ex = failure.error()
.map(err -> new ProcessException("Failed to initialize the processor.", err))
.orElseGet(() -> new ProcessException("Failed to initialize the processor due to an unknown failure. Check application logs for more details."));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This branch is active only when a scheduler was shutdown, but no initialization error was provided. However, I didn't observe this behavior while testing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If that is the case, it seems worth mentioning. Mentioning "check the application logs for more details" is not helpful because it already appears in the logs, so the message should be shortened.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason behind "check the application logs for more details" is that the error text is visible in a bulletin.
When an initialization error is present, it's logged and, therefore available in the bulletin too.

When error is not available for some reason, the bulletin will just mention that initialization failed. However, additional details may be present in application logs, which aren't available in NiFi canvas.

Is it expected for the NiFi users to inspect application logs as well? If so, then the "check the application logs for more details" part is not needed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bulletin messages are limited in nature, and often require checking logs as a general practice. More recent adjustments have introduced stack trace visibility for Bulletins, but checking the logs is almost always necessary, so including the statement in a message is redundant.

@awelless awelless force-pushed the NIFI-15180_Log_initialization_errors_in_ConsumeKinesis branch from f2955c9 to d6cabe6 Compare November 14, 2025 15:49
Copy link
Contributor

@exceptionfactory exceptionfactory left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adjusting the approach and implementing waiting on successful initialization @awelless. This looks like a good way forward in general. I recommended a few adjustments.


final ProcessException ex = failure.error()
.map(err -> new ProcessException("Failed to initialize the processor.", err))
.orElseGet(() -> new ProcessException("Failed to initialize the processor due to an unknown failure. Check application logs for more details."));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If that is the case, it seems worth mentioning. Mentioning "check the application logs for more details" is not helpful because it already appears in the logs, so the message should be shortened.

Copy link
Contributor

@exceptionfactory exceptionfactory left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the responses, this looks close to completion, I noted a few more minor recommendations.

Copy link
Contributor

@exceptionfactory exceptionfactory left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working through the feedback @awelless, the latest version looks good. +1 merging

@exceptionfactory exceptionfactory merged commit 28589e4 into apache:main Nov 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants