NIFI-15156 Add support for discovering Listen Ports

[kevdoran]

Summary

NIFI-15156

Depends on apache/nifi-api#26 (NIFI-15155)

• Update Listen Processors and Controller Services to implement ListenComponent interface
• Update Listen Processors and Controller Services to identify ListenPortDefinition properties
• Add draft REST API Controller, Entity, and DTO for listing Listen Ports
• Refactor ListenSyslog so that the transport protocol for a given port property is static and no longer configurable at runtime while still allowing users to choose between UDP and TCP for the processor.
• Update Manifest serializers and deserializers to include ListenPortDefinition for PropertyDescriptors

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

• Apache NiFi Jira issue created

Pull Request Tracking

• Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
• Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

• Pull Request based on current revision of the main branch
• Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

• Build completed using ./mvnw clean install -P contrib-check
  • JDK 21
  • JDK 25

Licensing

• New dependencies are compatible with the Apache License 2.0 according to the License Policy
• New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

• Documentation formatting appears as expected in rendered files

[exceptionfactory]

Thanks for drafting these changes @kevdoran.

It looks like there are a number of dependency version regressions in the parent pom.xml that need to be reverted.

To keep of the scope of changes more manageable, it might be helpful to decouple the ListenSyslog property changes into a separate pull request. That would also serve to highlight those changes as different than just adding the new mapping.

I plan to review more closely after reviewing the API changes, but thought it would be helpful to note these items for now.

[kevdoran]

I simplified this branch to have the bare minimum for the following:

• Include Listen Port definitions in extension manifests, flow definitions, versioned flows, etc.
• REST API is able to report Listen Ports the framework can discover that the user is authorized for
• Framework is able to discover Listen Ports from components that implement new ListenComponent interface
• ListenHTTP is updated to support new interfaces for testing all of the above

I restructured the commits so that they are individually reviewable for each of the areas above.

To test the functionality:

1. Build this branch (skip contrib-check, I still have some cleanup to do_
2. Observe that the manifest file for ListenHTTP nifi-extension-bundles/nifi-standard-bundle/nifi-standard-nar/target/META-INF/docs/extension-manifest.xml includes the ListenPort definition in PropertyDescriptors:

# nifi-extension-bundles/nifi-standard-bundle/nifi-standard-nar/target/META-INF/docs/extension-manifest.xml
<property>
	<name>Listening Port</name>
	<displayName>Listening Port</displayName>
	<description>The Port to listen on for incoming connections</description>
	<required>true</required>
	<sensitive>false</sensitive>
	<expressionLanguageSupported>true</expressionLanguageSupported>
	<expressionLanguageScope>ENVIRONMENT</expressionLanguageScope>
	<dynamicallyModifiesClasspath>false</dynamicallyModifiesClasspath>
	<dynamic>false</dynamic>
	<listenPortDefinition>
		<transportProtocol>TCP</transportProtocol>
		<applicationProtocols>
			<applicationProtocol>http/1.1</applicationProtocol>
			<applicationProtocol>h2</applicationProtocol>
		</applicationProtocols>
	</listenPortDefinition>
</property>
<property>
	<name>health-check-port</name>
	<displayName>Listening Port for Health Check Requests</displayName>
	<required>false</required>
	<sensitive>false</sensitive>
	<expressionLanguageSupported>true</expressionLanguageSupported>
	<expressionLanguageScope>ENVIRONMENT</expressionLanguageScope>
	<dynamicallyModifiesClasspath>false</dynamicallyModifiesClasspath>
	<dynamic>false</dynamic>
	<listenPortDefinition>
		<transportProtocol>TCP</transportProtocol>
		<applicationProtocols>
			<applicationProtocol>http/1.1</applicationProtocol>
			<applicationProtocol>h2</applicationProtocol>
		</applicationProtocols>
	</listenPortDefinition>
</property>

3. Start NiFi and go to NiFi UI in browser: https://localhost:8443/nifi

NIFI_DIR=nifi-assembly/target/nifi-2.7.0-SNAPSHOT-bin/nifi-2.7.0-SNAPSHOT && ./$NIFI_DIR/bin/nifi.sh set-single-user-credentials admin ***** && $NIFI_DIR/bin/nifi.sh start && tail -f $NIFI_DIR/logs/nifi-*.log

4. Add one or more ListenHTTP Processors to the canvas. Try nesting them in Process Groups. Configure each to have Port property and optionally and Health Check port as well.

5. In the same browser (for auth cookie reasons) go to this new NiFi REST API url: https://localhost:8443/nifi-api/flow/listen-ports - you should see something like the following:

{
  "listenPorts": [
    {
      "portName": "Listening Port",
      "portNumber": 1111,
      "transportProtocol": "TCP",
      "applicationProtocols": [
        "http/1.1"
      ],
      "componentType": "Processor",
      "componentId": "4f328cb9-019a-1000-8f20-3fdb4fb59c8c",
      "componentName": "ListenHTTP",
      "componentClass": "org.apache.nifi.processors.standard.ListenHTTP",
      "parentGroupId": "4f31a853-019a-1000-25c7-55eeb0410bae",
      "parentGroupName": "NiFi Flow"
    },
    {
      "portName": "Listening Port for Health Check Requests",
      "portNumber": 2222,
      "transportProtocol": "TCP",
      "applicationProtocols": [
        "http/1.1"
      ],
      "componentType": "Processor",
      "componentId": "4f328cb9-019a-1000-8f20-3fdb4fb59c8c",
      "componentName": "ListenHTTP",
      "componentClass": "org.apache.nifi.processors.standard.ListenHTTP",
      "parentGroupId": "4f31a853-019a-1000-25c7-55eeb0410bae",
      "parentGroupName": "NiFi Flow"
    }
  ]
}

6. Download the flow definition JSON. Observe that the processor property descriptors contain the listen port definitions:

"health-check-port": {
    "name": "health-check-port",
    "displayName": "Listening Port for Health Check Requests",
    "identifiesControllerService": false,
    "sensitive": false,
    "dynamic": false,
    "listenPortDefinition":
    {
        "transportProtocol": "TCP",
        "applicationProtocols":
        [
            "http/1.1",
            "h2"
        ]
    }
},
"Listening Port": {
    "name": "Listening Port",
    "displayName": "Listening Port",
    "identifiesControllerService": false,
    "sensitive": false,
    "dynamic": false,
    "listenPortDefinition":
    {
        "transportProtocol": "TCP",
        "applicationProtocols":
        [
            "http/1.1",
            "h2"
        ]
    }
}

[exceptionfactory]

Thanks for the work on the implementation @kevdoran. I noted a few very minor details, but otherwise this should be ready to go after a few adjustments.

[kevdoran]

Thanks very much for the review @exceptionfactory! I agreed with your suggestions, and have pushed a commit that should address everything.

[exceptionfactory]

Thanks again for the work on this @kevdoran, looks good! +1 merging