Improve dataplane logs to allow better transfer process traceability

[bmg13]

WHAT

Allow consistent request tracking across control and data plane within one Connector of one company.

Currently, there's a disconnect between logs in control and data plane which makes it hard to debug and trace requests when there's a lot of traffic in the EDC. This feature looks into improving request tracking by e.g. adding TransferProcess Identifiers to data plane logs.

WHY

Increased usability due to better log readability and transfer process observability.

HOW

Initially, the proposal was to pull the dataplane-core module from Upstream to Tractus-X and include the transfer Process ID to the data plane logs (both success and failing scenarios). However, after given feedback, the proposal will be to simply update the logs in Upstream.

[bmg13]

@rafaelmag110, @ndr-brt or @lgblaumeiser can any of you assign this issue to me, please?

[ndr-brt]

Pull the dataplane-core module from Upstream to Tractus-X and include the transfer Process ID to the data plane logs (both success and failing scenarios).

please don't do that, if there are missing log info we could just go upstream, replicating a whole module downstream is definitely something we don't want to, as it will make maintenance harder and create divergence with upstream.

Please provide a list of log messages before with code lines where this should be done and we can propose it upstream.

[bmg13]

Pull the dataplane-core module from Upstream to Tractus-X and include the transfer Process ID to the data plane logs (both success and failing scenarios).

please don't do that, if there are missing log info we could just go upstream, replicating a whole module downstream is definitely something we don't want to, as it will make maintenance harder and create divergence with upstream.

Please provide a list of log messages before with code lines where this should be done and we can propose it upstream.

I understand and close the PR adding this change and updated this issue's description accordingly. Do we want me to create the issue Upstream?

The change in Upstream would be simple, in PipelineServiceImpl.java considering this log regarding transfer from a source to a destination can be changed to include the DataFlow Id, being the following an example of the new outcome.

DataFlow 7926e16e-7357-4032-8078-db08ecd9d67f Transferring from AmazonS3 to AmazonS3.

Additionally the error logs for unknown data source and unknown data sink can be updated to also improve traceability. The following are examples of those.

DataFlow 7926e16e-7357-4032-8078-db08ecd9d67f Unknown data source type AmazonS3.

DataFlow 7926e16e-7357-4032-8078-db08ecd9d67f Unknown data sink type AmazonS3.

[ndr-brt]

this is definitely something to be proposed upstream, is a 3 line change. Feel free to open the issue there.

[bmg13]

Issue created upstream and this one is closed due to it.