how to send authentication parameters incase of http dispatcher

[ganeshnikumbh]

Hello,

I am using spline with Databricks and sending the lineage to a Azure function using http lineage dispatcher. I want to know how to configure authentication (basic or azure function access keys) to send authorized request to Azure function.

[cerveada]

Hello,
there is no built in support for Azure authentication. I guess you probably need to set some http headers. It should be relatively easy to extends/modify the existing httpDispatcher to do what you want.

Here is some documentation about dispatchers:
https://github.com/AbsaOSS/spline-spark-agent#lineage-dispatchers

You probably want to start from HttpLineageDispatcher class and also look into RestClient and RestEndpoint.

[SP-ARUN]

@ganeshnikumbh did you get your result?

I believe, you were looking for a result which im searching for now. Kindly share you thoughts here, in case if you have got your expected result.

Looking forward.

[ganeshnikumbh]

@SP-ARUN we haven't tried anything yet. This is still being discussed in the team for now.. I will let you know if we find/build any solution as per @cerveada 's reply..

[wajda]

If you about writing a custom lineage dispatcher and don't know where to start from, you might find this seed/example project helpful. Just clone it and add your code. https://github.com/AbsaOSS/spline-getting-started/tree/main/spark-agent-extension-example