Support azure-devops-oidc for serviceConnection name instead of pipeline name

[abij]

We encountered an issue that the Federation Policy for AzureDevOps must be based on the pipeline name.

Current working situation

I think we should get a medal for getting the azure-devops-oidc actually working.

When you read this, you are also wondering how to get the org-id . You cannot find this anyware in the Azure DevOps interface, but simple solution is to run a build-pipeline with System Diagnostics enabled and look for the property: SYSTEM_COLLECTIONID, yes thats the one you need! Using this ID you can created the Federation Policy for Databricks Account:

Content of my oidc_policy.json:

{
  "oidc_policy": {
    "issuer": "https://vstoken.dev.azure.com/<org GUID>",
    "audiences": ["api://AzureADTokenExchange"],
    "subject": "p://<Org name>/<Project>/<Pipeline name>"
  }
}

And executing using:
databricks account service-principal-federation-policy create <databricks-id-of-spn> --json @oidc_policy.json

Requested way of working

We would like to have a policy based on the name of the ServiceConnection sc, not on the name of the pipeline p. This way we can add a federation policy looking like this:

Content of my oidc_policy.json:

{
  "oidc_policy": {
    "issuer": "https://vstoken.dev.azure.com/<org GUID>",
    "audiences": ["api://AzureADTokenExchange"],
    "subject": "sc://<Org name>/<Project>/<ServiceConnection name>"
  }
}

I think it's a matter of adding the optional property serviceConnectionId as referenced here: https://learn.microsoft.com/en-us/rest/api/azure/devops/distributedtask/oidctoken/create?view=azure-devops-rest-7.1.

The code which should be updated:

databricks-sdk-go/config/experimental/auth/oidc/azuredevops.go

Line 83 in fd4721f

requestUrl := fmt.Sprintf("%s/%s/_apis/distributedtask/hubs/%s/plans/%s/jobs/%s/oidctoken?api-version=7.2-preview.1",