From e0c92f52ff58305f23035ad1a485afd7d9886f38 Mon Sep 17 00:00:00 2001 From: adamrtalbot <12817534+adamrtalbot@users.noreply.github.com> Date: Thu, 9 Jul 2026 15:00:31 +0100 Subject: [PATCH 1/2] Add managed identity support for Azure Batch container registries Allow authenticating to a private container registry with a user-assigned managed identity via azure.registry.managedIdentityResourceId (or the AZURE_REGISTRY_MANAGED_RESOURCE_ID environment variable), as an alternative to userName/password. When set, the managed identity takes precedence. Generated by Claude Code Signed-off-by: adamrtalbot <12817534+adamrtalbot@users.noreply.github.com> --- docs/azure.mdx | 17 +++++++++ docs/reference/config.mdx | 8 ++++ .../cloud/azure/batch/AzBatchService.groovy | 18 ++++++--- .../cloud/azure/config/AzRegistryOpts.groovy | 15 +++++++- .../azure/batch/AzBatchServiceTest.groovy | 38 +++++++++++++++++++ .../azure/config/AzRegistryOptsTest.groovy | 34 +++++++++++++++++ 6 files changed, 124 insertions(+), 6 deletions(-) diff --git a/docs/azure.mdx b/docs/azure.mdx index d931b16205..c562ee25cc 100644 --- a/docs/azure.mdx +++ b/docs/azure.mdx @@ -550,6 +550,23 @@ Nextflow uses the following environment variables if the registry credentials ar - `AZURE_REGISTRY_PASSWORD`: the password for Azure Container Registry authentication ::: +Alternatively, you can authenticate to a private registry with a user-assigned [managed identity](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview) instead of a username and password: + +```groovy +azure { + registry { + server = '' // e.g., 'myregistry.azurecr.io' + managedIdentityResourceId = '' // e.g., '/subscriptions//resourceGroups//providers/Microsoft.ManagedIdentity/userAssignedIdentities/' + } +} +``` + +The value is the full ARM resource ID of the managed identity, not its client ID. When set, it takes precedence over `userName` and `password`, and can also be provided via the `AZURE_REGISTRY_MANAGED_RESOURCE_ID` environment variable. + +:::note +The managed identity must already be attached to the pool nodes. Nextflow cannot attach an identity when auto-creating a pool, so this option requires a pre-existing pool (or an identity attached out-of-band). +::: + Container images from public registries such as Docker Hub can be used without additional configuration, even when a private registry is specified in the `azure.registry` scope. The Docker runtime on Azure Batch VMs automatically uses the appropriate registry based on the container image specified for the task. ### VM images diff --git a/docs/reference/config.mdx b/docs/reference/config.mdx index 5add615d87..7918110153 100644 --- a/docs/reference/config.mdx +++ b/docs/reference/config.mdx @@ -605,6 +605,14 @@ The client ID for an Azure [managed identity](https://learn.microsoft.com/en-us/ When `true`, use the system-assigned [managed identity](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview) to authenticate Azure resources. Defaults to environment variable `AZURE_MANAGED_IDENTITY_SYSTEM`. +##### `azure.registry.managedIdentityResourceId` + +The ARM resource ID of a user-assigned managed identity used to authenticate to a private container registry, e.g. `/subscriptions//resourceGroups//providers/Microsoft.ManagedIdentity/userAssignedIdentities/`. When set, it takes precedence over `azure.registry.userName` and `azure.registry.password`. Defaults to environment variable `AZURE_REGISTRY_MANAGED_RESOURCE_ID`. + +:::note +The managed identity must already be attached to the pool nodes. Nextflow cannot attach an identity when auto-creating a pool, so this option requires a pre-existing pool (or an identity attached out-of-band). +::: + ##### `azure.registry.password` The password to connect to a private container registry. diff --git a/plugins/nf-azure/src/main/nextflow/cloud/azure/batch/AzBatchService.groovy b/plugins/nf-azure/src/main/nextflow/cloud/azure/batch/AzBatchService.groovy index b35fc0c41f..c1091f535c 100644 --- a/plugins/nf-azure/src/main/nextflow/cloud/azure/batch/AzBatchService.groovy +++ b/plugins/nf-azure/src/main/nextflow/cloud/azure/batch/AzBatchService.groovy @@ -35,6 +35,7 @@ import com.azure.compute.batch.models.BatchJobCreateContent import com.azure.compute.batch.models.BatchJobConstraints import com.azure.compute.batch.models.BatchJobUpdateContent import com.azure.compute.batch.models.BatchNodeFillType +import com.azure.compute.batch.models.BatchNodeIdentityReference import com.azure.compute.batch.models.BatchPool import com.azure.compute.batch.models.BatchPoolCreateContent import com.azure.compute.batch.models.BatchPoolInfo @@ -847,13 +848,20 @@ class AzBatchService implements Closeable { final registryOpts = config.registry() if( registryOpts && registryOpts.isConfigured() ) { - final containerRegistries = new ArrayList(1) - containerRegistries << new ContainerRegistryReference() + final registryRef = new ContainerRegistryReference() .setRegistryServer(registryOpts.server) - .setUsername(registryOpts.userName) - .setPassword(registryOpts.password) + if( registryOpts.usesManagedIdentity() ) { + registryRef.setIdentityReference( new BatchNodeIdentityReference().setResourceId(registryOpts.managedIdentityResourceId) ) + log.debug "[AZURE BATCH] Connecting Azure Batch pool to Container Registry '$registryOpts.server' using managed identity '$registryOpts.managedIdentityResourceId'" + } + else { + registryRef.setUsername(registryOpts.userName) + .setPassword(registryOpts.password) + log.debug "[AZURE BATCH] Connecting Azure Batch pool to Container Registry '$registryOpts.server'" + } + final containerRegistries = new ArrayList(1) + containerRegistries << registryRef containerConfig.setContainerRegistries(containerRegistries) - log.debug "[AZURE BATCH] Connecting Azure Batch pool to Container Registry '$registryOpts.server'" } final image = getImage(opts) diff --git a/plugins/nf-azure/src/main/nextflow/cloud/azure/config/AzRegistryOpts.groovy b/plugins/nf-azure/src/main/nextflow/cloud/azure/config/AzRegistryOpts.groovy index 40e8734534..3106ce53dd 100644 --- a/plugins/nf-azure/src/main/nextflow/cloud/azure/config/AzRegistryOpts.groovy +++ b/plugins/nf-azure/src/main/nextflow/cloud/azure/config/AzRegistryOpts.groovy @@ -48,6 +48,12 @@ class AzRegistryOpts implements ConfigScope { """) final String password + @ConfigOption + @Description(""" + The ARM resource ID of a user-assigned managed identity used to authenticate to a private container registry (e.g. `/subscriptions//resourceGroups//providers/Microsoft.ManagedIdentity/userAssignedIdentities/`). When set, it takes precedence over `userName` and `password`. Defaults to environment variable `AZURE_REGISTRY_MANAGED_RESOURCE_ID`. Note: the managed identity must already be attached to the pool nodes. + """) + final String managedIdentityResourceId + AzRegistryOpts() { this(Collections.emptyMap()) } @@ -57,14 +63,21 @@ class AzRegistryOpts implements ConfigScope { this.server = config.server ?: 'docker.io' this.userName = config.userName ?: env.get('AZURE_REGISTRY_USER_NAME') this.password = config.password ?: env.get('AZURE_REGISTRY_PASSWORD') + this.managedIdentityResourceId = config.managedIdentityResourceId ?: env.get('AZURE_REGISTRY_MANAGED_RESOURCE_ID') + } + + boolean usesManagedIdentity() { + return managedIdentityResourceId as boolean } boolean isConfigured() { + if( managedIdentityResourceId ) + return true if( userName && password ) return true if( !userName && !password ) return false - throw new IllegalArgumentException("Invalid Container Registry configuration - Make sure userName and password are set for Container Registry") + throw new IllegalArgumentException("Invalid Container Registry configuration - Make sure managedIdentityResourceId or userName and password are set for Container Registry") } } diff --git a/plugins/nf-azure/src/test/nextflow/cloud/azure/batch/AzBatchServiceTest.groovy b/plugins/nf-azure/src/test/nextflow/cloud/azure/batch/AzBatchServiceTest.groovy index 0cafb05aa0..bc8729acd8 100644 --- a/plugins/nf-azure/src/test/nextflow/cloud/azure/batch/AzBatchServiceTest.groovy +++ b/plugins/nf-azure/src/test/nextflow/cloud/azure/batch/AzBatchServiceTest.groovy @@ -24,6 +24,7 @@ import dev.failsafe.function.CheckedPredicate import com.azure.compute.batch.models.BatchPool import com.azure.compute.batch.models.BatchJobCreateContent +import com.azure.compute.batch.models.BatchSupportedImage import com.azure.compute.batch.models.ElevationLevel import com.azure.compute.batch.models.EnvironmentSetting import com.azure.core.exception.HttpResponseException @@ -463,6 +464,43 @@ class AzBatchServiceTest extends Specification { } + def 'should configure container registry with username and password' () { + given: + def exec = createExecutor(new AzConfig([registry: [server: 'reg.azurecr.io', userName: 'foo', password: 'bar']])) + AzBatchService svc = Spy(AzBatchService, constructorArgs: [exec]) + + when: + def vmConfig = svc.poolVmConfig(new AzPoolOpts()) + then: + 1 * svc.getImage(_) >> BatchSupportedImage.fromJson(com.azure.json.JsonProviders.createReader('{"nodeAgentSKUId":"sku","imageReference":{}}')) + and: + def registries = vmConfig.containerConfiguration.containerRegistries + registries.size() == 1 + registries[0].registryServer == 'reg.azurecr.io' + registries[0].username == 'foo' + registries[0].password == 'bar' + registries[0].identityReference == null + } + + def 'should configure container registry with managed identity' () { + given: + def RESOURCE_ID = '/subscriptions/sub/resourceGroups/rg/providers/Microsoft.ManagedIdentity/userAssignedIdentities/name' + def exec = createExecutor(new AzConfig([registry: [server: 'reg.azurecr.io', userName: 'foo', password: 'bar', managedIdentityResourceId: RESOURCE_ID]])) + AzBatchService svc = Spy(AzBatchService, constructorArgs: [exec]) + + when: + def vmConfig = svc.poolVmConfig(new AzPoolOpts()) + then: + 1 * svc.getImage(_) >> BatchSupportedImage.fromJson(com.azure.json.JsonProviders.createReader('{"nodeAgentSKUId":"sku","imageReference":{}}')) + and: 'managed identity takes precedence over username/password' + def registries = vmConfig.containerConfiguration.containerRegistries + registries.size() == 1 + registries[0].registryServer == 'reg.azurecr.io' + registries[0].identityReference.resourceId == RESOURCE_ID + registries[0].username == null + registries[0].password == null + } + def 'should check poolid' () { given: def exec = createExecutor() diff --git a/plugins/nf-azure/src/test/nextflow/cloud/azure/config/AzRegistryOptsTest.groovy b/plugins/nf-azure/src/test/nextflow/cloud/azure/config/AzRegistryOptsTest.groovy index b391bcbbc4..8b7af0528e 100644 --- a/plugins/nf-azure/src/test/nextflow/cloud/azure/config/AzRegistryOptsTest.groovy +++ b/plugins/nf-azure/src/test/nextflow/cloud/azure/config/AzRegistryOptsTest.groovy @@ -52,4 +52,38 @@ class AzRegistryOptsTest extends Specification { opts3.password == 'env-password' } + def 'should get managed identity resource id'() { + expect: 'config value is used' + new AzRegistryOpts([managedIdentityResourceId: 'res-id'], [:]).managedIdentityResourceId == 'res-id' + + and: 'env fallback is used when config absent' + new AzRegistryOpts([:], [AZURE_REGISTRY_MANAGED_RESOURCE_ID: 'env-res-id']).managedIdentityResourceId == 'env-res-id' + + and: 'config value wins over env' + new AzRegistryOpts([managedIdentityResourceId: 'res-id'], [AZURE_REGISTRY_MANAGED_RESOURCE_ID: 'env-res-id']).managedIdentityResourceId == 'res-id' + + and: 'null when neither set' + new AzRegistryOpts([:], [:]).managedIdentityResourceId == null + } + + def 'should validate isConfigured'() { + expect: + new AzRegistryOpts([:], [:]).isConfigured() == false + new AzRegistryOpts([userName: 'foo', password: 'bar'], [:]).isConfigured() == true + new AzRegistryOpts([managedIdentityResourceId: 'res-id'], [:]).isConfigured() == true + new AzRegistryOpts([managedIdentityResourceId: 'res-id'], [:]).usesManagedIdentity() == true + new AzRegistryOpts([userName: 'foo', password: 'bar'], [:]).usesManagedIdentity() == false + + when: 'managed identity takes precedence over incomplete user/password' + def opts = new AzRegistryOpts([managedIdentityResourceId: 'res-id', userName: 'foo'], [:]) + then: + opts.isConfigured() == true + opts.usesManagedIdentity() == true + + when: 'partial user/password throws' + new AzRegistryOpts([userName: 'foo'], [:]).isConfigured() + then: + thrown(IllegalArgumentException) + } + } From e4ccff2bff7ad4abb073ae6545ea8f2987b8bf76 Mon Sep 17 00:00:00 2001 From: Adam Talbot <12817534+adamrtalbot@users.noreply.github.com> Date: Mon, 13 Jul 2026 10:46:29 +0100 Subject: [PATCH 2/2] Apply suggestions from code review Co-authored-by: Chris Hakkaart Signed-off-by: Adam Talbot <12817534+adamrtalbot@users.noreply.github.com> --- docs/azure.mdx | 16 +--------------- docs/reference/config.mdx | 4 ++-- 2 files changed, 3 insertions(+), 17 deletions(-) diff --git a/docs/azure.mdx b/docs/azure.mdx index c562ee25cc..12db1841ee 100644 --- a/docs/azure.mdx +++ b/docs/azure.mdx @@ -550,22 +550,8 @@ Nextflow uses the following environment variables if the registry credentials ar - `AZURE_REGISTRY_PASSWORD`: the password for Azure Container Registry authentication ::: -Alternatively, you can authenticate to a private registry with a user-assigned [managed identity](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview) instead of a username and password: +To authenticate to a private registry with a user-assigned [managed identity](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview) instead of a username and password: -```groovy -azure { - registry { - server = '' // e.g., 'myregistry.azurecr.io' - managedIdentityResourceId = '' // e.g., '/subscriptions//resourceGroups//providers/Microsoft.ManagedIdentity/userAssignedIdentities/' - } -} -``` - -The value is the full ARM resource ID of the managed identity, not its client ID. When set, it takes precedence over `userName` and `password`, and can also be provided via the `AZURE_REGISTRY_MANAGED_RESOURCE_ID` environment variable. - -:::note -The managed identity must already be attached to the pool nodes. Nextflow cannot attach an identity when auto-creating a pool, so this option requires a pre-existing pool (or an identity attached out-of-band). -::: Container images from public registries such as Docker Hub can be used without additional configuration, even when a private registry is specified in the `azure.registry` scope. The Docker runtime on Azure Batch VMs automatically uses the appropriate registry based on the container image specified for the task. diff --git a/docs/reference/config.mdx b/docs/reference/config.mdx index 7918110153..5c0573f862 100644 --- a/docs/reference/config.mdx +++ b/docs/reference/config.mdx @@ -607,10 +607,10 @@ When `true`, use the system-assigned [managed identity](https://learn.microsoft. ##### `azure.registry.managedIdentityResourceId` -The ARM resource ID of a user-assigned managed identity used to authenticate to a private container registry, e.g. `/subscriptions//resourceGroups//providers/Microsoft.ManagedIdentity/userAssignedIdentities/`. When set, it takes precedence over `azure.registry.userName` and `azure.registry.password`. Defaults to environment variable `AZURE_REGISTRY_MANAGED_RESOURCE_ID`. +The ARM resource ID of a user-assigned managed identity for authenticating to a private container registry. For example, `/subscriptions//resourceGroups//providers/Microsoft.ManagedIdentity/userAssignedIdentities/`. When set, it takes precedence over `azure.registry.userName` and `azure.registry.password`. Defaults to the `AZURE_REGISTRY_MANAGED_RESOURCE_ID` environment variable. :::note -The managed identity must already be attached to the pool nodes. Nextflow cannot attach an identity when auto-creating a pool, so this option requires a pre-existing pool (or an identity attached out-of-band). +The managed identity must already be attached to the pool nodes. Nextflow cannot attach an identity when auto-creating a pool. This option requires a pre-existing pool with the identity attached. ::: ##### `azure.registry.password`