feat: Add reverse group lookup for LDAP servers that do not return memberOf

Author: JWuCines

docs/development/extensions-core/druid-basic-security.md

@@ -261,6 +261,18 @@ The valid credentials cache size. The cache uses a LRU policy.<br />
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;**Required**: No<br />
 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;**Default**: 100
 
+**`druid.auth.authenticator.MyBasicLDAPAuthenticator.credentialsValidator.groupBaseDn`**
+
+The base DN for searching LDAP groups. When set together with `groupSearch`, Druid performs a reverse group lookup to populate the `memberOf` attribute during authentication. This is needed when the LDAP server does not return `memberOf` in user search results. If not set, Druid relies on the `memberOf` attribute being returned directly by the user search.<br />
+&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;**Required**: No<br />
+&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;**Default**: null
+
+**`druid.auth.authenticator.MyBasicLDAPAuthenticator.credentialsValidator.groupSearch`**
+
+The LDAP search filter for finding groups that contain a user. The `%s` placeholder is replaced with the user's full DN. For example, `(uniqueMember=%s)` for `groupOfUniqueNames` or `(member=%s)` for `groupOfNames`.<br />
+&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;**Required**: No<br />
+&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;**Default**: null
+
 **`druid.auth.authenticator.MyBasicLDAPAuthenticator.skipOnFailure`**
 
 If true and the request credential doesn't exists or isn't fully configured in the credentials store, the request will proceed to next Authenticator in the chain.<br />

docs/operations/auth-ldap.md

@@ -64,7 +64,7 @@ memberOf: cn=mygroup,ou=groups,dc=example,dc=com
 You use this information to map the LDAP group to Druid roles in a later step. 
 
 :::info
- Druid uses the `memberOf` attribute to determine a group's membership using LDAP. If your LDAP server implementation doesn't include this attribute, you must complete some additional steps when you [map LDAP groups to Druid roles](#map-ldap-groups-to-druid-roles).
+ Druid uses the `memberOf` attribute to determine group membership. If your LDAP server does not return this attribute, you can either [map LDAP groups to Druid roles](#map-ldap-groups-to-druid-roles) manually or configure a [reverse group lookup](#group-search-reverse-lookup-configuration) to resolve groups automatically.
 :::
 
 ## Configure Druid for LDAP authentication
@@ -296,6 +296,27 @@ Complete the following steps to set up LDAPS for Druid. See [Configuration refer
 5. Restart Druid.
 
 
+## Group search reverse lookup configuration
+
+By default, Druid reads the `memberOf` attribute from the LDAP user entry to determine group membership. Some LDAP servers do not return `memberOf` because the feature is not enabled, it is stored as an operational attribute that Java JNDI cannot retrieve, or groups only store membership on the group entry itself. In these cases, group-based authorization denies all requests because no groups are found.
+
+To resolve this, configure a reverse group lookup so that Druid searches group entries to find which groups contain the user. Add the following properties to your `common.runtime.properties`:
+
+```
+druid.auth.authenticator.ldap.credentialsValidator.groupBaseDn=ou=Groups,dc=example,dc=com
+druid.auth.authenticator.ldap.credentialsValidator.groupSearch=(uniqueMember=%s)
+```
+
+Where:
+- `groupBaseDn`: The base DN under which your LDAP groups are stored.
+- `groupSearch`: The LDAP filter to find groups containing a user. The `%s` placeholder is replaced with the user's full DN (for example, `uid=myuser,ou=People,dc=example,dc=com`). Use `(uniqueMember=%s)` for `groupOfUniqueNames` or `(member=%s)` for `groupOfNames`.
+
+When these properties are set and the user search does not return a `memberOf` attribute, Druid automatically performs the reverse group lookup and populates `memberOf` in the authentication result. The authorizer processes these groups as usual, requiring no additional configuration.
+
+:::info
+ If your LDAP server does return `memberOf` directly, the reverse lookup is skipped.
+:::
+
 ## Troubleshooting tips
 
 The following are some ideas to help you troubleshoot issues with LDAP and LDAPS.

embedded-tests/src/test/java/org/apache/druid/testing/embedded/auth/BasicAuthLdapReverseGroupLookupTest.java

@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.testing.embedded.auth;
+
+import org.apache.druid.testing.embedded.EmbeddedResource;
+
+/**
+ * Runs the same LDAP auth tests as {@link BasicAuthLdapConfigurationTest} but with
+ * the reverse group lookup feature enabled ({@code groupBaseDn} and {@code groupSearch}).
+ *
+ * OpenLDAP (used in the test container) does not return {@code memberOf} in user search
+ * results by default. Without reverse group lookup, group-based authorization would fail
+ * because the {@code LDAPRoleProvider} cannot resolve group memberships. This test verifies
+ * that enabling reverse group lookup allows all group-based authorization to work correctly.
+ */
+public class BasicAuthLdapReverseGroupLookupTest extends BasicAuthLdapConfigurationTest
+{
+  @Override
+  protected EmbeddedResource getAuthResource()
+  {
+    return new LdapReverseGroupLookupAuthResource();
+  }
+}

embedded-tests/src/test/java/org/apache/druid/testing/embedded/auth/LdapReverseGroupLookupAuthResource.java

@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.druid.testing.embedded.auth;
+
+import org.apache.druid.java.util.common.StringUtils;
+import org.apache.druid.testing.embedded.EmbeddedDruidCluster;
+
+/**
+ * LDAP auth resource that additionally configures reverse group lookup via
+ * {@code groupBaseDn} and {@code groupSearch}. This is needed for LDAP servers
+ * (such as OpenLDAP) that do not return the {@code memberOf} attribute in user
+ * search results.
+ */
+public class LdapReverseGroupLookupAuthResource extends LdapAuthResource
+{
+  private static final String AUTHENTICATOR_NAME = "ldap";
+
+  @Override
+  public void onStarted(EmbeddedDruidCluster cluster)
+  {
+    super.onStarted(cluster);
+    cluster.addCommonProperty(
+        StringUtils.format("druid.auth.authenticator.%s.credentialsValidator.groupBaseDn", AUTHENTICATOR_NAME),
+        "ou=Groups,dc=example,dc=org"
+    ).addCommonProperty(
+        StringUtils.format("druid.auth.authenticator.%s.credentialsValidator.groupSearch", AUTHENTICATOR_NAME),
+        "(uniqueMember=%s)"
+    );
+  }
+}

extensions-core/druid-basic-security/src/main/java/org/apache/druid/security/basic/BasicAuthLDAPConfig.java

@@ -21,6 +21,8 @@
 
 import org.apache.druid.metadata.PasswordProvider;
 
+import javax.annotation.Nullable;
+
 public class BasicAuthLDAPConfig
 {
   private final String url;
@@ -33,6 +35,10 @@ public class BasicAuthLDAPConfig
   private final Integer credentialVerifyDuration;
   private final Integer credentialMaxDuration;
   private final Integer credentialCacheSize;
+  @Nullable
+  private final String groupBaseDn;
+  @Nullable
+  private final String groupSearch;
 
   public BasicAuthLDAPConfig(
       final String url,
@@ -46,6 +52,37 @@ public BasicAuthLDAPConfig(
       final Integer credentialMaxDuration,
       final Integer credentialCacheSize
   )
+  {
+    this(
+        url,
+        bindUser,
+        bindPassword,
+        baseDn,
+        userSearch,
+        userAttribute,
+        credentialIterations,
+        credentialVerifyDuration,
+        credentialMaxDuration,
+        credentialCacheSize,
+        null,
+        null
+    );
+  }
+
+  public BasicAuthLDAPConfig(
+      final String url,
+      final String bindUser,
+      final PasswordProvider bindPassword,
+      final String baseDn,
+      final String userSearch,
+      final String userAttribute,
+      final int credentialIterations,
+      final Integer credentialVerifyDuration,
+      final Integer credentialMaxDuration,
+      final Integer credentialCacheSize,
+      @Nullable final String groupBaseDn,
+      @Nullable final String groupSearch
+  )
   {
     this.url = url;
     this.bindUser = bindUser;
@@ -57,6 +94,8 @@ public BasicAuthLDAPConfig(
     this.credentialVerifyDuration = credentialVerifyDuration;
     this.credentialMaxDuration = credentialMaxDuration;
     this.credentialCacheSize = credentialCacheSize;
+    this.groupBaseDn = groupBaseDn;
+    this.groupSearch = groupSearch;
   }
 
   public String getUrl()
@@ -108,4 +147,21 @@ public Integer getCredentialCacheSize()
   {
     return credentialCacheSize;
   }
+
+  @Nullable
+  public String getGroupBaseDn()
+  {
+    return groupBaseDn;
+  }
+
+  @Nullable
+  public String getGroupSearch()
+  {
+    return groupSearch;
+  }
+
+  public boolean isGroupSearchConfigured()
+  {
+    return groupBaseDn != null && groupSearch != null;
+  }
 }

extensions-core/druid-basic-security/src/main/java/org/apache/druid/security/basic/authentication/validator/LDAPCredentialsValidator.java

@@ -41,6 +41,8 @@
 import javax.naming.Name;
 import javax.naming.NamingEnumeration;
 import javax.naming.NamingException;
+import javax.naming.directory.BasicAttribute;
+import javax.naming.directory.BasicAttributes;
 import javax.naming.directory.DirContext;
 import javax.naming.directory.InitialDirContext;
 import javax.naming.directory.SearchControls;
@@ -77,7 +79,9 @@ public LDAPCredentialsValidator(
       @JsonProperty("credentialIterations") Integer credentialIterations,
       @JsonProperty("credentialVerifyDuration") Integer credentialVerifyDuration,
       @JsonProperty("credentialMaxDuration") Integer credentialMaxDuration,
-      @JsonProperty("credentialCacheSize") Integer credentialCacheSize
+      @JsonProperty("credentialCacheSize") Integer credentialCacheSize,
+      @JsonProperty("groupBaseDn") String groupBaseDn,
+      @JsonProperty("groupSearch") String groupSearch
   )
   {
     this.ldapConfig = new BasicAuthLDAPConfig(
@@ -90,7 +94,9 @@ public LDAPCredentialsValidator(
         credentialIterations == null ? BasicAuthUtils.DEFAULT_KEY_ITERATIONS : credentialIterations,
         credentialVerifyDuration == null ? BasicAuthUtils.DEFAULT_CREDENTIAL_VERIFY_DURATION_SECONDS : credentialVerifyDuration,
         credentialMaxDuration == null ? BasicAuthUtils.DEFAULT_CREDENTIAL_MAX_DURATION_SECONDS : credentialMaxDuration,
-        credentialCacheSize == null ? BasicAuthUtils.DEFAULT_CREDENTIAL_CACHE_SIZE : credentialCacheSize
+        credentialCacheSize == null ? BasicAuthUtils.DEFAULT_CREDENTIAL_CACHE_SIZE : credentialCacheSize,
+        groupBaseDn,
+        groupSearch
     );
 
     this.cache = new LruBlockCache(
@@ -237,22 +243,92 @@ SearchResult getLdapUserObject(BasicAuthLDAPConfig ldapConfig, DirContext contex
           ldapConfig.getBaseDn(),
           StringUtils.format(ldapConfig.getUserSearch(), encodedUsername),
           sc);
+      final SearchResult userResult;
       try {
         if (!results.hasMore()) {
           return null;
         }
-        return results.next();
+        userResult = results.next();
       }
       finally {
         results.close();
       }
+
+      // Some LDAP servers do not return memberOf in search results. When memberOf is
+      // absent and group search configuration is provided, fall back to a reverse
+      // group lookup.
+      if (ldapConfig.isGroupSearchConfigured() && !hasMemberOfAttribute(userResult)) {
+        populateMemberOfFromGroupSearch(ldapConfig, context, userResult);
+      }
+
+      return userResult;
     }
     catch (NamingException e) {
       LOG.debug(e, "Unable to find user '%s'", username);
       return null;
     }
   }
 
+  private static boolean hasMemberOfAttribute(SearchResult userResult)
+  {
+    return userResult.getAttributes() != null
+        && userResult.getAttributes().get("memberOf") != null;
+  }
+
+  @SuppressWarnings("BanJNDI")
+  private void populateMemberOfFromGroupSearch(
+      BasicAuthLDAPConfig ldapConfig,
+      DirContext context,
+      SearchResult userResult
+  )
+  {
+    try {
+      final String userDn = userResult.getNameInNamespace();
+      final SearchControls sc = new SearchControls();
+      sc.setSearchScope(SearchControls.SUBTREE_SCOPE);
+      // Request only the DN by returning a minimal attribute. The group DN is
+      // obtained via getNameInNamespace() and does not depend on any returned attribute.
+      sc.setReturningAttributes(new String[]{"1.1"});
+
+      final String filter = StringUtils.format(ldapConfig.getGroupSearch(), encodeForLDAP(userDn, true));
+      final NamingEnumeration<SearchResult> groupResults = context.search(
+          ldapConfig.getGroupBaseDn(),
+          filter,
+          sc
+      );
+
+      final BasicAttribute memberOfAttr = new BasicAttribute("memberOf");
+      try {
+        while (groupResults.hasMore()) {
+          final SearchResult groupResult = groupResults.next();
+          final String groupDn = groupResult.getNameInNamespace();
+          memberOfAttr.add(groupDn);
+        }
+      }
+      finally {
+        groupResults.close();
+      }
+
+      if (memberOfAttr.size() > 0) {
+        if (userResult.getAttributes() != null) {
+          userResult.getAttributes().put(memberOfAttr);
+        } else {
+          final BasicAttributes attrs = new BasicAttributes(true);
+          attrs.put(memberOfAttr);
+          userResult.setAttributes(attrs);
+        }
+        LOG.debug(
+            "Populated memberOf for user '%s' with %d groups from reverse group search",
+            userDn,
+            memberOfAttr.size()
+        );
+      }
+    }
+    catch (NamingException e) {
+      LOG.error(e, "Exception during reverse group lookup, proceeding without group memberships");
+    }
+  }
+
   boolean validatePassword(BasicAuthLDAPConfig ldapConfig, LdapName userDn, char[] password)
   {
     InitialDirContext context = null;