-
Notifications
You must be signed in to change notification settings - Fork 25.2k
[Entitlements] Test ScopeResolver based on TestBuildInfo (parser + resolver) #127719
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ldematte
wants to merge
9
commits into
elastic:main
Choose a base branch
from
ldematte:entitlement/test-resolver
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
6555be6
WIP
ldematte d9f715d
TestBuildInfo parsing + resolver
ldematte 8dfc361
[CI] Auto commit changes from spotless
elasticsearchmachine d12e32a
tests + extract code base from URLs
ldematte b7e16cf
Merge branch 'entitlement/test-resolver' of github.com:ldematte/elast…
ldematte 22ff04e
Merge branch 'main' into entitlement/test-resolver
ldematte 78d8737
Merge remote-tracking branch 'upstream/main' into entitlement/test-re…
ldematte 8a40526
updated to reflect changes to https://github.com/elastic/elasticsearc…
ldematte 89f117e
Merge branch 'entitlement/test-resolver' of github.com:ldematte/elast…
ldematte File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
14 changes: 14 additions & 0 deletions
14
test/framework/src/main/java/org/elasticsearch/bootstrap/TestBuildInfo.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
/* | ||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
* or more contributor license agreements. Licensed under the "Elastic License | ||
* 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side | ||
* Public License v 1"; you may not use this file except in compliance with, at | ||
* your election, the "Elastic License 2.0", the "GNU Affero General Public | ||
* License v3.0 only", or the "Server Side Public License, v 1". | ||
*/ | ||
|
||
package org.elasticsearch.bootstrap; | ||
|
||
import java.util.List; | ||
|
||
record TestBuildInfo(String componentName, List<TestBuildInfoLocation> locations) {} |
12 changes: 12 additions & 0 deletions
12
test/framework/src/main/java/org/elasticsearch/bootstrap/TestBuildInfoLocation.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
/* | ||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
* or more contributor license agreements. Licensed under the "Elastic License | ||
* 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side | ||
* Public License v 1"; you may not use this file except in compliance with, at | ||
* your election, the "Elastic License 2.0", the "GNU Affero General Public | ||
* License v3.0 only", or the "Server Side Public License, v 1". | ||
*/ | ||
|
||
package org.elasticsearch.bootstrap; | ||
|
||
record TestBuildInfoLocation(String className, String moduleName) {} |
104 changes: 104 additions & 0 deletions
104
test/framework/src/main/java/org/elasticsearch/bootstrap/TestBuildInfoParser.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,104 @@ | ||
/* | ||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
* or more contributor license agreements. Licensed under the "Elastic License | ||
* 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side | ||
* Public License v 1"; you may not use this file except in compliance with, at | ||
* your election, the "Elastic License 2.0", the "GNU Affero General Public | ||
* License v3.0 only", or the "Server Side Public License, v 1". | ||
*/ | ||
|
||
package org.elasticsearch.bootstrap; | ||
|
||
import org.elasticsearch.core.SuppressForbidden; | ||
import org.elasticsearch.xcontent.ObjectParser; | ||
import org.elasticsearch.xcontent.ParseField; | ||
import org.elasticsearch.xcontent.XContentFactory; | ||
import org.elasticsearch.xcontent.XContentParser; | ||
import org.elasticsearch.xcontent.XContentParserConfiguration; | ||
import org.elasticsearch.xcontent.XContentType; | ||
|
||
import java.io.IOException; | ||
import java.io.InputStream; | ||
import java.net.URL; | ||
import java.util.ArrayList; | ||
import java.util.List; | ||
|
||
class TestBuildInfoParser { | ||
|
||
private static final String PLUGIN_TEST_BUILD_INFO_RESOURCES = "META-INF/plugin-test-build-info.json"; | ||
private static final String SERVER_TEST_BUILD_INFO_RESOURCE = "META-INF/server-test-build-info.json"; | ||
|
||
private static final ObjectParser<Builder, Void> PARSER = new ObjectParser<>("test_build_info", Builder::new); | ||
private static final ObjectParser<Location, Void> LOCATION_PARSER = new ObjectParser<>("location", Location::new); | ||
static { | ||
LOCATION_PARSER.declareString(Location::className, new ParseField("class")); | ||
LOCATION_PARSER.declareString(Location::moduleName, new ParseField("module")); | ||
|
||
PARSER.declareString(Builder::name, new ParseField("name")); | ||
PARSER.declareObjectArray(Builder::locations, LOCATION_PARSER, new ParseField("locations")); | ||
} | ||
|
||
private static class Location { | ||
private String className; | ||
private String moduleName; | ||
|
||
public void moduleName(final String moduleName) { | ||
this.moduleName = moduleName; | ||
} | ||
|
||
public void className(final String className) { | ||
this.className = className; | ||
} | ||
} | ||
|
||
private static final class Builder { | ||
private String name; | ||
private List<Location> locations; | ||
|
||
public void name(final String name) { | ||
this.name = name; | ||
} | ||
|
||
public void locations(final List<Location> locations) { | ||
this.locations = locations; | ||
} | ||
|
||
TestBuildInfo build() { | ||
return new TestBuildInfo(name, locations.stream().map(l -> new TestBuildInfoLocation(l.className, l.moduleName)).toList()); | ||
} | ||
} | ||
|
||
static TestBuildInfo fromXContent(final XContentParser parser) throws IOException { | ||
return PARSER.parse(parser, null).build(); | ||
} | ||
|
||
static List<TestBuildInfo> parseAllPluginTestBuildInfo() throws IOException { | ||
var xContent = XContentFactory.xContent(XContentType.JSON); | ||
List<TestBuildInfo> pluginsTestBuildInfos = new ArrayList<>(); | ||
var resources = TestBuildInfoParser.class.getClassLoader().getResources(PLUGIN_TEST_BUILD_INFO_RESOURCES); | ||
URL resource; | ||
while ((resource = resources.nextElement()) != null) { | ||
try (var stream = getStream(resource); var parser = xContent.createParser(XContentParserConfiguration.EMPTY, stream)) { | ||
pluginsTestBuildInfos.add(fromXContent(parser)); | ||
} | ||
} | ||
return pluginsTestBuildInfos; | ||
} | ||
|
||
static TestBuildInfo parseServerTestBuildInfo() throws IOException { | ||
var xContent = XContentFactory.xContent(XContentType.JSON); | ||
var resource = TestBuildInfoParser.class.getClassLoader().getResource(SERVER_TEST_BUILD_INFO_RESOURCE); | ||
// No test-build-info for server: this might be a non-gradle build. Proceed without TestBuildInfo | ||
if (resource == null) { | ||
return null; | ||
} | ||
try (var stream = getStream(resource); var parser = xContent.createParser(XContentParserConfiguration.EMPTY, stream)) { | ||
return fromXContent(parser); | ||
} | ||
} | ||
|
||
@SuppressForbidden(reason = "URLs from class loader") | ||
private static InputStream getStream(URL resource) throws IOException { | ||
return resource.openStream(); | ||
} | ||
} |
104 changes: 104 additions & 0 deletions
104
test/framework/src/main/java/org/elasticsearch/bootstrap/TestScopeResolver.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,104 @@ | ||
/* | ||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
* or more contributor license agreements. Licensed under the "Elastic License | ||
* 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side | ||
* Public License v 1"; you may not use this file except in compliance with, at | ||
* your election, the "Elastic License 2.0", the "GNU Affero General Public | ||
* License v3.0 only", or the "Server Side Public License, v 1". | ||
*/ | ||
|
||
package org.elasticsearch.bootstrap; | ||
|
||
import org.elasticsearch.core.SuppressForbidden; | ||
import org.elasticsearch.entitlement.runtime.policy.PolicyManager; | ||
import org.elasticsearch.logging.LogManager; | ||
import org.elasticsearch.logging.Logger; | ||
|
||
import java.net.MalformedURLException; | ||
import java.net.URL; | ||
import java.util.HashMap; | ||
import java.util.List; | ||
import java.util.Map; | ||
import java.util.function.Function; | ||
|
||
record TestScopeResolver(Map<String, PolicyManager.PolicyScope> scopeMap) { | ||
|
||
private static final Logger logger = LogManager.getLogger(TestScopeResolver.class); | ||
|
||
PolicyManager.PolicyScope getScope(Class<?> callerClass) { | ||
var callerCodeSource = callerClass.getProtectionDomain().getCodeSource(); | ||
assert callerCodeSource != null; | ||
|
||
var location = callerCodeSource.getLocation().toString(); | ||
var scope = scopeMap.get(location); | ||
if (scope == null) { | ||
logger.warn("Cannot identify a scope for class [{}], location [{}]", callerClass.getName(), location); | ||
return PolicyManager.PolicyScope.unknown(location); | ||
} | ||
return scope; | ||
} | ||
|
||
static Function<Class<?>, PolicyManager.PolicyScope> createScopeResolver( | ||
TestBuildInfo serverBuildInfo, | ||
List<TestBuildInfo> pluginsBuildInfo | ||
) { | ||
|
||
Map<String, PolicyManager.PolicyScope> scopeMap = new HashMap<>(); | ||
for (var pluginBuildInfo : pluginsBuildInfo) { | ||
for (var location : pluginBuildInfo.locations()) { | ||
var codeSource = TestScopeResolver.class.getClassLoader().getResource(location.className()); | ||
if (codeSource == null) { | ||
throw new IllegalArgumentException("Cannot locate class [" + location.className() + "]"); | ||
} | ||
try { | ||
scopeMap.put( | ||
getCodeSource(codeSource, location.className()), | ||
PolicyManager.PolicyScope.plugin(pluginBuildInfo.componentName(), location.moduleName()) | ||
); | ||
} catch (MalformedURLException e) { | ||
throw new IllegalArgumentException("Cannot locate class [" + location.className() + "]", e); | ||
} | ||
} | ||
} | ||
|
||
for (var location : serverBuildInfo.locations()) { | ||
var classUrl = TestScopeResolver.class.getClassLoader().getResource(location.className()); | ||
if (classUrl == null) { | ||
throw new IllegalArgumentException("Cannot locate class [" + location.className() + "]"); | ||
} | ||
try { | ||
scopeMap.put(getCodeSource(classUrl, location.className()), PolicyManager.PolicyScope.server(location.moduleName())); | ||
} catch (MalformedURLException e) { | ||
throw new IllegalArgumentException("Cannot locate class [" + location.className() + "]", e); | ||
} | ||
} | ||
|
||
var testScopeResolver = new TestScopeResolver(scopeMap); | ||
return testScopeResolver::getScope; | ||
} | ||
|
||
private static String getCodeSource(URL classUrl, String className) throws MalformedURLException { | ||
if (isJarUrl(classUrl)) { | ||
return extractJarFileUrl(classUrl).toString(); | ||
} | ||
var s = classUrl.toString(); | ||
return s.substring(0, s.indexOf(className)); | ||
} | ||
|
||
private static boolean isJarUrl(URL url) { | ||
return "jar".equals(url.getProtocol()); | ||
} | ||
|
||
@SuppressWarnings("deprecation") | ||
@SuppressForbidden(reason = "need file spec in string form to extract the inner URL form the JAR URL") | ||
private static URL extractJarFileUrl(URL jarUrl) throws MalformedURLException { | ||
String spec = jarUrl.getFile(); | ||
int separator = spec.indexOf("!/"); | ||
|
||
if (separator == -1) { | ||
throw new MalformedURLException(); | ||
} | ||
|
||
return new URL(spec.substring(0, separator)); | ||
} | ||
} |
75 changes: 75 additions & 0 deletions
75
test/framework/src/test/java/org/elasticsearch/bootstrap/TestBuildInfoParserTests.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,75 @@ | ||
/* | ||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
* or more contributor license agreements. Licensed under the "Elastic License | ||
* 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side | ||
* Public License v 1"; you may not use this file except in compliance with, at | ||
* your election, the "Elastic License 2.0", the "GNU Affero General Public | ||
* License v3.0 only", or the "Server Side Public License, v 1". | ||
*/ | ||
|
||
package org.elasticsearch.bootstrap; | ||
|
||
import org.elasticsearch.test.ESTestCase; | ||
import org.elasticsearch.xcontent.XContentFactory; | ||
import org.elasticsearch.xcontent.XContentParserConfiguration; | ||
import org.elasticsearch.xcontent.XContentType; | ||
|
||
import java.io.IOException; | ||
|
||
import static org.elasticsearch.test.LambdaMatchers.transformedItemsMatch; | ||
import static org.hamcrest.Matchers.contains; | ||
import static org.hamcrest.Matchers.is; | ||
|
||
public class TestBuildInfoParserTests extends ESTestCase { | ||
public void testSimpleParsing() throws IOException { | ||
|
||
var input = """ | ||
{ | ||
"name": "lang-painless", | ||
"locations": [ | ||
{ | ||
"class": "Location.class", | ||
"module": "org.elasticsearch.painless" | ||
}, | ||
{ | ||
"class": "org/objectweb/asm/AnnotationVisitor.class", | ||
"module": "org.objectweb.asm" | ||
}, | ||
{ | ||
"class": "org/antlr/v4/runtime/ANTLRErrorListener.class", | ||
"module": "org.antlr.antlr4.runtime" | ||
}, | ||
{ | ||
"class": "org/objectweb/asm/commons/AdviceAdapter.class", | ||
"module": "org.objectweb.asm.commons" | ||
} | ||
] | ||
} | ||
"""; | ||
|
||
try (var parser = XContentFactory.xContent(XContentType.JSON).createParser(XContentParserConfiguration.EMPTY, input)) { | ||
var testInfo = TestBuildInfoParser.fromXContent(parser); | ||
assertThat(testInfo.componentName(), is("lang-painless")); | ||
assertThat( | ||
testInfo.locations(), | ||
transformedItemsMatch( | ||
TestBuildInfoLocation::moduleName, | ||
contains("org.elasticsearch.painless", "org.objectweb.asm", "org.antlr.antlr4.runtime", "org.objectweb.asm.commons") | ||
) | ||
); | ||
|
||
assertThat( | ||
testInfo.locations(), | ||
transformedItemsMatch( | ||
TestBuildInfoLocation::className, | ||
contains( | ||
"Location.class", | ||
"org/objectweb/asm/AnnotationVisitor.class", | ||
"org/antlr/v4/runtime/ANTLRErrorListener.class", | ||
"org/objectweb/asm/commons/AdviceAdapter.class" | ||
) | ||
) | ||
); | ||
} | ||
} | ||
} |
48 changes: 48 additions & 0 deletions
48
test/framework/src/test/java/org/elasticsearch/bootstrap/TestScopeResolverTests.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,48 @@ | ||
/* | ||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
* or more contributor license agreements. Licensed under the "Elastic License | ||
* 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side | ||
* Public License v 1"; you may not use this file except in compliance with, at | ||
* your election, the "Elastic License 2.0", the "GNU Affero General Public | ||
* License v3.0 only", or the "Server Side Public License, v 1". | ||
*/ | ||
|
||
package org.elasticsearch.bootstrap; | ||
|
||
import org.elasticsearch.plugins.Plugin; | ||
import org.elasticsearch.test.ESTestCase; | ||
|
||
import java.util.List; | ||
|
||
import static org.hamcrest.Matchers.is; | ||
|
||
public class TestScopeResolverTests extends ESTestCase { | ||
|
||
public void testScopeResolverServerClass() { | ||
var testBuildInfo = new TestBuildInfo( | ||
"server", | ||
List.of(new TestBuildInfoLocation("org/elasticsearch/Build.class", "org.elasticsearch.server")) | ||
); | ||
var resolver = TestScopeResolver.createScopeResolver(testBuildInfo, List.of()); | ||
|
||
var scope = resolver.apply(Plugin.class); | ||
assertThat(scope.componentName(), is("(server)")); | ||
assertThat(scope.moduleName(), is("org.elasticsearch.server")); | ||
} | ||
|
||
public void testScopeResolverInternalClass() { | ||
var testBuildInfo = new TestBuildInfo( | ||
"server", | ||
List.of(new TestBuildInfoLocation("org/elasticsearch/Build.class", "org.elasticsearch.server")) | ||
); | ||
var testOwnBuildInfo = new TestBuildInfo( | ||
"test-component", | ||
List.of(new TestBuildInfoLocation("org/elasticsearch/bootstrap/TestBuildInfoParserTests.class", "test-module-name")) | ||
); | ||
var resolver = TestScopeResolver.createScopeResolver(testBuildInfo, List.of(testOwnBuildInfo)); | ||
|
||
var scope = resolver.apply(this.getClass()); | ||
assertThat(scope.componentName(), is("test-component")); | ||
assertThat(scope.moduleName(), is("test-module-name")); | ||
} | ||
} |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What happens if there is no policy for this specific module? This will just end up with an empty policy?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reading policies and parsing them will happen separately - I still have to do that. It will work like in production code, a policy can be missing and we just have the default.
Here it is about the resolver, mapping classes and locations to a scope.
But that's a good question on the build infos - what happens if there is no build info for a specific module or plugin? I assumed that build infos will be for all plugins and modules and server, and they will cover all modules; if we miss something, that will fall back to the "unknown" case - similar to what we do in prod code - and that will likely lead to a NotEntitledException.
I think this is correct, but if we feel that should never happen in tests maybe we should have an early
assert
?I'm going to add additional logging, but let me know if we should add an
assert
too.