-
Notifications
You must be signed in to change notification settings - Fork 157
Working With The Weaver
A weave class matches a target class from an external library and specifies how to modify its bytecode. Weave classes follow special rules that tell the weaver which bytecode needs to be changed and how. Once the weaver has finished, the weave class and the target class will have been combined into a composite class, which is the one that actually gets used in a customer’s application.
All Weave Classes start with a @Weave annotation at the class level identifying the classes that we want to instrument:
@Weave(type=MatchType.ExactClass, originalName=“com.example.Foo”)
class Foo_Instrumentation {
}
The originalName parameter is the fully qualified name of whatever we’re trying to instrument.
The type parameter is an enum with the possible values: ExactClass, BaseClass, Interface.
-
ExactClass: match only classes with the name originalName -
BaseClass: match any class that has a superclass with the name originalName -
Interface: match any class that implements an interface with name originalName
For more on match types and the rules they impose, see Inheritance Rules.
Our convention is to name weave classes using the format OriginalName_Instrumentation.
Our convention is to put weave classes in the same-named package of whatever they instrument. The benefit of doing this is it gives your weave class access to all the same classes that the underlying target class has access to. For example, the following weave class needs to use the authenticator field, which is of type JedisSafeAuthenticator:
@Weave(type = MatchType.ExactClass, originalName = "redis.clients.jedis.JedisPubSubBase")
public class JedisPubSubBase_Instrumentation {
private final JedisSafeAuthenticator authenticator = Weaver.callOriginal();
}
Unfortunately, JedisSafeAuthenticator is a package-private class. For this code to compile, our weave class has to have the same package name as the underlying JedisPubSubBase:
package redis.clients.jedis;
@Weave(type = MatchType.ExactClass, originalName = "redis.clients.jedis.JedisPubSubBase")
public class JedisPubSubBase_Instrumentation {
private final JedisSafeAuthenticator authenticator = Weaver.callOriginal();
}
Inside a weave class, you can choose methods of the target class that you want to instrument. These are called weave methods.
A weave method must have the exact same signature (name, modifiers, parameters, return type, and throws clause) as the corresponding method in the target class. If the weaver is unable to find a matching method from the target, the entire class will fail to weave and a weave violation will be generated. The signature is how the weaver identifies the target method, so you don’t need to label your weave method in any special way. You can instrument as many or as few methods from the target class as you like.
Within a weave method, we represent invocation of the original target method with the API Weaver.callOriginal(). For example, given the target class:
class Foo {
public String sayMyName(){
return “foo”;
}
}
we can instrument sayMyName() to make it return an upper case string as follows:
@Weave(type=MatchType.ExactClass, originalName=“com.example.Foo”)
class Foo_Instrumentation {
public String sayMyName(){
String name = Weaver.callOriginal();
return name.toUpperCase();
}
}
Every weave method must contain exactly one invocation of Weaver.callOriginal().
TODO
Sometimes, you might want to instrument a weave method by updating one of the parameters passed to the original method. For example, imagine that whenever handleRequest is called in the below code, we want to replace the original req param with our own custom wrapper:
class Handler {
public Request handleRequest(Request req){
…. do some processing
return req;
}
}
We can achieve that by simply reassigning the parameter value before doing Weaver.callOriginal():
@Weave(...)
class Handler_Instrumentation{
public Request handleRequest(Request req){
req = new CustomRequestWrapper(req);
return Weaver.callOriginal();
}
}
A weave class can introduce new private methods. These new methods must necessarily have a different signature than any of the private methods in the target class (otherwise, the weaver will find the private method from the target and start weaving it). New methods cannot call each other.
Like with methods, you can reference fields from the target class by using a matching field signature. This is useful when some part of your code needs a reference to the field. If the field is not final, you can leave it uninstantiated.
@Weave(type=MatchType.ExactClass, originalName=“com.example.Foo”)
class Foo_Instrumentation {
private String myFoo; //this is fine
}
If you need access to a final field from the target class, you must instantiate it with Weaver.callOriginal() or in a weaved constructor. If you don’t do this, then the compiler will reject the weave class.
@Weave(type=MatchType.ExactClass, originalName=“com.example.Foo”)
class Foo_Instrumentation {
private final String myFoo = Weaver.callOriginal();
}
Any new field (regardless of access modifier) that you wish to introduce to a class must be annotated with the @NewField annotation. If you don’t do this, your weave class will generate a Weave Violation.
The @NewField annotation allows you to use a field in your code seamlessly, just like you would any other field. Under the hood, it actually will be interpreted as a method call to a cache the agent maintains. The cache maps weaved objects to their new fields.
@Weave(type=MatchType.ExactClass, originalName=“com.example.Foo”)
class Foo_Instrumentation {
@NewField
public String useMeLater;
}
The weaver’s rules around inheritance are generally based on the same principle: the weaver can only weave bytecode that is actually present in a given classfile. It cannot weave methods that are implemented in and inherited from a parent class (because these methods will never be present in the child class’ bytecode).
The weaver prohibits weaving unimplemented methods in ExactClass matches altogether, and enforces somewhat looser rules for BaseClass and Interface matches:
| MatchType | Can weave | Can't Weave |
|---|---|---|
| ExactClass | Declared, concrete methods | Inherited methods* Abstract methods |
| BaseClass | Declared, concrete methods Abstract methods (whether declared or inherited) |
Inherited methods* |
| Interface | Declared, interface methods | Inherited interface methods from a superinterface |
*The restriction on inherited methods does not apply to overridden methods. Overridden methods do have a concrete implementation in the classfile, so they are okay to instrument.
These rules increase the chances (or, in the case of ExactClass matches, guarantee) that our weave methods actually match against a target class’ bytecode. Failing to comply with these rules will generate a Weave Violation.
This section is here to help explain how the weaver goes about pulling together instrumentation across the different MatchTypes. It isn't required reading for writing instrumentation, but is here as a reference.
Here is how the weaver finds what it needs to weave inside a target class. The code that does this is in PackageValidationResult:
- The weaver checks if this class is an exact match for a weave class of type ExactClass. If so, all weave methods will be instrumented.
- The weaver checks is this class is an exact match for a weave class of type BaseClass. If so, any weave methods that are implemented will be weaved.
- The weaver checks if this class has a superclass that matches a weave class of type BaseClass. If so, any weave methods that are implemented will be weaved.
- The weaver checks if this class implements an interface that matches a weave class of type Interface. If so, any weave methods that are implemented will be weaved.
Note the "that are implemented" qualifier: only methods implemented by the target class are eligible for weaving.
The weaver’s rules maximize our chances of weaving everything we intend to. However, the subtleties of Java inheritance still make it possible for classes to dodge instrumentation from time to time. Here is a strange example, showing how a class implementing an interface might not pick up all the instrumentation on the interface:
public interface InterfaceA {
void method1();
void method2();
}
public class Parent {
public void method2(){
System.out.println(“I got this from Parent”);
}
}
public class Child extends Parent implements InterfaceA {
public void method1(){
System.out.println(“I got this from Child”);
}
}
Child implements InterfaceA, but only implements method1 itself, having inherited method2 from Parent. Now suppose we want to instrument InterfaceA as follows:
@Weave(type=MatchType.Interface, originalName="com.example.InterfaceA”)
public class InterfaceA_Instrumentation {
public void method1(){
System.out.println(“I’ve instrumented method1”);
Weaver.callOriginal();
}
public void method2(){
System.out.println(“I’ve instrumented method2”);
Weaver.callOriginal();
}
}
This is perfectly legitimate instrumentation. The weaver will recognize that Child implements InterfaceA, and try to apply the relevant instrumentation. However, only the instrumentation on method1 will apply. This is because method2 is not present in Child’s bytecode.
The verifier is a tool used to insure that a particular instrumentation module only applies to a particular set of version ranges of the instrumented library. The verification ranges are defined in a closure in the modules build.gradle file:
verifyInstrumentation {
passesOnly 'com.datastax.oss:java-driver-core:[4.0.0,)'
excludeRegex ".*(rc|beta|alpha).*"
}
The passesOnly method defines the maven artifact identifier, followed by the version ranges that the instrumentation module should apply too. If the module applies to any version outside of this range, and verification error will be thrown when the verifier is run.
The verifier uses the standard Maven range syntax.
- [ and ] indicate an inclusive bound → [1.1, 2.0]
- ( and ) indicate an exclusive bound → (1.1, 2.0)
- a missing starting or ending version value denotes an unbounded range → (,4.0.0) or (1.0.0,)
In the example above, this module must only apply to com.datastax.oss:java-driver-core versions 4.0.0 or greater.
The verifier can be run from the command line for a specific instrumentation library:
./gradlew :instrumentation:<module_name>:verifyInstrumentation
For example: ./gradlew :instrumentation:mongodb-reactive-streams-4.2:verifyInstrumentation
If the verifier passes, the build will end with BUILD SUCCESSFUL. A failure will end with BUILD FAILED. Any failures will be dumped to stderr, along with the weave violation (if any) that caused the failure. To see a full list of artifact versions that passed or failed, open the module's build/verifier folder. There will be two files: passes.txt and failures.txt which will list the artifact versions that passed and failed.
-
excludeRegex- A regular expression that defines a pattern of artifact names that the verifier should exclude from a verification check. This can be handy for libraries that release milestones, betas or alpha artifacts to Maven. -
exclude- This is used to exclude a specific name and version of the artifact. For example: `exclude 'org.glassfish.main.web:web-core:7.0.0-M3'. -
passesandfails- These should rarely be used, but are necessary in some scenarios where a range of versions can't be used with thepassesOnlymethod. An example exists in the Glassfish-3 module.
Skip classes are classes added to an instrumentation module that, if present in the instrumented application, will prevent the instrumentation module from applying. These are used if multiple modules exist for a specific framework/library and it's possible that more than one module can apply (which is never something that should happen).
This is a real example from the agent: Couchbase instrumentation modules version 3.0.0 and version 3.4.3.
The way Couchbase is instrumented, the 3.0.0 version of the module would apply to versions of Couchbase 3.4.3 and greater, which is incorrect. In order to prevent this, a class needed to be found that exists in version 3.4.3 and above, but NOT in version 3.0.0 through 3.4.2. For Couchbase, a new inner class was introduced in version 3.4.3: com.couchbase.client.java.kv.SamplingScan$Built. To force the version 3.0.0 module to NOT apply to Couchbase 3.4.3 and above, we create a skip class in the 3.0.0 module that references the Built inner class:
package com.nr.instrumentation.couchbase;
import com.newrelic.api.agent.weaver.SkipIfPresent;
@SkipIfPresent(originalName = "com.couchbase.client.java.kv.SamplingScan$Built")
public class SamplingScan$Built_Skip {
}
@SkipIfPresent is the annotation that defines the skip class. The originalName property must be assigned the fully qualified name of the class that will be used as the skip class.
The convention for the name of the annotated class is <SkipClassName>_Skip. It can exist in any package inside the module source tree, but is usually put in a non-framework package (a package that holds utilities or helper classes).
By hand. There's not automated way (yet) to discover a proper skip class. Usually, it involves downloading the target artifacts of the framework/library, unzipping them and using some sort of diff tool to compare the folders. The command line utility diff is built in to Mac OS. A simple example of diff'ing two different folder using diff:
diff -rq ./342 ./343 > out.txt
This compares the ./342 and ./343 folders and dumps the results into the file out.txt.
@SkipIfPresent is the preferred way to control the versions that modules match. However, in some cases, you might be unable to find a suitable @SkipIfPresent class. In this case you'll have to get creative. Here are some other (hackier) options we have used:
- Sometimes, a newer version of a module must be developed because some non-weaved class that's used internally changes, for example a method name. In this case, it might be necessary to do a simple weave of a class that only exists in the new version of the module so the earlier version doesn't get instrumented. As an example, the lettuce 6.5 module had to be developed because the
name()method on theProtocolKeywordclass was removed in 6.5 and replaced withtoString().- Version 6.0 of the
io.lettuce.core.AbstractRedisAsyncCommandsclass - Version 6.5
io.lettuce.core.AbstractRedisAsyncCommandsclass - This was solved by doing an empty weave of io.lettuce.core.AbstractRedisAsyncCommands, which only exists in version 6.5. This prevents it from applying to 6.0.
- Version 6.0 of the
Weave violations are reported by the verifier tool. Weave violations are issues that prevent a @Weave class from properly applying to the target class. A full list of violations and their can be found here. The list below is provided for search-ability.
| Violation | Message |
|---|---|
| CLASS_WEAVE_IS_INTERFACE | @Weave classes can not be interfaces |
| CLASS_ACCESS_MISMATCH | Class access levels do not match |
| CLASS_NESTED_NONSTATIC_UNSUPPORTED | Non-static nested classes are not supported in @Weave classes |
| CLASS_EXTENDS_ILLEGAL_SUPERCLASS | @Weave classes must extend java.lang.Object or the same superclass as the original class |
| CLASS_IMPLEMENTS_ILLEGAL_INTERFACE | @Weave classes cannot implement interfaces that the original class does not implement |
| CLASS_NESTED_IMPLICIT_OUTER_ACCESS_UNSUPPORTED | Nested classes cannot implicitly access outer fields/methods in @Weave classes |
| CLASS_MISSING_REQUIRED_ANNOTATIONS | Class does not contain at least one of the required class annotations |
| ENUM_NEW_FIELD | Enums cannot add new fields |
| FIELD_TYPE_MISMATCH | Cannot match fields with the same name but different types |
| FIELD_FINAL_MISMATCH | Cannot match a non-final field with a final field |
| FIELD_FINAL_ASSIGNMENT | Cannot assign a value other than Weaver.callOriginal to a matched final field |
| FIELD_STATIC_MISMATCH | Cannot match a non-static field with a static field |
| FIELD_ACCESS_MISMATCH | Field access levels do not match |
| FIELD_PRIVATE_BASE_CLASS_MATCH | Cannot match a private field when using base class matching |
| FIELD_SERIALVERSIONUID_UNSUPPORTED | Cannot define a serialVersionUID field |
| METHOD_CALL_ORIGINAL_ALLOWED_ONLY_ONCE | @Weave methods may only invoke Weaver.callOriginal once |
| METHOD_CALL_ORIGINAL_ILLEGAL_RETURN_TYPE | Return type of Weaver.callOriginal does not match original return type |
| METHOD_EXACT_ABSTRACT_WEAVE | Exact matches cannot weave abstract methods |
| METHOD_BASE_CONCRETE_WEAVE | Base matches cannot weave concrete parent methods |
| METHOD_INDIRECT_INTERFACE_WEAVE | Cannot weave indirect interface methods |
| METHOD_MISSING_REQUIRED_ANNOTATIONS | Method does not contain at least one of the required method annotations |
| METHOD_NEW_CALL_ORIGINAL_UNSUPPORTED | New @Weave methods may not invoke Weaver.callOriginal |
| METHOD_NEW_INVOKE_UNSUPPORTED | Cannot invoke a new method from another new method |
| METHOD_NEW_ABSTRACT_UNSUPPORTED | Cannot define a new abstract method |
| METHOD_NEW_NON_PRIVATE_UNSUPPORTED | New methods must be declared private |
| METHOD_NATIVE_UNSUPPORTED | Native method weaving is unsupported |
| METHOD_STATIC_MISMATCH | Cannot match a non-static method with a static method |
| METHOD_ACCESS_MISMATCH | Method access levels do not match |
| METHOD_THROWS_MISMATCH | Method throws do not match. Ensure that the weave method only throws exceptions that exist in the original method signature |
| METHOD_SYNTHETIC_WEAVE_ILLEGAL | Cannot weave a synthetic method |
| METHOD_RETURNTYPE_MISMATCH | Matched methods must have the same return type |
| NON_VOID_NO_PARAMETERS_WEAVE_ALL_METHODS | @WeaveIntoAllMethods can only be applied to a method with no parameters and void return type |
| NON_STATIC_WEAVE_INTO_ALL_METHODS | @WeaveIntoAllMethods must be static |
| INIT_NEW_UNSUPPORTED | Cannot add new constructors in a @Weave class |
| INIT_WITH_ARGS_INTERFACE_MATCH_UNSUPPORTED | Only no-argument constructors are allowed in @Weave classes that match interfaces |
| INIT_WEAVE_ALL_NO_OTHER_INIT_ALLOWED | Only one constructor is allowed in @Weave classes when @WeaveAllConstructors is present |
| INIT_WEAVE_ALL_WITH_ARGS_PROHIBITED | Cannot apply @WeaveAllConstructors to constructor with arguments |
| INIT_ILLEGAL_CALL_ORIGINAL | Only matched members can be initialized with Weaver.callOriginal |
| CLINIT_MATCHED_FIELD_MODIFICATION_UNSUPPORTED | Cannot modify the value of a matched static field |
| CLINIT_FIELD_ACCESS_VIOLATION | Cannot access a matched private or protected static field during weave class initialization |
| CLINIT_METHOD_ACCESS_VIOLATION | Cannot call a matched private or protected static method during weave class initialization |
| MISSING_ORIGINAL_BYTECODE | Could not find original bytecode |
| BOOTSTRAP_CLASS | Cannot weave a bootstrap class without java.lang.instrument.Instrumentation |
| INVALID_REFERENCE | Code in the weave packages references a class in a way that does not match the original code |
| LANGUAGE_ADAPTER_VIOLATION | Non-Java violation. |
| UNEXPECTED_NEW_FIELD_ANNOTATION | Field is marked with @NewField, but is matched under the current ClassLoader |
| EXPECTED_NEW_FIELD_ANNOTATION | Field is not marked with @NewField, but is not matched under the current ClassLoader |
| ILLEGAL_CLASS_NAME | Cannot define a new class with the same name as an original class |
| INCOMPATIBLE_BYTECODE_VERSION | The major version of the weave bytecode is higher than the jvm supports. |
| SKIP_IF_PRESENT | Encountered illegal original class. |
| MULTIPLE_WEAVE_ALL_METHODS | Encountered more than one method with @WeaveIntoAllMethods |