[XABT] Avoid non-blittable types in native callback methods #9724

jpobst · 2025-01-29T22:56:13Z

Context: dotnet/java-interop#1296
Context: dotnet/java-interop#1027

Changing callback methods to blittable types (bool -> sbyte and char -> ushort) causes warnings when the marshal method classifier verifies that the marshal types match the native types:

Xamarin.Android.Common.targets(1502,3): Method 'System.SByte Java.Lang.Object::n_Equals_Ljava_lang_Object_(System.IntPtr,System.IntPtr,System.IntPtr)' doesn't match native callback signature (invalid return type: expected 'System.Boolean', found 'System.SByte')
Xamarin.Android.Common.targets(1502,3): Unable to find native callback method 'n_Equals_Ljava_lang_Object_' in type 'Java.Lang.Object', matching the 'System.Boolean Java.Lang.Object::Equals(Java.Lang.Object)' signature (jniName: 'equals') [Arch: X86_64; Assembly: obj/Release/android-x64/linked/Mono.Android.dll]
Xamarin.Android.Common.targets(1502,3): Method 'System.Boolean Java.Lang.Object::Equals(Java.Lang.Object)' will be registered dynamically [Arch: X86_64; Assembly: obj/Release/android-x64/linked/Mono.Android.dll]
Xamarin.Android.Common.targets(1502,3): Method 'System.SByte Java.Lang.Object::n_Equals_Ljava_lang_Object_(System.IntPtr,System.IntPtr,System.IntPtr)' doesn't match native callback signature (invalid return type: expected 'System.Boolean', found 'System.SByte')
Xamarin.Android.Common.targets(1502,3): Unable to find native callback method 'n_Equals_Ljava_lang_Object_' in type 'Java.Lang.Object', matching the 'System.Boolean Java.Lang.Object::Equals(Java.Lang.Object)' signature (jniName: 'equals') [Arch: Arm64; Assembly: obj/Release/android-arm64/linked/Mono.Android.dll]
Xamarin.Android.Common.targets(1502,3): Method 'System.Boolean Java.Lang.Object::Equals(Java.Lang.Object)' will be registered dynamically [Arch: Arm64; Assembly: obj/Release/android-arm64/linked/Mono.Android.dll]
Xamarin.Android.Common.targets(1502,3): Type 'Android.Runtime.JavaObject, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=84e04ff9cfb79065' will register some of its Java override methods dynamically. This may adversely affect runtime performance. See preceding warnings for names of dynamically registered methods.
Xamarin.Android.Common.targets(1502,3): [Arm64] Number of methods in the project that will be registered dynamically: 1
Xamarin.Android.Common.targets(1502,3): [X86_64] Number of methods in the project that will be registered dynamically: 1

Fix this by treating our blittable types as equivalent to their native types in the marshal method classifier.

Additionally, Mono.Android has "built-in delegates" that marshal bool types (eg: _JniMarshal_PP_Z). Because generator no longer creates these delegates (it will now create _JniMarshal_PP_B instead), we need to add them manually.

jonpryor · 2025-02-10T21:09:09Z

src/Xamarin.Android.Build.Tasks/Utilities/MarshalMethodsClassifier.cs

+
+			// Because these types are marshaled as different blittable types,
+			// we need to accept them as equivalent
+			static readonly Tuple<string, string>[] equivalent_types = new [] {


I would prefer the use of "named ValueTuple<…>":

static readonly (string Source, string Replacement)[] equivalent_types = new[]{ (Source: "System.Boolean", Replacement: "System.SByte"), (Source: "System.Char", Replacement: "System.UInt16"), };

Then TypeMatches() can use .Source instead of .Item1, etc.

jonpryor · 2025-02-10T21:52:06Z

src/Xamarin.Android.Build.Tasks/Utilities/MarshalMethodsClassifier.cs

+				(Source: "System.Char", Replacement: "System.UInt16"),
+			];
+
+			static bool TypeMatches (string type, string methodType)


The need for this entire method confuses me.

From the PR message:

Changing callback methods to blittable types (bool -> sbyte and char -> ushort) causes warnings when the marshal method classifier verifies that the marshal types match the native types:

Xamarin.Android.Common.targets(1502,3): Method 'System.SByte Java.Lang.Object::n_Equals_Ljava_lang_Object_(System.IntPtr,System.IntPtr,System.IntPtr)' doesn't match native callback signature (invalid return type: expected 'System.Boolean', found 'System.SByte')

but why? The invalid return type: expected message is from here:

android/src/Xamarin.Android.Build.Tasks/Utilities/MarshalMethodsClassifier.cs

Lines 211 to 214 in dbda951

if (String.Compare (returnType, method.ReturnType.FullName, StringComparison.Ordinal) != 0) {

log.LogWarning ($"Method '{method.FullName}' doesn't match native callback signature (invalid return type: expected '{returnType}', found '{method.ReturnType.FullName}')");

return false;

}

returnType is MapType(target.ReturnType), which uses the verbatimTypes set, and thus should be System.SByte (which is in verbatimTypes), but apparently isn't?

Why doesn't the existing code work? Why is this method needed?

If I'm understanding what this code does, it is trying to match bound methods to their native callback methods.

Thus, given this bound method:

[Register ("markSupported", "()Z", "GetMarkSupportedHandler")] public virtual unsafe bool MarkSupported () { ... }

It is trying to find this callback with a matching signature:

static bool n_MarkSupported (IntPtr jnienv, IntPtr native__this) { ... }

However, because the native callback is now sbyte instead of bool the return types no longer match and thus the "new" callback is rejected because bool != sbyte:

static sbyte n_MarkSupported (IntPtr jnienv, IntPtr native__this) { ... }

The method I added simply says that for this purpose bool and sbyte are the same.

Fixes: #1027 Context: dotnet/android@8bc7a3e Context: 356485e Context: dotnet/android#9724 `generator --codegen-target=XAJavaInterop1` -emitted marshal methods can involve non-[blittable][0] types, specifically `bool` and `char`. Consider: namespace Java.Nio.Charset; partial class CharsetEncoder { public virtual unsafe bool CanEncode (char c) => … static bool n_CanEncode_C (IntPtr jnienv, IntPtr native__this, char c) => … } Historically this hadn't mattered, because we always had a runtime marshaler. This started changing with the introduction of LLVM Marshal Methods in dotnet/android@8bc7a3e8: instead of the previous method registration approach, in which a bunch of `Delegate` instances were collected and passed to `JNIEnv::RegisterNatives()`, the idea was to instead emit LLVM-IR for a `Java_…` native symbol which would lookup and invoke a [`[UnmanagedCallersOnly]`][1]-attributed method, and `[UnmanagedCallersOnly]` requires that all parameter and return types be *blittable*. There were two potential solutions: 1. Add a wrapper around the marshal method which converts parameter and return types, or 2. Skip them entirely, and require use of `JNIEnv::RegisterNatives()` for types overriding Java methods which involve `bool` or `char`. dotnet/android@8bc7a3e8 went with (2). ((1) is complicated.) We now introduce solution (3): update `generator` so that the generated marshal methods always use blittable types: * We marshal `bool` as `sbyte` * We marshal `char` as `ushort`. `generator` has built in support for conversions when the native type does not match the managed type, via `ISymbol.ToNative()` and `ISymbol.FromNative()`. However this affects marshaling both ways: managed to Java and Java to managed. We do not want to affect managed to Java marshaling, so we introduce a new `ISymbol.OnlyFormatOnMarshal` property to control this. This also requires some changes in `dotnet/android`; see dotnet/android#9724. [0]: https://learn.microsoft.com/en-us/dotnet/framework/interop/blittable-and-non-blittable-types [1]: https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.unmanagedcallersonlyattribute?view=net-6.0

jonpryor · 2025-02-11T20:55:28Z

Draft commit message:

Bump to dotnet/java-interop/main@57f7bc84 (#9724)

Changes: https://github.com/dotnet/java-interop/compare/6bc87e8b55bc00ae1423a5ae92cf5db573fc76ed...57f7bc849f36ec7c3159cc7729c7276b8dcd3ca4

  * dotnet/java-interop@57f7bc84: [generator] Avoid non-blittable types in native callback methods (dotnet/java-interop#1296)
  * dotnet/java-interop@9369d4fd: [tests] fix `NU1510` warning as error (dotnet/java-interop#1306)

Context: https://github.com/dotnet/java-interop/commit/57f7bc849f36ec7c3159cc7729c7276b8dcd3ca4
Context: https://github.com/dotnet/java-interop/issues/1027

dotnet/java-interop@57f7bc84 updated
`generator --codegen-target=XAJavaInterop1` marshal methods to
contain only blittable types in parameter and return types.
Instead of `bool`, use `sbyte`; instead of `char`, use `ushort`.

This change causes warnings when `MarshalMethodsClassifier` verifies
that the marshal types match the native types:

	…/Xamarin.Android.Common.targets(1502,3): Method 'System.SByte Java.Lang.Object::n_Equals_Ljava_lang_Object_(System.IntPtr,System.IntPtr,System.IntPtr)' doesn't match native callback signature (invalid return type: expected 'System.Boolean', found 'System.SByte')
	…/Xamarin.Android.Common.targets(1502,3): Unable to find native callback method 'n_Equals_Ljava_lang_Object_' in type 'Java.Lang.Object', matching the 'System.Boolean Java.Lang.Object::Equals(Java.Lang.Object)' signature (jniName: 'equals') [Arch: X86_64; Assembly: obj/Release/android-x64/linked/Mono.Android.dll]
	…/Xamarin.Android.Common.targets(1502,3): Method 'System.Boolean Java.Lang.Object::Equals(Java.Lang.Object)' will be registered dynamically [Arch: X86_64; Assembly: obj/Release/android-x64/linked/Mono.Android.dll]
	…/Xamarin.Android.Common.targets(1502,3): Method 'System.SByte Java.Lang.Object::n_Equals_Ljava_lang_Object_(System.IntPtr,System.IntPtr,System.IntPtr)' doesn't match native callback signature (invalid return type: expected 'System.Boolean', found 'System.SByte')
	…/Xamarin.Android.Common.targets(1502,3): Unable to find native callback method 'n_Equals_Ljava_lang_Object_' in type 'Java.Lang.Object', matching the 'System.Boolean Java.Lang.Object::Equals(Java.Lang.Object)' signature (jniName: 'equals') [Arch: Arm64; Assembly: obj/Release/android-arm64/linked/Mono.Android.dll]
	…/Xamarin.Android.Common.targets(1502,3): Method 'System.Boolean Java.Lang.Object::Equals(Java.Lang.Object)' will be registered dynamically [Arch: Arm64; Assembly: obj/Release/android-arm64/linked/Mono.Android.dll]
	…/Xamarin.Android.Common.targets(1502,3): Type 'Android.Runtime.JavaObject, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=84e04ff9cfb79065' will register some of its Java override methods dynamically. This may adversely affect runtime performance. See preceding warnings for names of dynamically registered methods.
	…/Xamarin.Android.Common.targets(1502,3): [Arm64] Number of methods in the project that will be registered dynamically: 1
	…/Xamarin.Android.Common.targets(1502,3): [X86_64] Number of methods in the project that will be registered dynamically: 1

These warnings occur because the marshal method signature now differs
from the public API signature.  Consider:

	namespace Java.Lang;

	partial class Object {
	    static sbyte n_Equals_Ljava_lang_Object_ (IntPtr jnienv, IntPtr native__this, IntPtr native_obj) => …

	    [Register ("equals", "(Ljava/lang/Object;)Z", "GetEquals_Ljava_lang_Object_Handler")]
	    public virtual unsafe bool Equals (Java.Lang.Object? obj) => …
	}

`MarshalMethodsClassifier` uses the 3rd `connector` parameter to
`RegisterAttribute` to determine the name of the marshal method --
string processing `GetEquals_Ljava_lang_Object_Handler` to obtain
`n_Equals_Ljava_lang_Object_` -- and then checks that the parameter
and return types match between the two methods.

Now that we're emitting blittable types, they *don't* match, thus the
warning.

Fix this by treating our blittable types as equivalent to their
native types in `MarshalMethodsClassifier`.

Additionally, `Mono.Android` has "built-in delegates" that marshal
`bool` types, e.g. `_JniMarshal_PP_Z`.  Because `generator` no longer
creates these delegates -- it will now create `_JniMarshal_PP_B`
instead -- we need to add them manually.

Changes: dotnet/java-interop@6bc87e8...57f7bc8 * dotnet/java-interop@57f7bc84: [generator] Avoid non-blittable types in native callback methods (dotnet/java-interop#1296) * dotnet/java-interop@9369d4fd: [tests] fix `NU1510` warning as error (dotnet/java-interop#1306) Context: dotnet/java-interop@57f7bc8 Context: dotnet/java-interop#1027 dotnet/java-interop@57f7bc84 updated `generator --codegen-target=XAJavaInterop1` marshal methods to contain only blittable types in parameter and return types. Instead of `bool`, use `sbyte`; instead of `char`, use `ushort`. This change causes warnings when `MarshalMethodsClassifier` verifies that the marshal types match the native types: …/Xamarin.Android.Common.targets(1502,3): Method 'System.SByte Java.Lang.Object::n_Equals_Ljava_lang_Object_(System.IntPtr,System.IntPtr,System.IntPtr)' doesn't match native callback signature (invalid return type: expected 'System.Boolean', found 'System.SByte') …/Xamarin.Android.Common.targets(1502,3): Unable to find native callback method 'n_Equals_Ljava_lang_Object_' in type 'Java.Lang.Object', matching the 'System.Boolean Java.Lang.Object::Equals(Java.Lang.Object)' signature (jniName: 'equals') [Arch: X86_64; Assembly: obj/Release/android-x64/linked/Mono.Android.dll] …/Xamarin.Android.Common.targets(1502,3): Method 'System.Boolean Java.Lang.Object::Equals(Java.Lang.Object)' will be registered dynamically [Arch: X86_64; Assembly: obj/Release/android-x64/linked/Mono.Android.dll] …/Xamarin.Android.Common.targets(1502,3): Method 'System.SByte Java.Lang.Object::n_Equals_Ljava_lang_Object_(System.IntPtr,System.IntPtr,System.IntPtr)' doesn't match native callback signature (invalid return type: expected 'System.Boolean', found 'System.SByte') …/Xamarin.Android.Common.targets(1502,3): Unable to find native callback method 'n_Equals_Ljava_lang_Object_' in type 'Java.Lang.Object', matching the 'System.Boolean Java.Lang.Object::Equals(Java.Lang.Object)' signature (jniName: 'equals') [Arch: Arm64; Assembly: obj/Release/android-arm64/linked/Mono.Android.dll] …/Xamarin.Android.Common.targets(1502,3): Method 'System.Boolean Java.Lang.Object::Equals(Java.Lang.Object)' will be registered dynamically [Arch: Arm64; Assembly: obj/Release/android-arm64/linked/Mono.Android.dll] …/Xamarin.Android.Common.targets(1502,3): Type 'Android.Runtime.JavaObject, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=84e04ff9cfb79065' will register some of its Java override methods dynamically. This may adversely affect runtime performance. See preceding warnings for names of dynamically registered methods. …/Xamarin.Android.Common.targets(1502,3): [Arm64] Number of methods in the project that will be registered dynamically: 1 …/Xamarin.Android.Common.targets(1502,3): [X86_64] Number of methods in the project that will be registered dynamically: 1 These warnings occur because the marshal method signature now differs from the public API signature. Consider: namespace Java.Lang; partial class Object { static sbyte n_Equals_Ljava_lang_Object_ (IntPtr jnienv, IntPtr native__this, IntPtr native_obj) => … [Register ("equals", "(Ljava/lang/Object;)Z", "GetEquals_Ljava_lang_Object_Handler")] public virtual unsafe bool Equals (Java.Lang.Object? obj) => … } `MarshalMethodsClassifier` uses the 3rd `connector` parameter to `RegisterAttribute` to determine the name of the marshal method -- string processing `GetEquals_Ljava_lang_Object_Handler` to obtain `n_Equals_Ljava_lang_Object_` -- and then checks that the parameter and return types match between the two methods. Now that we're emitting blittable types, they *don't* match, thus the warning. Fix this by treating our blittable types as equivalent to their native types in `MarshalMethodsClassifier`. Additionally, `Mono.Android` has "built-in delegates" that marshal `bool` types, e.g. `_JniMarshal_PP_Z`. Because `generator` no longer creates these delegates -- it will now create `_JniMarshal_PP_B` instead -- we need to add them manually.

* main: Bump to dotnet/java-interop@57f7bc84 (#9724) [XABT] Fix Xamarin.Android.Build.Tasks.sln (#9782) [java-runtime, Mono.Android] Use `ApplicationRegistration.Context` (#9779) [Xamarin.Android.Build.Tasks] temporarily support .NET 8 in .NET 10 (#9777) Bump to dotnet/sdk@c0f88f2416 10.0.100-preview.2.25108.4 (#9775) [docs] update dotnetbuilds domain (#9772)

jpobst force-pushed the dev/jpobst/blittable branch 4 times, most recently from a23c154 to 38e8ca4 Compare February 4, 2025 18:12

jpobst changed the title ~~Test JI 1296.~~ [XABT] Avoid non-blittable types in native callback methods Feb 4, 2025

jpobst mentioned this pull request Feb 6, 2025

[generator] Avoid non-blittable types in native callback methods dotnet/java-interop#1296

Merged

jpobst force-pushed the dev/jpobst/blittable branch from 38e8ca4 to bffa04c Compare February 6, 2025 23:29

Test JI 1296.

58858cd

jpobst force-pushed the dev/jpobst/blittable branch from bffa04c to 58858cd Compare February 7, 2025 18:03

jpobst marked this pull request as ready for review February 10, 2025 18:54

jpobst requested review from grendello and jonpryor as code owners February 10, 2025 18:54

grendello approved these changes Feb 10, 2025

View reviewed changes

jonpryor reviewed Feb 10, 2025

View reviewed changes

Address review feedback.

12955f7

jonpryor reviewed Feb 10, 2025

View reviewed changes

Update JI.

a95a284

jonpryor merged commit c7f9f5c into main Feb 12, 2025
57 checks passed

jonpryor deleted the dev/jpobst/blittable branch February 12, 2025 01:26

github-actions bot locked and limited conversation to collaborators Mar 14, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[XABT] Avoid non-blittable types in native callback methods #9724

[XABT] Avoid non-blittable types in native callback methods #9724

jpobst commented Jan 29, 2025 •

edited

Loading

jonpryor Feb 10, 2025

jonpryor Feb 10, 2025

jpobst Feb 10, 2025

jonpryor commented Feb 11, 2025 •

edited

Loading

	if (String.Compare (returnType, method.ReturnType.FullName, StringComparison.Ordinal) != 0) {
	log.LogWarning ($"Method '{method.FullName}' doesn't match native callback signature (invalid return type: expected '{returnType}', found '{method.ReturnType.FullName}')");
	return false;
	}

[XABT] Avoid non-blittable types in native callback methods #9724

[XABT] Avoid non-blittable types in native callback methods #9724

Conversation

jpobst commented Jan 29, 2025 • edited Loading

jonpryor Feb 10, 2025

Choose a reason for hiding this comment

jonpryor Feb 10, 2025

Choose a reason for hiding this comment

jpobst Feb 10, 2025

Choose a reason for hiding this comment

jonpryor commented Feb 11, 2025 • edited Loading

jpobst commented Jan 29, 2025 •

edited

Loading

jonpryor commented Feb 11, 2025 •

edited

Loading