[Android] Library tests failing on CoreCLR Android

[kotlarmilos]

Description

These failures occurred while standing up the library tests on CoreCLR Android in #114148.

Disabled tests fall into two categories.

Globalization or other mobile‐platform issues

These tests have been disabled because they depend on invariant‐culture behaviors or platform APIs that aren’t fully supported on mobile targets:

System.Runtime\tests\System.Globalization.Tests\Invariant\Invariant.Tests.csproj
System.Net.Http\tests\FunctionalTests\System.Net.Http.Functional.Tests.csproj
System.Diagnostics.DiagnosticSource\tests\TestWithConfigSwitches\System.Diagnostics.DiagnosticSource.Switches.Tests.csproj
System.Reflection.MetadataLoadContext\tests\System.Reflection.MetadataLoadContext.Tests.csproj
System.Runtime\tests\System.Resources.ResourceManager.Tests\System.Resources.ResourceManager.Tests.csproj
System.Runtime\tests\System.Runtime.Tests\InvariantTimezone\System.Runtime.InvariantTimezone.Tests.csproj
System.Runtime.Loader\tests\System.Runtime.Loader.Tests.csproj
System.Runtime\tests\System.Text.Encoding.Tests\System.Text.Encoding.Tests.csproj
System.Runtime\tests\System.IO.Tests\System.IO.Tests.csproj
Microsoft.Bcl.Cryptography\tests\Microsoft.Bcl.Cryptography.Tests.csproj
System.Net.Security\tests\FunctionalTests\System.Net.Security.Tests.csproj
System.Net.WebSockets\tests\System.Net.WebSockets.Tests.csproj

Logs

System.Diagnostics.DiagnosticSource.Switches.Tests

1)  [FAIL] System.Diagnostics.Tests.ActivityTests.ActivityIdNonHierarchicalOverflow   Test name: System.Diagnostics.Tests.ActivityTests.ActivityIdNonHierarchicalOverflow
Assembly:  [System.Diagnostics.DiagnosticSource.Switches.Tests, Version=10.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51]
Exception messages: Assert.InRange() Failure: Value not in range
Range:  (55 - 56)
Actual: 27   Exception stack traces:    at System.Diagnostics.Tests.ActivityTests.ActivityIdNonHierarchicalOverflow() in /_/src/libraries/System.Diagnostics.DiagnosticSource/tests/TestWithConfigSwitches/ActivityTests.cs:line 31
at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
Execution time: 0.0333031
2)  [FAIL] System.Diagnostics.Tests.ActivityTests.IdFormat_ZeroTraceIdAndSpanIdWithHierarchicalFormat   Test name: System.Diagnostics.Tests.ActivityTests.IdFormat_ZeroTraceIdAndSpanIdWithHierarchicalFormat
Assembly:  [System.Diagnostics.DiagnosticSource.Switches.Tests, Version=10.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51]
Exception messages: Assert.Equal() Failure: Values differ
Expected: Hierarchical
Actual:   W3C   Exception stack traces:    at System.Diagnostics.Tests.ActivityTests.IdFormat_ZeroTraceIdAndSpanIdWithHierarchicalFormat() in /_/src/libraries/System.Diagnostics.DiagnosticSource/tests/TestWithConfigSwitches/ActivityTests.cs:line 95
at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
Execution time: 0.0041994
3)  [FAIL] System.Diagnostics.Tests.ActivityTests.IdFormat_HierarchicalIsDefault   Test name: System.Diagnostics.Tests.ActivityTests.IdFormat_HierarchicalIsDefault
Assembly:  [System.Diagnostics.DiagnosticSource.Switches.Tests, Version=10.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51]
Exception messages: Assert.Equal() Failure: Values differ
Expected: Hierarchical
Actual:   W3C   Exception stack traces:    at System.Diagnostics.Tests.ActivityTests.IdFormat_HierarchicalIsDefault() in /_/src/libraries/System.Diagnostics.DiagnosticSource/tests/TestWithConfigSwitches/ActivityTests.cs:line 87
at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
Execution time: 0.0006887
4)  [FAIL] System.Diagnostics.Tests.ActivityTests.IdGenerationInternalParent   Test name: System.Diagnostics.Tests.ActivityTests.IdGenerationInternalParent
Assembly:  [System.Diagnostics.DiagnosticSource.Switches.Tests, Version=10.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51]
Exception messages: Expected: |165c37dd244530c2361bd1b5eb6d94da.child1-1. || |165c37dd244530c2361bd1b5eb6d94da.1.
Actual: 00-165c37dd244530c2361bd1b5eb6d94da-5b0ea8fb01fb2e2a-00   Exception stack traces:    at System.AssertExtensions.AtLeastOneEquals[T](T expected1, T expected2, T value) in /_/src/libraries/Common/tests/TestUtilities/System/AssertExtensions.cs:line 584
at System.Diagnostics.Tests.ActivityTests.IdGenerationInternalParent() in /_/src/libraries/System.Diagnostics.DiagnosticSource/tests/TestWithConfigSwitches/ActivityTests.cs:line 54
at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
Execution time: 0.0264644

System.IO.Tests

lowmemorykiller: Kill 'com.android.deskclock' (2960), uid 10108, oom_adj 975 to free 76688kB
lowmemorykiller: Reclaimed 76688kB, cache(320948kB) and free(56036kB)-reserved(80816kB) below min(322560kB) for oom_adj 950
Zygote  : Process 2960 exited due to signal 9 (Killed)
InputMethodManagerService: Session failed to close due to remote exception
InputMethodManagerService: android.os.DeadObjectException
InputMethodManagerService:    at android.os.BinderProxy.transactNative(Native Method)
InputMethodManagerService:    at android.os.BinderProxy.transact(BinderProxy.java:510)
InputMethodManagerService:    at com.android.internal.view.IInputMethodSession$Stub$Proxy.finishSession(IInputMethodSession.java:419)
InputMethodManagerService:    at com.android.server.inputmethod.InputMethodManagerService.finishSessionLocked(InputMethodManagerService.java:2333)
InputMethodManagerService:    at com.android.server.inputmethod.InputMethodManagerService.clearClientSessionLocked(InputMethodManagerService.java:2324)
InputMethodManagerService:    at com.android.server.inputmethod.InputMethodManagerService.clearCurMethodLocked(InputMethodManagerService.java:2351)
InputMethodManagerService:    at com.android.server.inputmethod.InputMethodManagerService.onServiceDisconnected(InputMethodManagerService.java:2378)
InputMethodManagerService:    at android.app.LoadedApk$ServiceDispatcher.doDeath(LoadedApk.java:1967)
InputMethodManagerService:    at android.app.LoadedApk$ServiceDispatcher$RunConnection.run(LoadedApk.java:1982)
InputMethodManagerService:    at android.os.Handler.handleCallback(Handler.java:883)
InputMethodManagerService:    at android.os.Handler.dispatchMessage(Handler.java:100)
InputMethodManagerService:    at android.os.Looper.loop(Looper.java:214)
InputMethodManagerService:    at com.android.server.SystemServer.run(SystemServer.java:541)
InputMethodManagerService:    at com.android.server.SystemServer.main(SystemServer.java:349)
InputMethodManagerService:    at java.lang.reflect.Method.invoke(Native Method)
InputMethodManagerService:    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:492)
InputMethodManagerService:    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:908)
AndroidRuntime: Shutting down VM

System.Resources.ResourceManager.Tests

[FAIL] System.Resources.Tests.ResourceManagerTests.GetString_CultureFallback(key: "OneLoc", cultureName: "es", expectedValue: "Value-One(es)")
 Assert.Equal() Failure: Strings differ
                     ↓ (pos 9)
 Expected: "Value-One(es)"
 Actual:   "Value-One"
    at System.Resources.Tests.ResourceManagerTests.GetString_CultureFallback(String key, String cultureName, String expectedValue) in /_/src/libraries/System.Runtime/tests/System.Resources.ResourceManager.Tests/ResourceManagerTests.cs:line 96
    at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
    at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
[FAIL] System.Resources.Tests.ResourceManagerTests.GetString_CultureFallback(key: "OneLoc", cultureName: "es-ES", expectedValue: "Value-One(es)")
 Assert.Equal() Failure: Strings differ
                     ↓ (pos 9)
 Expected: "Value-One(es)"
 Actual:   "Value-One"
    at System.Resources.Tests.ResourceManagerTests.GetString_CultureFallback(String key, String cultureName, String expectedValue) in /_/src/libraries/System.Runtime/tests/System.Resources.ResourceManager.Tests/ResourceManagerTests.cs:line 96
    at InvokeStub_ResourceManagerTests.GetString_CultureFallback(Object, Span`1)
    at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

System.Runtime.InvariantTimezone.Tests

[FAIL] System.Tests.TimeZoneInfoTests.IsInvariant
 Assert.True() Failure
 Expected: True

System.Text.Tests

[FAIL] System.Text.Tests.EncodingGetEncodingTest.GetEncoding_WebName
 System.NotSupportedException : Support for UTF-7 is disabled. See https://aka.ms/dotnet-warnings/SYSLIB0001 for more information.
    at System.Text.Encoding.GetEncoding(Int32 codepage)
    at System.Text.Tests.EncodingGetEncodingTest.GetEncoding_WebName() in /_/src/libraries/System.Runtime/tests/System.Text.Encoding.Tests/Encoding/EncodingGetEncodingTests.cs:line 123
    at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
    at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] 
 
 Actual:   False
     [FAIL] System.Resources.Tests.ResourceManagerTests.GetString_CultureFallback(key: "OneLoc", cultureName: "es-MX", expectedValue: "Value-One(es-MX)")
 Assert.Equal() Failure: Strings differ
                     ↓ (pos 9)
 Expected: "Value-One(es-MX)"
 Actual:   "Value-One"
    at System.Resources.Tests.ResourceManagerTests.GetString_CultureFallback(String key, String cultureName, String expectedValue) in /_/src/libraries/System.Runtime/tests/System.Resources.ResourceManager.Tests/ResourceManagerTests.cs:line 96
    at InvokeStub_ResourceManagerTests.GetString_CultureFallback(Object, Span`1)
    at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

System.Text.RegularExpressions.Tests

[FAIL] System.Text.RegularExpressions.Tests.UpgradeToGeneratedRegexAnalyzerTests.CodeFixerDoesNotSimplifyStyle   Test name: System.Text.RegularExpressions.Tests.UpgradeToGeneratedRegexAnalyzerTests.CodeFixerDoesNotSimplifyStyle
Assembly:  [System.Text.RegularExpressions.Tests, Version=10.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51]
Exception messages: System.TypeInitializationException : The type initializer for 'System.Text.RegularExpressions.Tests.RegexGeneratorHelper' threw an exception.
---- System.ArgumentException : The value cannot be an empty string. (Parameter 'path')   Exception stack traces:    at System.Text.RegularExpressions.Tests.RegexGeneratorHelper.get_References() in /_/src/libraries/System.Text.RegularExpressions/tests/FunctionalTests/RegexGeneratorHelper.netcoreapp.cs:line 31
at System.Text.RegularExpressions.Tests.CSharpCodeFixVerifier`2.Test..ctor(ReferenceAssemblies references) in /_/src/libraries/System.Text.RegularExpressions/tests/FunctionalTests/CSharpCodeFixVerifier`2.cs:line 48
at System.Text.RegularExpressions.Tests.CSharpCodeFixVerifier`2.VerifyCodeFixAsync(String source, String fixedSource, ReferenceAssemblies references) in /_/src/libraries/System.Text.RegularExpressions/tests/FunctionalTests/CSharpCodeFixVerifier`2.cs:line 26
at System.Text.RegularExpressions.Tests.UpgradeToGeneratedRegexAnalyzerTests.CodeFixerDoesNotSimplifyStyle() in /_/src/libraries/System.Text.RegularExpressions/tests/FunctionalTests/UpgradeToGeneratedRegexAnalyzerTests.cs:line 836
--- End of stack trace from previous location ---
----- Inner Stack Trace -----
at System.ArgumentException.ThrowNullOrEmptyException(String argument, String paramName)
at System.IO.Strategies.FileStreamHelpers.ValidateArguments(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, Int64 preallocationSize)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Roslyn.Utilities.StandardFileSystem.OpenFile(String filePath, FileMode mode, FileAccess access, FileShare share)
at Roslyn.Utilities.CommonCompilerFileSystemExtensions.OpenFileWithNormalizedException(ICommonCompilerFileSystem fileSystem, String filePath, FileMode fileMode, FileAccess fileAccess, FileShare fileShare)
at Microsoft.CodeAnalysis.MetadataReference.CreateFromFile(String path, MetadataReferenceProperties properties, DocumentationProvider documentation)
at System.Text.RegularExpressions.Tests.RegexGeneratorHelper.CreateReferences() in /_/src/libraries/System.Text.RegularExpressions/tests/FunctionalTests/RegexGeneratorHelper.netcoreapp.cs:line 44
at System.Text.RegularExpressions.Tests.RegexGeneratorHelper..cctor() in /_/src/libraries/System.Text.RegularExpressions/tests/FunctionalTests/RegexGeneratorHelper.netcoreapp.cs:line 31
Execution time: 0.0875516

System.Net.Security.Tests

GetJNIEnv: Registering JNI thread detach. env ptr 0x738dbd214200. Key: -2147483632
 Background young concurrent copying GC freed 344674(18MB) AllocSpace objects, 3(60KB) LOS objects, 31% free, 12MB/18MB, paused 1.002ms total 207.817ms
DetachThreadFromJNI: Detaching thread from JNI
 Background young concurrent copying GC freed 149968(7387KB) AllocSpace objects, 0(0B) LOS objects, 12% free, 16MB/18MB, paused 625us total 172.622ms
GetJNIEnv: Registering JNI thread detach. env ptr 0x738db9fabc00. Key: -2147483632
DetachThreadFromJNI: Detaching thread from JNI
 javax.net.ssl.SSLPeerUnverifiedException: No peer certificates
   at com.android.org.conscrypt.ActiveSession.checkPeerCertificatesPresent(ActiveSession.java:314)
   at com.android.org.conscrypt.ActiveSession.getPeerCertificates(ActiveSession.java:180)
   at com.android.org.conscrypt.ExternalSession.getPeerCertificates(ExternalSession.java:106)
   at com.android.org.conscrypt.Java7ExtendedSSLSession.getPeerCertificates(Java7ExtendedSSLSession.java:129)
   at com.android.org.conscrypt.Java7ExtendedSSLSession.getPeerCertificates(Java7ExtendedSSLSession.java:31)

CoreCLR Android-specific failures

The following disabled tests are specific to CoreCLR Android:

Microsoft.VisualBasic.Tests.ErrObjectTests.Clear
MonoTests.System.Configuration.mapped_ExeConfiguration_null
MonoTests.System.Configuration.OpenExeConfiguration1_UserLevel_None
System.Threading.Tests.MutexTests - add a process-wide implementation of named mutex to CoreCLR on Mono to better compatibility with Mono
System.IO.Pipelines.Tests.CompleteWithLargeWriteThrows

[dotnet-policy-service]

Tagging subscribers to 'arch-android': @vitek-karas, @simonrozsival, @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

[vitek-karas]

The test System.Runtime.Loader.Tests.ContextualReflectionTest.AssemblyGetTypeStringSharedAssemblyDefault will not work on mobile. It requires access to assembly files -

runtime/src/libraries/System.Runtime.Loader/tests/ContextualReflection.cs

Line 96 in 2474c62

isolatedAlcAssembly = isolatedAlc.LoadFromAssemblyPath(defaultAlcAssembly.Location);

. Which will not work in packaged apps on mobile devices.

In general we should probably disable all tests which are also disabled in single-file modes - mobile is by default "single-file".

[akoeplinger]

do we know why these didn't fail on mono?

[kotlarmilos]

Tests that load assembly files fail on mobile due to the single-file host. I don’t have a good understanding why the globalization and diagnostics tests are failing on CoreCLR, but I suspect it’s due to a configuration mismatch.

[vitek-karas]

@akoeplinger comment is actually a really good point.
Did these tests pass on mono?

• If so, how did that work since in that case it's also single-file-like.