[6.0][Android][mono] XHarness exit code 78 PACKAGE_INSTALLATION_FAILURE

[carlossanlop]

Error Blob

{
  "ErrorMessage": "XHarness exit code: 78 (PACKAGE_INSTALLATION_FAILURE)",
  "BuildRetry": false,
  "ErrorPattern": "",
  "ExcludeConsoleLog": false
}

Reproduction Steps

Two failure variations, both showed up in the same job.

• Found in PR: [release/6.0] SyntaxValueProvider: avoid performance issue with syntax list containing many items #83742
• Queue: Build Android x64 Release AllSubsets_Mono
• Job: https://dev.azure.com/dnceng-public/public/_build/results?buildId=220445&view=logs&j=b6e5d97f-d4f8-506b-9e47-e3e32385754a&t=94ce2805-f3e2-5f9a-1d92-0da5613f733d

Failure 1

Affected tests:

• System.Private.Uri.Functional.Tests
  • Log file: https://helixre107v0xd1eu3ibi6ka.blob.core.windows.net/dotnet-runtime-refs-pull-83742-merge-86269b52b2af48c4a0/System.Private.Uri.Functional.Tests/1/console.b5411e7e.log?helixlogtype=result
• Microsoft.Extensions.Configuration.EnvironmentVariables.Tests
  • Log file: https://helixre107v0xd1eu3ibi6ka.blob.core.windows.net/dotnet-runtime-refs-pull-83742-merge-86269b52b2af48c4a0/Microsoft.Extensions.Configuration.EnvironmentVariables.Tests/1/console.317555ec.log?helixlogtype=result

Output 1 (expand)

[00:46:29] dbug: Executing command: '/datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb -s emulator-5554 start-server'
[00:46:29] dbug: 
[00:46:29] dbug: Executing command: '/datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb -s emulator-5554 install "/datadisks/disk1/work/A821097B/w/9F0308ED/e/Microsoft.Extensions.Configuration.EnvironmentVariables.Tests.apk"'
[00:46:29] fail: Error:
                 Exit code: 255
                 Standard Output:
                 Performing Push Install
                 adb: error: failed to get feature set: device offline
                 
                 Standard Error:
                 error: device offline
                 
                 
[00:46:29] crit: Install failure: Test command cannot continue
[00:46:29] info: Attempting to remove apk 'net.dot.Microsoft.Extensions.Configuration.EnvironmentVariables.Tests': 
[00:46:29] dbug: Executing command: '/datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb -s emulator-5554 uninstall net.dot.Microsoft.Extensions.Configuration.EnvironmentVariables.Tests'
[00:46:30] fail: Error: Exit code: 20
                 Standard Output:
                 
                 Standard Error:
                 error: device offline
                 - waiting for device -
                 cmd: Can't find service: package
                 
                 
[00:46:30] info: Attempting to remove apk 'net.dot.Microsoft.Extensions.Configuration.EnvironmentVariables.Tests': 
[00:46:30] dbug: Executing command: '/datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb -s emulator-5554 uninstall net.dot.Microsoft.Extensions.Configuration.EnvironmentVariables.Tests'
[00:46:30] fail: Error: Exit code: 20
                 Standard Output:
                 
                 Standard Error:
                 cmd: Can't find service: package
                 
                 
XHarness exit code: 78 (PACKAGE_INSTALLATION_FAILURE)

---

Failure 2

Affected tests:

• System.Security.SecureString.Tests
  • Log file: https://helixre107v0xd1eu3ibi6ka.blob.core.windows.net/dotnet-runtime-refs-pull-83742-merge-86269b52b2af48c4a0/System.Security.SecureString.Tests/1/console.0df13351.log?helixlogtype=result

Output 2 (expand)

[00:47:20] dbug: Executing command: '/datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb -s emulator-5554 shell getprop sys.boot_completed'
[00:47:20] dbug: sys.boot_completed = ''
[00:47:30] warn: Did not detect boot completion variable on device; variable used ('sys.boot_completed') may be incorrect or device may be in a bad state
[00:47:30] dbug: Executing command: '/datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb -s emulator-5554 version'
[00:47:30] dbug: Working with Android Debug Bridge version 1.0.41
                 Version 29.0.6-6198805
                 Installed as /datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb
                 
[00:47:30] info: Attempting to remove apk 'net.dot.System.Security.SecureString.Tests': 
[00:47:30] dbug: Executing command: '/datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb -s emulator-5554 uninstall net.dot.System.Security.SecureString.Tests'
[00:47:30] info: APK 'net.dot.System.Security.SecureString.Tests' not on device.
[00:47:30] info: Attempting to install /datadisks/disk1/work/A821097B/w/A6E409D9/e/System.Security.SecureString.Tests.apk: 
[00:47:30] dbug: Executing command: '/datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb -s emulator-5554 install "/datadisks/disk1/work/A821097B/w/A6E409D9/e/System.Security.SecureString.Tests.apk"'
[00:47:30] fail: Error:
                 Exit code: 1
                 Standard Output:
                 Performing Streamed Install
                 
                 Standard Error:
                 adb: failed to install /datadisks/disk1/work/A821097B/w/A6E409D9/e/System.Security.SecureString.Tests.apk: 
                 Exception occurred while executing:
                 java.lang.NullPointerException: Attempt to invoke virtual method 'void android.content.pm.PackageManagerInternal.freeStorage(java.lang.String, long, int)' on a null object reference
                 	at com.android.server.StorageManagerService.allocateBytes(StorageManagerService.java:3263)
                 	at android.os.storage.StorageManager.allocateBytes(StorageManager.java:2099)
                 	at android.os.storage.StorageManager.allocateBytes(StorageManager.java:2156)
                 	at com.android.server.pm.PackageInstallerSession.doWriteInternal(PackageInstallerSession.java:723)
                 	at com.android.server.pm.PackageInstallerSession.write(PackageInstallerSession.java:670)
                 	at android.content.pm.PackageInstaller$Session.write(PackageInstaller.java:951)
                 	at com.android.server.pm.PackageManagerShellCommand.doWriteSplit(PackageManagerShellCommand.java:2732)
                 	at com.android.server.pm.PackageManagerShellCommand.runInstall(PackageManagerShellCommand.java:1072)
                 	at com.android.server.pm.PackageManagerShellCommand.onCommand(PackageManagerShellCommand.java:169)
                 	at android.os.ShellCommand.exec(ShellCommand.java:104)
                 	at com.android.server.pm.PackageManagerService.onShellCommand(PackageManagerService.java:21729)
                 	at android.os.Binder.shellCommand(Binder.java:881)
                 	at android.os.Binder.onTransact(Binder.java:765)
                 	at android.content.pm.IPackageManager$Stub.onTransact(IPackageManager.java:4860)
                 	at com.android.server.pm.PackageManagerService.onTransact(PackageManagerService.java:4014)
                 	at android.os.Binder.execTransactInternal(Binder.java:1021)
                 	at android.os.Binder.execTransact(Binder.java:994)
                 
                 
[00:47:30] crit: Install failure: Test command cannot continue
[00:47:30] info: Attempting to remove apk 'net.dot.System.Security.SecureString.Tests': 
[00:47:30] dbug: Executing command: '/datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb -s emulator-5554 uninstall net.dot.System.Security.SecureString.Tests'
[00:47:30] info: APK 'net.dot.System.Security.SecureString.Tests' not on device.
[00:47:30] info: Attempting to remove apk 'net.dot.System.Security.SecureString.Tests': 
[00:47:30] dbug: Executing command: '/datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb -s emulator-5554 uninstall net.dot.System.Security.SecureString.Tests'
[00:47:30] info: APK 'net.dot.System.Security.SecureString.Tests' not on device.
XHarness exit code: 78 (PACKAGE_INSTALLATION_FAILURE)
+ exit 78

Report

Build	Definition	Test	Pull Request
989528	dotnet/runtime	Android.Device_Emulator.JIT.Test.WorkItemExecution	#113765
989488	dotnet/runtime	Android.Device_Emulator.JIT.Test.WorkItemExecution	#113666
989351	dotnet/runtime	System.Reflection.TypeExtensions.Tests.WorkItemExecution
984667	dotnet/runtime	Microsoft.Extensions.Diagnostics.Tests.WorkItemExecution
981321	dotnet/runtime	JIT_Methodical.WorkItemExecution
978095	dotnet/runtime	Common.Tests.WorkItemExecution
977774	dotnet/runtime	Microsoft.Extensions.Configuration.UserSecrets.Tests.WorkItemExecution
977393	dotnet/runtime	System.Security.Cryptography.OpenSsl.Tests.WorkItemExecution
975180	dotnet/runtime	System.Formats.Tar.Manual.Tests.WorkItemExecution
973794	dotnet/runtime	System.Reflection.Emit.Tests.WorkItemExecution
972197	dotnet/runtime	Microsoft.NETCore.Platforms.Tests.WorkItemExecution
969854	dotnet/runtime	System.Threading.RateLimiting.Tests.WorkItemExecution
958954	dotnet/runtime	Microsoft.Extensions.Logging.Tests.WorkItemExecution

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
2	4	12

Known issue validation

Build: 🔎
Result validation: ⚠️ Validation could not be done without an Azure DevOps build URL on the issue. Please add it to the "Build: 🔎" line.

Tagging subscribers to 'arch-android': @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Error Blob

{
  "ErrorMessage": "XHarness exit code: 78 (PACKAGE_INSTALLATION_FAILURE)",
  "BuildRetry": false,
  "ErrorPattern": "",
  "ExcludeConsoleLog": false
}

Reproduction Steps

Two failure variations, both showed up in the same job.

• Found in PR: [release/6.0] SyntaxValueProvider: avoid performance issue with syntax list containing many items #83742
• Queue: Build Android x64 Release AllSubsets_Mono
• Job: https://dev.azure.com/dnceng-public/public/_build/results?buildId=220445&view=logs&j=b6e5d97f-d4f8-506b-9e47-e3e32385754a&t=94ce2805-f3e2-5f9a-1d92-0da5613f733d

Failure 1

Affected tests:

• System.Private.Uri.Functional.Tests
  • Log file: https://helixre107v0xd1eu3ibi6ka.blob.core.windows.net/dotnet-runtime-refs-pull-83742-merge-86269b52b2af48c4a0/System.Private.Uri.Functional.Tests/1/console.b5411e7e.log?helixlogtype=result
• Microsoft.Extensions.Configuration.EnvironmentVariables.Tests
  • Log file: https://helixre107v0xd1eu3ibi6ka.blob.core.windows.net/dotnet-runtime-refs-pull-83742-merge-86269b52b2af48c4a0/Microsoft.Extensions.Configuration.EnvironmentVariables.Tests/1/console.317555ec.log?helixlogtype=result

Output 1 (expand)

[00:46:29] dbug: Executing command: '/datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb -s emulator-5554 start-server'
[00:46:29] dbug: 
[00:46:29] dbug: Executing command: '/datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb -s emulator-5554 install "/datadisks/disk1/work/A821097B/w/9F0308ED/e/Microsoft.Extensions.Configuration.EnvironmentVariables.Tests.apk"'
[00:46:29] fail: Error:
                 Exit code: 255
                 Standard Output:
                 Performing Push Install
                 adb: error: failed to get feature set: device offline
                 
                 Standard Error:
                 error: device offline
                 
                 
[00:46:29] crit: Install failure: Test command cannot continue
[00:46:29] info: Attempting to remove apk 'net.dot.Microsoft.Extensions.Configuration.EnvironmentVariables.Tests': 
[00:46:29] dbug: Executing command: '/datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb -s emulator-5554 uninstall net.dot.Microsoft.Extensions.Configuration.EnvironmentVariables.Tests'
[00:46:30] fail: Error: Exit code: 20
                 Standard Output:
                 
                 Standard Error:
                 error: device offline
                 - waiting for device -
                 cmd: Can't find service: package
                 
                 
[00:46:30] info: Attempting to remove apk 'net.dot.Microsoft.Extensions.Configuration.EnvironmentVariables.Tests': 
[00:46:30] dbug: Executing command: '/datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb -s emulator-5554 uninstall net.dot.Microsoft.Extensions.Configuration.EnvironmentVariables.Tests'
[00:46:30] fail: Error: Exit code: 20
                 Standard Output:
                 
                 Standard Error:
                 cmd: Can't find service: package
                 
                 
XHarness exit code: 78 (PACKAGE_INSTALLATION_FAILURE)

---

Failure 2

Affected tests:

• System.Security.SecureString.Tests
  • Log file: https://helixre107v0xd1eu3ibi6ka.blob.core.windows.net/dotnet-runtime-refs-pull-83742-merge-86269b52b2af48c4a0/System.Security.SecureString.Tests/1/console.0df13351.log?helixlogtype=result

Output 2 (expand)

[00:47:20] dbug: Executing command: '/datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb -s emulator-5554 shell getprop sys.boot_completed'
[00:47:20] dbug: sys.boot_completed = ''
[00:47:30] warn: Did not detect boot completion variable on device; variable used ('sys.boot_completed') may be incorrect or device may be in a bad state
[00:47:30] dbug: Executing command: '/datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb -s emulator-5554 version'
[00:47:30] dbug: Working with Android Debug Bridge version 1.0.41
                 Version 29.0.6-6198805
                 Installed as /datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb
                 
[00:47:30] info: Attempting to remove apk 'net.dot.System.Security.SecureString.Tests': 
[00:47:30] dbug: Executing command: '/datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb -s emulator-5554 uninstall net.dot.System.Security.SecureString.Tests'
[00:47:30] info: APK 'net.dot.System.Security.SecureString.Tests' not on device.
[00:47:30] info: Attempting to install /datadisks/disk1/work/A821097B/w/A6E409D9/e/System.Security.SecureString.Tests.apk: 
[00:47:30] dbug: Executing command: '/datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb -s emulator-5554 install "/datadisks/disk1/work/A821097B/w/A6E409D9/e/System.Security.SecureString.Tests.apk"'
[00:47:30] fail: Error:
                 Exit code: 1
                 Standard Output:
                 Performing Streamed Install
                 
                 Standard Error:
                 adb: failed to install /datadisks/disk1/work/A821097B/w/A6E409D9/e/System.Security.SecureString.Tests.apk: 
                 Exception occurred while executing:
                 java.lang.NullPointerException: Attempt to invoke virtual method 'void android.content.pm.PackageManagerInternal.freeStorage(java.lang.String, long, int)' on a null object reference
                 	at com.android.server.StorageManagerService.allocateBytes(StorageManagerService.java:3263)
                 	at android.os.storage.StorageManager.allocateBytes(StorageManager.java:2099)
                 	at android.os.storage.StorageManager.allocateBytes(StorageManager.java:2156)
                 	at com.android.server.pm.PackageInstallerSession.doWriteInternal(PackageInstallerSession.java:723)
                 	at com.android.server.pm.PackageInstallerSession.write(PackageInstallerSession.java:670)
                 	at android.content.pm.PackageInstaller$Session.write(PackageInstaller.java:951)
                 	at com.android.server.pm.PackageManagerShellCommand.doWriteSplit(PackageManagerShellCommand.java:2732)
                 	at com.android.server.pm.PackageManagerShellCommand.runInstall(PackageManagerShellCommand.java:1072)
                 	at com.android.server.pm.PackageManagerShellCommand.onCommand(PackageManagerShellCommand.java:169)
                 	at android.os.ShellCommand.exec(ShellCommand.java:104)
                 	at com.android.server.pm.PackageManagerService.onShellCommand(PackageManagerService.java:21729)
                 	at android.os.Binder.shellCommand(Binder.java:881)
                 	at android.os.Binder.onTransact(Binder.java:765)
                 	at android.content.pm.IPackageManager$Stub.onTransact(IPackageManager.java:4860)
                 	at com.android.server.pm.PackageManagerService.onTransact(PackageManagerService.java:4014)
                 	at android.os.Binder.execTransactInternal(Binder.java:1021)
                 	at android.os.Binder.execTransact(Binder.java:994)
                 
                 
[00:47:30] crit: Install failure: Test command cannot continue
[00:47:30] info: Attempting to remove apk 'net.dot.System.Security.SecureString.Tests': 
[00:47:30] dbug: Executing command: '/datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb -s emulator-5554 uninstall net.dot.System.Security.SecureString.Tests'
[00:47:30] info: APK 'net.dot.System.Security.SecureString.Tests' not on device.
[00:47:30] info: Attempting to remove apk 'net.dot.System.Security.SecureString.Tests': 
[00:47:30] dbug: Executing command: '/datadisks/disk1/work/A821097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.21404.1/runtimes/any/native/adb/linux/adb -s emulator-5554 uninstall net.dot.System.Security.SecureString.Tests'
[00:47:30] info: APK 'net.dot.System.Security.SecureString.Tests' not on device.
XHarness exit code: 78 (PACKAGE_INSTALLATION_FAILURE)
+ exit 78

Author:	carlossanlop
Assignees:	-
Labels:	arch-x64, blocking-clean-ci, os-android, area-VM-meta-mono, runtime-mono, Known Build Error
Milestone:	-