Notarization for arm64 version of app fails randomly but succeeds on retry

[praneetloke]

I have an Electron app that is built and notarized using electron-builder. Notarization for the arm64 version of the app fails randomly without any additional information. The only log I have is this:

...
electron-notarize:spawn spawning cmd: xcrun args: [
  'notarytool',
  'submit',
  '/var/folders/5q/zcc7tkt11f13mhczs1yfd4qc0000gn/T/electron-notarize-PsiiWM/****.zip',
  '--apple-id',
  '*********',
  '--password',
  '*********',
  '--team-id',
  '*********',
  '--wait',
  '--output-format',
  'json'
] opts: {}
...
...
electron-notarize:spawn cmd xcrun terminated with code: null
electron-notarize:helpers work failed
  ⨯ Failed to notarize via notarytool.  Failed with unexpected result: 

  failedTask=build stackTrace=Error: Failed to notarize via notarytool.  Failed with unexpected result: 
                                                                                 at /Users/runner/work/****/****/node_modules/@electron/notarize/src/notarytool.ts:115:13
    at Generator.next (<anonymous>)
    at fulfilled (/Users/runner/work/****/****/node_modules/@electron/notarize/lib/notarytool.js:28:58)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
From previous event:
    at processImmediate (node:internal/timers:483:21)
From previous event:
    at readDirectoryAndSign (/Users/runner/work/****/****/node_modules/app-builder-lib/src/macPackager.ts:496:29)
    at MacPackager.signApp (/Users/runner/work/****/****/node_modules/app-builder-lib/src/macPackager.ts:506:11)
    at MacPackager.doSignAfterPack (/Users/runner/work/****/****/node_modules/app-builder-lib/src/platformPackager.ts:346:21)
    at MacPackager.doPack (/Users/runner/work/****/****/node_modules/app-builder-lib/src/platformPackager.ts:331:7)
    at MacPackager.pack (/Users/runner/work/****/****/node_modules/app-builder-lib/src/macPackager.ts:215:9)
    at Packager.doBuild (/Users/runner/work/****/****/node_modules/app-builder-lib/src/packager.ts:459:9)
    at executeFinally (/Users/runner/work/****/****/node_modules/builder-util/src/promise.ts:12:14)
    at Packager.build (/Users/runner/work/****/****/node_modules/app-builder-lib/src/packager.ts:393:31)
    at executeFinally (/Users/runner/work/****/****/node_modules/builder-util/src/promise.ts:12:14)

There is no other information about the failure. In a successful execution, the exit code is 0 which is expected. I have set DEBUG=electron-notarize* as per the README in this repo but it doesn't seem to help with emitting any additional information about the failure. Any help/tips on diagnosing this would be appreciated.

EDIT Jan 2025:
See #219 (comment). If you are using GitHub Actions:

• and have noticed that the build fails randomly despite having accepted all pending Apple Terms
• and are using macos-latest
• downgrade your runner to macos-13.
  • macos-13 runner is an Intel-based runner with 14GB RAM.

[VerteDinde]

Hey @praneetloke, thanks for the report. Does this also repro with a basic repo? Also could you include the version of notarize that you're using?

[fempyrean]

@VerteDinde just to add to this, I'm also having a very similar issue. The error output is identical to the one reported, but in my case, the problematic architecture is x86, arm64 notarizes fine.

The notarization had been working fine for a long time but suddenly it stopped working, but I couldn't figure out exactly when it happened because the last build was from a couple months ago, so no logs for that.

Thought it could be related to GitHub upgrading their macos-latest runner to use macOS 14, but that happened at the start of 2024 and notarization was definitely working after that.

I'm running the latest version of notarize: 2.5.0.

[praneetloke]

Hello @VerteDinde, thank you for your response.

Does this also repro with a basic repo?

Unfortunately, I do not know if that's the case since the failure is sporadic right now. I might have an entire day of zero failures.

Also could you include the version of notarize that you're using?

I use electron-builder@25.1.8 which in turn uses @electron/notarize@2.5.0.

To add additional context, my CI, like @fempyrean, also runs on GH actions. I use macos-latest for the macOS builds which runs macOS 14 arm64 runners. I have a suspicion that macOS is perhaps terminating xcrun and hence the lack of a proper exit code (it's null)? However, I don't know of a way to prove this; it's just a theory. At this point, I am not sure where the issue lies.

These failures started about two months ago with version @electron/notarize@2.2.1 that we had been running since the beginning of this year. Interestingly, the stack trace was slightly different with that version and also the failure was flip-flopping between x64 and arm64 with the following stack trace:

electron-notarize:spawn spawning cmd: xcrun args: [
  'notarytool',
  'submit',
  '/var/folders/g6/rgtlsw6n123b0gt5483s5_cm0000gn/T/electron-notarize-ajOoEA/***.zip',
  '--apple-id',
  '*********',
  '--password',
  '*********',
  '--team-id',
  '*********',
  '--wait',
  '--output-format',
  'json'
] opts: {}
electron-notarize:spawn cmd xcrun terminated with code: null <----NOTE: null exit code.
electron-notarize:helpers work failed
  ⨯ Unexpected end of JSON input  failedTask=build stackTrace=SyntaxError: Unexpected end of JSON input
SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at /Users/runner/work/***/node_modules/@electron/notarize/src/notarytool.ts:79:25
    at Generator.next (<anonymous>)
    at fulfilled (/Users/runner/work/***/node_modules/@electron/notarize/lib/notarytool.js:28:58)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
From previous event:
    at processImmediate (node:internal/timers:478:21)
From previous event:
    at readDirectoryAndSign (/Users/runner/work/***/node_modules/app-builder-lib/src/macPackager.ts:478:29)
    at MacPackager.signApp (/Users/runner/work/***/node_modules/app-builder-lib/src/macPackager.ts:488:11)
    at MacPackager.doSignAfterPack (/Users/runner/work/***/node_modules/app-builder-lib/src/platformPackager.ts:336:21)
    at MacPackager.doPack (/Users/runner/work/***/node_modules/app-builder-lib/src/platformPackager.ts:321:7)
    at MacPackager.pack (/Users/runner/work/***/node_modules/app-builder-lib/src/macPackager.ts:198:9)
    at Packager.doBuild (/Users/runner/work/***/node_modules/app-builder-lib/src/packager.ts:445:9)
    at executeFinally (/Users/runner/work/***/node_modules/builder-util/src/promise.ts:12:14)
    at Packager._build (/Users/runner/work/***/node_modules/app-builder-lib/src/packager.ts:379:31)
    at Packager.build (/Users/runner/work/***/node_modules/app-builder-lib/src/packager.ts:340:12)
    at executeFinally (/Users/runner/work/***/node_modules/builder-util/src/promise.ts:12:14)

About three weeks ago, I updated to @electron/notarize@2.5.0 and the failure still occurs but only inarm64 now. Although I am not sure if that's just a coincidence.

[bradfair]

I'm having this issue also, and came across this reference: actions/runner-images#9811

I've added this step to the action I'm using, and running it a few times to see if it stabilizes:

      
      - name: xcode-select
        if: matrix.os == 'macos-latest'
        run: sudo xcode-select -s /Library/Developer/CommandLineTools

edit: womp womp.

2024-12-06T13:30:29.517Z electron-notarize:spawn cmd xcrun terminated with code: null
2024-12-06T13:30:29.517Z electron-notarize:helpers work failed
  ⨯ Failed to notarize via notarytool.  Failed with unexpected result: 

final edit: I think @praneetloke's suspicion about the xcrun crash is spot on. i found some other references yesterday that suggested changing back to macos-13 in the action runner, and I did that. The process is stable now, so I'm leaving it at that and moving on.

[praneetloke]

@bradfair

i found some other references yesterday that suggested changing back to macos-13 in the action runner, and I did that. The process is stable now, so I'm leaving it at that and moving on.

That's a great idea!

Also I took a look at GH's hosted runners and their macos-13 runners are Intel-based with 14GB RAM. Their M1 runners are half of that at 7GB. My guess is xcrun is terminated as a result of low memory sometimes. Though, again, I don't have a way to prove this right now.

[mifi]

Same thing just happened here on macos latest:

https://github.com/mifi/lossless-cut/actions/runs/12211950166/job/34069829185
I've had a scheduled job run every day and I believe today is the first time i've seen this error. also macos-latest (needed for apple silicon builds). Maybe we could impleemtn a retry?

Edit: it still happens intermittently. last week:

• https://github.com/mifi/lossless-cut/actions/runs/12765242934/job/35579071342
• https://github.com/mifi/lossless-cut/actions/runs/12732484808/job/35487735589
• https://github.com/mifi/lossless-cut/actions/runs/12707306521/job/35421953916
• https://github.com/mifi/lossless-cut/actions/runs/12649279112/job/35245346751

[praneetloke]

@mifi it looks like you are using electron-builder, it supports cross-platform compilation for arm64 on Intel-based macOS. However, if this was the first time you've seen a problem, you might want to check if you've got a pending Apple terms agreement waiting to be accepted in your Apple Developer Account. (I got one yesterday that I accepted.) If you've already accepted that, it would be helpful if you could observe your builds for a few days at least to see how often it fails with macos-latest, if at all.

[mifi]

got a pending Apple terms agreement waiting to be accepted in your Apple Developer Account.

That seems like it might have been the problem. I also got pending terms/agreements that day. Now it seems to be succeeding. Will keep a look at the job going forward

[Hawstein]

I encountered the exact same problem. This issue is unrelated to the Apple terms agreement. All terms in my Apple developer account have been accepted, yet I still encounter this problem.

• macOS 14.5
• electron-builder@25.1.8
• @electron/notarize@2.5.0

P.S. If I manually use xcrun notarytool to submit my app, the notarization is immediately accepted.

[codeofandrin]

macos-13 runner however is significantly slower. Takes 2 minutes longer for my whole pipeline (5min on macos-latest vs 7min on macos-13)

[fredluetkemeier]

Having this issue as well on macos-latest when using electron-forge. Trying to build a universal app, and it seems about 50% of the time it will just randomly fail. I've accepted the Apple terms so it isn't that.