Transient errors loading secrets

[AdrienPoupa]

Hello,

We're experimenting random failures in our workflows, for example:

2025-03-25T15:13:46.9378402Z ##[group]Run 1password/load-secrets-action@v2
2025-03-25T15:13:46.9378726Z with:
2025-03-25T15:13:46.9378938Z   export-env: true
2025-03-25T15:13:46.9379153Z   unset-previous: false
2025-03-25T15:13:46.9379364Z env:
2025-03-25T15:13:46.9388828Z   OP_SERVICE_ACCOUNT_TOKEN: ***
2025-03-25T15:13:46.9389154Z   SOME_KEY: op://github-actions-dev/path/to/key
2025-03-25T15:13:46.9389832Z ##[endgroup]
2025-03-25T15:13:47.0189283Z Authenticated with Service account.
2025-03-25T15:13:47.0258985Z [command]/usr/bin/sh -c /home/runner/work/_actions/1password/load-secrets-action/v2/install_cli.sh
2025-03-25T15:13:47.5210987Z Archive:  op.zip
2025-03-25T15:13:47.7078168Z   inflating: /tmp/tmp.JMycA1yFyu/op  
2025-03-25T15:13:47.7081889Z  extracting: /tmp/tmp.JMycA1yFyu/op.sig  
2025-03-25T15:13:47.7162093Z [command]/usr/bin/sh -c op env ls
2025-03-25T15:13:47.8109637Z SOME_KEY
2025-03-25T15:13:47.8139170Z Populating variable: SOME_KEY
2025-03-25T15:14:03.5049985Z ##[error]could not read secret 'op://github-actions-dev/path/to/key': could not get item github-actions-dev/cosmo: Server: (500) (Internal Server Error), Server experienced a fatal error.

It happens frequently enough that I notice it (think multiple times a day). Retrying the job usually works, but this is annoying.

Let me know if I can help debug this further!

[alyssaruth]

We are also seeing intermittent failures, though in our case the error claims it can't find an item even though it's definitely there:

Error: could not read secret 'op://SRE/75vbf4j6646ggmzx4xvalr3f6a/password': could not retrieve item 'SRE/75vbf4j6646ggmzx4xvalr3f6a': Found 0 item(s) in vault "uxhyhf5j3eluhb4fvgl3oflloi" with title "75vbf4j6646ggmzx4xvalr3f6a"

Again, happy to enable debug logging if it exists to help diagnose further 👍

[AdrienPoupa]

It happened again this morning, all our workflows crashed. Could we get a feedback from 1password please? @edif2008

[edif2008]

Hey @AdrienPoupa!
Yesterday there was a service disruption that affected signing in to 1password.com. Therefore, the workflows couldn't authenticate and thus they crashed. It has now been resolved. Apologies for the inconvenience.

[AdrienPoupa]

Thanks for the answer! That explains yesterday's issues. But it happens every now and then as well, even when there is no outage reported in the status page.

[genisd]

We're about to be onboarding this. But this issue is kind of worrysome. Transient errors happen but should now fail workflows.

Can we add a retry mechanism to this?

[volodymyrZotov]

Should be fixed in v3 as not it uses Typescript implementation and official github action libraries.

Please re-open this issue if you still experience the same errors with v3 as well.

[AdrienPoupa]

Reopening the issue as we're experiencing a similar issue after migrating:

Error: could not read secret 'op://path': error initializing client: Authentication: (failed to auth.LookupAuth), Network: (failed to request.DoUnencrypted), Server: (500) (Internal Server Error), Server experienced a fatal error.

@volodymyrZotov I don't have the permissions to reopen the issue unfortunately.

[Marton6]

@AdrienPoupa thanks for reaching out about this. This is related to an ongoing incident which currently impacts authenticating to 1Password.

You can follow the status page for more info: https://status.1password.com/

[volodymyrZotov]

@AdrienPoupa the outage is resolved now. Let us know if you still have any issues.

[AdrienPoupa]

Will do, thanks for the responsiveness!