Skip to content

Always use EDL user token to fetch s3 creds - #1183

Merged
betolink merged 2 commits into
mainfrom
always-use-token
Jan 29, 2026
Merged

Always use EDL user token to fetch s3 creds#1183
betolink merged 2 commits into
mainfrom
always-use-token

Conversation

@chuckwondo

@chuckwondo chuckwondo commented Jan 27, 2026

Copy link
Copy Markdown
Contributor

This doesn't change any functionality, but speeds up fetching of s3 credentials by about 4x based on very primitive benchmarking, due to always using an EDL (bearer) token for fetching temp creds, rather than using EDL username/password (basic auth):

from timeit import timeit
import earthaccess

auth = earthaccess.login()

# These 2 lines are necessary only when running against
# earthaccess code that does not include the code in this PR,
# due to issue 1181, which this PR also fixes.
setattr(auth, "username", None)
setattr(auth, "password", None)

endpoint = "https://data.nsidc.earthdatacloud.nasa.gov/s3credentials"

print(timeit(lambda: auth.get_s3_credentials(endpoint=endpoint), number=10))

Benchmark results running on my pre-M1 MacBook:

  • Without this PR: 36.76829030702356 (avg. of ~3.7s per s3 creds fetch)
  • With this PR: 8.482286745973397 (avg. of ~0.85s per s3 creds fetch)
  • Speedup: 36.76829030702356 / 8.482286745973397 ~= 4.33

The speedup is due to a reduction in the number of requests. In particular, using basic auth ends up making 4 times the number of requests than using a bearer token does, so the 4x speedup jives with reducing the number of requests by a factor of 4.

Fixes #1182
Fixes #1181

Pull Request (PR) draft checklist - click to expand
  • Please review our
    contributing documentation
    before getting started.
  • Populate a descriptive title. For example, instead of "Updated README.md", use a
    title such as "Add testing details to the contributor section of the README".
    Example PRs: #763
  • Populate the body of the pull request with:
  • Update CHANGELOG.md with details about your change in a section titled
    ## Unreleased. If such a section does not exist, please create one. Follow
    Common Changelog for your additions.
    Example PRs: #763
  • Update the documentation and/or the README.md with details of changes to the
    earthaccess interface, if any. Consider new environment variables, function names,
    decorators, etc.

Click the "Ready for review" button at the bottom of the "Conversation" tab in GitHub
once these requirements are fulfilled. Don't worry if you see any test failures in
GitHub at this point!

Pull Request (PR) merge checklist - click to expand

Please do your best to complete these requirements! If you need help with any of these
requirements, you can ping the @nsidc/earthaccess-support team in a comment and we
will help you out!

  • Add unit tests for any new features.
  • Apply formatting and linting autofixes. You can add a GitHub comment in this Pull
    Request containing "pre-commit.ci autofix" to automate this.
  • Ensure all automated PR checks (seen at the bottom of the "conversation" tab) pass.
  • Get at least one approving review.

📚 Documentation preview 📚: https://earthaccess--1183.org.readthedocs.build/en/1183/

@github-actions

github-actions Bot commented Jan 27, 2026

Copy link
Copy Markdown

Binder 👈 Launch a binder notebook on this branch for commit abda5ca

I will automatically update this comment whenever this PR is modified

Binder 👈 Launch a binder notebook on this branch for commit 14fa683

Comment thread earthaccess/auth.py
Comment on lines +103 to +104
self.username: str | None = None
self.password: str | None = None

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the crux of the fix for #1181, which also helps with the rest of the changes for #1182.

Comment thread earthaccess/search.py
Comment on lines +262 to +264
print( # noqa: T201
[method for method in dir(self) if method.startswith("_") is False]
)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ruff ended up reformatting, which then put the noqa comment on the wrong line, which then made ruff complain about the print call, so this is just to satisfy ruff.

@betolink betolink left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple questions in the code, this looks good and once get merge it we should release! @chuckwondo

Comment thread earthaccess/auth.py Outdated
Comment thread earthaccess/auth.py

@betolink betolink left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good @chuckwondo, I'm approving and waiting for another maintainer until tomorrow, if nobody has any objection I'll merge it in the morning.

@jhkennedy jhkennedy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me as well

@betolink
betolink merged commit 0b0d3e5 into main Jan 29, 2026
11 checks passed
@betolink
betolink deleted the always-use-token branch January 29, 2026 15:27
@github-project-automation github-project-automation Bot moved this to ✅ Done in earthaccess Mar 3, 2026
@mfisher87 mfisher87 removed this from earthaccess Mar 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Always use bearer token auth, not basic auth [BUG] Auth instances can fail to recognize that EARTHDATA_TOKEN environment variable is in play

4 participants