Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update-python-resources munges--extra-packages if it's a URL #18276

Closed
3 tasks done
ctaintor opened this issue Sep 7, 2024 · 2 comments
Closed
3 tasks done

update-python-resources munges--extra-packages if it's a URL #18276

ctaintor opened this issue Sep 7, 2024 · 2 comments
Labels
features New features

Comments

@ctaintor
Copy link
Contributor

ctaintor commented Sep 7, 2024

brew doctor output

Your system is ready to brew.

Verification

  • My "brew doctor output" above says Your system is ready to brew. and am still able to reproduce my issue.
  • I ran brew update twice and am still able to reproduce my issue.
  • This issue's title and/or description do not reference a single formula e.g. brew install wget. If they do, open an issue at https://github.com/Homebrew/homebrew-core/issues/new/choose instead.

brew config output

HOMEBREW_VERSION: 4.3.19-98-g9978c3d-dirty
ORIGIN: https://github.com/Homebrew/brew
HEAD: 9978c3d11ac9108940ee0976a362d0d5872e0916
Last commit: 15 hours ago
Core tap JSON: 07 Sep 07:55 UTC
Core cask tap JSON: 07 Sep 07:55 UTC
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_ARTIFACT_DOMAIN: https://<internal>
HOMEBREW_CASK_OPTS: []
HOMEBREW_DOCKER_REGISTRY_BASIC_AUTH_TOKEN: set
HOMEBREW_MAKE_JOBS: 10
HOMEBREW_PIP_INDEX_URL: https://<internal>
HOMEBREW_SORBET_RUNTIME: set
Homebrew Ruby: 3.3.4 => /opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.3.4_1/bin/ruby
CPU: 10-core 64-bit arm_blizzard_avalanche
Clang: 15.0.0 build 1500
Git: 2.46.0 => /opt/homebrew/bin/git
Curl: 8.7.1 => /usr/bin/curl
macOS: 14.6.1-arm64
CLT: 15.3.0.0.1.1708646388
Xcode: 15.2 => /Applications/Xcode-15.2.0.app/Contents/Developer
Rosetta 2: false

What were you trying to do (and why)?

I'm trying to use update-python-resources on an internal library which has internal dependencies. This is proving a pain for two reasons:

  • pypi_info hardcodes to https://pypi.org/pypi/
  • our artifact server (artifactory) does not support the JSON api :(

Rather than patch (or send a PR if wanted) brew to fallback to the simple api, I thought I could get away with adding the internal packages as URL arguments to --extra-packages since the code seems to support this.

What happened (include all command output)?

$ brew update-python-resources Formula/my-formula.rb --verbose --extra-packages "https://a.b.net/.../...0.2.31"
==> Retrieving PyPI dependencies for "<path-to-my-formula-package> https://a-b-net/.../...-0-2-31-tar-gz"...
  WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x106704810>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')': .../...-0-2-31-tar-gz
...-0-2-31-tar-gz (Caused by NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x106707090>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))

(note that the URL has been munged, changing all . to be -, obviously meaning that the hostname can't be resolved)

What did you expect to happen?

I did not expect that error

Step-by-step reproduction instructions (by running brew commands)

I believe you could reproduce this by adding a URL to a package at PyPi.

I am pretty sure that the offending code is [here](https://github.com/Homebrew/brew/blob/9978c3d11ac9108940ee0976a362d0d5872e0916/Library/Homebrew/utils/pypi.rb#L411). I tried changing it to:


    if name.start_with?("http")
      name
    else
      name.gsub(/[-_.]+/, "-").downcase
    end


and I was able to get farther along in the update (as in, it was able to read the metadata of the URL'd package). However, I will still need to implement the "fallback to simple api".
@ctaintor ctaintor added the bug Reproducible Homebrew/brew bug label Sep 7, 2024
@carlocab
Copy link
Member

carlocab commented Sep 7, 2024

This seems more like a feature request for --extra-packages to accept arbitrary URLs rather than just PyPI package names.

Could you answer the questions in the feature request template instead? https://github.com/Homebrew/brew/issues/new?assignees=&labels=features&projects=&template=feature.yml

@carlocab carlocab added features New features and removed bug Reproducible Homebrew/brew bug labels Sep 7, 2024
@ctaintor
Copy link
Contributor Author

ctaintor commented Sep 8, 2024

Opened #18282 to make it cleaner. Since --extra-packages did do some expected work when I passed a URL, it was unclear if this was a bug or a feature request

@ctaintor ctaintor closed this as completed Sep 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
features New features
Projects
None yet
Development

No branches or pull requests

2 participants