Use user's ~/.netrc for accessing mixin repositories

[cottsay]

Recent changes to GitHub raw content availability have made it more desirable to support authenticated requests to externally hosted content. Rather than adding specific logic for GitHub's authorization mechanisms, we can leverage the widely used .netrc format for adding an authorization header to our web requests.

As an example, a GitHub classic PAT with sufficient scopes can be configured and automatically used with a ~/.netrc like this:

machine raw.githubusercontent.com password gbp_XXXXXXXXX

Note that this scheme works for raw.githubusercontent.com, but may not work for github.com URLs (even when they redirect to raw.githubusercontent.com). Also worth noting is that .netrc files are implicitly used by the widely-used requests Python package, so there is precedent for their use in this context.

This change doesn't introduce any new dependencies, but does change default behavior.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 0% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 0.00%. Comparing base (e16e742) to head (73e0004).
⚠️ Report is 11 commits behind head on master.

Files with missing lines	Patch %	Lines
colcon_mixin/mixin/repository.py	0.00%	19 Missing ⚠️

Additional details and impacted files

@@          Coverage Diff           @@
##           master     #50   +/-   ##
======================================
  Coverage    0.00%   0.00%           
======================================
  Files          11      11           
  Lines         540     557   +17     
======================================
- Misses        540     557   +17     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[christophebedard]

Looks good to me.

I'm having a hard time testing that the ~/.netrc raw.githubusercontent.com token on my machine actually does something, though. I used a dummy password, which does seem to be used and unexpectedly works fine, so I'm not sure if a real token would actually do anything:

machine raw.githubusercontent.com
  password 123

If I change Basic to token and only include the password without the login (from here), I get a 404 unless I use a real token:

-                credentials = f'{entry[0]}:{entry[2]}'
-                auth = 'Basic ' + b64encode(credentials.encode()).decode()
+                credentials = f'{entry[2]}'
+                auth = 'token ' + credentials

[cottsay]

I observed similar behavior. The only way I was able to verify that it was in fact using the token was to set up a mixin repository that was in a private repository. I'm not sure why we don't get an error when the credentials are wrong.

I don't think we can use token authentication if we want to use a .netrc file, which (as far as I understand it) always performs basic auth. If we find that this general-purpose solution is insufficient to bypass the new rate limiting, we'll have to explore token auth, OAuth, or API redirection as a GitHub-specific solution.

[christophebedard]

Do you think other hosts would also need token instead of Basic? It doesn't scale, but we could always keep Basic base64($login:$password) as the default but change to token $password if the host is raw.githubusercontent.com or github.com.

[cottsay]

Let's verify whether this existing approach works first, then we can discuss GitHub-specific alternatives and workarounds if necessary.

(by that I mean verify that it can be used to get around the anonymous rate limiting)