Bug: `get_diff_of_pull_request` fails for private repositories

[aarogyarijal]

Bug Description

The get_diff_of_pull_request function in src/git_bob/_github_utilities.py isn't working for private GitHub repositories. It fetches pull request diffs just fine when the repository is public, but as soon as it's private, it fails.

Steps to Reproduce

1. Make sure git-bob is set up with a GitHub API key that has access to a private repository.
2. Open a pull request in that private repository.
3. Try to use any git-bob feature that relies on get_diff_of_pull_request (e.g., the commenting or reviewing features for pull requests in _ai_github_utilities.py).

Expected Behavior

We expect get_diff_of_pull_request to retrieve diffs from both public and private repositories without issue, assuming the authenticated PyGithub instance has the right permissions.

Actual Behavior

When trying to get a pull request diff from a private repository, the requests.get() call to pull_request.diff_url returns Error: 404 Not Found, which means the diff content can't be retrieved.

Root Cause

Maybe the problem is that the requests.get() call on line _github_utilities.py:622 doesn't include the necessary headers. See below link for more.

Additional Links

https://github.com/orgs/community/discussions/24460#discussioncomment-3244157

git-bob comment

[github-actions]

^{This message was generated by git-bob (version: 0.23.0, model: claude-3-5-sonnet-20241022, log), an experimental AI-based assistant. It can make mistakes and has limitations. Check its messages carefully.}

Hi @aarogyarijal,

thanks for reaching out! Unfortunately, I'm not allowed to respond to you directly.
I need approval from a person who has access.

Best,
git-bob

[haesleinhuepf]

Awesome, thanks for the bug report. Let's if git-bob comments a useful bug fix approach

[github-actions]

^{This message was generated by git-bob (version: 0.23.0, model: claude-3-5-sonnet-20241022, log), an experimental AI-based assistant. It can make mistakes and has limitations. Check its messages carefully.}

Looking at the bug report, the issue can be fixed by adding GitHub authentication to the requests.get call in the get_diff_of_pull_request function. Here's the fixed code:

def get_diff_of_pull_request(repository, pull_request):
    import requests

    Log().log(f"-> get_diff_of_pull_request({repository}, {pull_request})")
    repo = get_repository_handle(repository)
    access_token = os.getenv('GITHUB_API_KEY')

    pull_request = repo.get_pull(pull_request)

    headers = {'Authorization': f'token {access_token}'}
    response = requests.get(pull_request.diff_url, headers=headers)
    if response.status_code == 200:
        return response.text
    else:
        print("Error:", response.status_code, response.text)
        return ""

This should allow accessing diffs from private repositories.

[haesleinhuepf]

Ok sounds good, git-bob implement this!

[haesleinhuepf]

git-bob deploy

[github-actions]

^{This message was generated by git-bob (version: 0.23.1, model: claude-3-5-sonnet-20241022, log), an experimental AI-based assistant. It can make mistakes and has limitations. Check its messages carefully.}

Deployment report

Command

python setup.py sdist bdist_wheel

StdOut

running sdist
running egg_info
writing src/git_bob.egg-info/PKG-INFO
writing dependency_links to src/git_bob.egg-info/dependency_links.txt
writing entry points to src/git_bob.egg-info/entry_points.txt
writing requirements to src/git_bob.egg-info/requires.txt
writing top-level names to src/git_bob.egg-info/top_level.txt
reading manifest file 'src/git_bob.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'src/git_bob.egg-info/SOURCES.txt'
running check
creating git_bob-0.23.1
creating git_bob-0.23.1/src/git_bob
creating git_bob-0.23.1/src/git_bob.egg-info
creating git_bob-0.23.1/src/git_bob/data
creating git_bob-0.23.1/tests
copying files to git_bob-0.23.1...
copying LICENSE -> git_bob-0.23.1
copying README.md -> git_bob-0.23.1
copying pyproject.toml -> git_bob-0.23.1
copying requirements.txt -> git_bob-0.23.1
copying setup.cfg -> git_bob-0.23.1
copying setup.py -> git_bob-0.23.1
copying src/git_bob/__init__.py -> git_bob-0.23.1/src/git_bob
copying src/git_bob/_ai_github_utilities.py -> git_bob-0.23.1/src/git_bob
copying src/git_bob/_endpoints.py -> git_bob-0.23.1/src/git_bob
copying src/git_bob/_github_utilities.py -> git_bob-0.23.1/src/git_bob
copying src/git_bob/_gitlab_utilities.py -> git_bob-0.23.1/src/git_bob
copying src/git_bob/_logger.py -> git_bob-0.23.1/src/git_bob
copying src/git_bob/_terminal.py -> git_bob-0.23.1/src/git_bob
copying src/git_bob/_utilities.py -> git_bob-0.23.1/src/git_bob
copying src/git_bob.egg-info/PKG-INFO -> git_bob-0.23.1/src/git_bob.egg-info
copying src/git_bob.egg-info/SOURCES.txt -> git_bob-0.23.1/src/git_bob.egg-info
copying src/git_bob.egg-info/dependency_links.txt -> git_bob-0.23.1/src/git_bob.egg-info
copying src/git_bob.egg-info/entry_points.txt -> git_bob-0.23.1/src/git_bob.egg-info
copying src/git_bob.egg-info/requires.txt -> git_bob-0.23.1/src/git_bob.egg-info
copying src/git_bob.egg-info/top_level.txt -> git_bob-0.23.1/src/git_bob.egg-info
copying src/git_bob/data/__init__.py -> git_bob-0.23.1/src/git_bob/data
copying src/git_bob/data/slide_template.pptx -> git_bob-0.23.1/src/git_bob/data
copying tests/test_agent.py -> git_bob-0.23.1/tests
copying tests/test_endpoints.py -> git_bob-0.23.1/tests
copying tests/test_github_utilities.py -> git_bob-0.23.1/tests
copying tests/test_gitlab_utilities.py -> git_bob-0.23.1/tests
copying tests/test_logger.py -> git_bob-0.23.1/tests
copying tests/test_utilities.py -> git_bob-0.23.1/tests
copying src/git_bob.egg-info/SOURCES.txt -> git_bob-0.23.1/src/git_bob.egg-info
Writing git_bob-0.23.1/setup.cfg
creating dist
Creating tar archive
removing 'git_bob-0.23.1' (and everything under it)
running bdist_wheel
running build
running build_py
creating build/lib/git_bob
copying src/git_bob/_terminal.py -> build/lib/git_bob
copying src/git_bob/_logger.py -> build/lib/git_bob
copying src/git_bob/_utilities.py -> build/lib/git_bob
copying src/git_bob/_gitlab_utilities.py -> build/lib/git_bob
copying src/git_bob/_endpoints.py -> build/lib/git_bob
copying src/git_bob/_github_utilities.py -> build/lib/git_bob
copying src/git_bob/__init__.py -> build/lib/git_bob
copying src/git_bob/_ai_github_utilities.py -> build/lib/git_bob
creating build/lib/git_bob/data
copying src/git_bob/data/__init__.py -> build/lib/git_bob/data
copying src/git_bob/data/slide_template.pptx -> build/lib/git_bob/data
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/git_bob
copying build/lib/git_bob/_terminal.py -> build/bdist.linux-x86_64/wheel/./git_bob
copying build/lib/git_bob/_logger.py -> build/bdist.linux-x86_64/wheel/./git_bob
copying build/lib/git_bob/_utilities.py -> build/bdist.linux-x86_64/wheel/./git_bob
copying build/lib/git_bob/_gitlab_utilities.py -> build/bdist.linux-x86_64/wheel/./git_bob
copying build/lib/git_bob/_endpoints.py -> build/bdist.linux-x86_64/wheel/./git_bob
creating build/bdist.linux-x86_64/wheel/git_bob/data
copying build/lib/git_bob/data/slide_template.pptx -> build/bdist.linux-x86_64/wheel/./git_bob/data
copying build/lib/git_bob/data/__init__.py -> build/bdist.linux-x86_64/wheel/./git_bob/data
copying build/lib/git_bob/_github_utilities.py -> build/bdist.linux-x86_64/wheel/./git_bob
copying build/lib/git_bob/__init__.py -> build/bdist.linux-x86_64/wheel/./git_bob
copying build/lib/git_bob/_ai_github_utilities.py -> build/bdist.linux-x86_64/wheel/./git_bob
running install_egg_info
Copying src/git_bob.egg-info to build/bdist.linux-x86_64/wheel/./git_bob-0.23.1-py3.13.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/git_bob-0.23.1.dist-info/WHEEL
creating 'dist/git_bob-0.23.1-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'git_bob/__init__.py'
adding 'git_bob/_ai_github_utilities.py'
adding 'git_bob/_endpoints.py'
adding 'git_bob/_github_utilities.py'
adding 'git_bob/_gitlab_utilities.py'
adding 'git_bob/_logger.py'
adding 'git_bob/_terminal.py'
adding 'git_bob/_utilities.py'
adding 'git_bob/data/__init__.py'
adding 'git_bob/data/slide_template.pptx'
adding 'git_bob-0.23.1.dist-info/licenses/LICENSE'
adding 'git_bob-0.23.1.dist-info/METADATA'
adding 'git_bob-0.23.1.dist-info/WHEEL'
adding 'git_bob-0.23.1.dist-info/entry_points.txt'
adding 'git_bob-0.23.1.dist-info/top_level.txt'
adding 'git_bob-0.23.1.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel

StdErr

/opt/hostedtoolcache/Python/3.13.5/x64/lib/python3.13/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
!!

        ********************************************************************************
        Please consider removing the following classifiers in favor of a SPDX license expression:

        License :: OSI Approved :: BSD License

        See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
        ********************************************************************************

!!
  self._finalize_license_expression()
/opt/hostedtoolcache/Python/3.13.5/x64/lib/python3.13/site-packages/setuptools/_distutils/cmd.py:90: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        By 2025-Oct-31, you need to update your project and remove deprecated calls
        or your builds will no longer be supported.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()

Command

twine upload dist/*

StdOut

Uploading distributions to https://upload.pypi.org/legacy/
Uploading git_bob-0.23.1-py3-none-any.whl

  0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/114.2 kB • --:-- • ?
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.2/114.2 kB • 00:00 • 5.3 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.2/114.2 kB • 00:00 • 5.3 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.2/114.2 kB • 00:00 • 5.3 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.2/114.2 kB • 00:00 • 5.3 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.2/114.2 kB • 00:00 • 5.3 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.2/114.2 kB • 00:00 • 5.3 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.2/114.2 kB • 00:00 • 5.3 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.2/114.2 kB • 00:00 • 5.3 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.2/114.2 kB • 00:00 • 5.3 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.2/114.2 kB • 00:00 • 5.3 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.2/114.2 kB • 00:00 • 5.3 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.2/114.2 kB • 00:00 • 5.3 MB/s
Uploading git_bob-0.23.1.tar.gz

  0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/125.3 kB • --:-- • ?
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.3/125.3 kB • 00:00 • 218.3 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.3/125.3 kB • 00:00 • 218.3 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.3/125.3 kB • 00:00 • 218.3 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.3/125.3 kB • 00:00 • 218.3 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.3/125.3 kB • 00:00 • 218.3 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.3/125.3 kB • 00:00 • 218.3 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.3/125.3 kB • 00:00 • 218.3 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.3/125.3 kB • 00:00 • 218.3 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.3/125.3 kB • 00:00 • 218.3 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.3/125.3 kB • 00:00 • 218.3 MB/s

View at:
https://pypi.org/project/git-bob/0.23.1/

StdErr

[haesleinhuepf]

@aarogyarijal thanks again for the bug-report. Would you mind trying if git-bob 0.23.1 now works for your use-case?

[aarogyarijal]

@haesleinhuepf Thanks for the quick fix and deployment of git-bob 0.23.1!

I've tested the latest version with my private repository, and I'm still encountering an "Error: 404 Not Found" when the get_diff_of_pull_request function is called.

After further investigation and reviewing the GitHub API documentation, it appears there are two key aspects to this issue:

1. The URL used for the request: The pull_request.diff_url that PyGithub provides (e.g., https://github.com/OWNER/REPO/pull/PULL_NUMBER.diff) is primarily a web UI URL. While it works for private repos in a browser (where GitHub handles your session authentication), directly calling it with requests.get() for a private repository often results in a "404 Not Found" or "401 Unauthorized" error.
   Instead, we might need to use the explicit GitHub REST API endpoint for pull requests: https://api.github.com/repos/{owner}/{repo}/pulls/{pull_number}.

2. The Accept header for diff content: As documented in the GitHub Docs for Pull Requests under "Custom Media Types" (and referenced in the GitHub Community discussion here), to specifically get the diff content from this API endpoint, a particular Accept header is required.

Therefore, the solution needs to combine both: target the correct API endpoint and include the necessary Accept header.

Could we please update the requests.get() call within get_diff_of_pull_request (in src/git_bob/_github_utilities.py) to use the API endpoint and include the Accept: application/vnd.github.v3.diff header?

The requests call should be constructed like this:

import requests
import os

# Assuming repository_full_name is 'owner/repo' and pull_request_number is the PR ID
api_url = f"https://api.github.com/repos/{repository_full_name}/pulls/{pull_request_number}"

access_token = os.getenv('GITHUB_API_KEY')
headers = {
    "Accept": "application/vnd.github.v3.diff", # This tells GitHub we want the diff format
    "Authorization": f"token {access_token}"    # This handles authentication
}

response = requests.get(api_url, headers=headers)

[haesleinhuepf]

Git-bob implement this modification

[haesleinhuepf]

Hi @aarogyarijal , thanks for the code snippet! This is part of git-bob 0.24.0 . Feel free to send a pull-request next time you know the solution to an issue. ;-)

Thanks again!