feat: sync feature set pygithub (#60)

kaosx5s · coreycaverly · zejunintuit · web-flow · commit 8c22fdd5a092 · 2026-02-27T10:24:28.000-08:00
* fix: drop support for outdated python * fix: update pygithub to support 2.2.0 or higher * fix: update tests for newer pytest * fix: check pygithub 2.4.0 support * fix: check pygithub 2.6.1 support * fix: set lazy to be false by default for backwards compatibility * fix: remove completed param for pygithub 2.6.0 support * fix: add required requester for pygithub 2.6.0 support * fix: fix param typo * fix: make requester a magicmock * fix: update tests for new lazy param * fix: update lazy from str to bool * feat: add support for pool_size and passing down lazy correctly * fix: correct issue with pr creation and pygithub drift * fix: update for issue #59 * revving for release (#57) Signed-off-by: Corey Caverly <corey.caverly@autodesk.com> * Updating workflow for python publish pipeline (#58) * Allow path define on get files (#42) * Add support for setting a path on get_files Signed-off-by: kaosx5s <jeremy.xfives@gmail.com> * fix repo test after target change in 3.6.0 Signed-off-by: kaosx5s <> --------- Signed-off-by: kaosx5s <jeremy.xfives@gmail.com> Signed-off-by: kaosx5s <> Co-authored-by: kaosx5s <> Signed-off-by: Corey Caverly <corey.caverly@autodesk.com> * Removed transient line (#50) Signed-off-by: Corey Caverly <corey.caverly@autodesk.com> * Revving version for release (#51) Signed-off-by: Corey Caverly <corey.caverly@autodesk.com> Co-authored-by: Corey Caverly <corey.caverly@autodesk.com> Signed-off-by: Corey Caverly <corey.caverly@autodesk.com> * Update docker-publish.yml (#52) * Update docker-publish.yml Removing Dockerhub and just doing ghcr Signed-off-by: Corey Caverly <corey.caverly@autodesk.com> * Revving for release Signed-off-by: Corey Caverly <corey.caverly@autodesk.com> * Decided to remove and not just comment out Signed-off-by: Corey Caverly <corey.caverly@autodesk.com> --------- Signed-off-by: Corey Caverly <corey.caverly@autodesk.com> * retry on timeout (#54) * retry on timeout Signed-off-by: zliu3 <zejun_liu@intuit.com> * add unit test and throw exception on timeout Signed-off-by: zliu3 <zejun_liu@intuit.com> * update README.md Signed-off-by: zliu3 <zejun_liu@intuit.com> * add test to make sure retry on exception, add a good case Signed-off-by: zliu3 <zejun_liu@intuit.com> --------- Signed-off-by: zliu3 <zejun_liu@intuit.com> Co-authored-by: zliu3 <zejun_liu@intuit.com> Signed-off-by: Corey Caverly <corey.caverly@autodesk.com> * feat: drop support for older python versions, support up to 2.6.1 for pygithub (#56) * fix: drop support for outdated python * fix: update pygithub to support 2.2.0 or higher * fix: update tests for newer pytest * fix: check pygithub 2.4.0 support * fix: check pygithub 2.6.1 support * fix: set lazy to be false by default for backwards compatibility * fix: remove completed param for pygithub 2.6.0 support * fix: add required requester for pygithub 2.6.0 support * fix: fix param typo * fix: make requester a magicmock * fix: update tests for new lazy param * fix: update lazy from str to bool Signed-off-by: Corey Caverly <corey.caverly@autodesk.com> * revving for release Signed-off-by: Corey Caverly <corey.caverly@autodesk.com> * Revving python version for publish Signed-off-by: Corey Caverly <corey.caverly@autodesk.com> * Updating readme to reflect new location of docker images Signed-off-by: Corey Caverly <corey.caverly@autodesk.com> * Revving for new release Signed-off-by: Corey Caverly <corey.caverly@autodesk.com> --------- Signed-off-by: kaosx5s <jeremy.xfives@gmail.com> Signed-off-by: kaosx5s <> Signed-off-by: Corey Caverly <corey.caverly@autodesk.com> Signed-off-by: zliu3 <zejun_liu@intuit.com> Co-authored-by: Jeremy <1028480+kaosx5s@users.noreply.github.com> Co-authored-by: zejunintuit <zejunliuajk@gmail.com> Co-authored-by: zliu3 <zejun_liu@intuit.com> * fix: typo in comment for github exception handling * fix: update github actions for newer setuppython verisons * fix: remove errant store_true on args --------- Signed-off-by: Corey Caverly <corey.caverly@autodesk.com> Signed-off-by: kaosx5s <jeremy.xfives@gmail.com> Signed-off-by: kaosx5s <> Signed-off-by: zliu3 <zejun_liu@intuit.com> Co-authored-by: kaosx5s <> Co-authored-by: Corey Caverly <coreycaverly@users.noreply.github.com> Co-authored-by: zejunintuit <zejunliuajk@gmail.com> Co-authored-by: zliu3 <zejun_liu@intuit.com>
diff --git a/.github/workflows/publish-python.yml b/.github/workflows/publish-python.yml
@@ -9,9 +9,9 @@ jobs:
     name: Create release-artifacts
     runs-on: ubuntu-latest
     steps:
-    - uses: actions/checkout@v1
+    - uses: actions/checkout@v6
     - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v1
+      uses: actions/setup-python@v6
       with:
         python-version: 3.11
     - name: Install dependencies
diff --git a/.github/workflows/python-app.yml b/.github/workflows/python-app.yml
@@ -11,9 +11,9 @@ jobs:
         python-version: [3.9, '3.10', 3.11, 3.12, 3.13]
 
     steps:
-    - uses: actions/checkout@v1
+    - uses: actions/checkout@v6
     - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v1
+      uses: actions/setup-python@v6
       with:
         python-version: ${{ matrix.python-version }}
     - name: Install dependencies
@@ -25,5 +25,5 @@ jobs:
         pytest
     - name: Publish CodeCov
       uses: codecov/codecov-action@v1
-      with: 
+      with:
         token: ${{ secrets.codecovToken }}
diff --git a/gordian/gordian.py b/gordian/gordian.py
@@ -80,6 +80,28 @@ def get_basic_parser():
         dest='pr_labels',
         help='List of space separated label names you wish to add to your pull request(s)'
     )
+    parser.add_argument(
+        '--draft',
+        required=False,
+        default=False,
+        dest='draft',
+        help='Create a draft pull request'
+    )
+    parser.add_argument(
+        '--lazy',
+        required=False,
+        default=False,
+        dest='lazy',
+        help='Use lazy loading for the Github client'
+    )
+    parser.add_argument(
+        '--pool-size',
+        required=False,
+        default=None,
+        action='store',
+        dest='pool_size',
+        help='Pool size for the Github client'
+    )
     parser.add_argument(
         '-F', '--file',
         required=False,
@@ -190,15 +212,23 @@ def transform(args, transformations, repositories, pr_description, pr_created_ca
                 branch=args.branch,
                 semver_label=args.semver_label,
                 target_branch=args.target_branch,
-                fork=args.fork
+                fork=args.fork,
+                lazy=args.lazy,
+                pool_size=args.pool_size
         )
         for transformation in transformations:
             transformation(args, repo).run()
         if repo.dirty:
             repo.bump_version(args.dry_run)
             if not args.dry_run:
                 try:
-                    pull_request = repo.create_pr(args.pr_message, pr_description, args.target_branch, args.pr_labels)
+                    pull_request = repo.create_pr(
+                        pr_message=args.pr_message,
+                        pr_body=pr_description,
+                        target_branch=args.target_branch,
+                        labels=args.pr_labels,
+                        draft=args.draft
+                    )
                     pull_request_urls.append(pull_request.html_url)
                     if pr_created_callback is not None:
                         logger.debug(f'Calling post pr created callback with: {pull_request}, {repo.branch_name}')
diff --git a/gordian/repo.py b/gordian/repo.py
@@ -1,5 +1,4 @@
-from github import Github
-from github import GithubException
+from github import Github, GithubException, BadCredentialsException, RateLimitExceededException, UnknownObjectException
 import datetime
 import logging
 import os
@@ -14,7 +13,7 @@
 
 class Repo:
 
-    def __init__(self, repo_name, github_api_url=None, branch=None, github=None, files=None, semver_label=None, target_branch='master', fork=False, token=None, username=None, password=None, lazy=False):
+    def __init__(self, repo_name, github_api_url=None, branch=None, github=None, files=None, semver_label=None, target_branch='master', fork=False, token=None, username=None, password=None, lazy=False, pool_size=None):
         if github_api_url is None:
             self.github_api_url = BASE_URL
         else:
@@ -29,10 +28,10 @@ def __init__(self, repo_name, github_api_url=None, branch=None, github=None, fil
 
             if token:
                 logger.debug('Using git token for authentication')
-                self._github = Github(base_url=self.github_api_url, login_or_token=token, lazy=lazy)
+                self._github = Github(base_url=self.github_api_url, login_or_token=token, lazy=lazy, pool_size=pool_size)
             else:
                 logger.debug('Using git username and password for authentication')
-                self._github = Github(base_url=self.github_api_url, login_or_token=username, password=password, lazy=lazy)
+                self._github = Github(base_url=self.github_api_url, login_or_token=username, password=password, lazy=lazy, pool_size=pool_size)
 
         if files is None:
             files = []
@@ -126,15 +125,26 @@ def _set_target_branch(self, target_branch, source_branch=None):
             self.branch_name = f"refs/heads/{datetime.datetime.now().strftime('%Y-%m-%d-%H%M%S.%f')}"
             self.source_branch = self.target_ref
 
-    @retry((GithubException, TimeoutError), tries=3, delay=1, backoff=2)
+    @retry((GithubException, TimeoutError, RateLimitExceededException), tries=3, delay=1, backoff=2)
     def _get_repo_contents(self, path):
         try:
             logger.debug(f'Fetching repo contents {path}...')
             return self._source_repo.get_contents(path, self.target_branch)
+
+        # https://docs.github.com/en/rest/repos/contents?apiVersion=2022-11-28#get-repository-content--status-codes
+        except BadCredentialsException as e:
+            logger.info(f'Error fetching repo contents due to bad credentials: {e}')
+            raise e
+        except RateLimitExceededException as e:
+            logger.info(f'Error fetching repo contents due to rate limit: {e}')
+            raise e
+        except UnknownObjectException as e:
+            logger.info(f'Error fetching repo contents due to unknown object: {e}')
+            raise e
         except GithubException as e:
-            if e.status == 404:
-                raise e
+            # generic catch all for any other github exceptions
             logger.info(f'Error fetching repo contents: {e}')
+            raise e
         except TimeoutError as e:
             logger.info(f'Error fetching repo contents: {e}')
             raise e
@@ -221,12 +231,14 @@ def delete_file(self, file, message, dry_run=False):
             branch=self.branch_name
         )
 
-    def create_pr(self, pr_message, pr_body, target_branch, labels=[]):
+    def create_pr(self, pr_message, pr_body, target_branch, labels=[], draft=False):
+        # https://github.com/PyGithub/PyGithub/blob/v2.8.1/github/Repository.py#L1822 , we need to stay in-line with the PyGithub library
         pr = self._target_repo.create_pull(
-            pr_message,
-            pr_body,
-            target_branch,
-            f'{self._source_repo.owner.login}:{self.branch_name}'
+            title=pr_message,
+            body=pr_body,
+            base=target_branch,
+            head=f'{self._source_repo.owner.login}:{self.branch_name}',
+            draft=draft
         )
         if labels:
             pr.set_labels(*labels)
diff --git a/tests/test_gordian.py b/tests/test_gordian.py
@@ -7,7 +7,7 @@
 class TestGordian(unittest.TestCase):
 
     class Args(object):
-        def __init__(self, config_file='./tests/fixtures/test_config.yaml', dry_run = False):
+        def __init__(self, config_file='./tests/fixtures/test_config.yaml', dry_run = False, lazy = False, pool_size = None, draft = False):
             self.config_file = config_file
             self.major = False
             self.minor = False
@@ -22,15 +22,18 @@ def __init__(self, config_file='./tests/fixtures/test_config.yaml', dry_run = Fa
             self.description = ''
             self.description_file = None
             self.fork = False
+            self.lazy = lazy
+            self.pool_size = pool_size
+            self.draft = draft
 
     def test_apply_transformations_without_changes(self):
         with patch('gordian.gordian.Repo') as RepoMock, patch('gordian.transformations.Transformation') as TransformationMockClass:
             instance = RepoMock.return_value
             instance.dirty = False
             apply_transformations(TestGordian.Args(), [TransformationMockClass])
             RepoMock.assert_has_calls([
-                call('testOrg/TestService1', github_api_url=None, branch='test', semver_label=None, target_branch='master', fork=False),
-                call('testOrg/TestService2', github_api_url=None, branch='test', semver_label=None, target_branch='master', fork=False)
+                call('testOrg/TestService1', github_api_url=None, branch='test', semver_label=None, target_branch='master', fork=False, lazy=False, pool_size=None),
+                call('testOrg/TestService2', github_api_url=None, branch='test', semver_label=None, target_branch='master', fork=False, lazy=False, pool_size=None)
             ])
 
     def test_apply_transformations_with_changes(self):
@@ -39,15 +42,49 @@ def test_apply_transformations_with_changes(self):
             instance.dirty = True
             apply_transformations(TestGordian.Args(), [TransformationMockClass])
             RepoMock.assert_has_calls([call().bump_version(False), call().bump_version(False)], any_order=True)
-            RepoMock.assert_has_calls([call().create_pr('test', '', 'master', ANY), call().create_pr('test', '', 'master', ANY)], any_order=True)
+            RepoMock.assert_has_calls([
+                call().create_pr(pr_message='test', pr_body='', target_branch='master', labels=ANY, draft=False),
+                call().create_pr(pr_message='test', pr_body='', target_branch='master', labels=ANY, draft=False)], any_order=True)
 
     def test_apply_transformations_with_changes_dry_run(self):
         with patch('gordian.gordian.Repo') as RepoMock, patch('gordian.transformations.Transformation') as TransformationMockClass:
             instance = RepoMock.return_value
             instance.dirty = True
             apply_transformations(TestGordian.Args(dry_run=True), [TransformationMockClass])
             RepoMock.assert_has_calls([call().bump_version(True), call().bump_version(True)], any_order=True)
-            self.assertNotIn(call().repo.create_pr('test', '', 'master', ANY), RepoMock.mock_calls)
+            self.assertNotIn(call().repo.create_pr(pr_message='test', pr_body='', target_branch='master', labels=ANY, draft=False), RepoMock.mock_calls)
+
+    def test_apply_transformations_with_changes_draft(self):
+        with patch('gordian.gordian.Repo') as RepoMock, patch('gordian.transformations.Transformation') as TransformationMockClass:
+            instance = RepoMock.return_value
+            instance.dirty = True
+            apply_transformations(TestGordian.Args(draft=True), [TransformationMockClass])
+            RepoMock.assert_has_calls([call().bump_version(False), call().bump_version(False)], any_order=True)
+            RepoMock.assert_has_calls([
+                call().create_pr(pr_message='test', pr_body='', target_branch='master', labels=ANY, draft=True),
+                call().create_pr(pr_message='test', pr_body='', target_branch='master', labels=ANY, draft=True)], any_order=True)
+
+    def test_apply_transformations_with_changes_lazy(self):
+        with patch('gordian.gordian.Repo') as RepoMock, patch('gordian.transformations.Transformation') as TransformationMockClass:
+            instance = RepoMock.return_value
+            instance.dirty = True
+            apply_transformations(TestGordian.Args(lazy=True), [TransformationMockClass])
+            RepoMock.assert_has_calls([call().bump_version(False), call().bump_version(False)], any_order=True)
+            RepoMock.assert_has_calls([
+                call('testOrg/TestService1', github_api_url=None, branch='test', semver_label=None, target_branch='master', fork=False, lazy=True, pool_size=None),
+                call().create_pr(pr_message='test', pr_body='', target_branch='master', labels=ANY, draft=False),
+                call().create_pr(pr_message='test', pr_body='', target_branch='master', labels=ANY, draft=False)], any_order=True)
+
+    def test_apply_transformations_with_changes_pool_size(self):
+        with patch('gordian.gordian.Repo') as RepoMock, patch('gordian.transformations.Transformation') as TransformationMockClass:
+            instance = RepoMock.return_value
+            instance.dirty = True
+            apply_transformations(TestGordian.Args(pool_size=10), [TransformationMockClass])
+            RepoMock.assert_has_calls([call().bump_version(False), call().bump_version(False)], any_order=True)
+            RepoMock.assert_has_calls([
+                call('testOrg/TestService1', github_api_url=None, branch='test', semver_label=None, target_branch='master', fork=False, lazy=False, pool_size=10),
+                call().create_pr(pr_message='test', pr_body='', target_branch='master', labels=ANY, draft=False),
+                call().create_pr(pr_message='test', pr_body='', target_branch='master', labels=ANY, draft=False)], any_order=True)
 
     def test_apply_transformations_with_changes_and_callback(self):
         with patch('gordian.gordian.Repo') as RepoMock, patch('gordian.transformations.Transformation') as TransformationMockClass:
@@ -60,8 +97,8 @@ def test_apply_transformations_with_changes_and_callback(self):
             pull_request = RepoMock.return_value.create_pr.return_value
             RepoMock.assert_has_calls([call().bump_version(False), call().bump_version(False)], any_order=True)
             RepoMock.assert_has_calls([
-                call().create_pr('test', '', 'target_branch', ANY),
-                call().create_pr('test', '', 'target_branch', ANY)],
+                call().create_pr(pr_message='test', pr_body='', target_branch='target_branch', labels=ANY, draft=False),
+                call().create_pr(pr_message='test', pr_body='', target_branch='target_branch', labels=ANY, draft=False)],
                 any_order=True
             )
             callback_mock.assert_has_calls([
@@ -77,7 +114,9 @@ def test_apply_transformations_with_changes_default_labels(self):
             gordian_args.pr_labels = []
             apply_transformations(gordian_args, [TransformationMockClass])
             RepoMock.assert_has_calls([call().bump_version(False), call().bump_version(False)], any_order=True)
-            RepoMock.assert_has_calls([call().create_pr('test', '', 'master', ANY), call().create_pr('test', '', 'master', ANY)], any_order=True)
+            RepoMock.assert_has_calls([
+                call().create_pr(pr_message='test', pr_body='', target_branch='master', labels=ANY, draft=False),
+                call().create_pr(pr_message='test', pr_body='', target_branch='master', labels=ANY, draft=False)], any_order=True)
             self.assertNotIn(call()._repo.create_pull().set_labels(ANY), RepoMock.mock_calls)
 
     def test_apply_transformations_with_changes_custom_description(self):
@@ -89,4 +128,6 @@ def test_apply_transformations_with_changes_custom_description(self):
             gordian_args.description_file = './tests/fixtures/pr_description'
             apply_transformations(gordian_args, [TransformationMockClass])
             RepoMock.assert_has_calls([call().bump_version(False), call().bump_version(False)], any_order=True)
-            RepoMock.assert_has_calls([call().create_pr('test', description, 'master', ANY), call().create_pr('test', description, 'master', ANY)], any_order=True)
+            RepoMock.assert_has_calls([
+                call().create_pr(pr_message='test', pr_body=description, target_branch='master', labels=ANY, draft=False),
+                call().create_pr(pr_message='test', pr_body=description, target_branch='master', labels=ANY, draft=False)], any_order=True)
diff --git a/tests/test_repo.py b/tests/test_repo.py
