Skip to content

[dependencies] update 3rd party dependencies #407

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 46 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
9225f3a
[setup.py] remove 'pip' from requires, unsafe
apalala Sep 8, 2021
05ed6c1
[dependencies] pip-compile --upgrade setup.py and tests/requirements.in
apalala Sep 8, 2021
25a5446
[dependencies] pip compile --upgrade setup.py
apalala Sep 8, 2021
f4a9ffc
[dependencies] remove requirements.in as it duplicates setup.py
apalala Sep 8, 2021
450b73d
[dependencies] enable TravisCI on branch
apalala Sep 8, 2021
402dcf4
[cli][bug] fix incorrect parameter passed for `click.prompt(default=)`
apalala Sep 8, 2021
878a967
[travisci] use Python 3.9 + apply to all branches
apalala Sep 8, 2021
3dd95a7
[setup.py] unpin click
apalala Sep 8, 2021
9342d08
[ci] add github pipeline/action for builds
apalala Sep 8, 2021
4df40f0
[dependencies] pin click==7.1.2 for Python 2.7 support
apalala Sep 8, 2021
8df9044
[travisci] restore active branches
apalala Sep 8, 2021
81c3fc8
[github] install other Python versions in action
apalala Sep 8, 2021
d6577e5
[github] bugfix in actions
apalala Sep 8, 2021
02489bf
[tox] drop py36
apalala Sep 8, 2021
0f4558f
[tox] drop py27
apalala Sep 8, 2021
8e316dd
[github] try to install other Python versions
apalala Sep 8, 2021
02bc076
Bump version: 2.13.0 → 2.14.0
apalala Sep 8, 2021
f4f98ba
update from incorrect commit in master
apalala Sep 8, 2021
109ee19
Merge branch 'master' into update_dependencies
apalala Sep 8, 2021
6e8caa9
[tests][cli] skip tests involving `click.invoke(input=)`
apalala Sep 8, 2021
616cdd7
[dependencies] unpin dependencies in setup.py
apalala Sep 8, 2021
8c4b490
[setup.py] drop support for 2.7, 3.5, 3.6
apalala Sep 8, 2021
911cd58
[versioning] revert version changes
apalala Sep 9, 2021
db8f39a
[ci] remove experimental Github Actions
apalala Sep 9, 2021
abf7150
[setup.py] revert version change + unpin requirements
apalala Sep 9, 2021
7b83121
[versioning] restore version
apalala Sep 9, 2021
015be38
[test] include py310
apalala Oct 20, 2021
3213221
[requirements] update + force updated python-scrapinghub from branch
apalala Oct 20, 2021
3811292
Bump version: 2.13.0 → 3.0.0
apalala Oct 20, 2021
55e4131
[setup.py] patch for branch version of python-scrapinghub
apalala Oct 20, 2021
d99ed0f
[setup.py] revert latest change
apalala Oct 20, 2021
227cc64
[setup.py] pin click to >=7.0
apalala Oct 23, 2021
b47afd0
[tox] test on py310
apalala Oct 23, 2021
fc441d4
Merge remote-tracking branch 'refs/remotes/origin/update_dependencies…
apalala Oct 23, 2021
c586b58
[test] clean up tox.ini
apalala Oct 23, 2021
b3f158a
[setup.py] fix syntax error
apalala Oct 23, 2021
cddc023
ignore PyCharm configuration dir .idea/
apalala Oct 23, 2021
cdeaef0
[test] refactor requirments files for library version consistency
apalala Oct 23, 2021
3c98c78
[tests] resolve deprecation in pyyaml.yaml.load()
apalala Oct 23, 2021
fddef8e
[test] resolve self.assertRaisesRegex deprecation
apalala Oct 23, 2021
99f5670
[utils] solve SafeconfigParser deprecation
apalala Oct 23, 2021
b28c03f
[requirements] use the branch version of python-scrapinghub
apalala Oct 23, 2021
962ac7f
[make] requirements-test.txt depends on requirements-test.in
apalala Oct 23, 2021
e1482ac
[tests] disable test_can_deploy_an_egg_from_pypi()
apalala Oct 23, 2021
22d0ad1
[requirements] force branch version of python-scrapinghub
apalala Oct 23, 2021
90886f3
[tests] resolve deprecation warnings
apalala Oct 23, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 2.13.0
current_version = 3.0.0
commit = True
tag = True
tag_name = v{new_version}
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -127,4 +127,7 @@ tags
# Persistent undo
[._]*.un~

# PyCharm
.idea/

# End of https://www.gitignore.io/api/vim,python
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
language: python
python: 3.6
python: 3.9
sudo: false
env:
- TOX_ENV=py27
Expand Down
12 changes: 12 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@

start:
echo "Hello World!"

requirements: requirements-test.txt

requirements-test.txt: requirements-test.in requirements.txt
pip-compile -qU requirements-test.in

requirements.txt: setup.py
pip-compile -qU setup.py

13 changes: 13 additions & 0 deletions requirements-test.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
-r requirements.txt
python-dateutil
mock
pytest
pytest-cov
flake8
pipenv

# latest python-scrapinghub
git+https://[email protected]/scrapinghub/python-scrapinghub.git@update_py310_minus_py27

# CVE-2020-29651
py>=1.10.0
115 changes: 115 additions & 0 deletions requirements-test.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
#
# This file is autogenerated by pip-compile with python 3.9
# To update, run:
#
# pip-compile requirements-test.in
#
attrs==21.2.0
# via pytest
backports.entry-points-selectable==1.1.0
# via virtualenv
certifi==2021.10.8
# via
# -r requirements.txt
# pipenv
# requests
charset-normalizer==2.0.7
# via
# -r requirements.txt
# requests
click==8.0.3
# via -r requirements.txt
coverage[toml]==6.0.2
# via pytest-cov
distlib==0.3.3
# via virtualenv
docker==5.0.3
# via -r requirements.txt
filelock==3.3.1
# via virtualenv
flake8==4.0.1
# via -r requirements-test.in
idna==3.3
# via
# -r requirements.txt
# requests
iniconfig==1.1.1
# via pytest
mccabe==0.6.1
# via flake8
mock==4.0.3
# via -r requirements-test.in
msgpack==1.0.2
# via scrapinghub
packaging==21.0
# via pytest
pipenv==2021.5.29
# via -r requirements-test.in
platformdirs==2.4.0
# via virtualenv
pluggy==1.0.0
# via pytest
py==1.10.0
# via
# -r requirements-test.in
# pytest
pycodestyle==2.8.0
# via flake8
pyflakes==2.4.0
# via flake8
pyparsing==3.0.0
# via packaging
pytest==6.2.5
# via
# -r requirements-test.in
# pytest-cov
pytest-cov==3.0.0
# via -r requirements-test.in
python-dateutil==2.8.2
# via -r requirements-test.in
pyyaml==6.0
# via -r requirements.txt
requests==2.26.0
# via
# -r requirements.txt
# docker
# scrapinghub
retrying==1.3.3
# via
# -r requirements.txt
# scrapinghub
git+https://[email protected]/scrapinghub/python-scrapinghub.git@update_py310_minus_py27
# via
# -r requirements-test.in
# -r requirements.txt
six==1.16.0
# via
# -r requirements.txt
# python-dateutil
# retrying
# scrapinghub
# virtualenv
toml==0.10.2
# via
# -r requirements.txt
# pytest
tomli==1.2.1
# via coverage
tqdm==4.62.3
# via -r requirements.txt
urllib3==1.26.7
# via
# -r requirements.txt
# requests
virtualenv==20.9.0
# via pipenv
virtualenv-clone==0.5.7
# via pipenv
websocket-client==1.2.1
# via
# -r requirements.txt
# docker

# The following packages are considered to be unsafe in a requirements file:
# pip
# setuptools
14 changes: 0 additions & 14 deletions requirements.in

This file was deleted.

50 changes: 25 additions & 25 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,43 +1,43 @@
#
# This file is autogenerated by pip-compile
# This file is autogenerated by pip-compile with python 3.9
# To update, run:
#
# pip-compile
# pip-compile setup.py
#
certifi==2020.6.20
certifi==2021.10.8
# via requests
chardet==3.0.4
charset-normalizer==2.0.7
# via requests
click==7.0
# via -r requirements.in
docker==4.2.2
# via -r requirements.in
idna==2.10
click==8.0.3
# via shub (setup.py)
docker==5.0.3
# via shub (setup.py)
idna==3.3
# via requests
pyyaml==5.4
# via -r requirements.in
requests==2.24.0
pyyaml==6.0
# via shub (setup.py)
requests==2.26.0
# via
# -r requirements.in
# docker
# scrapinghub
# shub (setup.py)
retrying==1.3.3
# via
# -r requirements.in
# scrapinghub
scrapinghub==2.3.1
# via -r requirements.in
six==1.15.0
# shub (setup.py)
#scrapinghub==2.3.1
# via shub (setup.py)
git+https://[email protected]/scrapinghub/python-scrapinghub.git@update_py310_minus_py27
six==1.16.0
# via
# -r requirements.in
# docker
# retrying
# scrapinghub
toml==0.10.1
# via -r requirements.in
tqdm==4.55.1
# via -r requirements.in
urllib3==1.25.9
# shub (setup.py)
toml==0.10.2
# via shub (setup.py)
tqdm==4.62.3
# via shub (setup.py)
urllib3==1.26.7
# via requests
websocket-client==0.57.0
websocket-client==1.2.1
# via docker
13 changes: 5 additions & 8 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

setup(
name='shub',
version='2.13.0',
version='3.0.0',
packages=find_packages(exclude=('tests', 'tests.*')),
url=about['DOCS_LINK'],
description='Scrapinghub Command Line Client',
Expand All @@ -29,15 +29,14 @@
include_package_data=True,
zip_safe=False,
install_requires=[
'click==7.0',
'click>=7.0',
'docker',
'pip',
'PyYAML',
'pyyaml',
'retrying',
'requests',
'scrapinghub>=2.3.1',
'six>=1.7.0',
'tqdm==4.55.1',
'tqdm',
Comment on lines -40 to +39
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably keep some limit, assuming we do not support just any version.

'toml',
],
classifiers=[
Expand All @@ -46,12 +45,10 @@
'Natural Language :: English',
'License :: OSI Approved :: BSD License',
'Programming Language :: Python',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3.10',
'Operating System :: OS Independent',
'Environment :: Console',
'Topic :: Internet :: WWW/HTTP',
Expand Down
2 changes: 1 addition & 1 deletion shub/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
__version__ = '2.13.0'
__version__ = '3.0.0'


# Links to documentation to use over the project sources
Expand Down
4 changes: 2 additions & 2 deletions shub/config.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
from __future__ import absolute_import
import netrc
import os
import warnings
from collections import namedtuple

import click
Expand Down Expand Up @@ -326,7 +325,8 @@ def _select_image_for_project(self, target, project):

def get_target(self, target, auth_required=True):
"""Return (project_id, endpoint, apikey) for given target."""
warnings.warn("get_target is deprecated, use get_target_conf instead")
# WARNING: deprecating is just annoying because this library's unit tests still use this method
# warnings.warn("get_target is deprecated, use get_target_conf instead")
targetconf = self.get_target_conf(target, auth_required=auth_required)
return (
targetconf.project_id,
Expand Down
8 changes: 4 additions & 4 deletions shub/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
import time

from collections import deque
from six.moves.configparser import SafeConfigParser
from six.moves.configparser import ConfigParser
from distutils.spawn import find_executable
from distutils.version import LooseVersion, StrictVersion
from glob import glob
Expand Down Expand Up @@ -484,9 +484,9 @@ def inside_project():


def get_config(use_closest=True):
"""Get Scrapy config file as a SafeConfigParser"""
"""Get Scrapy config file as a ConfigParser"""
sources = get_sources(use_closest)
cfg = SafeConfigParser()
cfg = ConfigParser()
cfg.read(sources)
return cfg

Expand All @@ -505,7 +505,7 @@ def get_sources(use_closest=True):


def get_scrapycfg_targets(cfgfiles=None):
cfg = SafeConfigParser()
cfg = ConfigParser()
cfg.read(cfgfiles or [])
baset = dict(cfg.items('deploy')) if cfg.has_section('deploy') else {}
targets = {}
Expand Down
48 changes: 0 additions & 48 deletions tests/requirements-py2.txt

This file was deleted.

Loading