Allow compatible dependency versions

[bmos]

What's Changed

• Set all direct dependencies to be more permissive. This will avoid us having to approve each dependabot version update that isn't breaking compatibility.

[github-actions]

Coverage report

This PR does not seem to contain any modification to coverable code.

[austinweisgrau]

Frankly the range of changes here is a bit diffuse for a single PR. I agree with the title that the changes to requirements seem to be the most consequential. I think they are potentially problematic for two reasons:

• having ranges for so many packages means that pip will potentially have to spend a LOT longer trying to find compatible versions for everything
• this is a bit less concerning, but in practice you can't always rely on major versions to prevent breaking changes. pinning exact versions is a way to make dependencies more reliable

If people are having issues with the package versions, we already have a solution for that - the friendly dependencies install allows for more liberal package versioning (defined in setup.py). there are tradeoffs with pinning exact versions, and the current setup allows users to choose which side of the tradeoffs they prefer (friendly dependencies or not). this PR gets rid of that choice.

I think it could still be valid, but it's a consequential package design choice point.

[bmos]

I tend to agree (and would generally just set this to a recent version of each with the next minor version as the max), but wanted to demonstrate the diverse range of dependency versions which pass all tests.

[bmos]

Regarding the pip install times, uv installs it very quickly in a brand new venv, but you're right that the pip install takes a while. It takes just over twice as long to install.

[bmos]

I took the changes that weren't necessary to the primary purpose of this PR and moved them to new PRs

[austinweisgrau]

It seems like parsons limited dependencies/extras doesn't work with uv for some reason, which makes me more partial to this PR, although it's still a bit indirect. Seems like we need a comprehensive fix for dependency management. But yeah in the meantime this PR would be helpful for using uv with parsons.

[bmos]

That is so strange because we do test limited dependencies in our GitHub Actions workflow...
What happens when you try and use it locally?

[austinweisgrau]

ahh you're right i'm probably just doing something wrong

[fgregg]

what about ditching the requirements file entirely and doing something like this in setup.py

def main():

    install_requires = [
        "petl",
        ...,
        "simplejson",
    ]
    extras_require = {
        "airtable": ["pyairtable"],
        ...,
        "twilio": ["twilio"],
    }
    extras_require["all"] = sorted({lib for libs in extras_require.values() for lib in libs})
    
    limited_deps = os.environ.get("PARSONS_LIMITED_DEPENDENCIES", "")
    if limited_deps.strip().upper() not in ("1", "YES", "TRUE", "ON"):
        install_requires.extend(extras_require['all'])
        extras_require = {"all": []}

that way you are only needing to manage dependencies in one place?

[bmos]

what about ditching the requirements file entirely and doing something like this in setup.py
...

Personally, I'm very hesitant to use that approach.
There are two common approaches I see for requirements (pyproject.toml and requirements.txt) and that's neither of them.

I do like the functionality though!

[fgregg]

Right now, the two ways to install all the dependencies will install different sets of dependencies.

PARSONS_LIMITED_DEPENDENCIES pip install parsons[all]

and

pip install parsons

will not install the same exact set of requirements, because PARSONS_LIMITED_DEPENDENCIES pip install parsons[all] will install the dependencies specified in the extras_required block of setup.py, and pip install parsons will install the dependencies specified in the requirements.txt file.

this does not seem good.

if we are very, very careful to keep the requirements.txt file up and the setup.py extras_require up to date we may be able to have the two ways of install all the dependencies install the same, but that seems actually very hard.

I agree that our setup.py is strange, and I look forward to world where we don't need to support the dynamic behavior of PARSONS_LIMITED_DEPENDENCIES. I see my suggested change as a step torwards that.

When pip install parsons[all] is the default, we will only have pyproject.toml and no setup.py or requirements.txt. We can't ditch setup.py yet because we need the dynamic behavior of PARSONS_LIMITED_DEPENDENCIES, but we can go ahead and ditch requirements.txt now.

[austinweisgrau]

I do think it is more idiomatic for something like the PARSONS_LIMITED_DEPENDENCIES behavior to be the default, and users always need to specify extras or [all] if they want them. This can all be achieved in pyproject.toml rather than setup.py or requirements.txt. Here's an example: pypa/setuptools#3627 (comment)