Release 1.0

[danchr]

From an outside perspective, versioning of Dulwich releases seems rather arbitrary:

• Despite being a rather old library, all releases start with 0.x.y indicating that it's somehow “not done”.
• The distinction between major and minor releases is irrelevant, as Dulwich has never had a major release.
• Minor releases are expected to contain API changes — although ideally, they should be backwards compatible.
• Patch releases should only fix bugs, and should never change APIs.

Yet, patch releases frequently contain breaking changes, such as #1891 which changed the return type of client.get_credentials_from_store() from a tuple to a generator of tuples. As the maintainer of a project depending on Dulwich, this is quite annoying. Not even guarding on minor releases is sufficient; I should probably mark it as incompatible with any future release.

(Now hg-git has also had a fair amount of usages of functions prefixed with _, i.e. private ones. When those break, that's our fault. But that's expected; what's annoying is changes to exposed functions, with no regards to backward compatibily, and in a release supposed to be compatible.)

[jelmer]

Hi Dan,

Thanks for the bug report. I do appreciate this can be quite annoying.

Strictly speaking we are following semver today, it says:

Major version zero (0.y.z) is for initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable.

Dulwich has always been pre 1.0 because it was never a complete git implementation (until very recently). It for example didn't support merge or rebase until 6 months ago. It's also seen quite a lot of churn recently to get to a more complete implementation. It may be time to re-evaluate that and consider doing a 1.0 release.

The versioning philosophy for Dulwich isn't written down anywhere, but it's been roughly that we bump the minor version when introducing changes that could break higher level library users (i.e. those that rely on porcelain, repo, etc). It's hard (though mostly costly, certainly not impossible) to avoid breaking more deep/low level users that implement interfaces like bzr-git or hg-git do. If we were strict about semver here, almost every release would be a minor version bump.

Some random thoughts:

• the current approach is meant to be helpful for a particular group of users, though admittedly that group or the policy isn't well documented
• if we were to go for strict semver (i.e. release 1.0), we should consider making a number of functions and modules private first
• should we just be more careful about breaking obscure APIs in dulwich, providing backwards compatibility stubs/etc instead? Or is what we're doing fine and it should just be better reflect in versioning?
• we haven't really documented what would be required for 1.0 anywhere; some of the obvious gaps have been filled now that merge and rebase are supported. Is there anything left?

I'll have a think and will follow up here, and am also curious if you have any additional thoughts.

[danchr]

I do understand that maintaining a stable API is a daunting task, especially with volunteer labour. I'd definitely appreciate that, if some magic fairy made it happen, but I also doubt it ever will 😁

The problem is more one of communication. I not that versed in whether the implementation is complete, but 0.x usually also means that something isn't ready for consumption and distribution — but even if the API is a bit fluent, the actual functionality has been quite good for a fairly long time. It's just very annoying not to be able to assume anything about future compatibility.

So yeah, a 1.0 would be really awesome — and making more functions, methods and classes private might actually be a good thing! That way, at least we'd know we (might) be doing the wrong thing…

If you wanted a summary of what I'd expect, it'd be something like:

• N.x.y — what you currently do for minor releases; all APIs may break.
• x.N.y — likely to break bzr-git and hg-git. Tries to be reasonably compatible.
• x.y.N — only bugs and security issues.

[jelmer]

Concretely I think the following needs to happen:

• Consistently use ObjectID, Ref and RawObjectID as types across the API Make Ref and ObjectID newtypes for improved typing #2008
• Fix dulwich fails to clone lfs files having names with special character when using smudge command #1878
• Address porcelain.status() output different type of file names. #889 (will involve API changes)
• Possibly address Dulwich Git history contains invalid objects #620 at the same time (we've already got the main branch with rewritten history)
• Land Add comprehensive SHA256 support for Git repositories #1604

I'll also have a think about which additional APIs we might want to clean up or make private.

[jelmer]

I've released 0.25.0; we'll possibly do another 0.25.1 with bug fixes if anything arises.

Furthermore, I'd like to address these for 1.0:

• replace gpg dependency to python-gnupg #1369
• porcelain.status() output different type of file names. #889
• Dulwich Git history contains invalid objects #620