Umbrella crate issues

[seunlanlege]

There are quite a few problems with the umbrella crate although it is a good step in the right direction. There are a few issues

1. Umbrella crate doesn't work with macros

I recently updated one of our libraries (polytope-labs/sc-simnode#48) to the umbrella crate only to discover it is incompatible with the macros in:

• sp-runtime-interface
• cumulus-pallet-parachain-system

2. Missing features for individual crates

The umbrella crate does not allow for using crate-specific features, for example sp-core has the feature: full_crypto, sp-io has the feature: disable_panic_handler. And there are a lot more crates with their own features which are critical to downstream dependencies. None of these features are included in the umbrella crate. There needs to be script which pulls in all of the features of the individual crates in to the umbrella crate.

3. Umbrella crate has a very weird semver on crates.io

This is to me, perhaps the most important issue. The latest umbrella crate has a version v0.7.0 which corresponds to stable2409. But this versioning strategy is uncorrellated with the actual github versioning which would be 1.16.0. It's important that we use the same version everywhere to avoid confusion and dependency hell for downstream users. The umbrella crate should follow the official github release versioning. This would mean publishing the correct version for the polkadot-sdk crate that corresponds with it's github tag. Eg instead of 0.8.0 for stable2412 this should be v1.17.0.

This also allows for publishing patch versions, in between stable releases.

[aurexav]

3 also be reported here

https://substrate.stackexchange.com/questions/12099/versioning-convention

[boundless-forest]

I strongly support 3.

It's very confusing to understand the relationship between the various version numbers and the polkadot-sdk version when examining a project.

For example, can you tell me which version of polkadot-sdk is used for this project if you see the following dependencies in the Cargo.toml file:

sc-network-common = { version = "0.44.0" }
sc-network-sync = { version = "0.44.0" }
sp-api = { version = "34.0.0", default-features = false }
sp-database = { version = "10.0.0" }
sp-genesis-builder = { version = "0.15.1", default-features = false }

Currently, I prefer using the git dependency to specify the polkadot-sdk version. It's more explicit and easier to understand.

[ggwpez]

About 3:

We tried to get rid of the numbers (like 1.16), but it is not possible since many systems expect version numbers. So now we are stuck with those legacy numbers and the new naming schema (stableYYMM).
Aligning them seems like a good idea, although we cannot do it in a way that respects semver, since the umbrella crate will mostly always have breaking changes.
@EgorPopelyaev @Morganamilo, ideas? We can bump major on that thing, but then it would be out of sync with the GitHub release number again...

2:

There needs to be script which pulls in all of the features of the individual crates in to the umbrella crate.

Yea we can do this.

1:

umbrella crate only to discover it is incompatible with the macros in

These macros are buggy then and should know where to find their dependencies, should be able to fix it.

[ggwpez]

We could also go the other way and do 24.09.0 as version number for the umbrella crate to link it up to stable2409. The we could also still do patch releases.

[seunlanlege]

We could also go the other way and do 24.09.0 as version number for the umbrella crate to link it up to stable2409. The we could also still do patch releases.

The issue is, this versioning doesn't make sense for 3rd party crates that depend on polkadot-sdk. I think we need a solution that serves both.

3rd party crates should be able to communicate what Polkadot-sdk version they support and still make minor and patch releases

[gui1117]

Versioning the crate with 1.16 would be inconsistent with semver traditionally used in crates.

Maybe the cleanest is 2412.0.0
Then for 2412-1, we can use 2412.1.0

[seunlanlege]

Then for 2412-1, we can use 2412.1.0

But these are supposed to be patch releases. Minor version bumps are for breaking changes

[seunlanlege]

Any updates with this? Other polkadot-sdk library maintainers are also making the same complaints. I'd say this issue is very existential for the success of the polkadot-sdk project.

polkadot-evm/frontier#1560 (comment)

For some reason, stable2409-3 which is really v1.16.3 was published as polkadot-sdk v0.9.0 whose previous version v0.7.0. Shouldn't the update to the umbrella crate have been a patch version?

This is system is absolutely untennable. It's important to get this right because git dependencies cannot benefit from the stable releases and their subsequent patch versions without requiring ecosystem libraries to update as well.

Recommended reading

https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#tilde-requirements

[ggwpez]

We brought this up yesterday in the OpenDev call, outcome was something like 2412.0.0, 202411.0.0 or similar.

But these are supposed to be patch releases. Minor version bumps are for breaking changes

Minor version changes are for additions. They dont break stuff.

Maybe the cleanest is 2412.0.0
Then for 2412-1, we can use 2412.1.0

Sounds good to me.

[ggwpez]

👀 Version Poll 📜

Poll for parachain devs and other devs downstream of the SDK. What versioning schema do you want for the Umbrella crate?
Example for the stable2412 release; please post a comment with your Project name and an option:

• A) 2412.0.0 / 202412.0.0
• B) 17.1.x
• C) 1.17.0
• D) Other (please specify)

[seunlanlege]

Project name: Hyperbridge

Strongly prefer B) 17.1.x. It Aligns well with the default cargo semver semantics and is familiar to everyone switching from git dependencies.

[aurexav]

Option A makes sense to me immediately; I understand its meaning without needing to think further. In contrast, the other options require a moment of reflection before I can fully grasp their meaning.

[gui1117]

What does 1 mean in 17.1? Would there be 17.2 for the release of polkadot 2.17.0?

If we don't expect more than 99 minor versions, we can also write 117.x.x