Upgrade to c-kzg 2.1.0 and alloy-primitives 1.0

[cakevm]

Status: Sadly upgrading c-kzg brings a performance degradation that let the simulator CI job failing. Increasing GENESIS_DELAY helps, but I wonder why this is happening now.

Issue Addressed

Update c-kzg from v1 to v2. My motivation here is that alloy-consensus now uses c-kzg in v2 and this results in a conflict when using lighthouse in combination with latest alloy. I tried also to disable the czkg feature in alloy, but the conflict persisted.

See here for the alloy update to c-kzg v2: alloy-rs/alloy#2240

Error:

error: failed to select a version for `c-kzg`.
...
versions that meet the requirements `^1` are: 1.0.3, 1.0.2, 1.0.0

the package `c-kzg` links to the native library `ckzg`, but it conflicts with a previous package which links to `ckzg` as well:
package `c-kzg v2.1.0`
    ... which satisfies dependency `c-kzg = "^2.1"` of package `alloy-consensus v0.13.0`
    ... which satisfies dependency `alloy-consensus = "^0.13.0"` of package ...
...

Proposed Changes

• Upgrade alloy-consensus to 0.14.0 and disable all default features
• Upgrade c-kzg to v2.1.0
• Upgrade alloy-primitives to 1.0.0
• Adapt the code to the new API c-kzg
• There is now NO_PRECOMPUTE as my understand from https://github.com/ethereum/c-kzg-4844/pull/545/files we should use 0 here as new_from_trusted_setup_no_precomp does not precomp. But maybe it is misleading. For all other places I used RECOMMENDED_PRECOMP_WIDTH because 8 is matching the recommendation.
• BYTES_PER_G1_POINT and BYTES_PER_G2_POINT are no longer public in c-kzg
• I adapted two tests that checking for the Attestation bitfield size. But I could not pinpoint to what has changed and why now 8 bytes less. I would be happy about any hint, and if this is correct. I found related a PR here: Bump SSZ version for larger bitfield SmallVec #6915
• Use same fields names, in json, as well as c-kzg and rust_eth_kzg for g1_monomial, g1_lagrange, and g2_monomial

Additional Info

• The copy and flattening into the data structure for the KzgSettings seems suboptimal. The whole data structure could be already prepared for this like in c-kzg.
• It seem that the c-kzg lib would provide already those constants (e.g. with c_kzg::ethereum_kzg_settings) and maybe the TrustedSetup could be simplified in the future (e.g. removing it and provide the user to load custom KzgSettings from json.

PS: I did not tested this with a real network, I hope that the test coverage from other crates would highlight if kzg is broken with this change. I just compared by hand that (e.g. g1_monomial) is correctly read. I am open to add test for the correct loading of the kzg, but maybe it make sense to seek to a deeper integration of c-kzg in another step, because it requires much more changes. Or it will be removed with this issue here: #6107 (comment)

[CLAassistant]

All committers have signed the CLA.

[pawanjay176]

Looking good
Not sure how urgently we want this to be in unstable. I would prefer not to have this in any of the stable releases that go into the pectra releases for now

[cakevm]

Thank you for taking the time to have a look. It’s not a priority. I came across this as a transient dependency and initially thought updating the library would be a quick win. But given the issues that have come up, that doesn’t seem to be the case. I’m fine with either closing this PR or leaving it open until there’s a decision on whether c-kzg will be removed.

[michaelsproul]

I'm going to try a narrower and more targeted cargo update, as the Cargo.lock file is currently very noisy and hard to review.

[jimmygchen]

Note that we don't currently use c-kzg4844 for PeerDAS, so the methods above from c-kzg aren't being used. We are using the rust-eth-kzg for PeerDAS support, and we're already doing precomputation there:

lighthouse/crypto/kzg/src/lib.rs

Lines 95 to 100 in 6ee8a7d

	let context = DASContext::new(
	&peerdas_trusted_setup,
	rust_eth_kzg::UsePrecomp::Yes {
	width: rust_eth_kzg::constants::RECOMMENDED_PRECOMP_WIDTH,
	},
	);

We should be able to just use NO_PRECOMPUTE for c-kzg until we decide to support using c-kzg for PeerDAS (#6107). Otherwise we end up doing precomputation for both libraries, and it would slow down startup time and double the memory usage to cache the precomputation results.

[michaelsproul]

Good catch! The simulator tests are failing, and it's probably because of the startup time regression.

[jimmygchen]

Right, from the link provided, it looks like it still takes 1.69s to precompute, even when precompute is set to 0:
https://github.com/ethereum/c-kzg-4844/pull/545/files

I think it would be ok to bump up genesis delay slightly to make this work.

[cakevm]

@jimmygchen Awesome! I absolutely missed that. Thank you for the insights about the different kzg libs. I adapted it to use always NO_PRECOMPUTE and increased the GENESIS_DELAY. Using now 38 instead of 32 seems to be the sweet spot.