Consider: A facility for tracking Integration

[gavofyork]

At present, Accumulation is the highest level service compute phase which is explicitly acknowledged by JAM. However for Cross-service Work-Item Entanglement, it is important that any changes to the entangled services are synchronised and premised on expected changes being enacted in all entangled services.

If Accumulation is infallible and conditionless (i.e. will always make the same, expected, change regardless of what other WPs might have been Accumulated before), then it is enough to place all entangled Work Items in a single WP.

However, most Accumulation code (not least the CoreChains and CorePlay services) will not be conditionless and the repercussions of entanglement must only be enacted once all services have ascertained that all other entangled services are each committed to enact them.

An on-chain facility to pub/sub for the integration of Work Items would help achieve this without excessive use of transfer and peek.

[sourabhniyogi]

From your comment on #129 "a WP might imply a block is valid and finalisable yet when it gets to Accumulate, some other WP has since been accumulated which has finalized a block which is mutually exclusive with that of the first WP" motivating "Integration" -- where I believe you envision the CoreChains JAM service to not commit finalized blocks in accumulate until integration happens, right?

In my idea of a parallel ETH L1/L2 validation JAM service for ORUs, following @rphmeier's suggestion to aggressively validate ORUs state transitions on all forks (both L1 and L2) in refine, I am envisioning an integration-less accumulate writing validated blocks to JAM State whether these validated ORU blocks are finalized or not. Then ANOTHER "finalizer" service can bring in work items from
(a) ETH L1's finality checkpoints (every 13 mins)
and
(b) ZK proofs of finality from prover networks optimized for ORUs that provides real finality beyond (a):

• Succinct's SP1
• Risc Zeros Kailua
• probably others ...

and basically does the "integration" work:

• establishing chainhood
• doing the bookkeeping for the finalized chain only
• forgetting blocks only on non-canonical chains

The true finality of ORUs is not based on (a) but really a social time period (of 7 days). But with (b), it can be reduced to under an hour ($50MM/yr for 50 ETH ORUs from OP Succinct, plausibly). JAM's beefy root for finalization would not be based on (a) alone (since its not really finalized) but (b). We would not delay writes to JAM State for validating blocks just because (b) is not there though ... because doing it based on (a) is more practical for rollups without the hefty ZK prover budget. I don't see the need to block the results of WP being written to JAM State waiting for (a) [13 mins] or (b) [1 hour] by requiring integration.

Are non-Polkadot rollups basically not needing the integration you envisioned because we want to see the validation on all possible forks for ORUs? What did I miss?

[gavofyork]

Integration is always needed in order to have secure inter-service interactions. Integration is used to ensure that any predicates which cause a state transition to become accumulated into a service's state are themselves valid. If you don't validate them prior to integrating them into your (final) state and they turn out to be unfounded then you're screwed.

In your language, do ensure that you do not conflate finality (essentially a fork-choice) with correctness (validating the transition). I'm not familiar with the details of these Eth L2s, but ZK is often used in L2s to prove correctness (potentially helpful for scaling) rather than finality (primarily for bridging).

Finally, validating L2 transitions alone wouldn't need integration since they're necessarily going to be independent of any other services with which they might happen to be co-scheduled.