Design `v2` "non-Eth" (chain, F3, state) RPC APIs that are F3-aware

[BigLep]

Done Criteria

There is a public document that lists proposed:

1. APIs being added
2. APIs being removed
3. APIs being modified
   as a result of F3. We should be able to share this with a partner who will use them and give feedback on them

The groups in scope are:

• Chain
• F3
• State

Eth is not in scope because we already know what to do from an implementation regard (and that is happening in #12988)

Why Important

1. Ensure we implement the right thing
2. Enable integraters to plan for how they'll adapt with the advent of F3

Notes

1. Exchanges (e.g., Coinbase through Zondax) are a target audience of this work.
2. There is existing work in this area to draw from: https://www.notion.so/filecoindev/WIP-API-updates-w-F3-106dc41950c1801789bef5eca507efa5 . That said it isn't currently crisp on our current proposal. We don't want readers to have to pick through to try and guess what our proposal is.
3. A key challenge to solve is a clean way for the user to specify the tipset they want to use. We have to introduce a new API paradigm to users but we should be careful not to break too hard so everyone has to rewrite stuff. We could lean on the Eth paradigm to make bridging easier which is why proposals to date have been a mash-up of Filecoin and Eth tipset selectors.
4. This design document doesn't block implementation work. There is code to write that will be independent of the exact APIs we settle on. For example, the tipset selection needs to be plumbing through everywhere we execute the backends of the APIs so we don’t have to have two versions of everything. v2 and v1 (and therefore v0) should share code paths, but the “which tipset” argument is now different. Refactoring to promote reuse is going to be tricky given the current internal architecture and the way we use the DI (uber/ux) tooling to assemble Lotus.
5. This effort is not getting bundled in the "Common Node" effort. v2 APIs established here can be consided for the "Common Node" effort in the future, but for now we want to be able to move fast and learn from user interaction on the ideal APIs to have. Once they are stabalized, they can be considered for "Common Node" inclusion.
6. We should set the expectation is that the v2 APIs are experimental and opt in at this point and will likely change based on feedback.

[masih]

"Common Node" RPC API, and Lotus V2 APIs are not necessarily the same. I think it might be worth discussing how much we want the latter to depend on the former. The key trade-off to point out here is time to completion. Common API is a slow train, and needs a lot of cross ecosystem coordination.

I would like to distill priorities here, please.

[rvagg]

Agreed, we need to narrow down some essentials to kick off with. Aside from the Eth APIs which are straightforward we need a core suite of APIs that are minimally useful so we can kick this off without going whole hog. Plus we also need to give ourselves lattitude to introduce changes to these APIs so we're not entirely bound by v1 patterns because they no longer quite fit.

Some personal thoughts, which need to be validated:

1. Eth APIs are fairly straightforward for v2, we just move "finalized" and "safe" to f3 finality instead of 900 and (the arbitrary) 30. There is "latest" and "pending" to consider, I think I'd prefer to leave them alone but perhaps those could be shifted in some way too.
2. Chain accessors - get-head and get-tipset operations, these are the most interesting for initial API work which is why I've been experimenting with them in my PRs. I think we need a replacement of ChainHead that lets you specify what you what for "head", going with "finalized" and "latest" to match Eth might be good here but I also thought that while we're taking flexible parameters then we should just make a one-stop-shop for get-tipset here. In feat(api): v2 API integrating with F3 #12719 I experimented with an TipSetSelector that let you do this - either a string specifier, a specific tipset. In https://www.notion.so/filecoindev/WIP-API-updates-w-F3-106dc41950c1801789bef5eca507efa5?pvs=4#152dc41950c1804cacaae1d5e79df324 I proposed going a step further by letting this parameter specify: a string or an epoch number or a specific tipset, then an optional "anchor" (like the current APIs that let you find a TipSet starting from another TipSet and working back from that chain) and a "previous" boolean so we could do ByHeight and AfterHeight in one API for null rounds.
3. Chain utilities - messages etc. probably less important
4. State utilities - pretty much anything starting with State* and I think once we have nailed down something like a TipSetSelector we just apply that to all of these that we consider worth bringing into a /v2 world, and this is where "Common Node API" comes in because we currently have too many state accessors and need to narrow them down. We can work on polishing the direct actor state access pattern like was used in feat: expose market ProviderSectors access on state-types abstraction #12978 to avoid the need to have specific functions that fetch the specific thing you need. But this is tricky because actor state accessors are easy in Go but they're going to be really hard in any other language that doesn't have all the fancy wrappers and go-state-types that Go does.

I think once you can get access to the tipset that is "finalized" in an easy way then you can use that to pass down to any API that takes a tipset (like the State* calls), so those become lower priority. So working out a nice, clean and ergonomic way to expose that to both Go and JSON-RPC users should be the priority here.

I'm also of the mind that Filecoin is better served by leaning into existing API patterns, we're past the ability to dictate terms so if we can borrow from other ecosystems where there's developer affordance then we may as well, Ethereum being the obvious one that we're already borrowing from so we may as well use it as inspiration so developers don't have to make such a big leap to grok our APIs.

[rvagg]

The other thing to keep in mind here is that we should probably ship this with an opt-in flag initially and mark it all as "experimental" so we give ourselves heaps of room to break stuff if it doesn't work. So let's just ship stuff and iterate. The hardest part of this I see is all the refactoring we need to do to enable code reuse between v0, v1 and v2.

[masih]

going a step further

What is the value proposition in combining all calls the return a tipset into one? Why is it better than GetTipsetByKey, GetTipsetByHeight, GetLatestTipset, GetFinalizedTipset, etc.? I am curious to understand the learnings from the existing v1 apis that follow this pattern please.