[RFC] fixpoint iteration support

[carljm]

This PR removes the existing unwind-based cycle fallback support (a plus for WASM compatibility), and replaces it with support for fixpoint iteration of cycles.

To opt in to fixpoint iteration, provide two additional arguments to salsa::tracked on the definition of a tracked function: cycle_initial and cycle_fn. The former is a function which should provide a provisional starting value for fixpoint iteration on this query, and the latter is a function which has the opportunity, after each iteration that failed to converge, to decide whether to continue iterating or fallback to some fixed value. See the added test in cycle_fixpoint.rs for details.

Usability points that should be covered in the documentation:

• With the old cycle fallback, it was sufficient to avoid panic for at least one query in a cycle to define a cycle fallback. With fixpoint iteration, to avoid cycle panics you must define cycle_fn and cycle_initial on every query that might end up as the "head" of a cycle (that is, queried for its value while it is already executing.)
• It is entirely possible to define cycle_fn and cycle_initial so as to cause iteration to diverge and never terminate; it's up to the user to avoid this. Techniques to avoid this include a) ensuring that cycles will converge, by defining the initial value and the queries themselves monotonically (for example, in a type-inference scenario, the initial value is the bottom, or empty, type, and types will only widen, never narrow, as the cycle iterates -- thus the cycle must eventually converge to the top type, if nowhere else), and/or b) with a larger hammer, by ensuring that cycle_fn respects the iteration count it is given, and always halts iteration with a fallback value if the count reaches some "too large" value.
• It's also entirely possible to define cycle_fn and cycle_initial such that memoized results can vary depending only on the order in which queries occur. Avoid this by minimizing the number of tracked functions that support fixpoint iteration and ensuring initial values and fallback values are consistent among tracked functions that may occur in a cycle together.
• You can call Salsa queries from within your cycle_fn and cycle_initial queries, but if the query you call re-enters the same cycle, it could lead to unexpected behavior. Take care what queries you call inside cycle recovery functions.

This is an RFC pull request to get initial reviewer feedback on the design and implementation. Remaining TODO items:

• add tests for more complex cycles:
  • nested (multiple head) cycles
  • cycles with multiple paths back to the same cycle head
• add tests for cross-thread cycles
• add tests that call queries in cycle recovery functions
• test in red-knot and validate it works there
• performance improvements
  • lazy creation of initial-value memo?
• documentation

[netlify]

✅ Deploy Preview for salsa-rs canceled.

Name	Link
🔨 Latest commit	7678d13
🔍 Latest deploy log	https://app.netlify.com/sites/salsa-rs/deploys/67aef3bef6a153000825e8d1

[codspeed-hq]

CodSpeed Performance Report

Merging #603 will not alter performance

_{Comparing carljm:fixpoint (7678d13) with master (ea1b2bd)}

Summary

✅ 9 untouched benchmarks
🆕 1 new benchmarks

Benchmarks breakdown

	Benchmark	BASE	HEAD	Change
🆕	converge_diverge	N/A	190.2 µs	N/A

[nikomatsakis]

This is very cool! (Admittedly, I say this pre-review.)

[MichaReiser]

This looks great. I left a few comments where I struggled understanding the implementation or had smaller suggestions.

[carljm]

In writing more comprehensive tests for this, I realized that it needs some changes to correctly handle multi-revision scenarios; taking it to Draft mode until I get that fixed.

[carljm]

Ok, multiple-revision cases are now fixed, and we now populate the initial provisional value only lazily, in case a cycle is actually encountered, which should reduce the number of memos created by quite a lot.

Also added a bunch of tests, including multiple-revision cases and one test involving durability. Still need to add cross-thread cycle tests.

[MichaReiser]

The lazy creation of the initial value is a neat improvement. Nice for taking the time to work on it !

[MichaReiser]

The benchmarks show a 4-5% regression. It seems that we're now resizing some hash maps more often. Are we reporting more tracked reads than before? Could you take a look what's causing it?

[carljm]

Initial experiments using this in the red-knot type checker are promising: astral-sh/ruff#14029

Not yet using it for loopy control flow in that PR, but there are cycles in the core type definitions of Python builtins and standard library, which we previously had a hacky fallback in place for using Salsa's previous cycle fallback support. Moving over to fixpoint iteration just worked, and fixed the type of a builtin impacted by the cycle.

On the downside, it is a performance regression. Need to do more work there.

[carljm]

Using an option box for cycle heads doesn't seem to have improved the overall picture in codspeed; slightly worse if anything. It did of course reduce memo size. Will look into perf more tomorrow.

[MichaReiser]

Using an option box for cycle heads doesn't seem to have improved the overall picture in codspeed; slightly worse if anything. It did of course reduce memo size. Will look into perf more tomorrow.

That's surprising, considering that no Benchmark uses cycle handling

[carljm]

I did add one new benchmark that uses cycle handling. But that of course isn't showing any increase or decrease since it's new in this PR.

[carljm]

I realized one mistake I made that could be increasing cost for code that doesn't use fixpoint; my CycleHeads::extend implementation always allocates a hashset even if we are extending with nothing. Will fix this tomorrow and see how much it helps.

[MichaReiser]

Yes. We could also use None to signify "no cycle heads" vs dereferencing the Box to see if it is empty.

I also noticed that we now have some Option<CycleHeads> usages which boils down to a Option<Option<Box<FxHashSet>>>: Do we need the double-option or would one level of Option (just knowing whether it is empty or not) be sufficient?

[carljm]

We could also use None to signify "no cycle heads" vs dereferencing the Box to see if it is empty.

I think that's what I'm already doing? Where do you see an unnecessary dereference of the Box?

I also noticed that we now have some Option<CycleHeads> usages which boils down to a Option<Option<Box<FxHashSet>>>: Do we need the double-option

Yeah I was going to ask about this / play with it more. This is in a case where I want to use a reference to a memo's CycleHeads so I don't have to clone it. But if the memo is verified_final I need to pass something to signify "no cycle heads", and I can't pass a reference to the memo's actual cycle heads since it will not be empty (we could maybe empty it when we set verified_final but that would require putting it behind a lock.) So I couldn't immediately find a better way than Option<&CycleHeads>, but open to suggestions.

I could unwrap the CycleHeads and pass an Option<&FxHashSet<DatabaseKeyIndex>> instead.

[carljm]

It looks like fixing the extend issue brought the benchmarks roughly back in line with where they were before, ranging from 0 (or slight positive on one benchmark) to -6%, with most benchmarks in the -3% to -6% range.

[carljm]

Ok benchmarks are now slightly improved (-2% to -5% instead of -3% to -6%). Would still like to improve more than that if we can.