Allow semantic rule execution despite compilation errors thanks to Scala 3.5+ best-effort compilation

[tribbloid]

Some options may cause scalafix to generate code that no longer compiles, e.g. In this document:

https://scalacenter.github.io/scalafix/docs/rules/ExplicitResultTypes.html

It is noted that:

rewriteStructuralTypesToNamedSubclass: .... Beware that this option may produce code that no longer compiles ...

As another example, ExplicitResultTypes may also have a tendency to generate type annotations that are widened from singleton types, dependent types and duck types, which also breaks compilation.

A simple solution would be to modify code speculatively, which uses a trial-and-error search algorithm to backtrack to part of the modification that breaks compilation:

• immediately after modifying, rebuild part of the AST and semantic DB to ensure that the outcome is equivalent to the old version.
• if this verification fails, revert modification and revert to previous AST or any kind of IR
• if the verification also fails even after reverting, find all previous changes made to the code that has affected type resolving of the current scope, and try reverting and and re-verify in inversed chronological order.

This could apply options as much as it could, even if the option is highly experimental and volatile

[bjaglin]

Such a trial-and-error strategy is used by scala3-migrate, an sbt plugin applying Scalafix patches independently using the Scalafix evaluation API so that it can verify that compilation succeeds from one iteration to another.

If we were to take a similar approach in Scalafix, I'd rather see this implemented in the same fashion within the build tool (sbt-scalafix), considering that (re)compiling & generating SemanticDB from Scalafix itself turned out to be challenging and was removed.

In any case, I wouldn't say that the implementation is simple.

[tribbloid]

Hmmm, good point, most people are already using it within the sbt plugin. Maybe all it takes is just a interactive inter-process interface that can zoom into a granular part of the semanticDB?

I'm also curious about the granularity of scala3-migrate, ideally it could also convert as much as it could and ask the user for minimal manual override. So this new mode could benefit both

[tribbloid]

OK now metaLS has been mostly stable for a while, maybe we can enable this for Scala 3 with fail-tolerant semanticDB indexing.

Scala 2 can wait indefinitely

[bjaglin]

Indeed, thanks for the bump, we should now be able to rely on all the heavy lifting available in the compiler thanks to https://dotty.epfl.ch/docs/internals/best-effort-compilation.html. I will rename the ticket accordingly.

I am just discovering the feature, but it might be as "simple" as

1. Injecting -Ybest-effort when sbt-scalafix prepares semanticdb files on scala 3.5+ projects (we already mess with scalacOptions so that should be easy) and proceed with scalafix invocation despite compilation error (probably under a flag? I am afraid we will have many useless scalafix invocations throwing exotic exceptions when semanticdb information is poor and/or the rule relies on class files)
   • That should allow semantic rules that only depend on semanticdb information to work (when the semanticdb info is good enough obviously)
   • Removing dependence to class files in the semantic APIs to leverage TASTy (or BETASTy in that case) would be nice independently of this particular ticket, but it's a much bigger effort...
2. Run the PC with -Ywith-best-effort-tasty whenever -Ybest-effort is found in the scalacOptions provided by the scalafix invoker
   • That should allow ExplicitResultTypes to run, provided (1) or similar is in place for the build tool invoking scalafix

@tgodzik did I get that right?

[bjaglin]

Obviously I wrote (1) above with sbt-scalafix in mind, but I wonder if it doesn't work out of the box when scalafix is ran through metals with -Dmetals.enable-best-effort=true (I would have to look at how compilation is triggerd ahead of a scalafix run).

That won't help with ExplicitResultTypes since (2) is still required, but it would still make all other semantic rules (most importantly OrganizeImports) work despite some compilations errors.

[tribbloid]

most people only run scalafix once or twice, so time constraint is not important at all. Many people would even delegate it to openAI o3 to run for a few hours without human intervention. So invoking the compiler many times to collect information and figure out a solution is totally acceptable

May I suggest an alternative: why is semantic APIs bad? It is part of the compiler that can be easily constructed in runtime. This version obviously lose a lot of functionality due to broken files, but it still can load well-formed types and give you the right hint. After the changes are made, you can easily get the diff between compiler errors before/after the fix

[bjaglin]

most people only run scalafix once or twice, so time constraint is not important at all. Many people would even delegate it to openAI o3 to run for a few hours without human intervention. So invoking the compiler many times to collect information and figure out a solution is totally acceptable

Agreed, but I wasn't implying that it was a bad thing, and the approach I described above does not need to call the compiler several times anyway.

May I suggest an alternative: why is semantic APIs bad? It is part of the compiler that can be easily constructed in runtime. This version obviously lose a lot of functionality due to broken files, but it still can load well-formed types and give you the right hint. After the changes are made, you can easily get the diff between compiler errors before/after the fix

I am afraid I am not following. What alternative are you suggesting? And alternative to what?

[tgodzik]

Obviously I wrote (1) above with sbt-scalafix in mind, but I wonder if it doesn't work out of the box when scalafix is ran through metals with -Dmetals.enable-best-effort=true (I would have to look at how compilation is triggerd ahead of a scalafix run).

I think it should work, though I need to experiment a bit. The biggest issue is that we made it mostly work for Bloop for now as we are still ironing things out. We also create additional separate directory not to confuse those betasty artifacts with standard ones. I am not sure if we can make it without changing some sbt internals or at least changing more things. We will need to take a look into that at some point.

[tribbloid]

@bjaglin I'm talking about:

2. Removing dependence to class files in the semantic APIs to leverage TASTy (or BETASTy in that case) would be nice independently of this particular ticket, but it's a much bigger effort...

we don't need third-party library that only rely on TASTy (e.g. https://github.com/scalacenter/tasty-query) for type inspection and inference, TASTy as a serialiser is good enough: Scala 3 runtime compiler can use the type info directly from compile-time compiler given proper serialisation

[bjaglin]

I see, thanks for the clarification!

The Scalafix API was not built against the compiler, but rather the (portable) output of it, namely classfiles and semanticdb. ExplicitResultTypes is an exception, as it requires information about scopes that can't easily be captured statically, so it starts its own presentation compiler.

In Scala 3 world, this was one of the selling point of Scalafix, resulting in portability and relative accessibility thanks to the Scalameta trees.

Now that Scala 3 got a stable presentation compiler, we could definitely argue that Scalafix should expose it. However, for Scalafix to remain efficient on large codebases, I believe we still need the existing API that does not actively depend on the compiler, and I was just making the point that we would benefit from NOT relying on class files but rather TASTy/BETASTy for that API.

Maybe what you imply is that we could also remove the dependency to semanticdb along the way, but I don't see a high ROI in doing that, especially because it is now a first-class citizen in a compiler (still emitted with best effort compilation!), and is actively used by the Scala tooling ecosystem - so not going anywhere.

[tribbloid]

exactly, so no argument here. I'm always against building 2 systems and fixing everything twice for the same purpose