Support Scala 3/Dotty

[Kazark]

Dotty, the Scala 3.x compiler, is beginning to mature. It is supposed to drop around the end of 2020. The community is poised to hop onto it pretty fast, and some libraries are already releasing Scala 3 versions. We'll need support for it in Emacs at some point.

One of the questions is, do we pretend that Scala 2 & 3 are really the same thing? I.e. do we just mash all the features together? Or, do we have some kind of switch by which you can set your Scala version/we detect it from SBT/other? There are syntax changes, but also semantic changes. The website linked above has a good amount of documentation.

I'm happy to help with this effort if needed. I am still relatively new to Emacs but learning all the time.

[hvesalai]

Personally I haven't yet looked much into Dotty.

Is there any abstract or concise presentation of the syntax changes? I think whether we pretend that Scala 2 and 3 are the same thing or not depends on how many and how large the changes are.

[hvesalai]

The current version of scala-mode was implemented against the Scala Language Specification 2.11, which had a nice BNF. Is there a BNF for Dotty?

[hvesalai]

And to answer my own question, yes there is https://dotty.epfl.ch/docs/internals/syntax.html

[Kazark]

Opened #161 as a simple starting point.

[Kazark]

Here's the next thing I'm encountering, after the PR above. Because the enum { ... } syntax for declaring ADTs is new, but the case keyword is not, Emacs is getting confused about where to put the cursor when I start a new line. I'm getting tabbed in too far:

I'm investigating this but it's going to require me to understand the code more deeply than the first PR. Pointers would be appreciated.

[Kazark]

I'm thinking scala-syntax:newlines-disabled-p is the right place to start looking at that problem.

[Kazark]

Wow, I am going to have to understand a lot more before I'm going to know how to change this function. I've got the general idea of what's going there but I'm not close yet on grokking it enough to feel safe changing it.

[hvesalai]

Yeah, you need to understand the logic of the indentation engine. But you are correct, that's the place to check if the case is inside enum block.

[hvesalai]

Here's the things you should understand:

1. syntax-ppss is a function that gives you the parser state at a given point point, which is the current cursor point, unless otherwise specified. The result is stored into a variable called state
2. the (nth 1 state) returns the first element of the parser state, which happens to be the position of the block opening curly brace. This is stored into a variable called parenthesisPos
3. the line (goto-char parenthesisPos) jumps to that point

So after that statement you are in a position where you would have { if you were in an enum block. Also the first non-comment or whitespace token left of that position would be enum. So what you have to do here is to check that it is so. You can do that by first calling scala-syntax:backward-sexp and then checking if the thing at point is enum by calling (looking-at scala-syntax:enum-re), where scala-syntax:enum-re is a new variable you have defined (see scala-syntax:case-re for an example on how to do that)

[Kazark]

@hvesalai Thanks so much for the explanation! That jives with what I was beginning to understand, but fills in some really critical details. In fact, that's probably all I need to get this working. If I get it working, I'll open a separate PR (unless you want me to bundle it in with my other one and just start to make one large Scala 3 PR).

[hvesalai]

Make separate small PRs where possible.

[datalek]

Hi! I'm also interested to have the support to dotty. Probably I'm missing something (sorry, I'm pretty new to emacs) but should we get the support to it pretty easily thanks to emacs-metals and lsp-mode module?

Can I help somehow to get it available?

[hvesalai]

@fommil can you give your educated guess about this topic? I know you know a thing or two about it. :-)

[fommil]

me, when people ask how easy it would be to retrospectively add significant whitespace to an Emacs major mode:

I've written a major mode from scratch with a significant whitespace indenter, and it was a lot of fun to learn all the Emacs APIs and I learnt some lessons too about how I'd do it the next next; but I think I'd need a really strong reason to go down that route, e.g. Scala 3 completely dominating over Scala 2 and me needing to use it for my job. I'm still unsure if Scala 3 will see any serious industry adoption beyond open source hobbyists and EPFL academics.

[hvesalai]

Uh-huh. I didn't know that Dotty has significant whitespace and I'm shocked to hear it has. What the ... were they thinking? I'll just bury my head in the sand and hope it goes away.