refactor: leverage Prettier's AstPath and comment attachment

[jtkiesel]

What changed with this PR:

The printer is totally refactored to leverage Prettier's own AstPath and comment attachment to reduce the amount of work done by our own code (including a reduction of ~1700 lines of code). Also improved type safety a bit, specifically by making CstElement a union type of IToken and all possible CstNode types, and in general aligned formatting a bit closer to Prettier's own formatting for JavaScript/TypeScript (one of those happens to be #720).

Because I wanted to prove that it would be easy to resolve issues with comments after refactoring to use Prettier's built-in comment attachment, I also integrated the implementation for #534, while fixing the comment issues I was having in #716.

Because this PR directly leverages Prettier's printing library, there are a few other issues that I have not yet verified, but may be able to be closed with the merge of this PR, including #487 and #660.

Because I renamed some files from *.js to *.ts, the full diff of the commits in this PR makes it appear as though the *.js files were deleted, but I renamed them in the 1st commit to preserve their git history. Because of this, we should avoid squashing this PR's commits when merging, if we want to keep that history.

Something to consider: I removed the java-parser package's custom comment attachment capability, and instead simply attach all comments to the comments field of the root node, which is what Prettier expects to allow it to move the comments to the appropriate nodes itself. This would be considered a breaking change to the java-parser package, and so should probably result in a major version increment, if we were to keep this change (though I don't really know how we've been treating breaking changes to the parser in the past). If instead, we determine that we wish to continue supporting our own custom comment attachment in the parser itself, then we could bring it back, and simply make it configurable to disable it. I'm open to either approach, but would obviously prefer to gut the code and reduce our maintenance burden if we can.

Example

Options

--experimental-operator-position start

Input

class Example {

  void example() {
    var rotateX =
      (RANGE / rect.height) * refY -
      (RANGE / 2) * getXMultiplication(rect.width);
  }

  void nestedForStatement() {
    for (SomeClass<?> elem : elements) for (SomeClass<
      ?
    > elem : elements) for (SomeClass<?> elem : elements) doSomeThing();
  }
}

Output

class Example {

  void operatorPositionStart() {
    var rotateX =
      (RANGE / rect.height) * refY
      - (RANGE / 2) * getXMultiplication(rect.width);
  }

  void nestedForStatement() {
    for (SomeClass<?> elem : elements)
      for (SomeClass<?> elem : elements)
        for (SomeClass<?> elem : elements) doSomeThing();
  }
}

Relative issues or prs:

Closes #534
Closes #592
Closes #720
Obsoletes #595
Obsoletes #716
Obsoletes #719

[clementdessoude]

Hello @jtkiesel ! Great work also here. It seems that it will certainly lessen the maintaining burden and ease comments handling.

I didn't had the time to fully review the MR this morning (there are quite some changes). I think I should be able to continue (and hopefully finish it) before Tuesday evening.

Let me know if you do not agree with some of the comments, we can discuss them ;)

[jtkiesel]

Thank you for taking the time to review, @clementdessoude! I've resolved/responded to all your comments, happy to continue discussing as well. I'll look forward to the rest of your review! Thanks again for taking the time, I know it's a large PR. 😅

[jtkiesel]

Just made some minor changes to improve the types on both the parser and the printer. Made it clear via the types that only the root node from the parser now contains comments, and added comments to the types on the printer side (where Prettier is now attaching them to the appropriate nodes). Also renamed some types/functions in the printer, because "node" vs "token" was a bit confusing due to Prettier using "node" to describe all elements of the AST, so I went with "non-terminal" and "terminal".

[clementdessoude]

I still have a few files to review, but I made some progress 😆

[clementdessoude]

Hey @jtkiesel ! Finally had some time to finish reviewing the PR ! I have to admit that there are some areas of the code where I didn't go in complete details but I trust the unit test enough to be ok with it 👍

Just have some questions about the path.getNode but otherwise looks good to me !

[clementdessoude]

What does this 14 means ?

[jtkiesel]

getNode() allows you to obtain an ancestor node by index up the stack. So this is obtaining the ancestor 14 parents up, which because of the children field on every Java node means it's actually 7 nodes up the stack. The parent and grandparent values on AstPath use this function internally. Here we're obtaining this specific ancestor in order to check if this TextBlock literal is a descendant of a VariableInitializer or BinaryExpression with an assignment operator.

These are unfortunately a bit fragile in that if the ancestor chain to these particular nodes changes in length, we wouldn't get a compilation error, but we do have unit tests covering these cases, so we would still catch the issue in that case.

[clementdessoude]

what are those path.getNode(14) and path.getNode(8) ?

[jtkiesel]

See #731 (comment) for generic explanation. In this case we're obtaining these specific ancestors (7 and 4 nodes up the stack) to check if this parentheses expression is the descendent of a BinaryExpression in a Guard or ReturnStatement.

[jtkiesel]

@DanielFran Bounty claimed: https://opencollective.com/generator-jhipster/expenses/254462

[murdos]

Hi @jtkiesel,
Do you plan to do a release with the new features that this PR is bringing? I was thinking e.g. to #534
Thanks!

[DanielFran]

@DanielFran Bounty claimed: https://opencollective.com/generator-jhipster/expenses/254462

@jtkiesel approved