Skip to content

Handling of await and yield not context-dependent #303

Open
@Maxdamantus

Description

@Maxdamantus

The following piece of code is valid but it is parsed incorrectly:

async function foo() {
    await /REGEX/g;
    yield /DIVISION/g;
}

function* bar() {
    await /DIVISION/g;
    yield /REGEX/g;
}

async function* baz() {
    await /REGEX/g;
    yield /REGEX/g;
}

function div3(await, yield, g) {
    return await/yield/g;
}

The output of tree-sitter parse is the following:

(program [0, 0] - [18, 0]
  (function_declaration [0, 0] - [3, 1]
    name: (identifier [0, 15] - [0, 18])
    parameters: (formal_parameters [0, 18] - [0, 20])
    body: (statement_block [0, 21] - [3, 1]
      (expression_statement [1, 4] - [1, 19]
        (await_expression [1, 4] - [1, 18]
          (regex [1, 10] - [1, 18]
            pattern: (regex_pattern [1, 11] - [1, 16])
            flags: (regex_flags [1, 17] - [1, 18]))))
      (expression_statement [2, 4] - [2, 22]
        (yield_expression [2, 4] - [2, 21]
          (regex [2, 10] - [2, 21]
            pattern: (regex_pattern [2, 11] - [2, 19])
            flags: (regex_flags [2, 20] - [2, 21]))))))
  (generator_function_declaration [5, 0] - [8, 1]
    name: (identifier [5, 10] - [5, 13])
    parameters: (formal_parameters [5, 13] - [5, 15])
    body: (statement_block [5, 16] - [8, 1]
      (expression_statement [6, 4] - [6, 22]
        (await_expression [6, 4] - [6, 21]
          (regex [6, 10] - [6, 21]
            pattern: (regex_pattern [6, 11] - [6, 19])
            flags: (regex_flags [6, 20] - [6, 21]))))
      (expression_statement [7, 4] - [7, 19]
        (yield_expression [7, 4] - [7, 18]
          (regex [7, 10] - [7, 18]
            pattern: (regex_pattern [7, 11] - [7, 16])
            flags: (regex_flags [7, 17] - [7, 18]))))))
  (generator_function_declaration [10, 0] - [13, 1]
    name: (identifier [10, 16] - [10, 19])
    parameters: (formal_parameters [10, 19] - [10, 21])
    body: (statement_block [10, 22] - [13, 1]
      (expression_statement [11, 4] - [11, 19]
        (await_expression [11, 4] - [11, 18]
          (regex [11, 10] - [11, 18]
            pattern: (regex_pattern [11, 11] - [11, 16])
            flags: (regex_flags [11, 17] - [11, 18]))))
      (expression_statement [12, 4] - [12, 19]
        (yield_expression [12, 4] - [12, 18]
          (regex [12, 10] - [12, 18]
            pattern: (regex_pattern [12, 11] - [12, 16])
            flags: (regex_flags [12, 17] - [12, 18]))))))
  (function_declaration [15, 0] - [17, 1]
    name: (identifier [15, 9] - [15, 13])
    parameters: (formal_parameters [15, 13] - [15, 30]
      (ERROR [15, 14] - [15, 29]
        (await_expression [15, 14] - [15, 29]
          (ERROR [15, 19] - [15, 20])
          (yield_expression [15, 21] - [15, 29]
            (ERROR [15, 26] - [15, 27])
            (identifier [15, 28] - [15, 29])))))
    body: (statement_block [15, 31] - [17, 1]
      (return_statement [16, 4] - [16, 25]
        (await_expression [16, 11] - [16, 24]
          (regex [16, 16] - [16, 24]
            pattern: (regex_pattern [16, 17] - [16, 22])
            flags: (regex_flags [16, 23] - [16, 24])))))))
test.js 0 ms    (ERROR [15, 14] - [15, 29])

foo and bar are incorrectly parsed such that occurrences of /DIVISION/g are interpreted as regex literals instead of as sequences of division operators. Handling this correctly depends on special treatment of await within async functions (since it's not a reserved word in other contexts), and special treatment of yield within generator functions (since it's otherwise only a reserved word in "use strict" contexts).

div3 incorrectly fails to parse. The body should similarly contain two division operations.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions