Performance issue on relatively small file

[Porges]

Describe the bug

Parsing the file located here with yaml.parse(str, {merge:true}) takes a very long time. Could there be something non-linear in the length of the file?

Currently it's under 500k:

$ wc bibliography.yaml
 18610  48360 489877 bibliography.yaml

To Reproduce
Steps to reproduce the behaviour.

const yaml = require('yaml');
const fs = require('fs');

fs.readFile("bibliography.yaml", 'utf8', (_, data) => {
    console.time('yaml-parse');
    const loaded = yaml.parse(data, {merge: true});
    console.timeEnd('yaml-parse');
});

Output:

yaml-parse: 10.501s

Expected behaviour
A clear and concise description of what you expected to happen.

By comparison, doing the same thing with js-yaml takes only:

const yaml = require('js-yaml');
const fs = require('fs');

fs.readFile("bibliography.yaml", 'utf8', (_, data) => {
    console.time('yaml-load');
    const loaded = yaml.load(data);
    console.timeEnd('yaml-load');
});

Output:

yaml-load: 125.814ms

Versions (please complete the following information):

• Environment: Node v18.17.1
• yaml: 2.4.1

Additional context
Add any other context about the problem here.

[eemeli]

Interesting. Thank you for sharing the file that's demonstrating this! Looks like the slowdown is during the document -> JS conversion. Probably due to the alias resolution?

I'll see when I can find time to dig into this properly; might take a while.

[rejetto]

loading a 3.7MB file of a simple single-level dictionary was taking 77 seconds on my machine fast mac (m1 max).
i unexpectedly solved it by reading the file line by line, and parsing each line. Now it's 3 seconds, 25x difference. 👀
of course this is not a solution unless you are in my same situation, but i wanted to share my experience, as it can shed some light on the problem.

[eemeli]

@rejetto Do you mean parsing each line as a separate YAML document?

[rejetto]

yes, invoking parse on the single line.
It's weird, and i don't have time to dig it more, but since it's hard to expect i thought good to share this information.
my dictionary is made of a file path for the key and a number for the value.

[PolyMeilex]

As far as I can tell, this is simply the cost of using generators everywhere.

js-yaml can fully parse my files faster than yaml can lex them, not because the code is bad but just because V8 is spending all its time popping generator scopes in and out, and is also not able to do any meaningful optimalizations.

Sadly this is a dealbreaker, my startup time went from 380ms to 10s after porting from js-yaml to yaml

[eemeli]

I'm going to close this, thanks to the improvements by @PolyMeilex in #612 speeding up the parse for bibliography.yaml by a factor of about 5x. It's still slower than it ought to be, but less horrible.

The excessive use of generators during parsing is probably the next biggest source of slowness; fixing that is a breaking change, though, and so something to do as a part of #601.