Skip to content

Commit ea29761

Browse files
committed
stdlib: Fix O(n^2) algorithm in erl_eval:extended_parse_exprs/1
`erl_eval:extended_parse_exprs/1` exhibited O(n^2) time complexity when passed more and more tokens, with both erts and AtomVM. The reason seems to be related to the `try <expr> of ... <recurse> catch _:_ -> <recurse otherwise> end.` pattern, where `<expr>` would usually fail. By replacing `<expr>` (in this case `unscannable/1`) with some expression that usually succeeds, evaluated complexity is back to O(n) which it should be for this algorithm. The script to evaluate the complexity can be found here: https://gist.github.com/pguyot/1aa53791a819709f147e2ad55aadb279 With OTP 28.1.1: ``` === Results Summary === Size | Tokens | Avg (ms) | Min (ms) | Max (ms) | StdDev | Ratio -----|--------|----------|----------|----------|--------|------- 512 | 1026 | 4 | 3 | 4 | 1 | 0.31/0.5 1024 | 2050 | 13 | 12 | 13 | 0 | 0.27/0.5 2048 | 4098 | 49 | 48 | 49 | 1 | 0.26/0.5 4096 | 8194 | 188 | 187 | 189 | 1 | 0.25/0.5 8192 | 16386 | 739 | 736 | 743 | 2 | - === Complexity Analysis === Expected behavior for doubling size: - O(n): 2x time - O(n^2): 4x time - O(n^3): 8x time Size 512 -> 1024: time ratio 3.25 (between O(n) and O(n^2)) Size 1024 -> 2048: time ratio 3.77 (between O(n) and O(n^2)) Size 2048 -> 4096: time ratio 3.84 (between O(n) and O(n^2)) Size 4096 -> 8192: time ratio 3.93 (between O(n) and O(n^2)) ``` With this change, it is both much faster and it exhibits O(n). ``` === Results Summary === Size | Tokens | Avg (ms) | Min (ms) | Max (ms) | StdDev | Ratio -----|--------|----------|----------|----------|--------|------- 512 | 1026 | 1 | 1 | 1 | 0 | 0.50/0.5 1024 | 2050 | 2 | 1 | 3 | 1 | 0.50/0.5 2048 | 4098 | 4 | 3 | 5 | 0 | 0.50/0.5 4096 | 8194 | 8 | 7 | 10 | 1 | 0.50/0.5 8192 | 16386 | 16 | 14 | 19 | 2 | - === Complexity Analysis === Expected behavior for doubling size: - O(n): 2x time - O(n^2): 4x time - O(n^3): 8x time Size 512 -> 1024: time ratio 2.00 (approximately O(n)) Size 1024 -> 2048: time ratio 2.00 (approximately O(n)) Size 2048 -> 4096: time ratio 2.00 (approximately O(n)) Size 4096 -> 8192: time ratio 2.00 (approximately O(n)) ``` Signed-off-by: Paul Guyot <[email protected]>
1 parent 0d588b9 commit ea29761

File tree

1 file changed

+12
-6
lines changed

1 file changed

+12
-6
lines changed

lib/stdlib/src/erl_eval.erl

Lines changed: 12 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2012,11 +2012,15 @@ tokens_fixup([T|Ts]=Ts0) ->
20122012
end.
20132013

20142014
token_fixup(Ts) ->
2015-
{AnnoL, NewTs, FixupTag} = unscannable(Ts),
2016-
String = lists:append([erl_anno:text(A) || A <- AnnoL]),
2017-
_ = validate_tag(FixupTag, String),
2018-
NewAnno = erl_anno:set_text(fixup_text(FixupTag), hd(AnnoL)),
2019-
{{string, NewAnno, String}, NewTs}.
2015+
case unscannable(Ts) of
2016+
{AnnoL, NewTs, FixupTag} ->
2017+
String = lists:append([erl_anno:text(A) || A <- AnnoL]),
2018+
_ = validate_tag(FixupTag, String),
2019+
NewAnno = erl_anno:set_text(fixup_text(FixupTag), hd(AnnoL)),
2020+
{{string, NewAnno, String}, NewTs};
2021+
false ->
2022+
{hd(Ts), tl(Ts)}
2023+
end.
20202024

20212025
unscannable([{'#', A1}, {var, A2, 'Fun'}, {'<', A3}, {atom, A4, _},
20222026
{'.', A5}, {float, A6, _}, {'>', A7}|Ts]) ->
@@ -2033,7 +2037,9 @@ unscannable([{'#', A1}, {var, A2, 'Port'}, {'<', A3}, {float, A4, _},
20332037
{[A1, A2, A3, A4, A5], Ts, port};
20342038
unscannable([{'#', A1}, {var, A2, 'Ref'}, {'<', A3}, {float, A4, _},
20352039
{'.', A5}, {float, A6, _}, {'>', A7}|Ts]) ->
2036-
{[A1, A2, A3, A4, A5, A6, A7], Ts, reference}.
2040+
{[A1, A2, A3, A4, A5, A6, A7], Ts, reference};
2041+
unscannable(_) ->
2042+
false.
20372043

20382044
expr_fixup({string,A,S}=T) ->
20392045
try string_fixup(A, S, T) of

0 commit comments

Comments
 (0)