Add LazyHTML.Tree.postreduce/3 and LazyHTML.Tree.prereduce/3

[ypconstante]

Phoenix LiveView is calling LazyHTML.Tree.postwalk in many situations in which it just needs to traverse the tree, without modifying its nodes.

This PR adds a LazyHTML.Tree.postreduce/3 function to optimize this kind of scenario in which the tree won't be modified.

Benchmark done using Floki benchmark HTML files.

##### With input big #####
Name                               ips        average  deviation         median         99th %
postreduce                      2.61 K        0.38 ms    ±15.07%        0.37 ms        0.67 ms
prereduce_multiple_calls        2.50 K        0.40 ms    ±14.83%        0.38 ms        0.68 ms
prereduce_single_call           0.40 K        2.51 ms    ±53.18%        3.35 ms        4.52 ms
postwalk                       0.191 K        5.23 ms    ±10.22%        5.12 ms        6.86 ms

Comparison: 
postreduce                      2.61 K
prereduce_multiple_calls        2.50 K - 1.04x slower +0.0169 ms
prereduce_single_call           0.40 K - 6.53x slower +2.12 ms
postwalk                       0.191 K - 13.63x slower +4.85 ms

Memory usage statistics:

Name                        Memory usage
postreduce                           0 B
prereduce_multiple_calls             0 B - 1.00x memory usage +0 B
prereduce_single_call           888352 B - ∞ x memory usage +888352 B
postwalk                       3419856 B - ∞ x memory usage +3419856 B

**All measurements for memory usage were the same**

##### With input medium #####
Name                               ips        average  deviation         median         99th %
postreduce                      7.40 K      135.04 μs    ±21.94%      129.29 μs      248.41 μs
prereduce_multiple_calls        6.93 K      144.29 μs    ±19.99%      138.53 μs      262.11 μs
prereduce_single_call           1.65 K      607.70 μs    ±58.48%      347.00 μs     1321.31 μs
postwalk                        1.05 K      948.35 μs    ±58.40%     1056.99 μs     2407.36 μs

Comparison: 
postreduce                      7.40 K
prereduce_multiple_calls        6.93 K - 1.07x slower +9.25 μs
prereduce_single_call           1.65 K - 4.50x slower +472.65 μs
postwalk                        1.05 K - 7.02x slower +813.30 μs

Memory usage statistics:

Name                        Memory usage
postreduce                           0 B
prereduce_multiple_calls             0 B - 1.00x memory usage +0 B
prereduce_single_call           287392 B - ∞ x memory usage +287392 B
postwalk                       1101864 B - ∞ x memory usage +1101864 B

**All measurements for memory usage were the same**

##### With input small #####
Name                               ips        average  deviation         median         99th %
postreduce                     39.84 K       25.10 μs    ±33.82%       23.98 μs       55.25 μs
prereduce_multiple_calls       37.64 K       26.57 μs    ±31.73%       25.14 μs       58.37 μs
prereduce_single_call          19.42 K       51.49 μs    ±18.62%       49.72 μs      103.44 μs
postwalk                       16.01 K       62.45 μs    ±23.47%       56.06 μs      124.10 μs

Comparison: 
postreduce                     39.84 K
prereduce_multiple_calls       37.64 K - 1.06x slower +1.47 μs
prereduce_single_call          19.42 K - 2.05x slower +26.39 μs
postwalk                       16.01 K - 2.49x slower +37.35 μs

Memory usage statistics:

Name                        Memory usage
postreduce                          0 KB
prereduce_multiple_calls            0 KB - 1.00x memory usage +0 KB
prereduce_single_call           56.98 KB - ∞ x memory usage +56.98 KB
postwalk                       218.60 KB - ∞ x memory usage +218.60 KB

read_file = fn name ->
  __ENV__.file
  |> Path.dirname()
  |> Path.join(name)
  |> File.read!()
  |> LazyHTML.from_document()
  |> LazyHTML.to_tree()
end

inputs = %{
  "big" => read_file.("big.html"),
  "medium" => read_file.("medium.html"),
  "small" => read_file.("small.html")
}

defmodule Bench do
  def prereduce_single_call([[{tag, attrs, children} | rest1] | rest2], acc, fun) do
    acc = fun.({tag, attrs, children}, acc)
    prereduce_single_call([children, rest1 | rest2], acc, fun)
  end

  def prereduce_single_call([[node | rest1] | rest2], acc, fun) do
    acc = fun.(node, acc)
    prereduce_single_call([rest1 | rest2], acc, fun)
  end

  def prereduce_single_call([[] | rest], acc, fun), do: prereduce_single_call(rest, acc, fun)

  def prereduce_single_call([{tag, attrs, children} | rest], acc, fun) do
    acc = fun.({tag, attrs, children}, acc)
    prereduce_single_call([children | rest], acc, fun)
  end

  def prereduce_single_call([node | rest], acc, fun) do
    acc = fun.(node, acc)
    prereduce_single_call(rest, acc, fun)
  end

  def prereduce_single_call([], acc, _fun), do: acc

  def prereduce_multiple_calls([{tag, attrs, children} | rest], acc, fun) do
    acc = fun.({tag, attrs, children}, acc)
    acc = prereduce_multiple_calls(children, acc, fun)
    prereduce_multiple_calls(rest, acc, fun)
  end

  def prereduce_multiple_calls([node | rest], acc, fun) do
    acc = fun.(node, acc)
    prereduce_multiple_calls(rest, acc, fun)
  end

  def prereduce_multiple_calls([], acc, _fun), do: acc

  def postreduce([], acc, _fun), do: acc

  def postreduce([node | rest], acc, fun) do
    acc = postreduce(node, acc, fun)
    postreduce(rest, acc, fun)
  end

  def postreduce({tag, attrs, children}, acc, fun) do
    acc = postreduce(children, acc, fun)
    fun.({tag, attrs, children}, acc)
  end

  def postreduce(node, acc, fun) do
    fun.(node, acc)
  end

  def postwalk([], acc, _fun), do: {[], acc}

  def postwalk([node | rest], acc, fun) do
    case postwalk(node, acc, fun) do
      {nodes, acc} when is_list(nodes) ->
        {rest, acc} = postwalk(rest, acc, fun)
        {nodes ++ rest, acc}

      {node, acc} ->
        {rest, acc} = postwalk(rest, acc, fun)
        {[node | rest], acc}
    end
  end

  def postwalk({tag, attrs, children}, acc, fun) do
    {children, acc} = postwalk(children, acc, fun)
    fun.({tag, attrs, children}, acc)
  end

  def postwalk(node, acc, fun) do
    fun.(node, acc)
  end
end

Benchee.run(
  %{
    "prereduce_single_call" => fn tree ->
      Bench.prereduce_single_call(tree, 0, fn _node, acc -> acc + 1 end)
    end,
    "prereduce_multiple_calls" => fn tree ->
      Bench.prereduce_multiple_calls(tree, 0, fn _node, acc -> acc + 1 end)
    end,
    "postreduce" => fn tree ->
      Bench.postreduce(tree, 0, fn _node, acc -> acc + 1 end)
    end,
    "postwalk" => fn tree ->
      {_tree, acc} = Bench.postwalk(tree, 0, fn node, acc -> {node, acc + 1} end)
      acc
    end
  },
  inputs: inputs,
  pre_check: :all_same,
  time: 10,
  memory_time: 2
)

[josevalim]

In Elixir, Macro.traverse is both prewalk and postwalk combined. So I think we should pick a different name here... maybe postwalk_reduce?

[jonatanklosko]

postvisit, postscan, since they better indicate read-only?

[josevalim]

postscan is also fine by me. Or postreduce.

[jonatanklosko]

I like postreduce!

[ypconstante]

8e33368
renamed to postreduce

[josevalim]

No need for the do_traverse btw. You can implement them all directly as clauses of traverse.

[ypconstante]

8e33368

[josevalim]

Btw, I applied this patch to LiveView, and it seems they need a prereduce or prescan. All other uses of postwalk are indeed traversing the tree: phoenixframework/phoenix_live_view#3937

[jonatanklosko]

Thank you!

[josevalim]

Just to be clear, this won't work for LV. We need a prereduce there.

[jonatanklosko]

@ypconstante let's add both then :D

[ypconstante]

Done, also updated the benchmark to show the difference between adding children to the list of items to reduce vs just calling the reduce multiple times. Previous benchmark showed pre order traversal as being slower, but it was just the algorithm difference, performance is basically the same for both approaches.