Skip to content

Use string slice in ascii_string and binary_string - #155

Merged
josevalim merged 1 commit into
dashbitco:masterfrom
dkuku:dk/string-slice
Aug 11, 2026
Merged

Use string slice in ascii_string and binary_string#155
josevalim merged 1 commit into
dashbitco:masterfrom
dkuku:dk/string-slice

Conversation

@dkuku

@dkuku dkuku commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

It probably requires a mention that it can reference keep a reference to the parsed binary - currently bytes combinator does the same.

Example parser:

  defmodule MyParser do
    import NimbleParsec

    defparsec :ident, ascii_string([?a..?z], min: 1), debug: true
  end

expands to:

Before (master) — 6 functions, 8 clauses:
  defp ident__0(rest, acc, stack, context, line, offset) do
    ident__1(rest, [], [acc | stack], context, line, offset)
  end

  defp ident__1(<<x0, rest::binary>>, acc, stack, context, comb__line, comb__offset) when x0 >= 97 and x0 <= 122 do
    ident__2(rest, [<<x0::integer>>] ++ acc, stack, context, comb__line, comb__offset + 1)
  end

  defp ident__1(rest, _acc, _stack, context, line, offset) do
    {:error, "expected ASCII character in the range \"a\" to \"z\"", rest, context, line, offset}
  end

  defp ident__2(<<x0, rest::binary>>, acc, stack, context, comb__line, comb__offset) when x0 >= 97 and x0 <= 122 do
    ident__4(rest, [x0] ++ acc, stack, context, comb__line, comb__offset + 1)
  end

  defp ident__2(rest, acc, stack, context, line, offset) do
    ident__3(rest, acc, stack, context, line, offset)
  end

  defp ident__4(rest, acc, stack, context, line, offset) do
    ident__2(rest, acc, stack, context, line, offset)
  end

  defp ident__3(rest, user_acc, [acc | stack], context, line, offset) do
    _ = user_acc
    ident__5(rest, [List.to_string(:lists.reverse(user_acc))] ++ acc, stack, context, line, offset)
  end

  defp ident__5(rest, acc, _stack, context, line, offset) do
    {:ok, acc, rest, context, line, offset}
  end
  
After (this branch) — 3 functions, 4 clauses:
  defp ident__0(<<x0, rest::binary>> = orig, acc, stack, context, line, offset) when x0 >= 97 and x0 <= 122 do
    ident__1(rest, orig, acc, stack, context, line, offset)
  end

  defp ident__0(rest, _acc, _stack, context, line, offset) do
    {:error, "expected ASCII character in the range \"a\" to \"z\"", rest, context, line, offset}
  end

  defp ident__1(<<x0, rest::binary>>, orig, acc, stack, context, line, offset) when x0 >= 97 and x0 <= 122 do
    ident__1(rest, orig, acc, stack, context, line, offset)
  end

  defp ident__1(rest, orig, acc, stack, context, line, offset) do
    len = byte_size(orig) - byte_size(rest)
    ident__2(rest, [binary_part(orig, 0, len) | acc], stack, context, line, offset + len)
  end

  defp ident__2(rest, acc, _stack, context, line, offset) do
    {:ok, acc, rest, context, line, offset}
  end

Benchmark result (I will include the benchmark code in a stacked pr that can be closed):


case              old(ns)  new(ns)  speedup  control   red.  red.diff
ascii len=1          60.7     53.6   1.132x   0.995x     13        -7
ascii len=4         105.4     62.0   1.699x   1.000x     17        -8
ascii len=8         122.1     67.7   1.803x   0.997x     21        -8
ascii len=32        393.6     97.5   4.038x   0.991x     48       -11
ascii len=128      1017.4    198.1   5.135x   1.018x    204       -71
ascii len=1024     6879.2   1177.2   5.844x   1.008x   1998      -969
opt len=1            67.3     44.4   1.516x   0.993x     13        -7
opt len=128        1007.2    258.2   3.902x   1.006x    224       -91
utf8 len=8          184.1     75.5   2.439x   0.970x     21        -8
utf8 len=128       1688.9    228.4   7.395x   1.007x    229       -96
bounded 2..8        114.5     64.0   1.790x   0.996x     19        -9
repeat 200x6ch    23323.4  11937.6   1.954x   1.004x   4170     -1953
repeat 200x40ch   55509.4  19811.8   2.802x   0.999x  11896     -2824
error (min fail)     36.0     34.5   1.042x   1.000x      5         0

speedup   old/new wall clock, min-of-21. Only trust it if it clears `control`.
control   the old side timed twice. This run's noise floor.
red.diff  reductions for one call, new minus old. Exact: 0 means both revisions
          perform exactly the same work, whatever the clock says.

Assisted-By: Claude Opus 5

@josevalim
josevalim merged commit 4abe35a into dashbitco:master Aug 11, 2026
0 of 2 checks passed
@josevalim

Copy link
Copy Markdown
Member

Btw, would you be so kind to send a PR that updates CI? :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants