Skip to content

beam_bounds: Tighten bounds for band, bor, and bxor - #11005

Closed
NelsonVides wants to merge 1 commit into
erlang:masterfrom
NelsonVides:finer_beam_bounds
Closed

beam_bounds: Tighten bounds for band, bor, and bxor#11005
NelsonVides wants to merge 1 commit into
erlang:masterfrom
NelsonVides:finer_beam_bounds

Conversation

@NelsonVides

@NelsonVides NelsonVides commented Apr 13, 2026

Copy link
Copy Markdown
Contributor

Add fallback clauses for bitwise operations when operand ranges exceed NUM_BITS or involve sign combinations not previously handled. Values that exceed NUM_BITS are widened to +inf/-inf via a new cap/1 helper to keep bounds within the representable range.

These tighter bounds enable the compiler to eliminate dead branches in code that performs comparisons after bitwise operations. For example, given:

    %% bxor with one negative, one non-negative: result is always negative.
    bxor_negative(X, Y) when is_integer(X), X >= 0,
                             is_integer(Y), Y < 0 ->
        Z = X bxor Y,
        case Z < 0 of
            true -> negative;
            false -> non_negative
        end.

    %% band with both negative: result is always negative.
    band_neg(X, Y) when is_integer(X), X < 0,
                        is_integer(Y), Y < 0 ->
        Z = X band Y,
        case Z < 0 of
            true -> negative;
            false -> non_negative
        end.

The compiler previously emitted:

    {function, bxor_negative, 2, 4}.
      {label,3}.
        {line,[{location,"test_bounds_example.erl",16}]}.
        {func_info,{atom,test_bounds_example},{atom,bxor_negative},2}.
      {label,4}.
        {test,is_integer,{f,3},[{x,0}]}.
        {test,is_ge,{f,3},[{tr,{x,0},{t_integer,any}},{integer,0}]}.
        {test,is_integer,{f,3},[{x,1}]}.
        {test,is_ge,{f,3},[{integer,-1},{tr,{x,1},{t_integer,any}}]}.
        {line,[{location,"test_bounds_example.erl",18}]}.
        {gc_bif,'bxor',
                {f,0},
                2,
                [{tr,{x,0},{t_integer,{0,'+inf'}}},
                 {tr,{x,1},{t_integer,{'-inf',-1}}}],
                {x,0}}.
        {test,is_ge,{f,5},[{integer,-1},{tr,{x,0},{t_integer,any}}]}.
        {move,{atom,negative},{x,0}}.
        return.
      {label,5}.
        {move,{atom,non_negative},{x,0}}.
        return.

    {function, band_neg, 2, 7}.
      {label,6}.
        {line,[{location,"test_bounds_example.erl",25}]}.
        {func_info,{atom,test_bounds_example},{atom,band_neg},2}.
      {label,7}.
        {test,is_integer,{f,6},[{x,0}]}.
        {test,is_ge,{f,6},[{integer,-1},{tr,{x,0},{t_integer,any}}]}.
        {test,is_integer,{f,6},[{x,1}]}.
        {test,is_ge,{f,6},[{integer,-1},{tr,{x,1},{t_integer,any}}]}.
        {line,[{location,"test_bounds_example.erl",27}]}.
        {gc_bif,'band',
                {f,0},
                2,
                [{tr,{x,0},{t_integer,{'-inf',-1}}},
                 {tr,{x,1},{t_integer,{'-inf',-1}}}],
                {x,0}}.
        {test,is_ge,{f,8},[{integer,-1},{tr,{x,0},{t_integer,any}}]}.
        {move,{atom,negative},{x,0}}.
        return.
      {label,8}.
        {move,{atom,non_negative},{x,0}}.
        return.

and it now emits:

    {function, bxor_negative, 2, 4}.
      {label,3}.
        {line,[{location,"test_bounds_example.erl",16}]}.
        {func_info,{atom,test_bounds_example},{atom,bxor_negative},2}.
      {label,4}.
        {test,is_integer,{f,3},[{x,0}]}.
        {test,is_ge,{f,3},[{tr,{x,0},{t_integer,any}},{integer,0}]}.
        {test,is_integer,{f,3},[{x,1}]}.
        {test,is_ge,{f,3},[{integer,-1},{tr,{x,1},{t_integer,any}}]}.
        {move,{atom,negative},{x,0}}.
        return.

    {function, band_neg, 2, 6}.
      {label,5}.
        {line,[{location,"test_bounds_example.erl",25}]}.
        {func_info,{atom,test_bounds_example},{atom,band_neg},2}.
      {label,6}.
        {test,is_integer,{f,5},[{x,0}]}.
        {test,is_ge,{f,5},[{integer,-1},{tr,{x,0},{t_integer,any}}]}.
        {test,is_integer,{f,5},[{x,1}]}.
        {test,is_ge,{f,5},[{integer,-1},{tr,{x,1},{t_integer,any}}]}.
        {move,{atom,negative},{x,0}}.
        return.

Note, I'm assuming two complement's integer representation, for which I'm 99% sure it is the case 🤔

Add fallback clauses for bitwise operations when operand ranges exceed
NUM_BITS or involve sign combinations not previously handled. Values
that exceed NUM_BITS are widened to +inf/-inf via a new cap/1 helper to
keep bounds within the representable range.

For band:
- One non-negative operand: result in [0, cap(B)].
- Both strictly negative: result in [-inf, min(B, D)].
- Both straddling zero: result in [-inf, cap(max(B, D))].
- Both non-negative exceeding NUM_BITS: [0, cap(min(B, D))].

For bor:
- Both non-negative exceeding NUM_BITS: [0, +inf].
- Both non-positive exceeding NUM_BITS: [-inf, 0].

For bxor:
- Both non-negative exceeding NUM_BITS: [0, +inf].
- One strictly negative, one non-negative: [-inf, -1].

These tighter bounds enable the compiler to eliminate dead branches in
code that performs comparisons after bitwise operations. For example,
given:

    %% bxor with one negative, one non-negative: result is always negative.
    bxor_negative(X, Y) when is_integer(X), X >= 0,
                             is_integer(Y), Y < 0 ->
        Z = X bxor Y,
        case Z < 0 of
            true -> negative;
            false -> non_negative
        end.

    %% band with both negative: result is always negative.
    band_neg(X, Y) when is_integer(X), X < 0,
                        is_integer(Y), Y < 0 ->
        Z = X band Y,
        case Z < 0 of
            true -> negative;
            false -> non_negative
        end.

The compiler previously emitted:

    {function, bxor_negative, 2, 4}.
      {label,3}.
        {line,[{location,"test_bounds_example.erl",16}]}.
        {func_info,{atom,test_bounds_example},{atom,bxor_negative},2}.
      {label,4}.
        {test,is_integer,{f,3},[{x,0}]}.
        {test,is_ge,{f,3},[{tr,{x,0},{t_integer,any}},{integer,0}]}.
        {test,is_integer,{f,3},[{x,1}]}.
        {test,is_ge,{f,3},[{integer,-1},{tr,{x,1},{t_integer,any}}]}.
        {line,[{location,"test_bounds_example.erl",18}]}.
        {gc_bif,'bxor',
                {f,0},
                2,
                [{tr,{x,0},{t_integer,{0,'+inf'}}},
                 {tr,{x,1},{t_integer,{'-inf',-1}}}],
                {x,0}}.
        {test,is_ge,{f,5},[{integer,-1},{tr,{x,0},{t_integer,any}}]}.
        {move,{atom,negative},{x,0}}.
        return.
      {label,5}.
        {move,{atom,non_negative},{x,0}}.
        return.

    {function, band_neg, 2, 7}.
      {label,6}.
        {line,[{location,"test_bounds_example.erl",25}]}.
        {func_info,{atom,test_bounds_example},{atom,band_neg},2}.
      {label,7}.
        {test,is_integer,{f,6},[{x,0}]}.
        {test,is_ge,{f,6},[{integer,-1},{tr,{x,0},{t_integer,any}}]}.
        {test,is_integer,{f,6},[{x,1}]}.
        {test,is_ge,{f,6},[{integer,-1},{tr,{x,1},{t_integer,any}}]}.
        {line,[{location,"test_bounds_example.erl",27}]}.
        {gc_bif,'band',
                {f,0},
                2,
                [{tr,{x,0},{t_integer,{'-inf',-1}}},
                 {tr,{x,1},{t_integer,{'-inf',-1}}}],
                {x,0}}.
        {test,is_ge,{f,8},[{integer,-1},{tr,{x,0},{t_integer,any}}]}.
        {move,{atom,negative},{x,0}}.
        return.
      {label,8}.
        {move,{atom,non_negative},{x,0}}.
        return.

and it now emits:

    {function, bxor_negative, 2, 4}.
      {label,3}.
        {line,[{location,"test_bounds_example.erl",16}]}.
        {func_info,{atom,test_bounds_example},{atom,bxor_negative},2}.
      {label,4}.
        {test,is_integer,{f,3},[{x,0}]}.
        {test,is_ge,{f,3},[{tr,{x,0},{t_integer,any}},{integer,0}]}.
        {test,is_integer,{f,3},[{x,1}]}.
        {test,is_ge,{f,3},[{integer,-1},{tr,{x,1},{t_integer,any}}]}.
        {move,{atom,negative},{x,0}}.
        return.

    {function, band_neg, 2, 6}.
      {label,5}.
        {line,[{location,"test_bounds_example.erl",25}]}.
        {func_info,{atom,test_bounds_example},{atom,band_neg},2}.
      {label,6}.
        {test,is_integer,{f,5},[{x,0}]}.
        {test,is_ge,{f,5},[{integer,-1},{tr,{x,0},{t_integer,any}}]}.
        {test,is_integer,{f,5},[{x,1}]}.
        {test,is_ge,{f,5},[{integer,-1},{tr,{x,1},{t_integer,any}}]}.
        {move,{atom,negative},{x,0}}.
        return.
@github-actions

github-actions Bot commented Apr 13, 2026

Copy link
Copy Markdown
Contributor

CT Test Results

    2 files    335 suites   8m 59s ⏱️
  872 tests   866 ✅ 6 💤 0 ❌
5 719 runs  5 713 ✅ 6 💤 0 ❌

Results for commit 5564208.

♻️ This comment has been updated with latest results.

To speed up review, make sure that you have read Contributing to Erlang/OTP and that all checks pass.

See the TESTING and DEVELOPMENT HowTo guides for details about how to run test locally.

Artifacts

// Erlang/OTP Github Action Bot

@bjorng bjorng added the team:VM Assigned to OTP team VM label Apr 13, 2026
@bjorng bjorng added this to the 30.0 milestone Apr 13, 2026
@bjorng bjorng self-assigned this Apr 13, 2026
@bjorng

bjorng commented Apr 19, 2026

Copy link
Copy Markdown
Contributor

Thanks for your pull request. It is looks good to me but we don't plan to merge it. The reason is that I've been working on similar optimizations on and off since last year. The result is #11038. However, your pull request inspired me to replace large integers with -inf or +inf, which simplified the code and eliminated many special cases. Many thanks for that idea!

@NelsonVides

Copy link
Copy Markdown
Contributor Author

Wow, nice, thank you so much for your comments and very happy to have contributed something good nevertheless!

Love your new PR! Gonna take some time to study the change in your PR, compiler code is a big thing to me yet but sure I'll learn a bunch of cool stuff! 🤓

@NelsonVides
NelsonVides deleted the finer_beam_bounds branch April 20, 2026 07:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

team:VM Assigned to OTP team VM

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants