Skip to content

Fix UInt8/UInt16 masking: switch scrutinee, ETernary, ECast#741

Open
mtzguido wants to merge 1 commit into
FStarLang:masterfrom
mtzguido:707
Open

Fix UInt8/UInt16 masking: switch scrutinee, ETernary, ECast#741
mtzguido wants to merge 1 commit into
FStarLang:masterfrom
mtzguido:707

Conversation

@mtzguido
Copy link
Copy Markdown
Member

@mtzguido mtzguido commented May 3, 2026

Three issues:
1- The ESwitch case passed the scrutinee through mk_expr, bypassing
mk_arith masking.
2- mk_arith's catch-all marked ETernary as atomic, hiding non-atomic
arithmetic branches from consumers (switch, comparisons).
3- ECast applied the cast to the widened uint32 value without masking,
leaking overflow bits on widening casts.

Fix: for UInt8/UInt16 switch scrutinees, route through mk_arith and mask non-atomic results. Add an ETernary case to mk_arith that recurses into both branches, returning is_atomic only when both are atomic. For ECast, route UInt8/UInt16 sources through mk_arith and mask before casting.

Add SmallIntSwitch.fst, SmallIntTernary.fst, SmallIntCast.fst regression tests.

Fixes #707

--

Hi Jonathan, this is coauthored with Claude. It fixes #707 (which is legit) and a few other variants. The three tests fail before this patch. I'm not sure the style of the patch is that pleasant, since it seems easy to forget about this mk_arith dance and reintroduce variants of this bug later.

Three issues:
1- The ESwitch case passed the scrutinee through mk_expr, bypassing
   mk_arith masking.
2- mk_arith's catch-all marked ETernary as atomic, hiding non-atomic
   arithmetic branches from consumers (switch, comparisons).
3- ECast applied the cast to the widened uint32 value without masking,
   leaking overflow bits on widening casts.

Fix: for UInt8/UInt16 switch scrutinees, route through mk_arith and mask
non-atomic results. Add an ETernary case to mk_arith that recurses into
both branches, returning is_atomic only when both are atomic. For ECast,
route UInt8/UInt16 sources through mk_arith and mask before casting.

Add SmallIntSwitch.fst, SmallIntTernary.fst, SmallIntCast.fst regression
tests.

Fixes FStarLang#707

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Collaborator

@protz protz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, thanks.

Thinking more about my comment regarding the ternary operator, e.g. (false ? f () : 1) >> 1. I think it should be have like a binary operator and I think w1 || w2 is correct.

Comment thread lib/AstToCStar.ml
that every subexpression operates over unsigned int until the final
cast, or until a mask is needed to preserve semantics. *)
mk_expr env false false e, true, false (* C++: a constant that is wider than the intended type, but in an initializer list, is ok *)
| ETernary (c, t, f) ->
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are you expecting things of the form return e1 + e2 ? e3 : e4?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean with an operation in the condition? I don't think so, they must have bool type.

Comment thread lib/AstToCStar.ml
let c = mk_expr env false false c in
let t, a1, w1 = mk_arith env t in
let f, a2, w2 = mk_arith env f in
CStar.Ternary (c, t, f), a1 && a2, w1 || w2
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if one of the branches was widened but not the other, shouldn't we widen both? feels like we should assert that w1 = w2 or do something

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a conservative choice so I think it is correct

Comment thread lib/AstToCStar.ml
CStar.Op (K.op_of_poly_comp c)
| ECast (e, t) ->
CStar.Cast (mk_expr env false e, mk_type env t)
(* When the source is UInt8/UInt16 and contains arithmetic, the widened
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

@mtzguido
Copy link
Copy Markdown
Member Author

mtzguido commented May 9, 2026

I have to say I'm a bit confused about the widening. It's only relevant for C++ initializers if I understand correctly? And it seems like an addition of u8 constants is not considered widened, but the addition would be promoted to int?

This generates a bad diff in Eurydice, I'll look into it.

tests>  size_t where_clauses_simple_add_81_6c(Eurydice_arr_96 x, size_t y)
tests>  {
tests> -  return (size_t)x.data[0U] + y + (size_t)1U;
tests> +  return (size_t)(uint32_t)x.data[0U] + y + (size_t)1U;
tests>  }

I wonder if a different design for all this would be more desirable, actually: keep all casts explicit during AST->C*, and then do a pass to remove the superfluous ones. With enough rules, I think we could remove all the casts in the expression above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[DeepTest] UInt8/UInt16 switch scrutinee not masked after wrapping arithmetic

2 participants