Guard symbolic-length vectors against 1x1 matrices instead of scalarizing

R's length-1-array recycling drops the array dims only when the vector's
length is not 1; for a length-1 vector the 1x1 dims are kept. Scalarizing
whenever the length was not *provably* 1 answered the compile-time
question "is the length 1?" with "no" when the truth was "unknown", so
a symbolic-length vector that turned out to have length 1 at run time
returned a dimensionless vector where R returns a 1x1 matrix -- a silent
shape divergence.

Scalarize only when the length is statically known and not 1 (the cases
where R itself drops the dims, including length 0). Symbolic lengths fall
through to the vector-matrix rule: a runtime guard requires length 1, the
result is a 1x1 matrix, and longer vectors raise an error where R would
recycle (a deprecated behavior in R).

Found by codex review (fable-final round); reproduces on upstream main.

Author: mns-nordicals

NEWS.md

@@ -15,12 +15,22 @@
   out of bounds. Statically unequal lengths are now a compile-time error;
   lengths that cannot be verified at compile time are checked at run time.
 
-- `1x1`-matrix operands now follow R's rules: in arithmetic they are
-  treated as scalars (which R allows, with a deprecation warning), while in
-  comparisons and `&`/`|` they are treated as one-row matrices, so
-  mismatched shapes are rejected — matching R, which raises an error.
-  Previously a `1x1` matrix was treated as a scalar everywhere, so e.g.
-  `x < m` with `m` a `1x1` matrix returned answers where R errors.
+- `1x1`-matrix operands now follow R's rules: in arithmetic against a
+  vector of statically known length other than 1 they are treated as
+  scalars and the result is a plain vector (which R allows, with a
+  deprecation warning), while in comparisons and `&`/`|` they are treated
+  as one-row matrices, so mismatched shapes are rejected — matching R,
+  which raises an error. When the vector's length is only known at run
+  time, the result's shape would depend on that value (R keeps the `1x1`
+  dims for a length-1 vector and drops them otherwise), so arithmetic
+  also takes the one-row-matrix rule: a runtime check requires length 1
+  and the result is a `1x1` matrix; longer vectors raise an error where
+  R would recycle. Previously a `1x1` matrix was scalarized in arithmetic
+  and not shape-checked at all in comparisons and `&`/`|`, so e.g.
+  `x < m` with `m` a `1x1` matrix failed to build with a Fortran rank
+  mismatch instead of a quickr error, an operand needing a cast failed to
+  build even in arithmetic, and a symbolic-length `x + m` returned a plain
+  vector where R returns a `1x1` matrix.
 
 # quickr 0.3.0
 

R/r2f-operators-helpers.R

@@ -128,6 +128,13 @@ dim_is_one <- function(x) {
   is_wholenumber(x) && identical(as.integer(x), 1L)
 }
 
+# Check if a dimension expression is statically known and not 1. Symbolic
+# dimensions are FALSE: "not provably 1" is not "provably not 1".
+# Used by: maybe_reshape_vector_matrix()
+dim_known_not_one <- function(x) {
+  is_wholenumber(x) && !identical(as.integer(x), 1L)
+}
+
 # Check if a Fortran value is a 1x1 matrix.
 # Used by: r2f-arithmetic.R, r2f-logical.R
 is_one_by_one <- function(x) {
@@ -238,10 +245,15 @@ scalarize_matrix <- function(mat) {
 # size guard through `hoist`.
 #
 # `scalarize_one_by_one` mirrors R's split over length-1 arrays: arithmetic
-# recycles a 1x1 matrix against a longer vector (deprecated in R but still
-# the behavior), while comparisons and & | error. Strict callers pass FALSE
-# so the 1x1 falls through to the vector-matrix rule and is rejected or
-# guarded like any other 1-row matrix.
+# recycles a 1x1 matrix against a vector of statically known length != 1
+# (deprecated in R but still the behavior: R drops the array dims). When
+# the vector's length is only known at run time, the result's shape would
+# depend on that value -- R keeps the 1x1 dims for a length-1 vector and
+# drops them otherwise -- so the 1x1 falls through to the vector-matrix
+# rule: a runtime guard requires length 1 and the result is a 1x1 matrix,
+# an error where R would recycle. Comparisons and & | error in R itself,
+# so strict callers pass FALSE and the 1x1 always takes the vector-matrix
+# path.
 # Used by: r2f-arithmetic.R, r2f-logical.R
 maybe_reshape_vector_matrix <- function(
   left,
@@ -281,7 +293,7 @@ maybe_reshape_vector_matrix <- function(
       is_one_by_one(left)
   ) {
     right_len <- dim_or_one(right, 1L)
-    if (!dim_is_one(right_len)) {
+    if (dim_known_not_one(right_len)) {
       left <- scalarize_via_hoist(left)
       left_rank <- 0L
     }
@@ -292,7 +304,7 @@ maybe_reshape_vector_matrix <- function(
       is_one_by_one(right)
   ) {
     left_len <- dim_or_one(left, 1L)
-    if (!dim_is_one(left_len)) {
+    if (dim_known_not_one(left_len)) {
       right <- scalarize_via_hoist(right)
       right_rank <- 0L
     }

tests/testthat/test-recycling.R

@@ -149,6 +149,31 @@ test_that("1x1 matrix operands follow R: arithmetic scalarizes, strict ops rejec
   expect_error(qcmp(c(1, 2, 3), matrix(5)), "matrix first dimension")
 })
 
+test_that("1x1 matrix with a symbolic-length vector keeps R's shape", {
+  # The result's shape depends on the runtime length: R keeps the 1x1
+  # dims for a length-1 vector and drops them for any other length, so no
+  # static decision can be right for both. Scalarizing regardless (the
+  # old behavior) silently returned a dimensionless vector where R
+  # returns a 1x1 matrix. Symbolic lengths now take the vector-matrix
+  # rule instead: a runtime guard requires length 1 and the result is a
+  # 1x1 matrix; longer vectors error where R would recycle (deprecated).
+  fn <- function(m, x) {
+    declare(type(m = double(1, 1)), type(x = double(n)))
+    m + x
+  }
+  qfn <- quick(fn)
+  expect_identical(qfn(matrix(2), 3), fn(matrix(2), 3))
+  expect_error(qfn(matrix(2), c(1, 2, 3)), "matrix first dimension")
+
+  rev_fn <- function(x, m) {
+    declare(type(x = double(n)), type(m = double(1, 1)))
+    x + m
+  }
+  qrev <- quick(rev_fn)
+  expect_identical(qrev(3, matrix(2)), rev_fn(3, matrix(2)))
+  expect_error(qrev(c(1, 2, 3), matrix(2)), "matrix first dimension")
+})
+
 test_that("guard text is pinned (one snapshot per mechanism)", {
   fn <- function(a, b) {
     declare(type(a = double(n)), type(b = double(m)))