Description
There are currently inconsistencies in whether normal Array behaviors respect 32- or 53-bit length maximums.
The spec defines two kinds of index:
An integer index is a String-valued property key … whose numeric value is either +0 or a positive integer ≤ 253 - 1. An array index is an integer index whose numeric value i is in the range +0 ≤ i < 232 - 1.
Creating a new Array using the Array constructor predictably throws a RangeError exception if trying to create an array with greater than 232 - 1 elements.
However, Array.from and Array.prototype.splice both only make checks against 253 - 1.
The current behavior of both is to silently transition from pushing elements onto the array into a mode of setting normal string-named properties (and thus not updating the array's length) because it causes the use of the Array Exotic Object [[DefineOwnProperty]] internal method. This leads to effects which I personally deem unintuitive, e.g.
// let someIterable be an iterable that emits 'abc' 5000000000 times
let arr = Array.from(someIterable);
arr.length; // 4294967295
arr[50000000000]; // "abc"
new Array(5000000000); // RangeError(And let arr = [...someIterator] would produce yet a different outcome; however, that behavior is subject to change given #1124.)
This would lead to a silent loss of values whose property keys were indices greater than 2**32-1 if the resulting array were copied again through its iterator. E.g., Array.from(Array.from(someIterable)), where one might otherwise expect that Array.from should be idempotent.
It is worth noting that Array.prototype.splice will cause a RangeError exception to be thrown, but only after it has already mutated the underlying array, and only as a consequence of explicitly setting the length property in step 19 (which eventually triggers a different path in the same exotic [[DefineOwnProperty]]), a step which the immediately-following spec note claims is "unneccessary starting in ES2015".
To demonstrate,
> let arr = Array(2**32 - 1)
[ <4294967295 empty items> ]
> arr.splice(2**32 - 1, 0, 1)
RangeError: Invalid array length
at Array.splice (<anonymous>)
> arr[2**32 - 1]
1
(node v10.15.0 / v8 6.8.275.32-node.45)
The conversations in #1095 and #1124 gave some related insight into this, though the specific issues addressed in those (to do with ArrayAccumulation) and the issue I bring up here seem to have a common root cause in the difference between allowed array lengths, namely the use of 32- vs 53-bit integers.
I have not been able to find background discussion on why normal Arrays were chosen to be limited to a length of 232 - 1, given that ECMAScript has always (at least since ES1 in 1997) had an IEEE 754 Number type with 52-bit mantissa (53 bits of precision); though it seems like a logical enough choice, especially for engine implementations for which 32-bit length values might be convenient for Array operations.
However, given the interoperability issues the 32-vs-53-bit disparity seems to be causing, would it be worth proposing that normal Arrays be extended to allow a maximum index of 253 - 1 to match their TypedArray and other integer-indexed counterparts? E.g., effectively getting rid of array index in favor of unifying with integer index.
Or, if that were not tractable, what about having Array.from throw once it encounters the (2**32 - 1)st value from an iterable (step 5.e.i)? And similarly change step 8 of Array.prototype.splice.
Disclaimer: I know that most of this is currently more theory than application; I hit memory limits plenty of times while trying to run related experiments. However, I find it conceivable that future resource-rich environments may encounter some scenarios in which they want to work with an array of length greater than ~4.3 billion. And, appropriateness of such use aside, I also think that it would be conceptually cleaner for all species of arrays to have the same limit on maximum length.
Many thanks ahead of time to anyone who might point out if/where I'm missing any pieces of the puzzle, historical background, etc!