Iterator bytecode field name is wrong in Scala 3.8.0 library

[alexdupre]

Apparently some classes in the published scala standard library, in particular some collections, have not been (re-)compiled correctly. The issue seems to have been triggered by a change in the Iterator object, from:

 private[this] val _empty: Iterator[Nothing]

to:

 private val _empty: Iterator[Nothing]

Some classes, like the Range, seem to still point to the previously generated bytecode field name.

Compiler version

3.8.0

Minimized code

//> using scala 3.8.0

@main def test = {
  val i = (1 to 1).grouped(1)
  println(i.next())
  println(i.next())
}

Output

Range 1 to 1
Exception in thread "main" java.lang.NoSuchFieldError: Class scala.collection.Iterator$ does not have member field 'scala.collection.Iterator scala$collection$Iterator$$_empty'
        at scala.collection.immutable.Range$$anon$3.next(Range.scala:562)
        at scala.collection.immutable.Range$$anon$3.next(Range.scala:553)
        at x$package$.test(x.scala:6)
        at test.main(x.scala:3)

Expectation

Range 1 to 1
Exception in thread "main" java.util.NoSuchElementException: next on empty iterator
        at scala.collection.Iterator$$anon$19.next(Iterator.scala:973)
        at scala.collection.Iterator$$anon$19.next(Iterator.scala:971)
        at scala.collection.immutable.Range$$anon$3.next(Range.scala:507)
        at scala.collection.immutable.Range$$anon$3.next(Range.scala:498)
        at x$package$.test(x.scala:6)
        at test.main(x.scala:3)

[WojciechMazur]

The issue seems is coused by approach with dealing to lack of specialization in Scala 2, where certain classess are copied from Scala 2 binaries in project/ScalaLibraryPlugin.scala - this allowed to preserve binary compatibility of PUBLIC methods and was checked against combined Scala 2.13.16 and Scala 3.7.4 artifacts.

However, the Scala 2 has an JVM Bytecode Optimizer which can can inline references to private fields, that's the case for private[this] val _empty which does not have an accessor generated, but rather is accessed directly which is then used by @inline def empty[T]

Combination of these introduces an inlined, direct reference to the private field, which has a different name when compiled using Scala 3. By being PRIVATE it was not detected by MiMa and slipped through the testing rig.

We'd need to workaround it, either manipulate even more bytecode, are have a dedicated version of Scala 2 stdlib artifacts that it's safe for coping to Scala 3 (unoptimized)

[sjrd]

Nice! Thanks for the report.

I wonder if we should take classes from the unoptimized Scala 2 library jars.

[WojciechMazur]

Reproducers based on the linked scalacenter/sbt-missinglink#54 - these match all of the 5 cases highlighted by sbt-missinglink

ArrayOps.ReverseItarator

> Array.empty[Int].reverseIterator.hasNext
val res8: Boolean = false
                                                                                                                                                                                         
> Array.empty[Int].reverseIterator.next
java.lang.NoSuchFieldError: Class scala.collection.Iterator$ does not have member field 'scala.collection.Iterator scala$collection$Iterator$$_empty'
  at scala.collection.ArrayOps$ReverseIterator.next(ArrayOps.scala:151)
  ... 30 elided                                                                                                                                                                         

empty Range.grouped

scala> Range(0,0).iterator.next // correct
java.util.NoSuchElementException: next on empty iterator
  at scala.collection.Iterator$$anon$19.next(Iterator.scala:991)
  at scala.collection.Iterator$$anon$19.next(Iterator.scala:991)
  at scala.collection.immutable.RangeIterator.next(Range.scala:710)
  at scala.collection.immutable.RangeIterator.next(Range.scala:709)
  ... 30 elided
                                                                                                                                                                                         
scala> Range(0,0).grouped(1).hasNext
java.lang.NoSuchFieldError: Class scala.collection.Iterator$ does not have member field 'scala.collection.Iterator scala$collection$Iterator$$_empty'
  at scala.collection.immutable.Range.grouped(Range.scala:550)
  ... 30 elided
                                                                                                                                                                                         
scala> Range(0,0).grouped(1).next
java.lang.NoSuchFieldError: Class scala.collection.Iterator$ does not have member field 'scala.collection.Iterator scala$collection$Iterator$$_empty'
  at scala.collection.immutable.Range.grouped(Range.scala:550)
  ... 30 elided

non-empty Range.grouped

scala> val x = Range(0, 1).grouped(1)
val x: Iterator[Range] = non-empty iterator
                                                                                                                                                                                         
scala> x.next
val res33: Range = Range(0)
                                                                                                                                                                                         
scala> x.next
java.lang.NoSuchFieldError: Class scala.collection.Iterator$ does not have member field 'scala.collection.Iterator scala$collection$Iterator$$_empty'
  at scala.collection.immutable.Range$$anon$3.next(Range.scala:562)
  at scala.collection.immutable.Range$$anon$3.next(Range.scala:553)
  ... 30 elided

empty Range.inits

cala> val x = Range(0,0).inits
val x: Iterator[Range] = non-empty iterator
                                                                                                                                                                                         
scala> x.next
val res26: Range = Range()
                                                                                                                                                                                         
scala> x.next
java.lang.NoSuchFieldError: Class scala.collection.Iterator$ does not have member field 'scala.collection.Iterator scala$collection$Iterator$$_empty'
  at scala.collection.immutable.Range$$anon$2.next(Range.scala:506)
  at scala.collection.immutable.Range$$anon$2.next(Range.scala:497)
  ... 30 elided

empty Range.tails

scala> val y = Range(0,0).tails
val y: Iterator[Range] = non-empty iterator
                                                                                                                                                                                         
scala> y.next
val res28: Range = Range()
                                                                                                                                                                                         
scala> y.next
java.lang.NoSuchFieldError: Class scala.collection.Iterator$ does not have member field 'scala.collection.Iterator scala$collection$Iterator$$_empty'
  at scala.collection.immutable.Range$$anon$1.next(Range.scala:491)
  at scala.collection.immutable.Range$$anon$1.next(Range.scala:482)
  ... 30 elided