[Java] `ClassCastException` when deserializing certain records (interpreter mode)

[gudzpoz]

Search before asking

• I had searched in the issues and found no similar issues.

Version

Fury: 'org.apache.fury:fury-core:0.10.2'
JDK: GraalVM 23.0.2+7.1 (build 23.0.2+7-jvmci-b01)

Component(s)

Java

Minimal reproduce step

    @Test
    public void testNumberListRecord() {
        Fury fury = Fury.builder()
                .withLanguage(Language.JAVA)
                .withAsyncCompilation(true)
                .build();
        fury.register(NumberListRecord.class);
        fury.register(IntArrayList.class);
        byte[] bytes = fury.serialize(new NumberListRecord(1L, new ArrayList<>()));
        fury.deserialize(bytes); // <-- ClassCastException
    }

    record NumberListRecord(Number n, List<Object> list) {
    }

What did you expect to see?

The test above passes.

What did you see instead?

Cannot cast java.util.ArrayList to java.lang.Number
java.lang.ClassCastException: Cannot cast java.util.ArrayList to java.lang.Number
	at java.base/java.lang.Class.cast(Class.java:4139)
	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:735)
	at org.apache.fury.serializer.ObjectSerializer.read(ObjectSerializer.java:241)
	at org.apache.fury.Fury.readDataInternal(Fury.java:990)
	at org.apache.fury.Fury.readRef(Fury.java:874)
	at org.apache.fury.Fury.deserialize(Fury.java:806)
	at org.apache.fury.Fury.deserialize(Fury.java:727)
	at party.iroiro.juicemacs.elisp.runtime.pdump.DumpUtilsTest.testNumberListRecord(DumpUtilsTest.java:164)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1597)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1597)

Anything Else?

I have also tested with the following records:

    record NumberListRecord(Integer n, List<Object> list) {
        // This one passes the test.
    }
    record NumberListRecord(AutoClosable n, List<Object> list) {
        // This one fails.
    }

Also, setting withAsyncCompilation(false) also removes the error. So the bug seems to be in the interpreted serializer.

Are you willing to submit a PR?

• I'm willing to submit a PR!

[chaokunyang]

I will fix it tonight.

[chaokunyang]

I can't reproduce it with code:

  @Test
  public void testNumberListRecord() {
    Fury fury = Fury.builder().withCodegen(false).build();
    fury.register(NumberListRecord.class);
    fury.register(IntArrayList.class);
    byte[] bytes = fury.serialize(new NumberListRecord(1L, new ArrayList<>()));
    Fury fury1 = Fury.builder().withCodegen(true).build();
    fury1.register(NumberListRecord.class);
    fury1.register(IntArrayList.class);
    fury1.deserialize(bytes);
    byte[] bytes1 = fury1.serialize(fury1.deserialize(bytes));
    System.out.println(fury.deserialize(bytes1));;
  }

  @Test
  public void testNumberListRecord1() {
    for (int i = 0; i < 100; i++) {
      Fury fury = Fury.builder()
        .withAsyncCompilation(true)
        .build();
      fury.register(NumberListRecord.class);
      fury.register(IntArrayList.class);
      byte[] bytes = fury.serialize(new NumberListRecord(1L, new ArrayList<>()));
      System.out.println(fury.deserialize(bytes));; // <-- ClassCastException
    }
  }

  static class IntArrayList extends ArrayList<Integer> {}

  record NumberListRecord(Number n, List<Object> list) {
  }

[gudzpoz]

Here is a more complete reproducer: https://github.com/gudzpoz/fury-bug-reproducer/
And here is the output of a GitHub action running the reproducer, which also fails: https://github.com/gudzpoz/fury-bug-reproducer/actions/runs/15301721519/job/43043763896
Both my machine and the action runner are x64. Stepping through the code, it seems it is getting the order of fields wrong. Maybe it is an architecture/JVM-specific issue?

[chaokunyang]

Fury will sort fields for serialization in DescripptorGrouper, could you check whether it work for you?

[chaokunyang]

I case across similiar error in https://github.com/apache/fury/actions/runs/15303134055/job/43048773047?pr=2261:

Error:  Tests run: 1350, Failures: 2, Errors: 0, Skipped: 3, Time elapsed: 65.68 s <<< FAILURE! - in TestSuite
Error:  testTargetHasLessFieldComparedToSourceClass(org.apache.fury.serializer.compatible.DifferentPOJOCompatibleSerializerTest)  Time elapsed: 0.016 s  <<< FAILURE!
org.apache.fury.exception.DeserializationException: Deserialize failed, read objects are: [org.apache.fury.serializer.compatible.classes.ClassMissingField@55e041cd, org.apache.fury.serializer.compatible.classes.ClassCompleteField@ca2a348b, subclass, subclass2, [a], [a], {a=b}, {a=b}]
	at org.apache.fury.serializer.compatible.DifferentPOJOCompatibleSerializerTest.testTargetHasLessFieldComparedToSourceClass(DifferentPOJOCompatibleSerializerTest.java:65)
Caused by: java.lang.IndexOutOfBoundsException: readerIndex(140) + length(3402) exceeds size(337): org.apache.fury.memory.MemoryBuffer$BoundChecker@1d4d69b6
	at org.apache.fury.serializer.compatible.DifferentPOJOCompatibleSerializerTest.testTargetHasLessFieldComparedToSourceClass(DifferentPOJOCompatibleSerializerTest.java:65)

Error:  testTargetHasLessFieldComparedToSourceClass(org.apache.fury.serializer.compatible.DifferentPOJOCompatibleSerializerWithRegistrationTest)  Time elapsed: 0.004 s  <<< FAILURE!
org.apache.fury.exception.DeserializationException: Deserialize failed, read objects are: [org.apache.fury.serializer.compatible.classes.ClassMissingField@78b2a0a7, org.apache.fury.serializer.compatible.classes.ClassMissingField@e6ea3da9, subclass, [a], [a], {a=b}, {a=b}]
	at org.apache.fury.serializer.compatible.DifferentPOJOCompatibleSerializerWithRegistrationTest.testTargetHasLessFieldComparedToSourceClass(DifferentPOJOCompatibleSerializerWithRegistrationTest.java:65)
Caused by: java.lang.IndexOutOfBoundsException: readerIndex(143) + length(1481457006) exceeds size(294): org.apache.fury.memory.MemoryBuffer$BoundChecker@543e724e
	at org.apache.fury.serializer.compatible.DifferentPOJOCompatibleSerializerWithRegistrationTest.testTargetHasLessFieldComparedToSourceClass(DifferentPOJOCompatibleSerializerWithRegistrationTest.java:65)

[gudzpoz]

No, that seems a separate issue. I cloned fury and tested against the main branch, only to find that the bug disappeared! It seems it has been fixed in #2146:

commit b3196e39a46c26b1e49f144283b1f22a398e2b7c
feat(java): unify java and xlang object serialization (#2146)
modified   java/fury-core/src/main/java/org/apache/fury/type/DescriptorGrouper.java
@@ -178,9 +190,9 @@ public class DescriptorGrouper {
     descriptors.addAll(getPrimitiveDescriptors());
     descriptors.addAll(getBoxedDescriptors());
     descriptors.addAll(getFinalDescriptors());
+    descriptors.addAll(getOtherDescriptors());
     descriptors.addAll(getCollectionDescriptors());
     descriptors.addAll(getMapDescriptors());
-    descriptors.addAll(getOtherDescriptors());
     return descriptors;
   }

However, the commit appears in neither fury-core v0.10.2 (https://repo1.maven.org/maven2/org/apache/fury/fury-core/0.10.2/fury-core-0.10.2-sources.jar) nor v0.10.3 (https://repo1.maven.org/maven2/org/apache/fury/fury-core/0.10.3/fury-core-0.10.3-sources.jar), causing issues in both versions. Is there any reason #2146 not being included in releases?

[chaokunyang]

It's too big and breaks binary compatibility, we can't include it in 0.10.x

[chaokunyang]

Would you like to create a PR to fix it in releases-0.10 branch?