How much longer for XStream?

[NicholasWMRitchie]

XStream is a critical part of my application's serialization procedures and has been a wonderful thing 👍 so I'm hesitant to ask such a downer of a question.

However, each subsequent version of Java seems to shut down more and more of the APIs on which XStream depends. I can't argue with the Java designer's desire to eliminate security flaws by eliminating problematic APIs, however, it does raise the question: How long can XStream survive? Are there alternative APIs that will permit re-instantiating objects while maintaining security? This can't only be a problem for XStream. What do other serialization APIs do?

I've been slowly implementing fixes as I've moved from JDK version to version. Each time, I've been able to find some work around using the XStream whitelist security API. However, I'm afraid that I might be at the end of the line.

I just upgraded from JDK 15 to JDK 16 and now serialization via XStream blows up spectacularly. The first error I get seems to be caused by this:

Unable to make field private final java.util.Comparator java.util.TreeMap.comparator accessible: module java.base does not "opens java.util" to unnamed module @2353b3e6

This would seem to be related to new restrictions implement in Java 16.

I'd like to know should I be considering alternative serialization schemes or will it be possible to address these problems within XStream?

[joehni]

Well, XStream mimics Java serialization, but it does stuff that is not available in public API and it even violates the Java specification (by calling for an object the ctor of the unserializable parent only). My guess is in combination with the security flaws, that it will be cut off in the long run also.

What does it mean for XStream? Actually it can still use reflection for types in modules that can be opened to it. You will be able to use it for your own types. For any stuff in the java.base module XStream will have to provide custom converters - reflection will no longer work. However, in consequence XStream will no longer be able to serialize some types out of the box at all (e.g. XStream will no longer be able to marshal a synchronized collection, because it cannot detect, what actual collection type is wrapped). Users of XStream will definitely have to write more custom converters in future.

[jyoshimi]

Hi all, and Jörg in particular. I'm also a long time and very happy xstream user. I teach a course with software using xstream. Students with java 16 can't run it, I think for the reasons brought up by the op. For now it's working by having them go back to java 14 or earlier, so we're good.

Longer term, you say we "will definitely have to write more custom converters in the future". Am I right that if that is done for most of the major java classes, that we can keep using it in the future?

I just want to make sure I have a sense of the long game here. I really don't want to lose xstream! I love how neat and tidy my code is and also how nice and readable the xml I generate is.

Thanks for the tireless efforts Jörg and xstream team, much appreciated.

[joehni]

At least for Java 16 you can set --illegal-access=permit.

As long as a module is open to XStream you will still be able to do most of the types in it. Obviously this is no longer true for java.base - here we'll need more converters in future. However, not everything can be handled in future. Think about synchronized collections:

List<String> list = Collections.synchronizedList(new LinkedList<>());

XStream has no possibility to detect the real type of the wrapped collection, it can no longer handle this object graph out of the box. You will have to write/register a converter on your own. This might even be true for a singleton wrapper, depending on the access method of the wrapped collection.

Another problem area are final fields. When XStream no longer can write into these by reflection, it won't be able to handle the type properly. This includes any non-static inner class type.

[celcius112]

For Java 16 you can probably make it works with the option --illegal-access=permit. However this command line switch is removed starting in Java 17 (more info here), making XStream incompatible with the latest LTS 😢. By transitivity, Jenkins CI is also incompatible 😄.

[timja]

By transitivity, Jenkins CI is also incompatible 😄.

Well, you can override it pretty easily with --add-opens, the Jenkins docker image will add it automatically for you:

jenkins/jenkins:2.314-jdk17-preview

(preview is to gather early feedback as not much testing has been done on it yet)

https://hub.docker.com/layers/jenkins/jenkins/2.314-jdk17-preview/images/sha256-30a195e4260fa86986fd9f1d98ed7c2c463d30a8d7291cf1bf2a3c7854a63aa6?context=explore

[celcius112]

Thanks for the heads up, we will try it

[timja]

Thanks for the heads up, we will try it

https://github.com/jenkinsci/docker for any issues with the image, https://issues.jenkins.io/ component core if there's an issue with Jenkins, label it with java17-compatibility, feel free to tag me for visibility

[galderz]

nexus-staging-maven-plugin, which depends on xstream, also having similar issues (see NEXUS-27902). I've added a comment there based on the hint above on using --add-opens but I've not tried it.

[kriegaex]

My project also has this problem on JDK 16+, despite XSTREAM.addPermission(AnyTypePermission.ANY); in my code:

Exception in thread "main" java.lang.ExceptionInInitializerError
	at com.thoughtworks.xstream.converters.collections.TreeMapConverter.unmarshal(TreeMapConverter.java:73)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72)
	at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:72)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:50)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:134)
	at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1391)
	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1376)
	at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1270)
	at de.scrum_master.galileo.Book.readConfig(Book.java:86)
	at de.scrum_master.galileo.Options.parse(Options.java:46)
	at de.scrum_master.galileo.OpenbookCleaner.processArgs(OpenbookCleaner.java:36)
	at de.scrum_master.galileo.OpenbookCleaner.main_aroundBody0(OpenbookCleaner.java:29)
	at de.scrum_master.galileo.OpenbookCleaner.main_aroundBody1$advice(OpenbookCleaner.java:12)
	at de.scrum_master.galileo.OpenbookCleaner.main(OpenbookCleaner.java:1)
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field private final java.util.Comparator java.util.TreeMap.comparator accessible: module java.base does not "opens java.util" to unnamed module @77847718
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:357)
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
	at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:177)
	at java.base/java.lang.reflect.Field.setAccessible(Field.java:171)
	at com.thoughtworks.xstream.core.util.Fields.locate(Fields.java:40)
	at com.thoughtworks.xstream.converters.collections.TreeMapConverter$Reflections.<clinit>(TreeMapConverter.java:135)
	... 16 more

The only workaround is to specify the JVM parameter --add-opens java.base/java.util=ALL-UNNAMED.

[kriegaex]

Related problems in Nexus Staging Maven Plugin:

• https://issues.sonatype.org/browse/OSSRH-66257
• https://issues.sonatype.org/browse/NEXUS-26993
• https://issues.sonatype.org/browse/NEXUS-27902