Possible ByteArrayModule issue

[lipisak]

Describe the bug

Hi falks,
after upgrade to Vaadin 24.8 I noticed that communication via RestTemplate doesn't work in some cases.

If I call endpoint, which returns DTO with byte[] field containing file data, the deserialization fails when there are more then 20000000 bytes.

The stacktrace:

com.fasterxml.jackson.core.exc.StreamConstraintsException: String value length (20054016) exceeds the maximum allowed (20000000, from StreamReadConstraints.getMaxStringLength()) at com.fasterxml.jackson.core.StreamReadConstraints._constructException(StreamReadConstraints.java:658) at com.fasterxml.jackson.core.StreamReadConstraints.validateStringLength(StreamReadConstraints.java:593) at com.fasterxml.jackson.core.util.ReadConstrainedTextBuffer.validateStringLength(ReadConstrainedTextBuffer.java:27) at com.fasterxml.jackson.core.util.TextBuffer.finishCurrentSegment(TextBuffer.java:1017) at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString2(UTF8StreamJsonParser.java:2544) at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishAndReturnString(UTF8StreamJsonParser.java:2520) at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:294) at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer._deserializeAnyScalar(JsonNodeDeserializer.java:666) at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:108) at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:24) at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342) at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:4944) at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:3165) at com.vaadin.hilla.parser.jackson.ByteArrayModule$ByteArrayDeSerializer.deserialize(ByteArrayModule.java:36) at com.vaadin.hilla.parser.jackson.ByteArrayModule$ByteArrayDeSerializer.deserialize(ByteArrayModule.java:29) at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:137) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:392) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:177) at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342) at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:2130) at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1500) at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType(AbstractJackson2HttpMessageConverter.java:397) at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.read(AbstractJackson2HttpMessageConverter.java:356) at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:105) at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:903) at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:801) at org.springframework.web.client.RestTemplate.postForObject(RestTemplate.java:518) at com.mediasol.dp.ui_v2.reporting.impl.ReportingServiceImpl.runPipelineWithFileResult(ReportingServiceImpl.java:107) at com.mediasol.dp.ui_v2.reporting.impl.RunReportDialogPresenterImpl.lambda$runPipeline$5(RunReportDialogPresenterImpl.java:163) at com.mediasol.solar.vaadin.views.async.UIAsyncRunnerImpl.lambda$runAsync$3(UIAsyncRunnerImpl.java:65) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) at org.springframework.security.concurrent.DelegatingSecurityContextRunnable.run(DelegatingSecurityContextRunnable.java:94) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1583)

From what I can see the problem is within ByteArrayModule for jackson, which uses plain ObjectMapper (String length constraint set to 20M).

I guess hilla module for jackson was added because of the new way of server-client communication (serialization/deserialization), but it also affects my spring backend rest communication, which is not good at all. Any suggestions for this?

Thank you for quick reply

JS

Expected-behavior

No response

Reproduction

Easiest way is to create rest template from spring and call for DTO, with byte[] field where the length of bytes is over 20_000_000 (over 20M)

System Info

Vaadin 24.8.3, Spring Boot 3.5.3.

[lipisak]

Hi again,
I guess the problem is in: EndpointControllerConfiguration

It defines bean ObjectMapper, we used the default one which is overridden by this one.

[cromoteca]

Hi, thanks for reporting this.
There was some work recently to unify the ObjectMapper used across our code. I'm sorry that it impacts your project.
I suggest to either replace the default mapper by defining the factory as you saw in EndpointControllerConfiguration, or to get our default mapper and make the necessary changes on it.
Let me know if that works for you.

[lipisak]

Hi,
Actually the problem is that internal object mapper used in deserializer is limited by size of byte array. That's the only actual limitation which I can see

[cromoteca]

It is probably better for us not to mess with default limits, although that limit easily sparks discussion, see FasterXML/jackson-core#1014 for example.
In my opinion, the best solution is to customize the mapper and set the limit you need. You can either replace the mapper, or set a parameter. The cited issue has an example of how to change the limit.

[lipisak]

It would be ok, i am not sure why but with no modifications before - it worked. Which means only default spring boot confoguration object mapper. We only added two custom modules which were kept even in your object mapper.

What I mean is that part of ByteArrayDeserializer that is part of hilla module is instance of object mapper (not configured by spring boot configuration) and this one (lets call it internal mapper) has 20M limit.

[platosha]

Let us investigate why the object mapper configuration is not respected.

[lipisak]

Hello @platosha,
is there any progress regarding this issue? I have workaround by providing my own ObjectMapper, but I am not sure if the ByteArrayModule wasn't anyhow important for Vaadin itself. It's part of Hilla and we don't use Hilla at all (yet).

Thank you
Best regards
JS

[platosha]

Hello,

can confirm that the internal mapper used in ByteArrayModule is limited. We don't have a fix for this yet.

Using a workaround through own ObjectMapper seems to be a good way to go. The ByteArrayModule isn't used anywhere except for Hilla endpoints, as far as I know.