Unexpected InvalidTypeIdException when parsing XML to POJO containing nested List<> , custom `TypeIdResolver`

[ankagar]

XML Parsing fails with the complain that type id is missing, even when it is defined in the input xml

When 'uid' attribute is completely removed from the input xml, then the parsing works.

Java Code where problem can be reproduced with jackson-dataformat-xml 2.11.2

package com.temp;

import java.io.IOException;
import java.util.List;

import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.annotation.JsonTypeInfo;
import com.fasterxml.jackson.databind.DatabindContext;
import com.fasterxml.jackson.databind.JavaType;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.annotation.JsonTypeIdResolver;
import com.fasterxml.jackson.databind.jsontype.impl.TypeIdResolverBase;
import com.fasterxml.jackson.dataformat.xml.XmlMapper;
import com.fasterxml.jackson.dataformat.xml.annotation.JacksonXmlElementWrapper;
import com.fasterxml.jackson.dataformat.xml.annotation.JacksonXmlProperty;

public class AutoParser {
    public static void main(String[] args) throws Exception {

        String xml = "" +
                "<Auto>\n" +
                "    <Object uid=\"1\" type=\"Engine\">\n" +
                "        <Object uid=\"2\" type=\"Chassis\"></Object>\n" +
                "        <Object uid=\"3\" type=\"Motor\"></Object>\n" +
                "    </Object>\n" +
                "    <Object uid=\"4\" type=\"Body\"></Object>\n" +
                "</Auto>";

        XmlMapper xmlMapper = new XmlMapper();
        Auto auto = xmlMapper.readValue(xml, Auto.class);
        xmlMapper.configure(SerializationFeature.INDENT_OUTPUT, true);
        System.out.println(xmlMapper.writeValueAsString(auto));
    }
}

class Auto {

    @JacksonXmlProperty(localName = "Object")
    @JacksonXmlElementWrapper(useWrapping = false)
    @JsonInclude(JsonInclude.Include.NON_EMPTY)
    List<CarParts> carParts;
}

@JsonTypeIdResolver(CarPartsResolver.class)
@JsonTypeInfo(use = JsonTypeInfo.Id.CUSTOM, include = JsonTypeInfo.As.PROPERTY, property = "type")
abstract class CarParts {

    @JacksonXmlProperty(isAttribute = true) // The is triggering the issue.
    String uid;

    @JacksonXmlProperty(localName = "Object")
    @JacksonXmlElementWrapper(useWrapping = false)
    @JsonInclude(JsonInclude.Include.NON_EMPTY)
    List<CarParts> carParts;
}

class Engine extends CarParts{}
class Chassis extends CarParts{}
class Motor extends CarParts{}
class Body extends CarParts{}

class CarPartsResolver extends TypeIdResolverBase {

    private JavaType superType;

    @Override
    public void init(JavaType javaType) {
        this.superType = javaType;
    }

    @Override
    public String idFromValue(Object o) {
        return idFromValueAndType(o, o.getClass());
    }

    @Override
    public String idFromValueAndType(Object o, Class<?> aClass) {
        return aClass.getSimpleName();
    }

    @Override
    public JavaType typeFromId(DatabindContext context, String id) throws IOException {
        Class<?> subType = null;
        switch (id) {
            case "Engine": subType = Engine.class; break;
            case "Chassis": subType = Chassis.class; break;
            case "Motor": subType = Motor.class; break;
            case "Body": subType = Body.class; break;
        }
        return context.constructSpecializedType(superType, subType);
    }

    @Override
    public JsonTypeInfo.Id getMechanism() {
        return JsonTypeInfo.Id.CUSTOM;
    }
}

Stacktrace

Exception in thread "main" com.fasterxml.jackson.databind.exc.InvalidTypeIdException: Missing type id when trying to resolve subtype of [simple type, class com.temp.CarParts]: missing type id property 'type' (for POJO property 'Object')
 at [Source: (StringReader); line: 3, column: 40] (through reference chain: com.temp.Auto["Object"]->java.util.ArrayList[0]->com.temp.Engine["Object"]->java.util.ArrayList[0])
	at com.fasterxml.jackson.databind.exc.InvalidTypeIdException.from(InvalidTypeIdException.java:43)
	at com.fasterxml.jackson.databind.DeserializationContext.missingTypeIdException(DeserializationContext.java:1794)
	at com.fasterxml.jackson.databind.DeserializationContext.handleMissingTypeId(DeserializationContext.java:1323)
	at com.fasterxml.jackson.databind.jsontype.impl.TypeDeserializerBase._handleMissingTypeId(TypeDeserializerBase.java:303)
	at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedUsingDefaultImpl(AsPropertyTypeDeserializer.java:166)
	at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:107)
	at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:254)
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:293)
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:250)
	at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:27)
	at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:138)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:293)

PS: Above code is able to parse following XMLs

<Auto>
    <Object type="Engine">
        <Object type="Chassis"></Object>
        <Object type="Motor"></Object>
    </Object>
    <Object type="Body"></Object>
</Auto>

<Auto>
    <Object uid="1" type="Engine"></Object>
    <Object uid="4" type="Body"></Object>
</Auto>

but fails on

<Auto>
    <Object uid="1" type="Engine">
        <Object uid="2" type="Chassis"></Object>
        <Object uid="3" type="Motor"></Object>
    </Object>
    <Object uid="4" type="Body"></Object>
</Auto>

[cowtowncoder]

Thank you for reporting this issue. I hope to look into this in near future.

[ankagar]

For the sake of completeness, I also tried parsing the above with 2.12.0-SNAPSHOT
(Implementation-Build-Date: 2020-09-30 21:05:13+0000)

The exception is not thrown, but the parsed POJO drops the nested List<>, resulting parsed xml is:

<Auto>
  <Object type="Engine" uid="1"/>
  <Object type="Body" uid="4"/>
</Auto>

[cowtowncoder]

@ankagar thank you for additional verification of 2.12.0-SNAPSHOT since there are many bug fixes since 2.11 (more than most minor releases).

[cowtowncoder]

Yes, I can reproduce this now with 2.12: as per earlier note will not throw exception but manages to lose content which is even worse.

NOTE: if order of uid and type attributes is changed (that is, Type Id attribute type comes first), test passes -- this suggest buffering plays a role.

[cowtowncoder]

Looking at this now... I think the problem is that code assumes that all attributes of an element that starts a Collection valued property need to be skipped (see issue #33) and that is part of the problem. But I think that this skipping should only occur for "wrapped" List case, and not for unwrapped: in latter case, element is for items which can have relevant attributes as is the case here.
Will need to see if there is a way to separate handling, and if so, resolve this problem.

[cowtowncoder]

I think that part of the problem is skipping of attributes within FromXmlParser.isExpectedStartArrayToken() as mentioned above, but it also appears that clearing of _nextToken plays a part. Have not been able to figure out logic to solve the first part much less rest of it: initial guess of checking for wrapping not working, at least not yet.

[cowtowncoder]

Looks like part of this is buffering: switching order of type and uid for first 2 elements actually makes test pass.
This is probably since buffering with TokenBuffer will by-pass all XML specific logic of FromXmlParser -- a known potential problem, but in this case actual problem it appears.

Will give up for now, hope to address eventually.