[Bug] Introspect not working with `flatten` and (possibly) `serde_with`

[Dzordzu]

Description

Structs with flattened argument are improperly introspected. Any introspection of such a struct results in the empty array. Cannot determine if serde_with causes unexpected behaviors, because it's dependent on the flatten

Unresolved issues

• flatten working as expected
• verify that serde_with prefix is working as expected

Example

ser.rs

// ser.rs

use serde::{Serialize, Deserialize};

serde_with::with_prefix!(my "my_)");

#[derive(Debug, Serialize, Deserialize)]
pub struct B {
    pub b: String,
    #[serde(rename="z")]
    pub b2: usize
}

#[derive(Debug, Serialize, Deserialize)]
pub struct A {
    pub a: usize,
    #[serde(flatten, with="my")]
    pub bx: B
}

#[derive(Debug, Serialize, Deserialize)]
pub struct A0 {
    pub a: usize,
    pub bx: B
}

#[derive(Debug, Serialize, Deserialize)]
pub struct A1 {
    #[serde(with="my")]
    pub a: usize
}

#[derive(Debug, Serialize, Deserialize)]
pub struct A2 {
    #[serde(flatten)]
    pub bx: B
}

#[derive(Debug, Serialize, Deserialize)]
pub struct A3 {
    #[serde(with="my")]
    pub bx: B
}

main.rs

// main.rs

use serde_aux::prelude::*;
mod ser;

fn main() {
    let b = ser::B {
        b: "AA".into(),
        b2: 22,
    };

    let a = ser::A {
        a: 2,
        bx: b.clone()
    };

    let a0 = ser::A0 {
        a: 2,
        bx: b.clone()
    };

    let _a1 = ser::A1 {
        a: 2,
    };

    let a2 = ser::A2 {
        bx: b.clone()
    };

    let a3 = ser::A3 {
        bx: b.clone()
    };

    println!("A = \t{}", serde_json::to_string_pretty(&a).unwrap());
    println!("A0 = \t{}", serde_json::to_string_pretty(&a0).unwrap());
    println!("A2 = \t{}", serde_json::to_string_pretty(&a2).unwrap());
    println!("A3 = \t{}", serde_json::to_string_pretty(&a3).unwrap());

    println!("A = \t{:?}", serde_introspect::<ser::A>());
    println!("A0 = \t{:?}", serde_introspect::<ser::A0>());
    println!("A1 = \t{:?}", serde_introspect::<ser::A1>());
    println!("A2 = \t{:?}", serde_introspect::<ser::A2>());
    println!("A3 = \t{:?}", serde_introspect::<ser::A3>());
}

stdout

# Output

A =     {
  "a": 2,
  "my_)b": "AA",
  "my_)z": 22
}
A0 =    {
  "a": 2,
  "bx": {
    "b": "AA",
    "z": 22
  }
}
A2 =    {
  "b": "AA",
  "z": 22
}
A3 =    {
  "bx": {
    "my_)b": "AA",
    "my_)z": 22
}

A =     []          # Wrong
A0 =    ["a", "bx"] # Control sample. This is always ok
A1 =                # Actually panics, my bad
A2 =    []          # Wrong
A3 =    ["bx"]      # Actually ok, my bad

[iddm]

Hi! Thank you for filing your issue! I am not aware of how it should work together (not usually use serde_with), can you please provide the examples with the expected results?

[Dzordzu]

Sure. Here you go. I've edited my bug report to include jsons

[Dzordzu]

So it MAY work with serde_with prefixing, but it certainly fails with the flattened structs

[iddm]

I have just noticed you use the serde_introspect crate, which is not a part of serde_aux. Also, I couldn't see any use of serde_aux here except for the use statements, which do not do anything independently. Perhaps you meant to file an issue in the serde_introspect crate instead?

[Dzordzu]

Que? There is no such a crate... Also https://github.com/iddm/serde-aux/blob/master/src/serde_introspection.rs#L35

I couldn't see any use of serde_aux here except for the use statements, which do not do anything independently

I literally took it from this repo. See docs

[iddm]

Que? There is no such a crate... Also https://github.com/iddm/serde-aux/blob/master/src/serde_introspection.rs#L35

I couldn't see any use of serde_aux here except for the use statements, which do not do anything independently

I literally took it from this repo. See docs

Oh, I apologise! It wasn't me who added it, and I had completely forgotten about its existence.

The serde attributes often mess up with everything else. It has been long known, and unfortunately, not many things can be fixed. From what I can tell right now, it seems that the problem is introduced by using the serde's flatten attribute. Additionally, the serde_with might be breaking things another way. It is going to require research, which I unfortunately have no time for. @lucatrv would you like to take a look at what might cause a problem here?

[Dzordzu]

It wasn't me who added it,

No worries, it's the beauty of the OS projects ;)

@lucatrv would you like to take a look at what might cause a problem here?

Thanks a lot!

And obviously, thanks for your effort in maintaining this project!

[Iizuki]

I too ran into this. I'm using the introspection to formulate DB queries, specifically what fields to fetch.

In this usecase it would be enough to just be able to tell the the presence of a flattened map ("catch the rest field"), as it can be interpreted as get all fields. E.g. SELECT * FROM ...

PS. Despite little hiccups like this, this is still a really helpful feature!

[iddm]

I too ran into this. I'm using the introspection to formulate DB queries, specifically what fields to fetch.

In this usecase it would be enough to just be able to tell the the presence of a flattened map ("catch the rest field"), as it can be interpreted as get all fields. E.g. SELECT * FROM ...

PS. Despite little hiccups like this, this is still a really helpful feature!

Hey there, thanks for jumping in! The procedural macro can always peek into the fields and iterate over them, certainly checking for the "rest" field. However, there are always cases when there might be two or more such map fields. I haven't yet taken a look at the code of this introspection module, I am on a few weeks of leave right now, but I think I did something like that in the past.