Skip to content

postcard-schema: maps with non-string keys don't work in cases that are supported by serde_json #275

@Calsign

Description

@Calsign

I'm attempting to switch a bunch of types from serializing with serde_json to postcard, with postcard-schema to support inspecting messages serialized with older schema versions.

serde_json doesn't support maps with non-string keys, and since postcard-dyn uses serde_json::Value it has this limitation as well. But serde_json does support non-string keys as long as those keys are primitive types which are trivially convertible to strings. Consider this example:

use std::collections::HashMap;

use postcard_schema::{Schema, schema::owned::OwnedNamedType};
use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize, Schema)]
struct Foobar {
    map: HashMap<u32, u32>,
}

fn main() {
    let foobar = Foobar {
        map: HashMap::from([(1, 1)]),
    };

    // this works
    let _json_encoded = serde_json::to_vec(&foobar).unwrap();

    let postcard_encoded = postcard::to_stdvec(&foobar).unwrap();
    let schema = OwnedNamedType::from(Foobar::SCHEMA);

    // this fails
    let _json_dyn = postcard_dyn::from_slice_dyn(&schema, &postcard_encoded).unwrap();
}

The serde_json path works but the postcard-dyn path doesn't. Output:

thread 'main' panicked at src/bin/ex1.rs:23:78:
called `Result::unwrap()` on an `Err` value: ShouldSupportButDont

serde_json implements serialization for all primitive types by converting them to strings, for example: https://github.com/serde-rs/json/blob/ce410dd77926181445321ce178fbc492a44328aa/src/ser.rs#L970

This is where postcard-dyn fails:

return Err(Error::ShouldSupportButDont);

Obviously full support for arbitrary non-string map keys would be ideal, but a smaller step would be for postcard-schema/postcard-dyn to not have this regression compared to serde_json. I think postcard-dyn can implement roughly equivalent functionality, but I'm not sure how difficult this is to do.

(Long-term it seems like the best solution would be to stop using serde_json::Value and use a forked version of that type with support for non-string map keys, and possibly other changes like bytearrays. I'm surprised I haven't seen this discussed elsewhere in this repo. That would add a lot of code surface to postcard-dyn so I understand not being eager to do that.)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions