How to create a symmetrical JSON class for reading and writing that is intuitive with [JsonExtensionData]?

[Swimburger]

We're experimenting with how we can provide the most intuitive, developer friendly experience for reading and writing additional properties for OpenAPI and are struggling to use [JsonExtensionData].

The ideal experience for end users would be

• As intuitive as creating an anonymous object or Dictionary<string, object?>
• Symmetrical, in both the signature of the property, but ideally also the concrete types
• The output is consistent whether the JSON number is short or big.

As it is today, there are three signatures to use with [JsonExtensionData]:

• IDictionary<string, JsonElement>:
  • Positive: This is great for reading properties and forcing the end user to very explicitly and verbosely pick the data type. For example, from a JSON number, you can't really predict whether a property will consistently return an int, double, decimal, float, etc, and the JsonElement forces the user to explicitly make that decision, fe JsonElement.GetUInt16().
  • Negative: To set the value you have to create a JsonElement which isn't intuitively created and I assume has poor performance characteristics, fe: recordForReading.AdditionalProperties["author"] = JsonDocument.Parse("\"J.R.R. Tolkien\"").RootElement
• IDictionary<string, object?>:
  • Positive: This is as intuitive as it can possibly be in .NET to write additional properties without needing to create JsonNode's or JsonElement's. Just put in data into the value and the JSON serializer takes care of properly writing it.
  • Negative:
    • When you're creating the additional properties, and immediately read them, the values have the original types you gave them, but when it is serialized and deserialized, the values are JsonElement's. Maybe this isn't actually a problem, but I'm worried this will surprise our users.
    • When deserializing JSON, the user sees object but the concrete type is always JsonElement which seems unintuive/deceiving. I would personally naively check for is string or is int etc.
• JsonObject:
  • Positive:
    • It's almost as intiutive as Dictionary<string, object?> because simple values are implicitly cast to JsonNode's, fe: recordSymmetrical.AdditionalProperties["author"] = "J.R.R. Tolkien".
    • The property is consistent and predictable a JsonObject.
  • Negative:
    • Some types can't be implicitly casted, fe:

      string[] tags = ["fantasy", "adventure"];
      recordSymmetrical.AdditionalProperties["tags"] = JsonValue.Create(tags);

    • JsonObject with ExtensionData cannot be used to write values currently, because of an open bug.

IMO, the best experience for reading is IDictionary<string, JsonElement> and JsonObject, while the best writing experience is IDictionary<string, object?>, and the best of both worlds is JsonObject if the bug were to be resolved.

additionalProperties in OpenAPI, the equivalent of ExtensionData is becoming more common needed with us gaining traction in the AI and embedding space (we work with Pinecone on their .NET SDK).

---

FYI, I wrote a little sample that plays with these different signatures here, and they have indexers to make it more intuitive.

For context, we generate code based on multiple spec formats (OpenAPI/AsyncAPI/proto) and merge them into intuitive SDKs, and I am personally invested in giving .NET devs the best SDKs possible balancing developer experience and performance. How would you recommend providing intuitive classes with extension data that can be used for both writing and reading?

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

[jeremyfiel]

Something to consider is the specification you are generating for may have different requirements for the underlying JSON Schema draft version and the behavior of the keyword used.

• OAS 3.0.x uses a subset/superset of JSON Schema Draft-04 and additionalProperties
• Async 2/3 use JSON Schema Draft-07 and additionalProperties
• OAS 3.1.x uses JSON Schema Draft 2020-12 and unevaluatedProperties which has slightly different behavior than additionalProperties.

I suppose the use of the different keyword for creating types has a different expectation than data validation where the behavior difference may be more pronounced, but it's something to be aware of.

[Swimburger]

• OAS 3.1.x uses JSON Schema Draft 2020-12 and unevaluatedProperties which has slightly different behavior than additionalProperties.

We're not concerned with validation at the moment, but that is good to know, thank you for bringing it up.

[Swimburger]

I can provide my own class and provide some helper methods which could make this more intuitive.

public record AdditionalProperties : AdditionalProperties<object?>;

public record AdditionalProperties<T> : IDictionary<string, T>
{
    private readonly Dictionary<string, T> _dictionary = new();

    public IDictionary<string, JsonElement> ToJsonElementDictionary(){...}
}

I kinda like this better, but still feel unsure if this is the best DX users should be getting.

[eiriktsarpalis]

When you're creating the additional properties, and immediately read them, the values have the original types you gave them, but when it is serialized and deserialized, the values are JsonElement's. Maybe this isn't actually a problem, but I'm worried this will surprise our users.

This is by design. TL;DR roundtripping object while preserving full type fidelity requires unchecked polymorphic deserialization which is a vector for remote code execution. While we could bake in shape recognition for basic primitives like int, bool or string it becomes hard to draw the line (e.g. why not extend to int[], or bool[][]). If you try throw in other common types like DateTimeOffset, double, or Guid it opens up a whole new can of worms. Who's to say if the JSON value 42 was intended to be handled as decimal, short, or double? Was that "2025-04-04" JSON string supposed to be DateTime, DateTimeOffset, DateOnly or just string? Json.NET has been doing quite a lot of that inference and it is known to create problems. The team decided at the time that it's best to keep things simple.

There are a few ways behaviour can be configured: either change the underlying representation using the JsonSerializerOptions.UnknownTypeHandling property or introduce a custom JsonConverter<object> that handles object deserialization in a desired manner.

JsonObject with ExtensionData cannot be used to write values currently, because of #97225.

Have you tried using Dictionary<string, JsonNode?> instead? This is the recommended way for attaching additional properties using JsonNode representations.

[Swimburger]

That's good to know. Thank you for clarifying that angle.
I'll try to figure out a way to make writing and reading a different experience so writing is as flexible as possible, but reading forces you to use JsonElement and the type is explicit and clear.
I think a custom converter may be necessary. Is there an example for replicating extension data in a custom converter? Does it play well with JsonExtensionData?

When I try using Dictionary<string, JsonNode?> or Dictionary<string, JsonNode>, I get this exception when trying to serialize:

The extension data property 'System.Collections.Generic.IDictionary`2[System.String,System.Text.Json.Nodes.JsonNode].AdditionalProperties' is invalid. It must implement 'IDictionary<string, JsonElement>' or 'IDictionary<string, object>', or be 'JsonObject'.

[gregsdennis]

@Swimburger I've written a POC project called Graeae to model OpenAPI descriptions using my JsonSchema.Net. it may not do exactly what you're looking for, but maybe you can find some inspiration there.

[Swimburger]

We figured out a pretty good solution, but we would love to get some feedback on it. Hopefully, this will help folks who are facing the same challenges working with OAS.

We have to generic classes, AdditionalProperties and ReadOnlyAdditionalProperties.
For unknown types, the type would be AdditionalProperties<object?> and ReadOnlyAdditionalProperties<JsonElement>.
For known types, it'll look like AdditionalProperties<int> and ReadOnlyAdditionalProperties<int>

The writeable additional properties are optimized for the writing experience, but AdditionalProperties has helper methods to help with making reading easier like

• ToJsonObject()
• ToJsonNode()
• ToJsonElement()
• ToJsonDocument()
• ToJsonElementDictionary()

These helper methods convert the extensions data directly to the respective types without making unnecessary conversions that target type (type param).
I could see us adding a Get<T>(string key) and TryGet<T>(string key) in the future, but we're leaving that out for now.

Both classes implement IDictionary<string, T> so users can interact with it as if it is a dictionary.

Our generated classes look like this:

public record RecordWithAdditionalPropertiesInts : IJsonOnDeserialized, IJsonOnSerializing
{
    // ... normal props ...
    
    [JsonExtensionData]
    private readonly IDictionary<string, object?> _extensionData = new Dictionary<string, object?>();

    [JsonIgnore] public AdditionalProperties<int> AdditionalProperties { get; } = new();

    void IJsonOnDeserialized.OnDeserialized()
    {
        AdditionalProperties.CopyFromExtensionData(_extensionData);
    }

    void IJsonOnSerializing.OnSerializing()
    {
        AdditionalProperties.CopyToExtensionData(_extensionData);
    }
}

public record RecordWithReadonlyAdditionalPropertiesInts : IJsonOnDeserialized
{
    // ... normal props ...
    
    [JsonExtensionData]
    private readonly IDictionary<string, JsonElement> _extensionData = new Dictionary<string, JsonElement>();

    [JsonIgnore] public ReadOnlyAdditionalProperties<int> AdditionalProperties { get; } = new();

    void IJsonOnDeserialized.OnDeserialized()
    {
        AdditionalProperties.CopyFromExtensionData(_extensionData);
    }
}

To avoid said security vector

TL;DR roundtripping object while preserving full type fidelity requires unchecked polymorphic deserialization which is a vector for remote code execution.

We avoid roundtripping by only converting to the desired type when the user requests it but not actually modifying the JSON extension data. This makes the implementation of AdditionalProperties and ReadOnlyAdditionalProperties a little more complicated, but it gives the user the optimal DX IMO. There's probably a performance hit, but we tried to minimize it.

Here's some examples of the usage:

var recordOfUnknown = new RecordOfUnknown
{
    // AdditionalProperties<object?>
    AdditionalProperties =
    {
        ["extra1"] = new { value = 42 },
        ["extra2"] = DateTime.Now,
        ["extra3"] = 99
    }
};

var recordOfBoolDictionaries = new RecordOfBoolDictionaries
{
    // AdditionalProperties<Dictionary<string, bool>>
    AdditionalProperties =
    {
        ["extra1"] = new Dictionary<string, bool>
        {
            ["key1"] = true,
            ["key2"] = false
        },
        ["extra2"] = new Dictionary<string, bool>
        {
            ["key1"] = true,
            ["key2"] = false
        },
    }
};

var recordOfInts = new RecordOfInts
{
    // AdditionalProperties<int>
    AdditionalProperties =
    {
        ["extra1"] = 42,
        ["extra2"] = 99
    }
};

object? extra1Value = recordOfUnknown.AdditionalProperties["extra1"]; // is an anonymous object, but on deserialization will be JsonElement
object? extra2Value = recordOfUnknown.AdditionalProperties["extra2"]; // is DateTime, but on deserialization will be JsonElement
object? extra3Value = recordOfUnknown.AdditionalProperties["extra3"]; // is 99, but on deserialization will be JsonElement
var jsonElements = recordOfUnknown.AdditionalProperties.ToJsonElementDictionary();
JsonElement extra1Element = jsonElements["extra1"];
JsonElement extra2Element = jsonElements["extra2"];
JsonElement extra3Element = jsonElements["extra3"];

bool extra1Key1 = recordOfBoolDictionaries.AdditionalProperties["extra1"]["key1"];
bool extra1Key2 = recordOfBoolDictionaries.AdditionalProperties["extra1"]["key2"];
bool extra2Key1 = recordOfBoolDictionaries.AdditionalProperties["extra2"]["key1"];

int extra1Int = recordOfInts.AdditionalProperties["extra1"];
int extra2Int = recordOfInts.AdditionalProperties["extra2"];

Here's the implementation of the additional properties classes:
https://github.com/Swimburger/AdditionalProperties/blob/54ef0ecc5a48822b62ecc6353fc12c871396cd0a/AdditionalProperties/AdditionalProperties.cs

It'd be great if we could tighten the code up more with a converter rather than using the JSON serialization callbacks, but this will do for now.

[eiriktsarpalis]

We avoid roundtripping by only converting to the desired type when the user requests it but not actually modifying the JSON extension data.

That should be fine. It only becomes a security problem when you start embedding type information on the JSON payload. For more information see https://github.com/pwntester/ysoserial.net.