Description
Describe the bug
If you include an array of values in a StructField metadata instance they will not serialize when .Json is called on the root StructType. They will however show on the fields JsonValue.
To Reproduce
Steps to reproduce the behavior:
var values = new[] { "Test1", "Test2" };
var metadata = new JObject(new JProperty("allowedValues", values));
var field = new StructField("TestField", new StringType(), true, metadata);
var fields = new StructType(new[] { field });
var json = fields.Json;
json will contain {"fields":[{"metadata":{"allowedValues":[]},"name":"TestField","nullable":true,"type":"string"}],"type":"struct"}
Expected behavior
I would expect allowedValues in the example above to contain Test1 and Test2.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
Microsoft.Spark 2.1.1
Additional context
The code that sorts the array in the serialization when you call DataType.Json does not take into account if the item is a simple string and thus returns an empty array:
spark/src/csharp/Microsoft.Spark/Interop/Ipc/JsonSerDe.cs
Lines 55 to 65 in 45d0058