parse_json in VRL doesn't support json-like strings (with single quotes)

[nomalord]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

I have encountered an issue with parsing json strings with single quotes:
the following VRL expression:

.bad = parse_json!(.bad)

parsing the following event:

{
    bad: "{'test': 'hello'}"
}

returns an error saying the json is invalid...

I also tested in the VRL playground
Please help :)
Thanks in advance

https://playground.vrl.dev/?state=eyJwcm9ncmFtIjoiLmJhZCA9IHBhcnNlX2pzb24hKC5iYWQpIiwiZXZlbnQiOm51bGwsImlzX2pzb25sIjpmYWxzZSwiZXJyb3IiOiJDb3VsZCBub3QgcGFyc2UgSlNPTiBldmVudDpcbntcbiAgICBiYWQ6IFwieyd0ZXN0JzogJ2hlbGxvJ31cIlxufSJ9

Configuration

Version

0.38

Debug Output

Example Data

No response

Additional Context

No response

References

No response

[pront]

Per https://www.json.org/json-en.html your JSON is invalid, double quotes are needed. Some other parsers might do extra processing to replace the ' with " hence the confusion. Internally, we are just delegating to serde_json as you can see here.

[jszwedko]

@nomalord What is generating that data? As @pront pointed out that isn't valid JSON and so I don't think parse_json should support parsing it, but maybe it is some related format that would support parsing via a different function?

[nomalord]

When writing a nested dictionary to a csv file using python, the values look something like:

next,is,dict,"{'test_field': 'test_value'}"

I used an enrichment table and wanted to parse the dict string to a json in VRL but got the aforementioned error