Streaming report produces invalid JSON for multiple violations on one request

[IMSoP]

Summary

When streaming to a file ("headless mode"), if a single request has multiple violations, the log file produced doesn't have a , between entries, so is not processable as JSON.

Steps to Reproduce

1. Download the "tictactoe" OAS sample from https://learn.openapis.org/examples/v3.1/tictactoe.html
2. Run wiretap --spec tictactoe.yaml --mock-mode --report-filename tictactoe-log.json --stream-report
3. Request http://localhost:9090/board/bad/input in a browser
4. Exit wiretap
5. Open tictactoe-log.json in an editor or IDE which can parse JSON
6. Observe that the file is not valid JSON

Sample output

This is the output I get, with wiretap v0.5.0 on Windows:

[{"message":"Path parameter 'column' is not a valid integer","reason":"The path parameter 'column' is defined as being an integer, however the value 'input' is not a valid integer","validationType":"parameter","validationSubType":"path","specLine":109,"specColumn":7,"howToFix":"Convert the value 'input' into an integer","requestPath":"/board/bad/input","specPath":"/board/{row}/{column}","requestMethod":"GET","parameterName":"column","specName":"tictactoe.yaml"}{"message":"Path parameter 'row' is not a valid integer","reason":"The path parameter 'row' is defined as being an integer, however the value 'bad' is not a valid integer","validationType":"parameter","validationSubType":"path","specLine":102,"specColumn":7,"howToFix":"Convert the value 'bad' into an integer","requestPath":"/board/bad/input","specPath":"/board/{row}/{column}","requestMethod":"GET","parameterName":"row","specName":"tictactoe.yaml"}]

Note the }{ in the middle - there is no comma between the two objects in the array.

It's slightly easier to see after pretty-printing with JetBrains PhpStorm:

[
  {
    "message": "Path parameter 'column' is not a valid integer",
    "reason": "The path parameter 'column' is defined as being an integer, however the value 'input' is not a valid integer",
    "validationType": "parameter",
    "validationSubType": "path",
    "specLine": 109,
    "specColumn": 7,
    "howToFix": "Convert the value 'input' into an integer",
    "requestPath": "/board/bad/input",
    "specPath": "/board/{row}/{column}",
    "requestMethod": "GET",
    "parameterName": "column",
    "specName": "tictactoe.yaml"
  }
  {
    "message": "Path parameter 'row' is not a valid integer",
    "reason": "The path parameter 'row' is defined as being an integer, however the value 'bad' is not a valid integer",
    "validationType": "parameter",
    "validationSubType": "path",
    "specLine": 102,
    "specColumn": 7,
    "howToFix": "Convert the value 'bad' into an integer",
    "requestPath": "/board/bad/input",
    "specPath": "/board/{row}/{column}",
    "requestMethod": "GET",
    "parameterName": "row",
    "specName": "tictactoe.yaml"
  }
]

Suggestion

Rather than trying to rewrite a valid JSON array dynamically, it might be sensible to use "JSON Lines" format. This is a commonly used format for logging and streaming applications, where each line can be produced independently with a full JSON serializer.

For the example above, the log would look like this:

{"message":"Path parameter 'column' is not a valid integer","reason":"The path parameter 'column' is defined as being an integer, however the value 'input' is not a valid integer","validationType":"parameter","validationSubType":"path","specLine":109,"specColumn":7,"howToFix":"Convert the value 'input' into an integer","requestPath":"/board/bad/input","specPath":"/board/{row}/{column}","requestMethod":"GET","parameterName":"column","specName":"tictactoe.yaml"}
{"message":"Path parameter 'row' is not a valid integer","reason":"The path parameter 'row' is defined as being an integer, however the value 'bad' is not a valid integer","validationType":"parameter","validationSubType":"path","specLine":102,"specColumn":7,"howToFix":"Convert the value 'bad' into an integer","requestPath":"/board/bad/input","specPath":"/board/{row}/{column}","requestMethod":"GET","parameterName":"row","specName":"tictactoe.yaml"}

[daveshanley]

I was thinking about this the other day! JSONL is the right solution, will add this to the roadmap.