JSON isn't being escaped correctly in at least one case

[ross-spencer]

Given the signature file sig1.xml:

NB. note the format name: Development\Signature

<?xml version="1.0" encoding="UTF-8"?>
<FFSignatureFile xmlns="http://www.nationalarchives.gov.uk/pronom/SignatureFile" Version="1767086900" DateCreated="2025-12-30T09:28:20+00:00">
  <InternalSignatureCollection>
  <InternalSignature ID="1" Specificity="Specific">
  <ByteSequence Reference="BOFoffset">
    <SubSequence MinFragLength="0" Position="1" SubSeqMaxOffset="0" SubSeqMinOffset="0">
      <Sequence>68656C6C6F</Sequence>
      <DefaultShift>6</DefaultShift>
      <Shift Byte="68">5</Shift>
      <Shift Byte="65">4</Shift>
      <Shift Byte="6C">2</Shift>
      <Shift Byte="6F">1</Shift>
    </SubSequence>
  </ByteSequence>
</InternalSignature>
</InternalSignatureCollection>
<FileFormatCollection>
  <FileFormat ID="1" Name="Development\Signature" PUID="dev/1" Version="1.0" MIMEType="application/octet-stream">
    <InternalSignatureID>1</InternalSignatureID>
    <Extension>ext</Extension>
  </FileFormat>
</FileFormatCollection></FFSignatureFile>

Stored in /tmp build with roy:

roy build --noreports -extend /tmp/sig1.xml

Identify the file hello.txt:

hello

We get valid YAML:

---
siegfried   : 1.11.2
scandate    : 2025-12-30T10:32:54+01:00
signature   : default.sig
created     : 2025-12-30T10:29:59+01:00
identifiers : 
  - name    : 'pronom'
    details : 'DROID_SignatureFile_V120.xml; container-signature-20240715.xml; built without reports; extensions: /tmp/sig1.xml'
---
filename : 'hello'
filesize : 6
modified : 2025-12-30T10:28:45+01:00
errors   : 
matches  :
  - ns      : 'pronom'
    id      : 'dev/1'
    format  : 'Development\Signature'
    version : '1.0'
    mime    : 'application/octet-stream'
    basis   : 'byte match at 0, 5'
    warning : 'extension mismatch'

But, we don't get valid JSON: sf -json hello

{"siegfried":"1.11.2","scandate":"2025-12-30T10:33:57+01:00","signature":"default.sig","created":"2025-12-30T10:29:59+01:00","identifiers":[{"name":"pronom","details":"DROID_SignatureFile_V120.xml; container-signature-20240715.xml; built without reports; extensions: /tmp/berg/sig1.xml"}],"files":[{"filename":"hello","filesize": 6,"modified":"2025-12-30T10:28:45+01:00","errors": "","matches": [{"ns":"pronom","id":"dev/1","format":"Development\Signature","version":"1.0","mime":"application/octet-stream","basis":"byte match at 0, 5","warning":"extension mismatch"}]}]}

JQ reporting the following:

jq: parse error: Invalid escape at line 1, column 456

This is simply due to the \ in Development\Signature.

We have an easy enough workaround, we can remove the slash. In JSON to escape this properly we need to recognize it and identify the slash, and then use \\.

Via: https://www.json.org/json-en.html

We need the following escaped:

escape
'"'
'\'
'/'
'b'
'f'
'n'
'r'
't'
'u' hex hex hex hex

We'd need to look which ones are missing from the JSON formatter. I can write test cases given a spare moment. (Alternatively, consider switching to the default tooling?)

[richardlehane]

Hey @ross-spencer thanks for reporting this and happy new year!

The JSON writer (https://github.com/richardlehane/siegfried/blob/main/pkg/writer/writer.go) only does escaping for the file name field at present, but this could be extended to the identifier fields too.

Rationale for not escaping strings from the identifier fields was that I considered these as trusted sources (i.e. from PRONOM, LOC), that generally don't contain characters that need escaping, and so didn't warrant that slight performance hit.

But I didn't consider the case you've hit of a backslash (forward slash?) inserted via roy.

I don't use the std JSON library in current implementation, a bit hacky and just do manual escaping via strings replacer, but would be interesting to benchmark the difference.

[ross-spencer]

Thanks Richard and Happy New Year as well!

I have a few things on the go but happy to try benchmarking the standard serialization libs when I have a moment.