Specific extension for raw mnx json files

[rpatters1]

I read somewhere either here or on the email group that the plan for the .mnx extension is to be a standard zip archive along the lines of .mxl that contains the raw json file along with other standardized items. For now, it seems, the raw json simply uses the the .json extension.

I am wondering if we should adopt an mnx-specific extension for raw json files, similar to the way raw music xml files are .musicxml rather than .xml. Some ideas I had were:

• musicjson (This is cumbersome, but it parallels musicxml which is also cumbersome.)
• mnjson
• mnj

Other options are available.

[mscuthbert]

I go back and forth on this -- one of the nice things about having your own extension is that double-click-to-open works, filtering by "appropriate files" works, etc. The downside of having your own extension is that tools that focus on JSON don't work as well -- so unless an editor knows that musicjson should be edited with .json tools (unlikely) you don't get the automatic syntax checking that a file extension helps with.

My feeling is slightly -1, because having its own extension will encourage (non-devs) to treat the uncompressed .json as an exchange format, and then Qs about uncompressed file size will emerge. Uncompressed file size is not a primary concern of the spec team (compressed file size is).

If we did have an extension, I would go with mnxjson -- .musicjson isn't directly analogous to .musicxml since MusicXML is the name of the standard while musicjson is not.

[rpatters1]

If we are going to adopt json as an approved raw extension for mnx, the specification should provide a quick and easy way for a consuming program to determine if this is, in fact, an mnx json file. I realize there are ways to do it now, including validating it against the schema. But a documented less intensive method would be helpful because:

• the consuming program may not have easy access to a fully featured schema validator.
• relying on the schema would require every consuming program to keep its copy of the schema up to date (or add the machinery to download it on the fly).
• the consuming program may wish to attempt to process the file anyway, even if the file does not fully validate.
• simply looking for top-level required branches such as mnx or global might be sufficient, but it would be better if an approved algorithm were documented.

[mscuthbert]

We discussed this at a steering meeting today and would like to propose this.

• raw uncompressed json for an mnx file have the double extension .mnx.json - for instance Symphony no. 2.mnx.json.
  • This will allow .mnx.json files to be opened with appropriate JSON tooling in most IDEs and Finder/Window-Managers
  • filename.endsWith('.mnx.json') or equivalent tools should be sufficient to choose a parsing tool for software that supports multiple formats. In Python pathlib.Path(filename).suffixes[-2:] == ['.mnx', '.json'] also works. Tools to match this should be available in nearly every language.
• For data/files arriving without a filename, we feel that the following should be sufficient to determine that a file is or intends to be uncompressedmnx data: (1) it parses as JSON, (2) it contains a key named mnx. (3) that key is an object (dictionary) which contains a key named version.
  • Of course it is possible that someone can construct a file that meets these criteria that is not in fact MNX, but the same is true for nearly every format. This should suffice to prevent accidental parsing of other JSON as MNX.
• The suffix .mnx (without .json after it) will be reserved for a compressed form of MNX still to be determined. It should not be used for now.
  • We welcome discussion on whether this should be a container format (in the form of MusicXML's .mxl or OpenDocument Format) or if MNX's ability to store multiple layouts makes this unnecessary.
  • If it is to be a container format, we welcome discussion on whether to use MusicXML or ODF's existing specifications (exactly or with tweaks) or create something new.
  • The advantage of using MusicXML's is that it works and has many implementations and is part of the MusicXML system that no one on the steering group feels needs updating.
  • The drawback is the mixture of encodings: writing container.xml XML files to specify the locations of JSON documents has a slight ick factor, and brings in security issues from two different systems.

[mscuthbert]

While writing that message, some things that we did not discuss at the meeting occur to me, so I'll put them here as my thoughts, not endorsed (nor rejected) by any consensus:

• MNX also needs media types (aka MIME types / file types / content-type).
  • Uncompressed MNX should be application/mnx+json per RFC 6839 to match things like Linked-Data's application/ld+json.
  • Depending on how the compressed/container format of MNX ends up being, it will probably be application/mnx+zip.
  • MusicXML predates RFC3023 (which established the application/X+Y types), so their media types application/vnd.recordare.musicxml and application/vnd.recordare.musicxml+xml are not exactly like this. However the community consensus (including Michael Good's) was that MusicXML should have been moved to application/musicxml+zip and application/musicxml+xml, but changing an existing IANA Media Type is very hard.
  • Media types only become official when registered with IANA. Currently not only is application/mnx available, no other media type contains the three-character string mnx anywhere.
• MNX JSON for exchange must be encoded as UTF-8 (per RFC8259).
  • Systems that are using MNX internally for state are, of course, allowed to use UTF-16 or UTF-32 (BE or LE) for their own purposes. But writing it out for exchange as anything but UTF-8 is an encoding error.

I don't imagine these to be controversial, but they just haven't been discussed outside my head (where I encode everything as UTF-9-Half-Width+Shift-JIS-BELEBE-Petscii-ISO-8859-1½-with-BOM-except-on-Tuesdays; so translation errors might take place)

[rpatters1]

I'm pretty sure at least some operating systems do not handle double extensions. If there is a way to set up an association for .mnx.json separate from .json in macOS, I don't see it. There are myriad uses for double extensions that have nothing to do with file type. This could create a mess.

For that reason, I think this approach is unwise. But perhaps it is mostly a moot point if we come up with an archival format to use with .mnx.

[rpatters1]

Also, if this is going to be adopted, the example files should renamed.