Document metadata

[oliverpaddock]

Is there a plan to support complex document metadata in MNX? I know there was issue #267 that addressed why title was omitted from the spec, but I think that MNX would benefit from a higher level "document" field that contains any document metadata that doesn't have to do with the music itself, such as "title", "composer", "arranger", "lyricist", "opus", and many more potential fields. It seems logical to separate these from the rest of the music implementation and I think differentiating between the "document" and the "score" or even "global" would be a good way to do it.

Happy to do a more in-depth spec on metadata but wanted to get a better idea of what is going on with development currently first.

[adrianholovaty]

Thanks for bringing this up. We don't have anything concrete planned yet, so we'd welcome any proposals — go for it!

Just to quickly dive into the weeds, I could see this structure living within the top-level "global" object or perhaps within a brand-new top-level object. And here's some prior art from MusicXML:

https://w3c.github.io/musicxml/musicxml-reference/elements/credit/
https://w3c.github.io/musicxml/musicxml-reference/elements/work/
https://w3c.github.io/musicxml/musicxml-reference/elements/identification/

[mscuthbert]

My thought is to have a single metadata object which contains various fields (title, credit/creators list, copyright, etc.) and have multiple places where the object can be placed so that it can function as a movement title when placed at a movement/"flow" break or attribute different lyric authorship to different songs in a collection by the same composer.

For compositions like a Baroque edition with realized basso-continuo, the composer and copyright attribution can differ not just from section to section but even staff to staff within a part (for those of us making databases of only out-of-copyright music, filtering out completions, realizations, etc. that have their own copyright is critical).

I strongly suggest we also look at the prior art of our friends and colleagues in the MEI project, because Metadata (informed by music librarians) is a place where that project particularly shines. https://music-encoding.org/guidelines/v4/content/metadata.html The Humdrum/Kern specification also thought deeply about metadata (https://www.humdrum.org/reference-records/index.html) though we would certainly NOT want to use their 3-letter codes.

I think that if possible it's best to adopt an existing open standard for metadata such as Dublin Core in JSON (DCMI) or JSON-LD so that there are one-to-one mapping possibilities with other databases/library catalogs etc. This gives the possibility of using existing parsers etc. saving time in implementing metadata conversion.

[oliverpaddock]

Here's my proposal for how to add metadata to MNX. Let me know if I can clarify any points or expand on anything:

Overview

• Add a new "metadata" section to root
• Add a new "metaFields" section to score object

In order to separate the concern of metadata from the actual written music, I propose to create a new root level "metadata" object. This not only separates metadata from music, but also makes an easily scannable root object that parsers can quickly reference without scanning through the rest of the file.

Within root.metadata are two fields, "records" and "assignments".

"records" hold the actual metadata, and "assignments" determines both where this metadata applies to and what language the metadata is written in for multi-language support.

Records

There are 4 types of records: work, expression, publication, and performance, each with their own scope limitations to provide a comprehensive metadata system.

work records represent metadata for a vertical range of music such as the full score or all staves in a collection of measures. Work record metadata doesn't overlap. For instance, you can assign a title or opus to a section of music but you can't have two titles overlap.

expression records target a specific range of music, both vertically and horizontally. This allows expression properties to be applied to both parts and measures. For instance, if desired, you could provide an orchestration credit in a full score to a specific part or a range of measures or a range of measures in one or more parts. Unlike work records, expression records can overlap, although some fields may not make sense to overlap.

publication records apply to the whole file and cannot be assigned to individual ranges. This is metadata like publisher and copyright, things that probably shouldn't have multiple instances within one file of music.

performance records are similar to expression records in that they can apply to both parts and measure ranges, but separate out a more likely less used set of metadata that relates specifically to the live performance or recording of the piece of music. If for some reason you needed to link the conductor of a piece or the venue it is being performed at, this would be the place for that.

All of these records are defined as a key-value pair in root.metadata.records as a dictionary to be accessed by root.metadata.assignments.

Assignments

The assignments section of metadata links these records to the actual music. Assigning a record is relatively simple, there are only 4 fields: record, location, parts, and language.

record is the dictionary key defined in "records" of the metadata record being assigned.

location is a set of measure rhythmic position objects to determine the range the record is applicable to. An omitted location tag applies the record to all measures.

parts is an array of part names to apply the record to. An omitted parts tag applies the record to all parts.

language is the language code that the record is written in. This allows you to have multiple languages of the same record, for example a copyright in both English and German would have 2 overlapping record assignments, one with "en" and the other with "de" in the language field. An omitted language tag becomes the default language if no language is sought out or the sought out language doesn't have a record.

As mentioned before, only expression and performance records apply to both "location" and "parts". The "parts" field is just omitted or ignored for "work" records, and both the "location" and "parts" fields are omitted or ignored for "publication" records.

Scores

This now embeds metadata into the MNX file, but there's one more step: displaying metadata in a score. Usually a score will have a title, list the composers, and include the copyright at the very least. To do this, I propose adding another field, "metaFields", to the Score object. This tells the engraver which metadata fields are to be displayed in that score, which are then linked back to the assignments and records to figure out which of those fields are being used, where they are, and what their values are.

The simplest example here to add a title to an MNX score would be this:

{
	"metadata": {
		"records": {
			"my-title" : {
				"type": "work",
				"title": "My Cool New Song"
			}
		},
		"assignments": [
			{
				"record": "my-title"
			}
		]
	}
	"scores": [
		{
			"name": "My New Score",
			"metaFields": ["title"]
		}
	]
}

One thing to note about this spec is that every field across all record types has a unique name, so any record field can be referenced inline in the metaFields array to be included without worrying about ambiguity.

Some things to figure out still:

• What are all the fields in each record type?
• What happens with overlapping fields in work records? If two titles are assigned to an overlapping section, which should be used? My instinct is that whichever one comes last in the list of assignments should take precedence. This behavior should be discouraged for the most part.

Possible pitfalls:

• Unable to specify subsets of metaFields to show or hide some but not all of the same field. This could be expanded to be more comprehensive if needed and reference the global "id" field of assignments instead of field names

Record fields

Here's a list of all the fields I think are a good starting point for each type of record, but I don't think this has full coverage for every possible situation so it probably needs to be expanded/changed.

Work Record Fields

• type (required): "work" - Identifies this as a work record
• title: string - Title of the work
• subtitle: string - Subtitle of the work
• movementTitle: string - Title of the movement
• altTitle: string - Alternative title
• composer: string or array of strings - Composer(s) of the work
• lyricist: string or array of strings - Lyricist(s) of the work
• librettist: string or array of strings - Librettist(s) of the work
• dateComposed: string - Date or date range when composed
• dateCompleted: string - Date when composition was completed
• dateRevised: string - Date when work was revised
• description: string - Description of the work
• genre: string - Genre of the work
• form: string - Musical form
• style: string - Musical style
• opus: string - Opus number
• workNumber: integer - Work number within opus
• catalogNumber: string - Catalog number (e.g., "K. 550", "BWV 1006")
• movementNumber: integer - Movement number within work
• iswc: string - International Standard Musical Work Code
• textLanguage: string - RFC 5646 language code for the text
• originalTextLanguage: string - RFC 5646 language code for original text
• workType: string - Type of work (e.g., "symphony", "concerto")
• isPartOf: string - Reference to parent work
• isDerivedFrom: string - Reference to source work

Expression Record Fields

• type (required): "expression" - Identifies this as an expression record
• arranger: string or array of strings - Arranger(s) of this expression
• orchestrator: string or array of strings - Orchestrator(s) of this expression
• transcriber: string or array of strings - Transcriber(s) of this expression
• editor: string or array of strings - Editor(s) of this expression
• reviser: string or array of strings - Reviser(s) of this expression
• copyist: string or array of strings - Copyist(s) of this expression
• dateArranged: string - Date when arranged
• dateOrchestrated: string - Date when orchestrated
• dateTranscribed: string - Date when transcribed
• dateRevised: string - Date when this expression was revised
• dateEdited: string - Date when edited
• note: string - General note about this expression
• isArrangementOf: string - Reference to source work
• isTranscriptionOf: string - Reference to source expression
• isReductionOf: string - Reference to source expression
• isOrchestrationOf: string - Reference to source expression
• isRealizationOf: string - Reference to source expression
• isRevisionOf: string - Reference to previous expression
• realizationType: string - Type of realization (e.g., "figured bass", "lead sheet")

Publication Record Fields

• type (required): "publication" - Identifies this as a publication record
• datePublished: string - Date of publication
• dateCopyrighted: string - Date of copyright
• dateIssued: string - Date issued
• dateReleased: string - Date released
• ismn: string - International Standard Music Number
• doi: string - Digital Object Identifier
• publisher: string - Publisher name
• distributor: string - Distributor name
• copyright: string - Copyright statement
• license: string - License type
• licenseUrl: string - URL to license text
• usageTerms: string - Usage terms and conditions
• rightsHolder: string - Rights holder name

Performance Record Fields

• type (required): "performance" - Identifies this as a performance record
• conductor: string - Conductor name
• performer: string or array of strings - Performer(s)
• venue: string - Performance venue
• location: string - Geographic location
• ensemble: string - Ensemble name
• performanceDate: string - Date of performance
• performanceStartTime: string - Performance start time
• performanceEndTime: string - Performance end time
• programNote: string - Program notes
• eventNote: string - Event-specific notes
• productionNote: string - Production notes
• performanceType: string - Type of performance (e.g., "live", "studio")

[mscuthbert]

I once again suggest that we look at existing open standards such as JSON-LD (whose community has been quite welcoming to me) rather than creating yet another metadata standard. catalogNumber -- what happens if there are multiple catalogues? or (w/ Mozart) different numbers for different editions of the same catalogue. Languages -- two-letter-code is enough for some things, but richer codes exist for, instance, en-us and en-uk use different spellcheckers. These are corner cases that have already been thought about by others.

I do think that the contribution to score metadata we need to make is here, as Oliver suggests, is determining what parts of the document metadata applies to (as w/ the basso continuo example above)

[oliverpaddock]

Here's what I've found in my research and why I might suggest that we don't use JSON-LD or rely on an existing standard, and also how we could still use JSON-LD in my proposed model:

There are several metadata schemas that exist, like schema.org's MusicComposition or MEI or MusicXML, as well as more general schemas like Dublin Core. But schemas like MEI have their own built in complexities that (in my opinion) go against the design patterns of MNX, and schema.org and Dublin Core are not feature-complete enough to be entirely relied on. Using one of these schemas to create a one-to-one mapping from MNX would mean limiting the metadata potential to what they have to offer.

There is the option of allowing multiple schemas with JSON-LD, but this creates ambiguity in the visual representation of the music. Schema.org uses "name" for the title of a music composition, while Dublin Core uses "title". Now we need some additional abstraction to let the parser know which one to use as the title of the piece, which is much more complicated than just defining a "title" field in the first place.

Additionally, I think that JSON-LD, while being an excellent way in itself to encode linked data, goes against the rest of the syntax and practices already implemented in MNX. Aside from dictionaries with custom keys (as with global.sounds) and _x and _c, the usable field names are clearly defined throughout the MNX spec. Using JSON-LD for only the metadata section adds a completely different complexity to the system that I think we would need to consider very strongly about to use at all.

We already have to implement an MNX-unique way of defining what metadata can go where, and it's not much of a leap further to simply define what those metadata fields are in the first place. I don't think that I came up with the best or ultimate list of metadata in my spec and at this point I would narrow it down and start with something as simple as title, composer, lyricist, and genre in the work metadata record and let the discussions be about what to include in each record type.

My proposal is more of the MNX-specific implementation of metadata and how it could be assigned to the file and scores and placed in specific locations when necessary. I do think it's a more complex base system than some other formats due to the potential of MNX to represent multiple works in one file.

Maybe there's some potential here to beef up the schema.org MusicComposition fields as MNX develops and then rely on that schema entirely, and maybe JSON-LD is the way we would want to go, specifying that MNX uses schema.org's MusicComposition for its metadata. In this case I think that JSON-LD would still fit into my proposal, the title example might look like this:

{
	"metadata": {
		"records": {
			"my-title" : {
				"@context": "https://schema.org",
				"@type": "MusicComposition",
				"name": "My Cool New Song",
			}
		},
		"assignments": [
			{
				"record": "my-title"
			}
		]
	}
	"scores": [
		{
			"name": "My New Score",
			"metaFields": ["name"]
		}
	]
}

I agree that maybe we shouldn't create a whole new metadata standard, but MNX will still require its own implementation to a point, and the existing standards are either too verbose or too simple as they stand.

As for the language codes, the lyric line metadata object already specifies using RFC 5646 which supports language code extensions, so that's what I would imagine would be used here as well.

[samuelbradshaw]

I like the idea of improving the Schema.org or JSON-LD schema to include what we need.
Since MNX is intended to be primarily a compressed format, should we consider breaking out separate scores and/or metadata into separate files within the compressed archive?

[ahankinson]

A note: JSON-LD is not a metadata format. It's a way of applying a context to JSON such that the JSON can be transformed into RDF (or any other Linked Data format.) It's a metadata format in the same way that "JSON" or "XML" are a metadata format. (In fact, it's probably closest in concept to "XML with a specified XSLT".)