Add “NOASSERTION” to the license expression syntax

[wking]

Like #49, but for NOASSERTION instead of NONE. The semantics would be:

NOASSERTION means:
(i) the SPDX License Expression author has attempted to but cannot reach a reasonable objective determination;
(ii) the SPDX License Expression author has made no attempt to determine this field; or
(iii) the SPDX License Expression author has intentionally provided no information (no meaning should be implied by doing so).

That matches our existing usage except for PackageLicenseInfoFromFiles and similar, where we currently drop (i). I don't think those consumers would suffer from the additional case, because I don't see an actionable distinction between those cases. When would you care about the distinction between “tried but gave up”, “did not try”, and “won't tell you”? If folks did care about those distinctions (which I think unlikely), we'd want to be using different tokens for each case.

Other divergent NOASSERTION consumers are:

• SnippetLicenseConcluded, which adds an additional case:

  the SPDX document creator is uncomfortable concluding a license, despite some license information being available;

  I don't think we need to bother with that one at all, since I can't think of a case where I could distinguish between it and the “cannot reach a reasonable objective determination” case, even for license expressions I write myself. But I haven't looked up the background motivation for this case, perhaps it is useful. If so, I don't see the harm in including it for all consumers.

• LicenseName, which is completely unrelated to license expressions.

[jeffmcaffer]

Just stumbled upon these issues (#49 and #50). In ClearlyDefined we are using SPDX identifiers like crazy (that's literally all we support) and have found there are four distinct cases:

1. There is a valid SPDX identifier. Great. Use it
2. There is some sort of license-ish thing but we can't figure out what it is or it does not have an SPDX id. == NOASSERTION
3. We looked as best we could and could not find anything license related == NONE
4. Did not look. == Leave the value blank, null, whatever

In the overall compliance workflow this corresponds to

1. Great. I can analyze the license terms and carry on
2. Sigh, I need to track down that license info, break out the lawyers and figure things out
3. Dang! Really? It's 2019 and there is no license?! Better go contact the project team and see what's up
4. Hmmm, I have basic work to do

These distinctions also help in the "curation" process to let subsequent users of the data understand the work that has (or has not) been done.

As for including these in expressions, we found the need to allow for these in expressions to support expression aggregation, simplification and comparison.

For example, we find some repo that has files with a mess of different licenses. We'd want to aggregate these by AND'ing them all together. At the same time you'd want to simplify to remove redundant expression clauses. Since any given file could have NOASSERTION or NONE license value, that should be carried through to get a possible final expression like MIT AND NOASSERTION AND NONE. The end user can then understand what work they have to do (see above).

@dabutvin did some work in ClearlyDefined expression parsing and satisfaction down this path.

[kestewart]

Use case from Gary: how know part of the information, but not all, so useful to understand what still need to be done. ie. may know that 20 different license apply, but there is part un-analyzed. So this is a good way to represent. There are other ways to represent.

[kestewart]

Steve ok with if there are use cases in the practice. Key don't want to mandate it as every package has "AND NOASSERTION". We don't want to see this abused. If empty file use case discussed - and it should not require "AND NOASSERTION" at the package level. Ok to put on the file.

Note: not clear about using "OR NOASSERTION" what use case would this represent?

[jeffmcaffer]

OR NOASSERTION, at the very least comes up through typos and the like. For example, 'GPL-3.0 OR MTI. Here the MTI should have been MIT but was mistyped. If we are to mechanically reason about this it would be great to convert it to GPL-3.0 OR NOASSERTION`. That way it just flows through the system like any other expression and in the end the user sees the NOASSERTION and deals with it as described above.

[kestewart]

Moving this to be discussed for 3.0 as it may result in incompatibilities, and assigning to Steve to weigh in on.