Incorrect Allowed @name values for @prop in documentation

[iMichaela]

Describe the bug

CROSS-REFERENCE BUG, INITIALLY: OSCAL-Reference/issue#6

There are many places in the OSCAL documentation lists incorrect values for the @name attribute on the prop field.

For example, prop[@name='marking'] is only supposed to be valid in the //metadata of each model; however, it is also incorrectly listed in the documentation as valid in many other places.

A search of the Catalog documentation shows eight additional occurrences of prop[@name='marking'] in places such as:

//metadata/revisions/prop
//metadata/role/prop
//metadata/location/prop
//metadata/party/prop
//metadata/responsible-party/prop
//param/prop (root, group, and control levels)
//control/prop (root, group, and control levels)
//part/prop (group and control levels)
//group/prop
//back-matter/resource/prop

Who is the bug affecting

Developers trying to properly implement OSCAL properties.

What is affected by this bug

Documentation, Metaschema

How do we replicate this issue

1. Visit the documentation page for any model.
2. Search the page for "marking" (three occurrences of marking per entry)
3. Observe prop[@name='marking'] as valid in places other than //metadata/prop

Expected behavior (i.e. solution)

Documentation for prop in each context should include only the actual accepted values for @name.

Other comments

This issue has existed since the pre 1.0.0 release candidates. To my knowledge no issue was created for it. I could not find one among the open issues.

Revisions

No response

[iMichaela]

Important comments are available under the original issue OSCAL-Reference/issue#6

Moving the earlier-assigned devs here.

[wendellpiez]

In my assessment (as I have remarked repeatedly, whenever consulted), the current design for documentation as a 'spill out' or rendering of a document tree, simply does not accommodate the semantics of 'allowed-values' listings in particular - and of constraints in general - especially given the way Metaschema constraints definitions are evolving.

I.e. I can corroborate the seriousness of this problem since it has been with us since the start, or at least since we settled on a 'good enough for now' docs design. That was fine at the time; now it is no longer 'good enough'. (Indeed on closer examination this usability problem is related to others I won't go into here.)

Accordingly the only way forward I see is this:

• Deal with the problem in the current design by reduction and simplification -- removing all constraints docs from the embedded (per-definition) docs.
• Then, supplement the docs set with a new generated HTML file, listing all constraints defined everywhere, and indexing them, not only by their contexts of definition but more importantly, by the sets of nodes (node types) to which they are applicable.
• Iterate this design with users working test samples

This is not a trivial work item and indeed #2 in itself will require significant effort starting with research spikes.

More importantly, none of this is a one-person job. It was already too ambitious as a two-person job (when it was not done in the current design). The second item especially (the design of an index to constraints by node type and context) is not trivial and ultimately may require more than a static index - i,e, Metapath-aware processing. The third item explicitly requires a team.

Am I wrong?

By an 'index' I mean being able to see all the constraints applicable to any OSCAL element by looking it up some sensible way. But even if you can find a metaschema definition for an element (in itself currently a problem, due to element name reuse in the current metaschema sources), you can't from this query alone find all the constraints applicable to any given case, for two reasons:

1. Such constraints might be defined anywhere in a metaschema, not only in definitions of nodes to which they are applicable or their parents, but actually anywhere
2. Constraints definitions also include arbitrary conditionals such a co-occurrent tests, which mean that which constraints apply to a node can never be known before runtime of the entire tree not just a local fragment (object)

Thus an index would show not only the constraints known to apply to a given node in an actual or abstracted case (element, object or property), but those that might apply depending on other conditions.

If none of the above made sense to you, that only shows how far we have to go.

A more radical approach would be to rip the constraints definitions out from OSCAL entirely and no longer require them for conformance. But I don't know how to do this while also acknowledging that obviously users need them (in some form), and guiding them or showing them how to implement such constraints themselves (i.e. exploring the same mountain range from the other direction). More fundamentally, it would mean giving up this particular tool to exert any leverage at all, meaning 'anything goes' would effectively be the rule.

[wendellpiez]

Just noting in passing that likelihood that there are almost certainly bugs among the current constraints definitions and possibly the tools as well -- but these need to be exposed with test cases before they can be corrected.

This is another reason to have a second implementation of the constraints validation available for testing.

[iMichaela]

The Discussion #1968 is also analyzing prop/@name constraints error in the assessment layer (discovered in the AP and reported int he OSCAL/Lobby)

[david-waltermire]

The property named marking was intended to be allowed anywhere a property is allowed. That is how the constraint is defined. This is a feature, not a bug. The bug is in the documentation being incorrect.

Please keep in mind that changing this is a backwards compatibility breaking change.

[iMichaela]

I think the documentation lists prop@name="marking" as a constraint, that does not allow other values (there is no allow-others="yes" -- see oscal_metadata_metaschema.xml (line 720)

    <constraint>
            <allowed-values target=".[has-oscal-namespace('http://csrc.nist.gov/ns/oscal')]/@name">
                <enum value="marking">A label or descriptor that is tied to a sensitivity or classification marking system. An optional class can be used to define the specific marking system used for the associated value.</enum>
            </allowed-values>
        </constraint>

This issue is a clone of @brian-ruf issue opened in OSCAL-Reference. He argues that marking was envisioned to be constraint only in metadata. I do not think there is any harm in allowing "marking" everywhere and preserve backwards compatibility, as long as other values are allowed in the metadata. Per my review, only keywords and marking are defined as allowed values. The catalog introduces 2 more allowed values in the metadata (see oscal_catalog_metaschema.xml)
It might be important to investigate if allowing other values is beneficial or not to the OSCAL adopters.

    <constraint>
            <allowed-values allow-other="yes" target=".[has-oscal-namespace('http://csrc.nist.gov/ns/oscal')]/@name">
                <enum value="marking">A label or descriptor that is tied to a sensitivity or classification marking system. An optional class can be used to define the specific marking system used for the associated value.</enum>
            </allowed-values>
        </constraint>

[david-waltermire]

@brian-ruf is not remembering this correctly. Please see this commit log and #600. The decision that was implemented was to "allow for a marking value to be provided anywhere that prop is included."

allow-other="no" does not work as you are indicating. See allow-other in the Metaschema specification.

allow-other="no" applies to the the expected value set, which is compiled from all matching allowed values statements. To avoid this aggregation, you need to use both allow-other="no" and @extension="none", which is not used in this case.

The OSCAL model documentation is not accurate around constraints, since it doesn't factor in the aggregation of allowed values across multiple allowed-value statements. This has been a long-standing known problem with the documentation generator.

[iMichaela]

Thank you for providing the decision record.
I was proposing allow-other="yes", not "no".
On a brief review, the documentation link does not explain how allow-other should be processed when:

• not set
• set to "no"
• set to "yes"