Fix validation constraints to properly deference and key on uri-reference values

[aj-stein-gsa]

Committer Notes

Closes #2102 and closes #2106.

All Submissions:

• Have you selected the correct base branch per Contributing guidance?
• Have you set "Allow edits and access to secrets by maintainers"?
• Have you checked to ensure there aren't other open Pull Requests for the same update/change?
• Have you squashed any non-relevant commits and commit messages? [instructions]
• Do all automated CI/CD checks pass?

By submitting a pull request, you are agreeing to provide this contribution under the CC0 1.0 Universal public domain dedication.

(For reviewers: The wiki has guidance on code review and overall issue review for completeness.)

Changes to Core Features:

• Have you added an explanation of what your changes do and why you'd like us to include them?
• Have you written new tests for your core changes, as applicable?
• Have you included examples of how to use your new feature(s)?
• Have you updated the OSCAL website and readme documentation affected by the changes you made? Changes to the OSCAL website can be made in the OSCAL-Pages and OSCAL_Reference repositories.

[aj-stein-gsa]

Per an update in #2103, I am proposing a more focused change to resolve this issue and that in #2102. I will update the PR text.

If possible, I would appreciate a review from the NIST Team.

[wandmagic]

nice work! this is true to the intention of the constraints and in line with previous examples self referential links prefixed by a hash.

[aj-stein-gsa]

@iMichaela have you had any time to review this proposed change? It does fix an overly generalized constraint and fixing it will help others, not just FedRAMP. We can work around this issue in the core models, but it means we have to make confusing duplicate requirements with special FedRAMP namespacing because upstream bugfixes if it cannot be accepted. Thanks.

[iMichaela]

NOTE: This constraint is applied to validate-by which has been changed /replaced 4+ years ago with validate . Unfortunately the change was not propagated everywhere. See src/metaschema/shared-constraints/allowed-values-component_component_link-rel.ent file.

[aj-stein-gsa]

The entity file can define values for a target of allowed-values, but not the target itself, which is what your comment describes. Can you explain what this means?

[iMichaela]

I am going to explain again what I see happening here:
In the file: src/metaschema/shared-constraints/allowed-values-component_component_link-rel.ent
the following shared constraints are listed:

<enum xmlns="http://csrc.nist.gov/ns/oscal/metaschema/1.0" value="depends-on">A reference to another component that this component has a dependency on.</enum>
<!-- CHANGED (BJR): validated-by to validation per  https://github.com/usnistgov/OSCAL/blob/metaschema-m4-integration/docs/content/documentation/schema/implementation-layer/validation-modeling.md -->
<enum xmlns="http://csrc.nist.gov/ns/oscal/metaschema/1.0" value="validation">A reference to another component of component-type=validation, that is a validation (e.g., FIPS 140-2) for this component</enum>
<enum xmlns="http://csrc.nist.gov/ns/oscal/metaschema/1.0" value="proof-of-compliance">A pointer to a validation record (e.g., FIPS 140-2) or other compliance information.</enum>
<!-- rel values to also be allowed in implemented-component -->
<enum xmlns="http://csrc.nist.gov/ns/oscal/metaschema/1.0" value="baseline-template">A reference to the baseline template used to configure the asset.</enum>
<enum xmlns="http://csrc.nist.gov/ns/oscal/metaschema/1.0" value="uses-service">This service is used by the referenced component identifier.</enum>
<enum xmlns="http://csrc.nist.gov/ns/oscal/metaschema/1.0" value="system-security-plan">A link to the system security plan of the external system.</enum>

Please note the comment about replacing validated-by with validation for the link/@rel allowed values.
See the reference:

We have a "validation" naming clash between the component/@type="validation" and link/@rel="validation". It is confusing but not a bug.
The bug is in the constraint definition because it is using validated-by which does not exist. In the image of the reference above, there is NO allowed value validated-by

[brian-ruf]

I understand what @iMichaela is saying here and agree. The changes @aj-stein-gsa proposed are essentially correct in that the constraints needed to be updated to look for the leading #; however, an additional change is required on new line number 621.

At minimum, this:
<index-has-key id="oscal-system-implementation-validated-by-index" name="index-system-implementation-component-uuid-validation" target="component/link[@rel='validated-by' and starts-with(@href,'#')]">

Must be changed to this:
<index-has-key id="oscal-system-implementation-validated-by-index" name="index-system-implementation-component-uuid-validation" target="component/link[@rel='validated' and starts-with(@href,'#')]">

We should also consider changing the key ID from oscal-system-implementation-validated-by-index to oscal-system-implementation-validated-index

[iMichaela]

@aj-stein-gsa - I agree with the fact that the faulty constraints are indicating the href should point to a 'raw' uuid, as posed to a proper URI (e.g. #uuid) .

Can you please help us understand:

1. why instead of removing the constraints as recommended in the cited issues (links with @rel "proof-of-compliance" constraint conflicting with @rel "validation" #2102 and Validation component index-has-key constraints do not reformat value before key lookups #2106) the PR is proposing to fix the constraints by enforcing the @href to be only a relative URI reference. Outside FedRAMPS's purpose when an URI with a required scheme is not a viable approach and why OSCAL should not permit it. I think it would be desirable to allow links to external resources.

2. why is the same approach proposed for the @link/rel="depends-on" limiting the href to relative URI?

Please note that the snippet of OSCAL SSP provided in the issue #2106 is taken from the validation test tutorial and it si valid per oscal-cli release 1.0.3 . No errors exist because the snippet is not exercising the faulty constraints.

<component uuid="11111111-0000-4000-a000-000000000001" type="hardware">
   <title>Product Name</title>
   <description>
      <p>Describe the product's function.</p>
   </description>
   <link rel="validation" href="#22222222-0000-4000-a000-000000000002" />
   <status state="operational" />
</component>
<component uuid="22222222-0000-4000-a000-000000000002" type="validation">
   <title>Validation Name</title>
   <description>
      <p>Describe the validation.</p>
   </description>
   <prop name="validation-type" value="fips-140-2" />
   <prop name="validation-reference" value="xxxx" />
   <link rel="validation-details" href="https://csrc.nist.gov/projects/cryptographic-module-validation-program/certificate/xxxx" />
   <status state="operational" />
</component>

The faulty constraint definitions are not exercised since it is not using component/link[@rel='validated-by'] or component/link[@rel='proof-of-compliance'] in the component with type="validation"

Errors are returned when validating an SSP with the following information:

<component uuid="11111111-0000-4000-a000-000000000001" type="hardware">
   <title>Product Name</title>
   <description>
      <p>Describe the product's function.</p>
   </description>
   <link rel="validation" href="#22222222-0000-4000-a000-000000000002" />
   <status state="operational" />
</component>
<component uuid="22222222-0000-4000-a000-000000000002" type="validation">
   <title>Validation Name</title>
   <description>
      <p>Describe the validation.</p>
   </description>
   <prop name="validation-type" value="fips-140-2" />
   <prop name="validation-reference" value="xxxx" />
   <link rel="proof-of-concept" href="https://csrc.nist.gov/projects/cryptographic-module-validation-program/certificate/xxxx" />
   <status state="operational" />
</component>

I also realized that the validated-by is no longer valid. It has been replaced 4+ years ago with validation See the comment above and the reference:
https://pages.nist.gov/OSCAL-Reference/models/v1.1.3/system-security-plan/xml-reference/#/system-security-plan/system-implementation/component. If we conclude the constraint should not be removed but rather fixed, could you also update it to reflect the change to validation (corrected).
Thank you.

[aj-stein-gsa]

@aj-stein-gsa - I agree with the fact that the faulty constraints are indicating the href should point to a 'raw' uuid, as posed to a proper URI (e.g. #uuid) .

Can you please help us understand:

1. why instead of removing the constraints as recommended in the cited issues (links with @rel "proof-of-compliance" constraint conflicting with @rel "validation" #2102 and Validation component index-has-key constraints do not reformat value before key lookups #2106) the PR is proposing to fix the constraints by enforcing the @href to be only a relative URI reference. Outside FedRAMPS's purpose when an URI with a required scheme is not a viable approach and why OSCAL should not permit it. I think it would be desirable to allow links to external resources.
2. why is the same approach proposed for the @link/rel="depends-on" limiting the href to relative URI?

I am not sure if we want to expand scope to 2 in this conversation, but I guess that is an alternative approach, but further strays from the tutorial you reference on test validation modeling. Re this PR and the whole issue at hand, the reason for the changes to indices values with the lookup is they, per the specification, read into a caching index. Those should only occur intra-document, as external URI values (which are acceptable, but not indexable) are not looked up intra-document, those values don't make sense. The proposed change ignores non-uri-reference values and formats the appropriate ones, implementing the specific intent of the constraint. There are other constraints like this in the models, and I adjust to follow the pattern missed here extant in several others.

Essentially you can throw away the index-has-key (where you ask why not delete it) and lose referential integrity or keep the referential integrity and check for the only relevant values (the UUID after the # symbol, looking up #uuid will result in a similar but different key value that is not there, hence the error). It would seem odd to reduce referential integrity and make developers not know about this require it and not make it actionable, right? So we either make it useful, or decrease model accuracy.

If you would like me to go over the relevant Metaschema specification and examples outside of this context, I am happy to do so.

Please note that the snippet of OSCAL SSP provided in the issue #2106 is taken from the validation test tutorial and it si valid per oscal-cli release 1.0.3 . No errors exist because the snippet is not exercising the faulty constraints.

The referenced snippets look identical, is there something I am missing or misunderstood here?

I also realized that the validated-by is no longer valid. It has been replaced 4+ years ago with validation See the comment above and the reference: https://pages.nist.gov/OSCAL-Reference/models/v1.1.3/system-security-plan/xml-reference/#/system-security-plan/system-implementation/component. If we conclude the constraint should not be removed but rather fixed, could you also update it to reflect the change to validate. Thank you.

Can you confirm this proposal is what you intend or use the GitHub suggestion feature to precisely identify the change you want? Your request does not align with the documentation, there is no component[@type="validate"'] in the linked component model docs and the tutorial you referenced you cite as a presumably canonical source in feedback requesting changes uses component[@type="validation"'].

[iMichaela]

Please note that the snippet of OSCAL SSP provided in the issue #2106 is taken from the validation test tutorial and it si valid per oscal-cli release 1.0.3 . No errors exist because the snippet is not exercising the faulty constraints.

The referenced snippets look identical, is there something I am missing or misunderstood here?

My understanding is that you identified the snippet you provided in the #2106 as not being valid and you want it to be valid. The snippet appears to be identical with the tutorial's example and I got no validation error with oscal-cli 1.0.3 . The tutorial sample is not using @link/rel="proof-of-compliance" or @link/rel="validate-by" (which is a bug !) but rather @link/rel="validation-details" like the example in your issue #2106 .

[iMichaela]

@aj-stein-gsa - I agree with the fact that the faulty constraints are indicating the href should point to a 'raw' uuid, as posed to a proper URI (e.g. #uuid) .
Can you please help us understand:

1. why instead of removing the constraints as recommended in the cited issues (links with @rel "proof-of-compliance" constraint conflicting with @rel "validation" #2102 and Validation component index-has-key constraints do not reformat value before key lookups #2106) the PR is proposing to fix the constraints by enforcing the @href to be only a relative URI reference. Outside FedRAMPS's purpose when an URI with a required scheme is not a viable approach and why OSCAL should not permit it. I think it would be desirable to allow links to external resources.
2. why is the same approach proposed for the @link/rel="depends-on" limiting the href to relative URI?

I am not sure if we want to expand scope to 2 in this conversation, but I guess that is an alternative approach, but further strays from the tutorial you reference on test validation modeling. Re this PR and the whole issue at hand, the reason for the

Putting aside the link/@rel="depends-on" discussion means we are not going to address it with this PR?

[iMichaela]

Your request does not align with the documentation, there is no component[@type="validate"']

That was a typo due to 2:00AM time. It should be `component[@type="validation"'.
To summarize what I am trying to bring forward is:

• the bug revealed by this PR that validated-by was long (prior to OSCAL 1.0.0 release) replaced with validation
• validation being used for component's type and for the link/@rel could be the root of the confusion and not understanding each other.

[iMichaela]

@aj-stein-gsa (sorry I forgot to ping you) -- I reviewed the documentation we currently have, the issues opened and the PR comments, and created the example below that is valid per oscal-cli v1.0.3. I think the example demonstrates what you want to accomplish.
NOTE: I avoided the validated-by and used the intended value validation. Is the example below meeting your requirements/expectation. Please feel free to point to the aspects I am missing.

        <component uuid="11111111-0000-4000-a000-000000000001" type="hardware">
            <title>Product Name</title>
            <description>
                <p>Describe the product's function.</p>
            </description>
            <link rel="validation" href="#22222222-0000-4000-a000-000000000002" />
            <status state="operational" />
        </component>
        <component uuid="22222222-0000-4000-a000-000000000002" type="validation">
            <title>Validation Name</title>
            <description>
                <p>Describe the validation.</p>
            </description>
            <prop name="validation-type" value="fips-140-2" />
            <prop name="validation-reference" value="xxxx" />
            <link rel="proof-of-compliance" href="#33333333-0000-4000-a000-000000000003" />
            <status state="operational" />
        </component>
...
      <back-matter>
        <resource uuid="33333333-0000-4000-a000-000000000003">
            <description>
                <p>FIPS 140-2 certificate.</p>
            </description>
            <rlink href="https://csrc.nist.gov/projects/cryptographic-module-validation-program/certificate/xxxx"/>
        </resource>
     </back-matter>

If the example illustrates what you want to accomplish, maybe all we/I need to do is to correct the bug and replace validated-by with validation in the constraint:

<index-has-key id="oscal-system-implementation-validated-by-index" name="index-system-implementation-component-uuid-validation" target="component/link[@rel='validated-by' and starts-with(@href,'#')]">
        <key-field target="@href" pattern="#(.*)"/>
 </index-has-key>

with:

<index-has-key id="oscal-system-implementation-validation-index" name="index-system-implementation-component-uuid-validation" target="component/link[@rel='validation' ]">
        <key-field target="@href" />
 </index-has-key>

@aj-stein-gsa -- Based on the example above, I have a question I would like to resurface: If FedRAMP wants to enforce url-references, can FedRAMP include the constraint locally? We can then keep core OSCAL without additional constraints that might cause backwards compatibility issues for the community.
I do plan on fixing the error I found.

[aj-stein-gsa]

@aj-stein-gsa -- Based on the example above, I have a question I would like to resurface: If FedRAMP wants to enforce url-references, can FedRAMP include the constraint locally? We can then keep core OSCAL without additional constraints that might cause backwards compatibility issues for the community. I do plan on fixing the error I found.

I apologize for the delayed reply. I was hoping if I did not distract from the highest priority PR for which I have asked for expedited review (2111), it would keep us focused on that. I should have followed up on this PR sooner.

To wit: if you do not focus this constraint and not correct it as I propose, anyone who uses the props for which I amended constraints will receive this error for a specification conformant tool when they use any @href that is a full URL. By adding the starts-with() to focus the target, it will make the results more specific and not cause an index miss and potentially throw an error. That or remove the constraint. However, index and index-has-key identify important cross-reference locations, so I am confident my proposed change is the least risky approach even if others want to use the alternative props or the ones I change here, it allows for different users to leverage any or all of them effectively.

[iMichaela]

SUM:

1. PR's request is for all component/link/@href to be a relative URI reference and it is using an example in the issues Validation component index-has-key constraints do not reformat value before key lookups #2106 which focuses on representing a FIPS 140 certification as a component in an SSP.

2. During the PR's review I identified an error through the example provided in Validation component index-has-key constraints do not reformat value before key lookups #2106. The example was not accurate. The claim that the oscal-cli returns errors when the FISP 140 certificate is documented as a system component was not reproduced with the NIST's oscal-cli & the example provided above by @iMichaela here.

If the community members are OK with enforcing the @herf to be a relative URI reference which points to the back-matter where the reference can be external or included in the back-matter, NIST does not have an issue with the PR. It is important for the community to assess if they have OSCAL artifacts (OSCAL SSPs) where they pass full URI in the component/link/@href since those artifacts will no longer be valid, and this is the concern NIST team expressed on behalf of OSCAL adopters from non-USG vertical markets (financial sector in AU, Canada, EU, other governments, other hyperscalers' services for non-USG customers).

This is where the OSCAL Foundation can help provide feedback that captures the view of ALL OSCAL constituents, hence bringing this PR for discussion there.

NOTE: NIST will proceed In the meantime with correcting the bug revealed while creating the example above, and replace validated-by with validation in the constraint:

<index-has-key id="oscal-system-implementation-validated-by-index" name="index-system-implementation-component-uuid-validation" target="component/link[@rel='validated-by' and starts-with(@href,'#')]">
        <key-field target="@href" pattern="#(.*)"/>
 </index-has-key>

with:

<index-has-key id="oscal-system-implementation-validation-index" name="index-system-implementation-component-uuid-validation" target="component/link[@rel='validation' ]">
        <key-field target="@href" />
 </index-has-key>

[brian-ruf]

Approved

[brian-ruf]

I understand what @iMichaela is saying here and agree. The changes @aj-stein-gsa proposed are essentially correct in that the constraints needed to be updated to look for the leading #; however, an additional change is required on new line number 621.

At minimum, this:
<index-has-key id="oscal-system-implementation-validated-by-index" name="index-system-implementation-component-uuid-validation" target="component/link[@rel='validated-by' and starts-with(@href,'#')]">

Must be changed to this:
<index-has-key id="oscal-system-implementation-validated-by-index" name="index-system-implementation-component-uuid-validation" target="component/link[@rel='validated' and starts-with(@href,'#')]">

We should also consider changing the key ID from oscal-system-implementation-validated-by-index to oscal-system-implementation-validated-index