Other Framework Assessment Methods Blocked

[brian-ruf]

Describe the bug

I am working with a catalog for PCI DSS v4.0.1.

The PCI DSS Framework explicitly defines three assessment methods:

• interview
• examine
• observe

See https://docs-prv.pcisecuritystandards.org/PCI%20DSS/Standard/PCI-DSS-v4_0_1.pdf Section 10, Testing Methods for PIC DSS Requirements.

While I realize NIST RMF views examine and observe as identical, the PCI Security Standards Council views them as separate and distinct.

The OSCAL metaschmea is hard-coded with the NIST RMF assessment methods in the NIST RMF namespace (separate from the NIST OSCAL namespace). Any attempt to use other assessment method types in other namespaces incorrectly generates WARNING messages.

NOTE: These are not ERROR messages, thus do not block the content from being declared valid by tools such as OSCAL-CLI; however, the warning is incorrect when using non-NIST frameworks.

Who is the bug affecting

Authors and consumers of framework catalogs that have assessment method types that differ from NIST RMF.

What is affected by this bug

OSCAL Content, Metaschema

How do we replicate this issue

Attempt to use a construct such as the following in an OSCAL Catalog:

        <control id="req-a.b.c">
          <title>Requirement Title</title>
          <part name="statement" id="req-a.b.c_smt">
              <p>Requirement statement.</p>
          </part>
          <part name="assessment-method" id="req-a.b.c_tst">
            <prop name="label" value="1.1.1" />
            <prop name="method" value="observe" ns="http://non-nist.org/ns/oscal" />
            <p>Observe something.</p>
          </part>
        </control>

Expected behavior (i.e. solution)

Treated as valid OSCAL with no warnings or errors when validated with tools such as the OSCAL-CLI

Other comments

As this only produces warnings and not errors, it is not a critical error; however, it highlights the need to separate NIST RMF constraints from core OSCAL syntax.

Revisions

No response

[iMichaela]

@brian-ruf - Thank you for raising this issue. Is the warning you document coded in the metaschema definition files or it is an 'oscal-cli' implementation issue? Did you have a chance to investigate it? If the issue you document is an oscal-cli implementation, then we need to move this issue to the respective repo.

I would like to raise a more fundamental problem: since PCI-DSS uses different definitions for the "interview" than RMF, how would a tool know how to perform a PCI interview vs RMF interview?

[brian-ruf]

@iMichaela those are fair questions.

On the question of whether this is coded into the OSCAL-CLI or actually part of OSCAL Metaschema, it is the latter.
As you can see at this link, the metaschema has an expect constraint on "assessment-method" parts to WARN if there is no prop with:

• the name "method"
• either the NIST RMF or NIST OSCAL namespace

The allowed-values constraint only allows this prop to have one of the three NIST RMF values: (EXAMINE, INTERVIEW, TEST)

<expect level="WARNING" target="part[has-oscal-namespace('http://csrc.nist.gov/ns/oscal') and @name=('assessment','assessment-method')]" test="prop[has-oscal-namespace(('http://csrc.nist.gov/ns/oscal','http://csrc.nist.gov/ns/rmf')) and @name='method']"/>
<allowed-values target="part[has-oscal-namespace('http://csrc.nist.gov/ns/oscal') and @name=('assessment','assessment-method')]/prop[has-oscal-namespace(('http://csrc.nist.gov/ns/oscal','http://csrc.nist.gov/ns/rmf')) and @name='method']/@value">
      <enum value="INTERVIEW">The process of holding discussions with individuals or groups of individuals within an organization to once again, facilitate assessor understanding, achieve clarification, or obtain evidence.</enum>
      <enum value="EXAMINE">The process of reviewing, inspecting, observing, studying, or analyzing one or more assessment objects (i.e., specifications, mechanisms, or activities).</enum>
      <enum value="TEST">The process of exercising one or more assessment objects (i.e., activities or mechanisms) under specified conditions to compare actual with expected behavior.</enum>
</allowed-values>

---

On the question of how to differentiate between frameworks, the namespace on the "method" property should indicate the framework.

NIST Example:

          <part name="assessment-method" id="req-a.b.c_tst">
            <prop name="label" value="1.1.1" />
            <prop name="method" value="TEST" ns="http://csirc.nist.gov/ns/rmf" />
            <p>Test something.</p>
          </part>

Other Framework Example:

          <part name="assessment-method" id="req-a.b.c_tst">
            <prop name="label" value="1.1.1" />
            <prop name="method" value="observe" ns="http://non-nist.org/ns/oscal" />
            <p>Observe something.</p>
          </part>

To break it down:

• This constraint should be removed from core OSCAL metaschema definitions.
• It (and other RMF-specific constraints) should be moved to a NIST RMF metaschema definition that is only applicable for people using OSCAL in the context of NIST RMF.
• Each additional framework should have its own metaschema definition.
• When the above constraint is moved to the NIST RMF metaschema the reference to the core OSCAL namespace should be dropped. Only the NIST RMF namespace should be present.
• It should be an error in the NIST RMF metaschema

It should look like this in the NIST RMF metaschema:

<expect level="ERROR" target="part[has-oscal-namespace('http://csrc.nist.gov/ns/oscal') and @name=('assessment','assessment-method')]" test="prop[has-oscal-namespace('http://csrc.nist.gov/ns/rmf') and @name='method']"/>
<allowed-values target="part[has-oscal-namespace('http://csrc.nist.gov/ns/oscal') and @name=('assessment','assessment-method')]/prop[has-oscal-namespace('http://csrc.nist.gov/ns/rmf') and @name='method']/@value">
   <enum value="INTERVIEW">The process of holding discussions with individuals or groups of individuals within an organization to once again, facilitate assessor understanding, achieve clarification, or obtain evidence.</enum>
   <enum value="EXAMINE">The process of reviewing, inspecting, observing, studying, or analyzing one or more assessment objects (i.e., specifications, mechanisms, or activities).</enum>
   <enum value="TEST">The process of exercising one or more assessment objects (i.e., activities or mechanisms) under specified conditions to compare actual with expected behavior.</enum>
</allowed-values>

Another framework, could have a similar constraint, and would also have the freedom to decide the allowed values - including whether the allowed values are upper or lower case:

<expect level="ERROR" target="part[has-oscal-namespace('http://csrc.nist.gov/ns/oscal') and @name=('assessment','assessment-method')]" test="prop[has-oscal-namespace('http://non-nist.org/ns/oscal') and @name='method']"/>
<allowed-values target="part[has-oscal-namespace('http://csrc.nist.gov/ns/oscal') and @name=('assessment','assessment-method')]/prop[has-oscal-namespace('http://non-nist.org/ns/oscal') and @name='method']/@value">
   <enum value="examine">The assessor critically evaluates ...</enum>
   <enum value="observe">The assessor watches ...</enum>
   <enum value="interview">The assessor converses with ...</enum>
</allowed-values>

This also makes it possible for both values to co-exist. For example, this would be valid in core OSCAL and within both frameworks:

         <part name="assessment-method" id="req-a.b.c_tst">
           <prop name="label" value="1.1.1" />
           <prop name="method" value="observe" ns="http://non-nist.org/ns/oscal" />
           <prop name="method" value="EXAMINE" ns="http://csirc.nist.gov/ns/rmf" />
           <p>Observe something.</p>
         </part>