[Compliance] Example PWS

[roscroft]

Framework

PWS (Performance Work Statement)

LLM Model

GPT-4o Mini

Documents

89243324RFE000084-PWS.pdf

Additional Context

No response

Confirmation

• I confirm that the uploaded documents do not contain sensitive PII or classified information

[roscroft]

🔍 Starting compliance analysis...

• Framework: PWS (Performance Work Statement)
• LLM: GPT-4o Mini
• Attachments: 1
• Started: 2026-01-14T08:51:45.083929+00:00

[roscroft]

📄 Downloaded 1 documents. Querying LLM...

[roscroft]

✅ Compliance Analysis Complete

🔴 Compliance Rate: 14.8%

Metric	Value
Framework	pws
LLM Model	gpt-4o-mini
Documents Analyzed	1
Chunks Indexed	495
Requirements Evaluated	27
Total Requirements	27

📎 See attached report below for full details.

[roscroft]

📄 compliance_report.md

# Compliance Report - Issue #5

## Metadata

| Field | Value |
|-------|-------|
| **Framework** | pws-soo-quality-framework |
| **Version** | 1.0.0 |
| **Generated** | 2026-01-14T08:54:01.816371+00:00 |
| **Model** | gpt-4o-mini |
| **Total Requirements** | 27 |
| **Evaluated Requirements** | 27 |
| **Skipped Requirements** | 0 |

## Executive Summary


This assessment evaluated **27** requirements.

### Overall Compliance: 14.8%

- **Compliant:** 4 requirements
- **Partial:** 0 requirements
- **Non-Compliant:** 23 requirements
- **Insufficient Evidence:** 0 requirements

**1 critical requirements are non-compliant.**

## Statistics

### By Status

| Status | Count | Percentage |
|--------|-------|------------|
| Compliant | 4 | 14.8% |
| Partial | 0 | 0.0% |
| Non-Compliant | 23 | 85.2% |
| Insufficient Evidence | 0 | 0.0% |
| Not Applicable | 0 | 0.0% |

### Confidence Analysis

- **Average Confidence:** 73.0%
- **High Confidence (≥80%):** 4 answers
- **Low Confidence (<50%):** 0 answers

## Detailed Results

### PWS-001

**Requirement:** Does the PWS/SOO include a concise current-state summary?

**Category:** Scope and Objectives | **Severity:** High

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not include a concise current-state summary as required by PWS-001.

**Evidence:**

1. [medium] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall be responsible for the implementation and verification of all cybersecurity controls for the HPC.

2. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall ensure all hosted servers are patched and maintained according to the most current vendor release levels.

3. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall support Security A&A for FISMA ATO (Low/Moderate) for HPC systems.

---

### PWS-003

**Requirement:** Does the PWS/SOO clearly define what is in-scope and out-of-scope?

**Category:** Scope and Objectives | **Severity:** Critical

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not clearly define what is in-scope and out-of-scope for the PWS/SOO.

**Evidence:**

1. [medium] unknown
   > the investigation of new operating systems, installation techniques and options, the maintenance and update options for new and existing operating systems

2. [medium] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > responsible for the activities associated with the investigation of new operating systems, installation techniques and options

3. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > appropriate cybersecurity remediations can be suggested and applied to allow research within a safe environment

---

### PWS-004

**Requirement:** Does the PWS/SOO state priorities and trade-offs among objectives?

**Category:** Scope and Objectives | **Severity:** Medium

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not explicitly state priorities and trade-offs among objectives as required by PWS-004.

**Evidence:**

1. [medium] unknown
   > The Contractor shall specifically be responsible for preparing and maintaining any operational drawings, architecture drawings, and diagrams which facilitate the documentation and understanding of the...

2. [medium] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > Analyze design constraints, trade-offs, and detailed system security.

3. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall ensure all hosted servers are patched and maintained according to the most current vendor release levels.

---

### PWS-005

**Requirement:** Does the PWS/SOO emphasize outcomes rather than activities?

**Category:** Scope and Objectives | **Severity:** High

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation primarily focuses on activities and responsibilities rather than emphasizing outcomes.

**Evidence:**

1. [high] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall contract for, interview, and hire qualified staff to successfully perform the activities defined in this PWS within the duration of the transition period.

2. [medium] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall be responsible for the implementation and verification of all cybersecurity controls for the HPC.

3. [medium] unknown
   > The Contractor shall specifically be responsible for preparing and maintaining any operational drawings, architecture drawings, and diagrams which facilitate the documentation and understanding of the...

---

### PWS-007

**Requirement:** Does the PWS/SOO acknowledge existing technical debt, data quality issues, or integration constraints?

**Category:** Technical Requirements | **Severity:** Medium

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not explicitly acknowledge existing technical debt, data quality issues, or integration constraints.

**Evidence:**

1. [medium] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall provide expertise for the development of a data warehouse ecosystem for the support, management, and security of research data.

2. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall establish a quality element within its organization that ensures compliance with applicable Federal mandates, contractual performance standards, and industry standards and best pr...

---

### PWS-008

**Requirement:** Does the PWS/SOO state software, tool, platform, or methodology preferences with rationale?

**Category:** Technical Requirements | **Severity:** Medium

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not explicitly state software, tool, platform, or methodology preferences with rationale.

**Evidence:**

1. [medium] unknown
   > The Contractor shall manage connection and configuration for the software license server located both in-HPC and outside of HPC Datacenter.

2. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall provide automated software detection and license monitoring services for all software.

---

### PWS-009

**Requirement:** Does the PWS/SOO explicitly indicate where contractor-proposed tools or methods are acceptable?

**Category:** Technical Requirements | **Severity:** Medium

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not explicitly indicate where contractor-proposed tools or methods are acceptable.

**Evidence:**

1. [medium] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall be responsible for the implementation and verification of all cybersecurity controls for the HPC.

2. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall ensure all hosted servers are patched and maintained according to the most current vendor release levels.

3. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall support Security A&A for FISMA ATO (Low/Moderate) for HPC systems.

---

### PWS-010

**Requirement:** Does each deliverable include the decision, action, or outcome it is meant to support?

**Category:** Deliverables and Schedule | **Severity:** High

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not explicitly state that each deliverable includes the decision, action, or outcome it is meant to support.

**Evidence:**

1. [medium] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The requirements shall include, but are not limited to: Be available to support employees through walk-in,

2. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > recognize the unique nature of research computing devices and perform all work on research computing devices (upgrades, repairs, patching, etc.)

3. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > provide support for Government Furnished Equipment (GFE) computers and NETL supported applications of employees working onsite or remotely

---

### PWS-011

**Requirement:** Does the PWS/SOO distinguish between fixed deliverables and those expected to evolve?

**Category:** Deliverables and Schedule | **Severity:** Medium

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not clearly distinguish between fixed deliverables and those expected to evolve, lacking explicit definitions or guidelines.

**Evidence:**

1. [medium] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > Specific requirements include but are not limited to: The Contractor shall proactively recommend changes and/or enhancements to the HPC systems.

2. [medium] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > Core Data Center Services requirements shall include but are not limited to: Maintain NETL’s enterprise monitoring tools.

---

### PWS-012

**Requirement:** Do timelines reflect real dependencies (Government inputs, review bandwidth, decision latency)?

**Category:** Deliverables and Schedule | **Severity:** High

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not clearly indicate whether timelines reflect real dependencies such as Government inputs, review bandwidth, and decision latency.

**Evidence:**

1. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The objective of the Activity is to provide contractor support for the

2. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > assistance. Since EDX does not deploy or manage endpoints, this primarily refers to requests for EDX services (user access/accounts)

---

### PWS-013

**Requirement:** Do financial deliverable timelines account for contract size, teaming structures, and reconciliation time?

**Category:** Deliverables and Schedule | **Severity:** Medium

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not explicitly address financial deliverable timelines in relation to contract size, teaming structures, and reconciliation time.

**Evidence:**

1. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall be responsible for the implementation and verification of all cybersecurity controls for the HPC.

2. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall ensure all hosted servers are patched and maintained according to the most current vendor release levels.

3. [low] unknown
   > The Contractor shall be responsible for managing certain EDX hardware and software licenses.

---

### PWS-014

**Requirement:** Does the PWS/SOO specify what will be Government-furnished (GFI/GFP/GFE)?

**Category:** Government-Furnished Resources | **Severity:** High

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not clearly specify what will be Government-furnished (GFI/GFP/GFE).

**Evidence:**

1. [high] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > Accountable and sensitive property currently issued to the incumbent Contractor for contract performance will be provided to the Contractor for performance of activities under this contract.

2. [medium] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > Provide support for Government Furnished Equipment (GFE) computers and NETL supported applications of employees working onsite or remotely.

---

### PWS-015

**Requirement:** Does the PWS/SOO indicate when Government-furnished resources will be available?

**Category:** Government-Furnished Resources | **Severity:** Medium

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not clearly indicate when Government-furnished resources will be available.

**Evidence:**

1. [medium] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > Accountable and sensitive property currently issued to the incumbent Contractor for contract performance will be provided to the Contractor for performance of activities under this contract.

2. [medium] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > Provide support for Government Furnished Equipment (GFE) computers and NETL supported applications of employees working onsite or remotely.

---

### PWS-017

**Requirement:** Does the PWS/SOO include a notional budget range or anticipated scale of effort?

**Category:** Pricing and Cost | **Severity:** High

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not include a notional budget range or anticipated scale of effort as required by PWS-017.

**Evidence:**

1. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall provide Ethernet Infrastructure Controllable via Intelligent Platform Management Interface (IPMI) or equivalent; connections to nodes must be 1Gbps or faster; Ethernet network mus...

2. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > Be available to support employees through walk-in, a single phone number, online portal, or chat, during business operating hours as applicable.

3. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > assistance. Since EDX does not deploy or manage endpoints, this primarily refers to requests for EDX services (user access/accounts), requests for information (including how-to instructions) regarding...

---

### PWS-018

**Requirement:** Does the PWS/SOO address anticipated variability or surge requirements?

**Category:** Pricing and Cost | **Severity:** Medium

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not explicitly address anticipated variability or surge requirements as per PWS-018.

**Evidence:**

1. [medium] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall perform all work within this Activity according to the requirements specified in the General Contract Requirements section.

2. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall specifically be responsible for managing certain EDX hardware and software lic.

---

### PWS-019

**Requirement:** Does the PWS/SOO specify staffing and clearance expectations?

**Category:** Pricing and Cost | **Severity:** High

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not specify staffing and clearance expectations as required by PWS-019.

**Evidence:**

1. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall provide a communication plan to ensure the lines of communication between the Contractor and the Government remain open and available.

2. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall be responsible for the implementation and verification of all cybersecurity controls for the HPC.

---

### PWS-021

**Requirement:** Does the PWS/SOO identify dependencies on Government systems or data?

**Category:** Pricing and Cost | **Severity:** Medium

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not explicitly identify dependencies on Government systems or data.

**Evidence:**

1. [medium] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall follow Federal Civilian standards and processes, for operating and maintaining all IT systems, applications, and projects under the purview of the contract.

2. [low] unknown
   > The Government has the right to access all IT systems supported by the Contractor.

---

### PWS-022

**Requirement:** Do performance standards measure mission impact and decision quality (not just task completion)?

**Category:** Performance Standards | **Severity:** High

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not explicitly address performance standards measuring mission impact and decision quality.

**Evidence:**

1. [medium] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall provide quality system, assurance, and control services in accordance with CMMI-DEV maturity level 3 practices and ISO 9000/9001 by planning, executing, and managing verification ...

2. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > Document any modification or deviation from organizationally defined security policies and standards.

---

### PWS-023

**Requirement:** Does the PWS/SOO include clearly defined acceptance criteria?

**Category:** Performance Standards | **Severity:** High

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not clearly define acceptance criteria as required by PWS-023.

**Evidence:**

1. [medium] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall be responsible for research information systems security functions for the Research environment.

2. [low] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall review the historical information identified in the solicitation and the minimum qualifications as identified in Section J, Attachment C – Position Qualifications of the contract ...

3. [low] unknown
   > The Contractor shall ensure software compliance and license management as well as warranty tracking such that all production hardware and software is always within the or

---

### PWS-024

**Requirement:** Does the PWS/SOO specify review timelines and service level expectations?

**Category:** Performance Standards | **Severity:** Medium

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not specify review timelines or service level expectations as required by PWS-024.

**Evidence:**

1. [medium] unknown
   > The Contractor shall ensure software compliance and license management as well as warranty tracking such that all production hardware and software is always within the or

2. [medium] unknown
   > The Contractor shall provide resource utilization and capacity planning support and shall be responsible for providing recommendations for equipment and service replacement, upgrade, or enhancement to...

---

### PWS-025

**Requirement:** Are objectives, deliverables, performance standards, and evaluation criteria aligned?

**Category:** Alignment | **Severity:** Critical

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation lacks clear alignment of objectives, deliverables, performance standards, and evaluation criteria.

**Evidence:**

1. [medium] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall manage A/V Services for standard conference rooms and VTC rooms, where requirements shall include but are not limited to: Provide Tier 2 support for standard conference rooms and ...

2. [medium] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > Provide subject matter expertise to develop requirements for collaboration system upgrades and implement the authorized upgrades in accordance with current technology standards, best practices, and al...

---

### PWS-026

**Requirement:** Does the PWS/SOO clearly communicate where flexibility exists vs. where constraints are fixed?

**Category:** Innovation | **Severity:** Medium

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not clearly communicate where flexibility exists versus where constraints are fixed, leading to ambiguity in compliance with requirement PWS-026.

**Evidence:**

1. [medium] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall follow Federal Civilian standards and processes, for operating and maintaining all IT systems, applications, and projects under the purview of the contract.

2. [medium] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall ensure all hosted servers are patched and maintained according to the most current vendor release levels.

3. [low] unknown
   > The specific transition activities are to be included in a Transition Plan, the contents of which are identified in Section L of the solicitation.

---

### PWS-027

**Requirement:** Does the PWS/SOO include explicit problem statements grounded in current-state challenges?

**Category:** Innovation | **Severity:** High

**Status:** Non-Compliant
**Confidence:** 🟡 70%

**Assessment:** The documentation does not explicitly include problem statements grounded in current-state challenges as required by PWS-027.

**Evidence:**

1. [medium] unknown
   > The Contractor shall specifically be responsible for maintaining the underlying EDX code and any explanatory and supporting documentation in a coherent and complete manner.

2. [medium] unknown
   > The Contractor shall specifically be responsible for preparing and maintaining any operational drawings, architecture drawings, and diagrams which facilitate the documentation and understanding of the...

---

### PWS-002

**Requirement:** Does the PWS/SOO identify known risks and constraints from market research?

**Category:** Scope and Objectives | **Severity:** High

**Status:** Compliant
**Confidence:** 🟢 90%

**Assessment:** The documentation identifies known risks and constraints through the incorporation of NETL’s Risk Management processes and the requirement to document modifications and deviations from security policies.

**Evidence:**

1. [high] unknown
   > The Contractor shall incorporate NETL’s Risk Management processes for the initiation of IT risks and issues including mitigation, acceptance, and closure through the Risk Review Board (RRB).

2. [high] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > Document any modification or deviation from organizationally defined security policies and standards as well as the resulting risks and any recommended mitigation actions.

---

### PWS-006

**Requirement:** Does the PWS/SOO describe the current technical environment (existing systems, tools, data sources, integrations)?

**Category:** Technical Requirements | **Severity:** High

**Status:** Compliant
**Confidence:** 🟢 90%

**Assessment:** The documentation describes the current technical environment, including systems, tools, and data sources relevant to the PWS/SOO.

**Evidence:**

1. [high] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall provide expertise for the development of a data warehouse ecosystem for the support, management, and security of research data.

2. [high] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > The Contractor shall be responsible for research information systems security functions for the Research environment.

---

### PWS-016

**Requirement:** Does the PWS/SOO address how remote/hybrid personnel will obtain access?

**Category:** Government-Furnished Resources | **Severity:** Medium

**Status:** Compliant
**Confidence:** 🟢 90%

**Assessment:** The documentation outlines procedures for remote personnel to obtain access, including support for remote desktop administration and VPN access with multi-factor authentication.

**Evidence:**

1. [high] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > Provide support for Government Furnished Equipment (GFE) computers and NETL supported applications of employees working onsite or remotely, either by phone or remote desktop administration.

2. [high] unknown
   > VPN access requires multi-factor authentication and is appropriately secure.

---

### PWS-020

**Requirement:** Does the PWS/SOO specify onsite vs. remote work location assumptions?

**Category:** Pricing and Cost | **Severity:** Medium

**Status:** Compliant
**Confidence:** 🟢 90%

**Assessment:** The documentation specifies that staff will support customers both onsite and remotely, indicating clear location assumptions.

**Evidence:**

1. [high] /tmp/cicero-jira/issue-5/documents/89243324RFE000084-PWS.pdf
   > it is expected that staff will be available to support customers both onsite and remotely within the hybrid work organization.

---