fix(ci): restore read-all at workflow level for Scorecard verification

[imran-siddique]

Fix OpenSSF Scorecard CI Failure

Root cause: The Scorecard API verification rejects workflows that have write permissions at the workflow level. Our recent security sweep (#303) changed permissions: read-all to explicit security-events: write + id-token: write at the workflow level, triggering this failure.

Error:

workflow verification failed: global perm is set to write: 
permission for id-token is set to write

Fix: Restore permissions: read-all at the workflow level. The write permissions (security-events: write, id-token: write) are already correctly scoped at the job level where Scorecard expects them.

Reference: https://github.com/ossf/scorecard-action#workflow-restrictions

[github-actions]

🤖 AI Agent: code-reviewer

Review Summary

This pull request addresses a CI failure caused by the OpenSSF Scorecard's stricter requirements for workflow-level permissions. The proposed fix reverts the workflow-level permissions to read-all while keeping the necessary write permissions (security-events: write, id-token: write) scoped at the job level. This change aligns with the Scorecard's expectations and resolves the issue without compromising security.

The PR also includes unrelated documentation updates, which are not directly tied to the CI fix but provide additional context about the project's compliance with OWASP and regulatory standards.

---

Feedback

🔴 CRITICAL

No critical issues identified in this PR.

---

🟡 WARNING

1. Unrelated Documentation Changes in CI Fix PR
   • The PR includes documentation updates (e.g., OWASP compliance and regulatory alignment) that are unrelated to the CI fix. While these changes are valuable, they should ideally be submitted in a separate PR to maintain a clear scope and facilitate focused reviews.
   • Action: Consider splitting the documentation changes into a separate PR to align with best practices for atomic commits.

---

💡 SUGGESTIONS

1. Add a Comment Explaining Workflow-Level Permissions

   • While the PR description explains the rationale for reverting to permissions: read-all, this context is not present in the workflow file itself. Future maintainers may not immediately understand why this change was necessary.
   • Action: Add a comment in the scorecard.yml file explaining why permissions: read-all is required at the workflow level for Scorecard verification.

   # Workflow-level permissions must be read-only for OpenSSF Scorecard verification.
   # Write permissions are scoped to the job level below.
   permissions: read-all

2. Add a Test or Validation for Workflow Permissions

   • To prevent future regressions, consider adding a validation step in the CI pipeline to ensure that workflow-level permissions remain compliant with Scorecard requirements.
   • Action: Investigate whether a pre-commit hook or a custom GitHub Action can be used to validate workflow permissions before merging changes.

3. Document CI/CD Workflow Permissions in README

   • The documentation does not currently explain the CI/CD workflow permissions strategy. Adding a brief section about the use of read-all at the workflow level and scoped write permissions at the job level would improve transparency.
   • Action: Add a section to the README or a dedicated CI/CD documentation file explaining the permissions strategy.

---

General Observations

• The fix is well-aligned with the OpenSSF Scorecard documentation and resolves the immediate issue without introducing new security risks.
• The documentation updates are well-written and provide valuable context for the project's compliance with OWASP and regulatory standards, but they should be decoupled from this PR for clarity.

---

Final Recommendation

• Merge this PR after addressing the suggestion to add a comment in the workflow file explaining the permissions: read-all change.
• Consider creating a separate PR for the documentation updates to maintain a clear scope for each PR.

[github-actions]

🤖 AI Agent: security-scanner

Security Analysis of the Pull Request

1. Prompt Injection Defense Bypass

No changes in this PR directly affect prompt injection defenses. The modifications are limited to CI/CD workflow permissions and documentation updates. No issues found.

2. Policy Engine Circumvention

The change restores permissions: read-all at the workflow level, which is a broader permission scope than the previous explicit permissions (security-events: write, id-token: write, etc.). However, the PR notes that write permissions are still scoped to the job level, which mitigates the risk of policy circumvention. The Scorecard verification enforces this restriction, ensuring that the workflow adheres to best practices. No issues found.

3. Trust Chain Weaknesses

The PR does not introduce any changes related to SPIFFE/SVID validation, certificate pinning, or other trust chain mechanisms. No issues found.

4. Credential Exposure

The change to permissions: read-all does not expose any credentials directly. However, broader permissions at the workflow level could increase the attack surface if a malicious actor gains access to the workflow. This is mitigated by the fact that write permissions are scoped to the job level. No issues found.

5. Sandbox Escape

This PR does not involve changes to sandboxing or container isolation mechanisms. No issues found.

6. Deserialization Attacks

No deserialization logic is introduced or modified in this PR. No issues found.

7. Race Conditions

The PR does not introduce any concurrency-related changes or time-of-check-to-time-of-use (TOCTOU) vulnerabilities. No issues found.

8. Supply Chain

The PR does not introduce new dependencies or modify existing ones. The Scorecard action is already part of the workflow, and no changes to its version or source are made. No issues found.

---

Rating of Findings

• 🔵 LOW: The restoration of permissions: read-all at the workflow level slightly broadens the permission scope. While this is necessary to pass OpenSSF Scorecard verification and is mitigated by scoping write permissions to the job level, it does increase the attack surface marginally.

---

Suggested Fix

1. Monitor Workflow Permissions: Ensure that the read-all permission at the workflow level does not inadvertently allow unauthorized access to sensitive resources. Regularly audit workflows for compliance with the principle of least privilege.
2. Enhance Documentation: Add a note in the repository's security documentation explaining why permissions: read-all is necessary and how job-level scoping mitigates potential risks.
3. Future Improvements: Investigate whether OpenSSF Scorecard can support more granular permissions at the workflow level in the future, to avoid requiring read-all.

---

Final Assessment

The changes in this PR are necessary to comply with OpenSSF Scorecard requirements and do not introduce significant security risks. The broader permission scope at the workflow level is mitigated by job-level scoping of write permissions. However, the slight increase in the attack surface warrants a LOW rating for potential risks.

[github-actions]

🤖 AI Agent: test-generator

🧪 Test Coverage Analysis

packages/agent-os/src/agent_os/cli/__init__.py

• ✅ Existing coverage:

  • Basic CLI initialization and argument parsing.
  • Core functionality for invoking CLI commands.
  • Handling of standard inputs and outputs for CLI operations.

• ❌ Missing coverage:

  • Edge cases for malformed CLI inputs (e.g., invalid flags, missing required arguments).
  • Concurrency scenarios where multiple CLI commands are invoked simultaneously.
  • Policy evaluation edge cases, such as conflicting policies or bypass attempts via CLI.
  • Trust scoring scenarios, including expired certificates or revoked trust.
  • Chaos experiments, such as partial failures or cascading errors during CLI execution.

• 💡 Suggested test cases:

  1. test_cli_invalid_arguments — Ensure the CLI gracefully handles invalid or missing arguments, providing appropriate error messages.
  2. test_cli_policy_conflict — Simulate a scenario where conflicting policies are applied via CLI commands and verify the resolution mechanism.
  3. test_cli_trust_expired_certificate — Test the CLI's behavior when interacting with a system using an expired certificate.
  4. test_cli_partial_failure — Introduce a partial failure (e.g., a timeout or network issue) during a CLI operation and verify proper error handling and recovery.
  5. test_cli_concurrent_invocations — Run multiple CLI commands simultaneously to check for race conditions or deadlocks.
  6. test_cli_injection_attempt — Attempt to inject malicious commands or inputs into the CLI and ensure they are properly sanitized and rejected.
  7. test_cli_oversized_payload — Pass an oversized payload via the CLI to test how the system handles large inputs.
  8. test_cli_policy_bypass_attempt — Simulate an attempt to bypass a policy via CLI and verify that the system enforces the policy correctly.
  9. test_cli_trust_revoked_certificate — Test the CLI's behavior when interacting with a system using a revoked certificate.
  10. test_cli_cascading_failure — Simulate a cascading failure scenario triggered by a CLI command and verify the system's resilience.

These test cases will help ensure robust coverage for the CLI functionality, particularly in edge cases related to policy evaluation, trust scoring, chaos experiments, concurrency, and input validation.

[github-actions]

🤖 AI Agent: docs-sync-checker

📝 Documentation Sync Report

Issues Found

• ❌ MCPAdapter.handle_message() in packages/agent-os/modules/control-plane/src/agent_control_plane/mcp_adapter.py — missing complete docstring for handle_message method.
• ❌ MCPAdapter._handle_tools_call() in packages/agent-os/modules/control-plane/src/agent_control_plane/mcp_adapter.py — missing type hints for parameters and return value.
• ❌ MCPAdapter._handle_resources_read() in packages/agent-os/modules/control-plane/src/agent_control_plane/mcp_adapter.py — missing type hints for parameters and return value.
• ⚠️ packages/agent-os/README.md — references to pip install agent-os need to be updated to pip install agent-os-kernel.
• ⚠️ CHANGELOG.md — no entry for the change from agent-os to agent-os-kernel package name.
• ⚠️ docs/tutorials/06-execution-sandboxing.md — references to pip install agent-os need to be updated to pip install agent-os-kernel.
• ⚠️ docs/tutorials/getting-started.md — references to pip install agent-os need to be updated to pip install agent-os-kernel.
• ⚠️ docs/tutorials/vscode-extension.md — references to pip install agent-os need to be updated to pip install agent-os-kernel.
• ⚠️ packages/agent-os/examples/ — example scripts and requirements.txt files need updates to reflect the package name change from agent-os to agent-os-kernel.

Suggestions

• 💡 Add a complete docstring for MCPAdapter.handle_message(message: Dict[str, Any]) -> Dict[str, Any] to explain its purpose, parameters, return values, and exceptions.
• 💡 Add type hints for MCPAdapter._handle_tools_call(params: Dict) -> Dict and MCPAdapter._handle_resources_read(params: Dict) -> Dict.
• 💡 Update README.md and all relevant documentation/tutorials to replace pip install agent-os with pip install agent-os-kernel.
• 💡 Add a CHANGELOG entry to document the change from agent-os to agent-os-kernel package name.
• 💡 Update all example scripts and requirements.txt files in packages/agent-os/examples/ to reflect the new package name agent-os-kernel.

Additional Notes

• The new docstrings added to MCPMessageType and MCPAdapter are detailed and well-written. However, some methods like handle_message, _handle_tools_call, and _handle_resources_read require additional type hints and/or docstring updates.
• The package name change from agent-os to agent-os-kernel is reflected in many files, but some documentation and examples still need to be updated.
• The new Link Check workflow and .lychee.toml configuration file are well-documented and do not require additional updates.

Action Items

1. Add missing docstrings and type hints to the identified methods in mcp_adapter.py.
2. Update all references to pip install agent-os in documentation and examples to pip install agent-os-kernel.
3. Add a CHANGELOG entry to document the package name change and any other relevant updates.

Once these issues are addressed, the documentation will be in sync.

[github-actions]

🤖 AI Agent: breaking-change-detector

🔍 API Compatibility Report

Summary

The recent changes in the microsoft/agent-governance-toolkit repository include modifications to workflow permissions and package installation instructions, primarily affecting the agent-os package. The most significant change is the renaming of the agent-os package to agent-os-kernel, which will impact users relying on the previous package name. Additionally, there are updates to documentation and examples reflecting this change. No breaking changes to public functions, classes, or methods were identified.

Findings

Severity	Package	Change	Impact
🔴	agent-os	Package renamed from agent-os to agent-os-kernel	Users must update their installation commands to avoid failures.
🟡	agent-os	Default installation command changed in documentation and examples	Users may experience confusion if they do not update their commands.

Migration Guide

To migrate from the old package name to the new one, users should replace any instances of pip install agent-os with pip install agent-os-kernel in their installation scripts and documentation. This change is crucial to ensure that the correct package is installed and to avoid any runtime errors related to missing packages.

For example:

• Change:

  pip install agent-os

• To:

  pip install agent-os-kernel

Ensure that all references in code, documentation, and CI/CD workflows are updated accordingly.

[github-actions]

🤖 AI Agent: security-scanner

Security Analysis of the Pull Request

1. Prompt Injection Defense Bypass

• Assessment: No changes in this PR directly affect prompt injection defenses. The changes primarily involve CI/CD configurations and documentation updates.
• Rating: 🔵 LOW
• Recommendation: No action required.

2. Policy Engine Circumvention

• Assessment: The PR modifies the GitHub Actions workflows to adjust permissions. The change from specific permissions (security-events: write, id-token: write) to read-all at the workflow level aligns with OpenSSF Scorecard requirements. However, this change could potentially allow unintended access to other resources in the repository if not properly scoped at the job level.
• Rating: 🟠 HIGH
• Attack Vector: If the read-all permission is abused or misconfigured, it could lead to unauthorized access to sensitive repository data or actions.
• Recommendation: Ensure that job-level permissions are explicitly scoped to the minimum required for each job. Perform a thorough review of all jobs to confirm that no unnecessary permissions are granted.

3. Trust Chain Weaknesses

• Assessment: No changes in this PR directly affect trust chain mechanisms such as SPIFFE/SVID validation or certificate pinning.
• Rating: 🔵 LOW
• Recommendation: No action required.

4. Credential Exposure

• Assessment: No credentials or secrets are exposed in this PR. The changes to the workflows and documentation do not include sensitive information.
• Rating: 🔵 LOW
• Recommendation: No action required.

5. Sandbox Escape

• Assessment: No changes in this PR directly affect sandboxing mechanisms or introduce potential escape vectors.
• Rating: 🔵 LOW
• Recommendation: No action required.

6. Deserialization Attacks

• Assessment: No changes in this PR involve deserialization of untrusted data.
• Rating: 🔵 LOW
• Recommendation: No action required.

7. Race Conditions

• Assessment: The PR does not introduce or modify any code that could lead to race conditions. The changes are limited to CI/CD workflows and documentation.
• Rating: 🔵 LOW
• Recommendation: No action required.

8. Supply Chain Risks

• Assessment: The PR includes updates to dependencies in the requirements.txt files and changes to the GitHub Actions workflows. Notably:
  • The zendesk-sdk and freshdesk-sdk dependencies were replaced with zenpy and freshdesk, respectively. These changes appear to address potential dependency confusion issues, as the original packages were not available on PyPI.
  • The hashlib-compat dependency was removed, which is appropriate since hashlib is part of Python's standard library and does not require installation.
  • The lycheeverse/lychee-action and actions/checkout actions are pinned to specific commit hashes, which is a good practice to prevent supply chain attacks.
• Rating: 🔵 LOW
• Recommendation: No further action required. The changes improve the security posture of the repository.

---

Summary of Findings

Category	Rating	Details	Recommendation
Prompt Injection Defense	🔵 LOW	No changes affecting prompt injection defenses.	No action required.
Policy Engine Circumvention	🟠 HIGH	read-all at workflow level could allow unintended access.	Ensure job-level permissions are scoped to the minimum required for each job.
Trust Chain Weaknesses	🔵 LOW	No changes affecting trust chain mechanisms.	No action required.
Credential Exposure	🔵 LOW	No credentials or secrets exposed.	No action required.
Sandbox Escape	🔵 LOW	No changes affecting sandboxing mechanisms.	No action required.
Deserialization Attacks	🔵 LOW	No changes involving deserialization of untrusted data.	No action required.
Race Conditions	🔵 LOW	No changes introducing potential race conditions.	No action required.
Supply Chain Risks	🔵 LOW	Dependency updates improve security posture.	No further action required.

---

Final Recommendation

• Approve with Caution: While the PR addresses the OpenSSF Scorecard CI failure and improves dependency security, the change to read-all permissions at the workflow level introduces a potential risk of policy engine circumvention. Ensure that all job-level permissions are explicitly scoped to the minimum required for each job.

[github-actions]

🤖 AI Agent: code-reviewer

Review Summary

This pull request addresses a critical issue with the OpenSSF Scorecard CI workflow by restoring permissions: read-all at the workflow level while maintaining scoped write permissions at the job level. Additionally, it includes several documentation updates, dependency corrections, and a significant renaming of the agent-os package to agent-os-kernel. The PR also introduces a new GitHub Actions workflow for link checking and updates the MCP adapter with improved documentation and functionality.

Below is a detailed review of the changes:

---

🔴 CRITICAL

1. Potential Security Issue: Missing Governance Check for _handle_prompts_list

   • File: mcp_adapter.py
   • Issue: The _handle_prompts_list method does not appear to include any governance checks before returning the list of prompts. This could lead to unauthorized access to sensitive prompt templates.
   • Recommendation: Add a governance check to ensure that the agent has the necessary permissions to access the list of prompts. For example:

     if not self.control_plane.check_permission(self.agent_context, ActionType.PROMPTS_LIST):
         raise PermissionError("Access to list prompts is denied by governance policy.")

2. Potential Sandbox Escape in _handle_resources_read

   • File: mcp_adapter.py
   • Issue: The _handle_resources_read method does not sanitize or validate the uri parameter before attempting to read the resource. This could allow an attacker to craft a malicious uri that accesses unauthorized files or resources.
   • Recommendation: Implement strict validation for the uri parameter, ensuring that it adheres to expected formats and does not allow directory traversal or access to unintended resources.

---

🟡 WARNING

1. Breaking Change: Package Renaming

   • Issue: The agent-os package has been renamed to agent-os-kernel. This is a breaking change for any users or systems that depend on the previous package name.
   • Recommendation: Clearly document this change in the release notes and provide a migration guide for users. Consider adding a deprecation warning in the old package (if it still exists) to inform users of the change.

2. Backward Compatibility for MCP Adapter

   • File: mcp_adapter.py
   • Issue: The updated MCP adapter introduces new arguments (tool_mapping, on_block, logger) to its constructor. This could break existing code that instantiates MCPAdapter without these arguments.
   • Recommendation: Ensure backward compatibility by providing default values for the new arguments. For example:

     def __init__(self, control_plane, agent_context, mcp_handler=None, tool_mapping=None, on_block=None, logger=None):
         ...

---

💡 SUGGESTIONS

1. Improved Documentation for MCP Adapter

   • File: mcp_adapter.py
   • Observation: The updated docstrings for MCPAdapter and its methods are clear and detailed. However, consider adding a high-level example of how the adapter integrates with the rest of the system (e.g., how it interacts with the control plane and the agent context).

2. Testing for MCP Governance Checks

   • File: mcp_adapter.py
   • Observation: The governance checks in methods like _handle_tools_call and _handle_resources_read are critical for security. However, there is no evidence in the PR that these checks are thoroughly tested.
   • Recommendation: Add unit tests to verify that governance checks are correctly enforced for all MCP methods, including edge cases (e.g., unknown tools, invalid URIs, etc.).

3. Link Checker Workflow

   • File: .github/workflows/link-check.yml
   • Observation: The new link-check workflow is a good addition for maintaining documentation quality. However, it is limited to Markdown files (**/*.md).
   • Recommendation: Consider extending the link checker to other file types that may contain links, such as HTML or JSON files.

4. Dependency Updates

   • File: requirements.txt in various examples
   • Observation: The PR corrects some dependency issues (e.g., replacing zendesk-sdk with zenpy and removing hashlib-compat). However, it would be beneficial to run a dependency audit to ensure all dependencies are up-to-date and free of known vulnerabilities.
   • Recommendation: Use tools like pip-audit or safety to perform a dependency audit and address any flagged issues.

5. Type Safety and Pydantic Validation

   • File: mcp_adapter.py
   • Observation: The MCP adapter methods rely on dictionary-based inputs (Dict[str, Any]) without explicit validation.
   • Recommendation: Use Pydantic models to validate the structure and types of the incoming JSON-RPC messages. This will improve type safety and reduce the risk of runtime errors.

6. Thread Safety

   • File: mcp_adapter.py
   • Observation: The MCP adapter does not appear to use any thread synchronization mechanisms. If the adapter is used in a multi-threaded environment, there could be race conditions when accessing shared state (e.g., registered_tools and registered_resources).
   • Recommendation: Use thread-safe data structures (e.g., threading.Lock or concurrent.futures.ThreadPoolExecutor) to ensure thread safety.

---

Final Assessment

• The PR addresses the immediate CI issue with OpenSSF Scorecard and introduces several improvements to the repository.
• However, there are critical security issues related to governance checks and potential sandbox escapes in the MCP adapter that must be addressed before merging.
• Additionally, the renaming of the agent-os package is a breaking change and should be clearly communicated to users.
• Several suggestions for improvement have been provided, including enhanced documentation, dependency audits, and type safety.

Recommendation: Address the critical and warning issues before merging. Consider implementing the suggested improvements to further enhance the quality and security of the codebase.