feat: add trust score visualization to CLI (#406)

[suyu-lily]

Description

Adds a trust-report CLI command that outputs a formatted trust score summary for registered agents.

Features added:

• New trust-report CLI command under agentmesh/cli/
• Display key trust metrics:
  • Agent DID
  • Trust score (0–1000)
  • Trust tier (Verified / Trusted / Standard / Probationary / Untrusted)
  • Successful / failed tasks (computed from execution history)
  • Last activity timestamp (based on most recent recorded event)
• Support multiple output formats:
  • Default table view for readability
  • --format json for programmatic usage

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update
• Maintenance (dependency updates, CI/CD, refactoring)
• Security fix

Package(s) Affected

• agent-os-kernel
• agent-mesh
• agent-runtime
• agent-sre
• agent-governance
• docs / root

Checklist

• My code follows the project style guidelines (ruff check)
• I have added tests that prove my fix/feature works
• All new and existing tests pass (pytest)
• I have updated documentation as needed
• I have signed the Microsoft CLA

Related Issues

Closes #406

[imran-siddique]

Thanks for the contribution, @suyu-lily! This is a solid start and follows the existing CLI patterns well. A few things to address before merge:

Blocker

• CLA: Please sign the Contributor License Agreement — merge is blocked until this is done. See the bot comment above.

Code issues

1. Missing newline at EOF — trust_cli.py is missing a trailing newline. Please add one.

2. Duplicated logic — The history categorization loop (splitting events by delta >= 0 vs < 0) is copy-pasted between the JSON and table branches. Please extract a helper, e.g.:

   def _categorize_tasks(history):
       successful = [h['event'] for h in history if h['delta'] >= 0]
       failed = [h['event'] for h in history if h['delta'] < 0]
       return successful, failed

3. Redundant conditional — Inside the if fmt == 'json': block, there's a nested if fmt == 'json': _output_json(data) that is always true. Remove the inner check and just call _output_json(data) directly.

4. delta >= 0 semantics — Zero-delta events are currently counted as successful tasks. Is that intentional? A zero-delta event might be neutral (no score change) rather than a success. Consider using delta > 0 instead, or add a comment explaining the choice.

Tests

5. Strengthen test_list_table — Currently it only checks for 'alpha' in output or 'Agent' in output. Please also verify that expected column headers appear (e.g., 'Score', 'Level', 'Last Activity').

Docs

6. The PR checklist shows documentation is not updated. If CLI docs exist for the trust subcommands, please update them to include report.

---

What looks good:

• Follows existing patterns (--format/--json flags, rich Table, _get_demo_peers())
• JSON output tests are solid (validates all expected keys)
• Help subcommand test properly updated

Happy to re-review once these are addressed!

[github-actions]

Welcome to the Agent Governance Toolkit! Thanks for your first pull request.
Please ensure tests pass, code follows style (ruff check), and you have signed the CLA.
See our Contributing Guide.

[github-actions]

🤖 AI Agent: docs-sync-checker — Issues Found

📝 Documentation Sync Report

Issues Found

• ❌ report_agents(fmt: str, json_flag: bool) in agentmesh/cli/trust_cli.py — missing docstring
• ⚠️ packages/agent-mesh/README.md — no mention of the new trust report CLI command
• ⚠️ CHANGELOG.md — no entry for the new trust report CLI command

Suggestions

• 💡 Add a docstring for report_agents(fmt: str, json_flag: bool) explaining its purpose, parameters, and behavior.
• 💡 Update packages/agent-mesh/README.md to include a description of the new trust report CLI command, its purpose, and usage examples (e.g., --format and --json options).
• 💡 Add an entry to CHANGELOG.md under a new version section, detailing the addition of the trust report CLI command and its features.

Additional Notes

• The new public API report_agents has type hints for its parameters, which is good.
• The example code in packages/agent-mesh/tests/test_trust_cli.py has been updated to include tests for the new trust report command, which is also good.

Please address the issues above to ensure documentation is fully in sync. Let me know if you need further assistance!

[github-actions]

🤖 AI Agent: breaking-change-detector — Summary

🔍 API Compatibility Report

Summary

This pull request introduces a new CLI command trust report to the agent-mesh package. The changes are additive and do not introduce any breaking changes to the existing API. The new functionality is well-tested and does not modify existing APIs or behaviors.

Findings

Severity	Package	Change	Impact
🔵	agent-mesh	Added trust report CLI command	New functionality, no breaking changes

Migration Guide

✅ No migration needed. This change is fully backward-compatible.

Notes

• Ensure the new trust report command is documented in the CLI reference documentation.
• Consider adding examples of usage for both table and JSON output formats to help users understand the new functionality.

[github-actions]

🤖 AI Agent: test-generator — `trust_cli.py`

🧪 Test Coverage Analysis

trust_cli.py

• ✅ Existing coverage:

  • The new trust report command is covered by the following test cases:
    • test_list_table: Verifies the default table output format.
    • test_list_json: Verifies the JSON output format and checks for the presence of expected keys in the output.
    • test_list_json_flag: Verifies the shorthand --json flag for JSON output.
  • These tests ensure the basic functionality of the trust report command, including its ability to handle different output formats and validate the structure of the JSON output.

• ❌ Missing coverage:

  • Edge cases for trust score boundaries (e.g., 0, 1000, and values in between).
  • Scenarios with no agents or empty history for agents.
  • Handling of malformed or unexpected data in the agent history (e.g., missing keys, invalid data types).
  • Behavior when _get_demo_peers() or _get_demo_history() returns None or raises exceptions.
  • Concurrency scenarios where multiple agents are being updated simultaneously.
  • Performance testing for a large number of agents (e.g., 10,000+ agents).
  • Validation of the last_activity timestamp format and its behavior when timestamps are missing or malformed.

• 💡 Suggested test cases:

  1. test_trust_report_empty_agents — Verify that the command handles scenarios where no agents are registered (empty peers list).
  2. test_trust_report_empty_history — Test behavior when agents have no recorded history (empty history list).
  3. test_trust_report_boundary_scores — Test trust score boundaries (e.g., 0, 1000, and intermediate values like 499 and 500) and ensure correct trust tier classification.
  4. test_trust_report_malformed_history — Simulate malformed or incomplete history data (e.g., missing delta or timestamp keys) and verify that the command handles it gracefully.
  5. test_trust_report_large_number_of_agents — Test performance and correctness when reporting on a large number of agents (e.g., 10,000+ agents).
  6. test_trust_report_concurrent_updates — Simulate concurrent updates to agent trust scores or history and ensure no race conditions or data inconsistencies occur.
  7. test_trust_report_invalid_format_option — Test behavior when an invalid value is passed to the --format option (e.g., --format xml).
  8. test_trust_report_missing_last_activity — Verify behavior when the last_activity timestamp is missing or invalid in the agent history.
  9. test_trust_report_partial_failure — Simulate a scenario where _get_demo_peers() or _get_demo_history() raises an exception for some agents and verify that the command handles it gracefully without crashing.
  10. test_trust_report_injection_attempt — Test for potential injection vulnerabilities by passing malicious input (e.g., SQL injection or script injection) in agent IDs or history data.

By implementing these additional test cases, the coverage for the new trust report functionality can be significantly improved, ensuring robustness and reliability in various edge cases and scenarios.

[github-actions]

🤖 AI Agent: contributor-guide — Welcome! 🎉

Welcome! 🎉

Hi @first-time-contributor! 👋 Welcome to the microsoft/agent-governance-toolkit community, and thank you for your contribution! We're thrilled to have you here. Your pull request to add trust score visualization to the CLI is a fantastic addition, and we appreciate the time and effort you've put into it. Let's dive into the review together!

---

What You Did Well 🌟

1. Clear and Descriptive PR Description: Your pull request description is thorough and well-structured. It clearly outlines the new trust-report command, its features, and the motivation behind it. This makes it easy for reviewers to understand the context and purpose of your changes.

2. Thoughtful Features: The ability to display trust metrics in both a table and JSON format is a great touch. It caters to both human readability and programmatic use cases, which is very thoughtful.

3. Tests Included: You've added tests for the new functionality, ensuring that the trust-report command works as expected. This demonstrates a strong commitment to maintaining the project's quality.

4. Code Readability: The code is well-organized and easy to follow. The use of helper functions like _get_demo_peers and _get_demo_history keeps the logic modular and clean.

---

Suggestions for Improvement ✨

1. Linting with Ruff:

   • While you mentioned that your code passes ruff check, I noticed a missing newline at the end of the trust_cli.py file. This is a small formatting issue but important for consistency. You can fix this by ensuring your editor automatically adds a newline at the end of files.

2. Test Coverage:

   • Your tests are great, but it might be helpful to add a test case for scenarios where there are no agents or no history for an agent. This ensures the CLI handles edge cases gracefully.

3. Documentation Update:

   • The new trust-report command should be documented in the relevant CLI documentation. This will help users discover and understand how to use this new feature. You can find more details on contributing to documentation in our CONTRIBUTING.md guide.

4. Conventional Commits:

   • Your commit message starts with feat:, which is great! However, it would be even better to include a more specific description of the feature in the commit message. For example: feat(cli): add trust-report command for trust score visualization.

5. Security Considerations:

   • Since this feature involves displaying trust scores and agent data, it's worth considering if any sensitive information might be exposed. For example:
     • Are there any privacy concerns with displaying agent IDs?
     • Should there be any access controls or warnings for this command?
   • If you've already considered these aspects, it would be helpful to mention them in the PR description.

---

Project Conventions 📚

Here are a few conventions we follow in this project to ensure consistency and quality:

1. Linting: We use ruff for linting, specifically with rules for E, F, and W codes. Run ruff check . to ensure your code adheres to these standards.

2. Testing: All tests should be placed in the packages/{name}/tests/ directory. You've done this correctly — great job! 🎉

3. Commit Messages: We follow the Conventional Commits specification. This helps us maintain a clean and meaningful Git history. Your feat: prefix is spot on, but consider adding more detail to the message.

4. Security: Any code that interacts with sensitive data or has security implications should be reviewed with extra care. If applicable, please provide details about how you've ensured the security of your changes.

---

Next Steps 🚀

1. Address the suggestions above:

   • Add a newline at the end of trust_cli.py.
   • Consider adding tests for edge cases (e.g., no agents or no history).
   • Update the CLI documentation to include the new trust-report command.
   • Optionally, refine your commit message to include more details.

2. Once you've made the updates, push your changes to the same branch. This will automatically update the pull request.

3. After you've made the changes, let us know by leaving a comment here. We'll review your updates and provide further feedback if needed.

4. Once everything looks good, we'll merge your PR and celebrate your contribution! 🎉

---

If you have any questions or need help with anything, don't hesitate to ask. We're here to support you! 😊

Thank you again for contributing to the microsoft/agent-governance-toolkit. We can't wait to see more of your amazing work in the future! 🚀

[github-actions]

🤖 AI Agent: security-scanner — Security Review of PR #406: `feat: add trust score visualization to CLI`

Security Review of PR #406: feat: add trust score visualization to CLI

This PR introduces a new trust-report CLI command to display a formatted trust score summary for registered agents. The feature includes both a table view and a JSON output format. Below is a security review of the changes, focusing on the critical areas outlined in the prompt.

---

Findings

1. Prompt Injection Defense Bypass

• Risk: N/A
• Analysis: This PR does not involve any user-input-driven prompt generation or LLM interactions. Therefore, prompt injection risks are not applicable.
• Rating: 🔵 LOW

---

2. Policy Engine Circumvention

• Risk: 🟡 MEDIUM
• Analysis: The _get_demo_peers() and _get_demo_history() functions are used to fetch agent data and history. However, there is no indication that these functions enforce access control or validate the authenticity of the data. If these functions are not properly secured, an attacker could potentially inject unauthorized or manipulated data into the trust report.
• Recommendation: Ensure that _get_demo_peers() and _get_demo_history() enforce strict access control and validate the integrity of the data. If these functions are mock implementations, ensure that the production equivalents are secure.

---

3. Trust Chain Weaknesses

• Risk: 🟠 HIGH
• Analysis: The trust-report command relies on the peer_did and trust_score values to generate the report. However, there is no evidence of cryptographic verification of the peer_did or the trust score's source. If these values can be tampered with, the trust report could be manipulated to show incorrect or malicious data.
• Recommendation: Ensure that the peer_did and trust_score are cryptographically verified using mechanisms like SPIFFE/SVID or certificate pinning. Add validation checks to confirm the authenticity of the data before generating the report.

---

4. Credential Exposure

• Risk: 🔵 LOW
• Analysis: The PR does not introduce any new handling of sensitive credentials, nor does it log sensitive information. The trust report only outputs agent metadata and trust metrics.
• Recommendation: Ensure that no sensitive information (e.g., private keys, tokens) is inadvertently included in the trust-report output.

---

5. Sandbox Escape

• Risk: 🔵 LOW
• Analysis: The CLI command does not execute untrusted code or interact with external systems in a way that could lead to a sandbox escape.
• Recommendation: None.

---

6. Deserialization Attacks

• Risk: 🟠 HIGH
• Analysis: The trust-report command processes agent history data using _get_demo_history(), which returns a list of dictionaries. If this data is deserialized from an untrusted source, it could be exploited for deserialization attacks.
• Recommendation: Ensure that any deserialization of data (e.g., JSON or other formats) is performed using safe libraries and that the data is validated against a strict schema before use.

---

7. Race Conditions

• Risk: 🟡 MEDIUM
• Analysis: The trust report aggregates data from multiple sources (_get_demo_peers() and _get_demo_history()). If these sources are updated concurrently, there could be a Time-of-Check to Time-of-Use (TOCTOU) issue, leading to inconsistent or incorrect reports.
• Recommendation: Implement locking mechanisms or ensure atomicity when fetching and processing data for the trust report.

---

8. Supply Chain

• Risk: 🟡 MEDIUM
• Analysis: The PR does not introduce new dependencies. However, the existing dependencies should be reviewed for potential supply chain risks, such as dependency confusion or typosquatting.
• Recommendation: Perform a dependency audit to ensure all imported libraries are from trusted sources and are up-to-date with security patches.

---

Summary of Findings

Category	Risk	Rating	Recommendation
Prompt Injection Defense	N/A	🔵 LOW	None
Policy Engine Circumvention	Data Integrity	🟡 MEDIUM	Enforce access control and validate data integrity in _get_demo_peers() and _get_demo_history().
Trust Chain Weaknesses	Data Authenticity	🟠 HIGH	Cryptographically verify peer_did and trust_score using SPIFFE/SVID or similar mechanisms.
Credential Exposure	N/A	🔵 LOW	Ensure no sensitive information is included in the trust report output.
Sandbox Escape	N/A	🔵 LOW	None
Deserialization Attacks	Data Validation	🟠 HIGH	Validate deserialized data against a strict schema to prevent malicious payloads.
Race Conditions	TOCTOU	🟡 MEDIUM	Ensure atomicity or implement locking mechanisms when fetching and processing data.
Supply Chain	Dependency Risks	🟡 MEDIUM	Perform a dependency audit to ensure all libraries are secure and up-to-date.

---

Recommendations for Fixes

1. Data Integrity and Authenticity:

   • Implement cryptographic verification for peer_did and trust_score values.
   • Add access control and data validation in _get_demo_peers() and _get_demo_history().

2. Deserialization Safety:

   • Use a strict schema to validate deserialized data from _get_demo_history().

3. Concurrency Safety:

   • Ensure atomic operations or use locking mechanisms when aggregating data for the trust report.

4. Dependency Audit:

   • Review all dependencies for potential supply chain risks and ensure they are up-to-date.

---

Final Rating: 🟠 HIGH

This PR introduces useful functionality but has potential security risks related to data authenticity, deserialization, and concurrency. Addressing these issues is critical to ensure the integrity and reliability of the trust-report feature.

[github-actions]

🤖 AI Agent: code-reviewer

Review of PR #406: Add Trust Score Visualization to CLI

Summary

This PR introduces a new trust-report CLI command to the agent-mesh package, allowing users to generate a trust score summary for registered agents. The command supports both table and JSON output formats. The implementation includes new functionality, tests, and integration into the CLI.

---

Key Areas of Focus

1. Policy Engine Correctness

• The trust-report command relies on _get_demo_peers() and _get_demo_history() to fetch agent data and their respective histories. These functions appear to be mock/demo implementations. If this is intended for production, ensure these functions are replaced with secure and validated data sources.
• 💡 SUGGESTION: Add validation to ensure that the trust score and task history data are consistent and cannot be tampered with. For example, verify that the trust score aligns with the recorded task history.

2. Trust/Identity

• The agent_id (DID) is displayed in the report. However, there is no validation or cryptographic verification of the DID's authenticity.
• 🔴 CRITICAL: If the agent_id is a DID, ensure that it is validated against its cryptographic signature to prevent impersonation or spoofing.
• 💡 SUGGESTION: Consider adding a column or flag to indicate whether the agent_id has been verified or attested.

3. Sandbox Escape Vectors

• The CLI command does not execute untrusted code or interact with external systems directly, so there are no immediate sandbox escape concerns.
• 💡 SUGGESTION: If the CLI will eventually process untrusted input (e.g., user-provided data), ensure proper sanitization and validation to prevent command injection or other vulnerabilities.

4. Thread Safety

• The implementation does not involve multithreading or concurrency, so no thread safety issues are apparent.

5. OWASP Agentic Top 10 Compliance

• 🔴 CRITICAL: The trust-report command outputs sensitive information (e.g., agent_id, trust scores, task history). Ensure that access to this command is restricted to authorized users only. If the CLI is used in shared environments, consider implementing authentication or access control mechanisms.
• 💡 SUGGESTION: Mask or redact sensitive information (e.g., partial agent_id) in the default table output, with an option to display full details if explicitly requested.

6. Type Safety and Pydantic Model Validation

• The data dictionary in the JSON output is constructed dynamically without validation. This could lead to runtime errors or inconsistencies.
• 💡 SUGGESTION: Use Pydantic models to define the structure of the JSON output. This ensures type safety and validates the data before outputting it.

7. Backward Compatibility

• 🟡 WARNING: Adding a new CLI command (trust report) is a non-breaking change but could affect users relying on the existing CLI structure. Ensure this addition is documented in the release notes and CLI documentation.

---

Code Review

Strengths

• The implementation is clean and adheres to the project's style guidelines.
• The use of click for CLI commands is consistent with the rest of the project.
• The table output is well-structured and user-friendly.
• The test coverage for the new functionality is adequate, with tests for both table and JSON output formats.

Issues and Suggestions

1. Duplicate Logic for Task History Parsing

   • The logic for parsing successful_tasks and failure_tasks is duplicated in both the JSON and table output sections.
   • 💡 SUGGESTION: Refactor this logic into a helper function to avoid duplication and improve maintainability.

   def _parse_task_history(history):
       successful_tasks = [h["event"] for h in history if h["delta"] >= 0]
       failure_tasks = [h["event"] for h in history if h["delta"] < 0]
       return successful_tasks, failure_tasks

   Then use it as:

   successful_tasks, failure_tasks = _parse_task_history(history)

2. Error Handling for Empty History

   • The code assumes that history[-1]["timestamp"] exists if history is not empty. This could raise an IndexError if history is malformed.
   • 💡 SUGGESTION: Add error handling or validation to ensure history contains valid entries.

   last_activity = history[-1]["timestamp"] if history and "timestamp" in history[-1] else "N/A"

3. Hardcoded Demo Functions

   • The _get_demo_peers() and _get_demo_history() functions are placeholders. If this feature is intended for production, these must be replaced with actual implementations.
   • 💡 SUGGESTION: Add a warning or TODO comment to indicate that these functions are not production-ready.

4. Test Coverage

   • The tests cover basic functionality but do not test edge cases, such as:
     • Agents with no task history.
     • Agents with malformed or incomplete data.
     • Large numbers of agents (performance testing).
   • 💡 SUGGESTION: Add tests for these edge cases to ensure robustness.

5. Output Format Consistency

   • The JSON output includes lists of task names for successful_tasks and failure_tasks, while the table output concatenates them into a single string. This inconsistency could confuse users.
   • 💡 SUGGESTION: Consider aligning the formats or documenting the difference clearly.

6. Missing Documentation

   • The PR does not include updates to the CLI documentation.
   • 🟡 WARNING: Update the CLI documentation to include details about the new trust-report command, its options, and examples of usage.

---

Final Assessment

• Functionality: The feature is well-implemented and provides useful insights into agent trust scores.
• Security: There are critical concerns regarding the validation of agent_id and access control for sensitive information.
• Code Quality: The code is clean but could benefit from minor refactoring and additional error handling.
• Testing: Basic tests are included, but more edge cases should be covered.
• Documentation: The new feature is not documented, which could lead to confusion for users.

---

Actionable Feedback Summary

1. 🔴 CRITICAL: Validate agent_id (DID) cryptographically to prevent spoofing or impersonation.
2. 🔴 CRITICAL: Restrict access to the trust-report command to authorized users. Consider masking sensitive information in the default output.
3. 💡 SUGGESTION: Refactor duplicate task history parsing logic into a helper function.
4. 💡 SUGGESTION: Add error handling for empty or malformed task history.
5. 💡 SUGGESTION: Replace _get_demo_peers() and _get_demo_history() with production-ready implementations or mark them as TODO.
6. 💡 SUGGESTION: Add tests for edge cases, such as agents with no history or malformed data.
7. 🟡 WARNING: Update CLI documentation to include the new trust-report command and its usage.

Once these issues are addressed, the feature will be more robust, secure, and user-friendly.

[suyu-lily]

@microsoft-github-policy-service agree