Skip to content

Add meta agent option and add continuous action space compatibility to async#297

Merged
Mark2000 merged 6 commits intodevelopfrom
feature/296-meta-agent
Jul 23, 2025
Merged

Add meta agent option and add continuous action space compatibility to async#297
Mark2000 merged 6 commits intodevelopfrom
feature/296-meta-agent

Conversation

@Mark2000
Copy link
Contributor

@Mark2000 Mark2000 commented Jul 18, 2025

Description

Closes #296

Allows for meta-agents to be created that concatenate the observation and action spaces of the satellites.

Also fixes some bugs in the async connectors for continuous action spaces.

⚠️Note to reviewer⚠️ Primarily concerned with changes to gym.py. RLlib utilities are not robustly tested/supported.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

How should this pull request be reviewed?

  • By commit
  • All changes at once

How Has This Been Tested?

Added tests for meta-agents.

Future Work

None.

Checklist

  • I have performed a self-review of my code
  • I have commented my code in hard-to-understand areas
  • I have made corresponding changes to the documentation and release notes
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works

Copilot AI review requested due to automatic review settings July 18, 2025 22:18
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds meta-agent support to the ConstellationTasking environment, allowing multiple satellites to be grouped under a single agent that controls them with concatenated observation and action spaces. Additionally, it fixes bugs related to continuous action space handling in async connectors and modernizes the PyTorch policy loading functionality.

  • Added meta-agent groupings functionality with observation/action space concatenation
  • Fixed continuous action space compatibility issues in async connectors
  • Updated PyTorch policy loading to use modern RLModule checkpoint format

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
tests/integration/test_int_full_environments.py Added test environment with meta-agent groupings and updated test to cover both regular and meta-agent scenarios
src/bsk_rl/utils/rllib/discounting.py Fixed continuous action space handling with new utility functions and improved NO_ACTION detection
src/bsk_rl/utils/rllib/init.py Modernized PyTorch policy loading to use RLModule checkpoint format instead of legacy approach
src/bsk_rl/gym.py Added core meta-agent implementation with observation/action space concatenation and satellite grouping logic
Comments suppressed due to low confidence (1)

@Mark2000 Mark2000 force-pushed the feature/296-meta-agent branch from e01b2b5 to f17dc71 Compare July 18, 2025 22:24
@Mark2000 Mark2000 force-pushed the feature/296-meta-agent branch from d88c656 to 4195ae0 Compare July 18, 2025 22:57
@Mark2000 Mark2000 merged commit b030aed into develop Jul 23, 2025
13 of 15 checks passed
@Mark2000 Mark2000 deleted the feature/296-meta-agent branch July 23, 2025 23:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Meta-agent environment

1 participant