Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
4506147
Implement ORB Worker Adapter
magniloquency Feb 10, 2026
783c10c
Add submit_tasks example to documentation and CI skip list
magniloquency Feb 10, 2026
9133f52
Merge branch 'main' into orb
magniloquency Feb 11, 2026
a1ff776
move orb/ and ami/ to driver/
magniloquency Feb 11, 2026
a46af96
remove superfluous checks in orb config
magniloquency Feb 11, 2026
e8ae3fc
move import
magniloquency Feb 11, 2026
2312d66
add a check for self._orb before returning machines
magniloquency Feb 11, 2026
89d964b
adjust security group rules
magniloquency Feb 11, 2026
248f2f7
move method
magniloquency Feb 11, 2026
0333ee7
rename no random worker ids to deterministic worker ids
magniloquency Feb 11, 2026
ce17847
add comment
magniloquency Feb 11, 2026
2796ead
run submit tasks in ci
magniloquency Feb 11, 2026
389a7b6
fix help text
magniloquency Feb 11, 2026
9eae58d
make _filter_data a static method
magniloquency Feb 11, 2026
d333899
refactor orb worker adapter polling to use constants
magniloquency Feb 11, 2026
562ec6a
flake8
magniloquency Feb 11, 2026
7a61c0e
don't touch skip examples file
magniloquency Feb 11, 2026
594342d
Merge branch 'main' of https://github.com/finos/opengris-scaler into orb
magniloquency Feb 12, 2026
350e840
bump minor version
magniloquency Feb 12, 2026
8ad0a61
Merge branch 'main' into orb
magniloquency Feb 13, 2026
a2d3856
docs: add worker adapter tutorials and update ORB integration details
magniloquency Feb 13, 2026
0329716
Merge branch 'orb' of https://github.com/magniloquency/scaler into orb
magniloquency Feb 13, 2026
c50fddb
refactor: move orb and ami drivers to src/scaler/drivers
magniloquency Feb 13, 2026
90f608a
Delete test.toml
magniloquency Feb 13, 2026
711e9e9
Merge branch 'main' into orb
magniloquency Feb 18, 2026
f789743
Refactor ORB worker adapter to use ZMQ-based protocol
magniloquency Feb 18, 2026
287cd7c
fix type error
magniloquency Feb 18, 2026
1235fae
fix type error
magniloquency Feb 18, 2026
8fda46c
Output ORB templates in snake_case
magniloquency Feb 20, 2026
5c5a76e
Simplify ORB driver config files
magniloquency Feb 20, 2026
2130431
Document default no-scaling policy and vanilla scaler example in work…
magniloquency Feb 20, 2026
2b54a3a
Merge branch 'main' into orb
sharpener6 Feb 23, 2026
4956370
Merge branch 'main' into orb
sharpener6 Feb 23, 2026
31de2fd
import documentation changes from #574
magniloquency Feb 24, 2026
8d893fe
Merge branch 'main' into orb
magniloquency Feb 25, 2026
759f21a
Merge main into orb, adopting worker_manager naming convention
magniloquency Mar 4, 2026
2928bef
Refactor ORB worker adapter to worker_manager_adapter naming and impr…
magniloquency Mar 4, 2026
d31f8ad
Refactor ORB config handling and simplify worker manager
magniloquency Mar 4, 2026
830d35f
Rename run_worker_adapter_orb to run_worker_manager_orb
magniloquency Mar 4, 2026
da6338c
Fix ORB template missing instance_types and broken region injection
magniloquency Mar 4, 2026
940f0d0
Use subnet_ids list field instead of subnet_id in ORB template
magniloquency Mar 4, 2026
ef54411
Add name field to ORBMachine to fix TypeError on deserialization
magniloquency Mar 4, 2026
e90f85c
Filter unknown keys when deserializing ORBMachine from dict
magniloquency Mar 4, 2026
15f3c42
Fix duplicate commands sent to adapter while previous command is in-f…
magniloquency Mar 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/actions/run-test/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,13 @@ runs:
run: |
uv pip install --system -r examples/applications/requirements_applications.txt
uv pip install --system -r examples/ray_compat/requirements.txt
readarray -t skip_examples < examples/skip_examples.txt
for example in "./examples"/*.py; do
filename=$(basename "$example")
if [[ " ${skip_examples[*]} " =~ [[:space:]]${filename}[[:space:]] ]]; then
echo "Skipping $example"
continue
fi
echo "Running $example"
python $example
done
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,6 @@ CMakeFiles/
.vs/
src/scaler/protocol/capnp/*.c++
src/scaler/protocol/capnp/*.h

orb/logs/
orb/metrics/
22 changes: 22 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,8 @@ The following table maps each Scaler command to its corresponding section name i
| `scaler_worker_adapter_native` | `[native_worker_adapter]` |
| `scaler_worker_adapter_fixed_native` | `[fixed_native_worker_adapter]` |
| `scaler_worker_adapter_symphony` | `[symphony_worker_adapter]` |
| `scaler_worker_manager_orb` | `[orb_worker_adapter]` |
| `scaler_worker_adapter_ecs` | `[ecs_worker_adapter]` |

### Practical Scenarios & Examples

Expand Down Expand Up @@ -466,6 +468,26 @@ where `deepest_nesting_level` is the deepest nesting level a task has in your wo
workload that has
a base task that calls a nested task that calls another nested task, then the deepest nesting level is 2.

## ORB (AWS EC2) integration

A Scaler scheduler can interface with ORB (Open Resource Broker) to dynamically provision and manage workers on AWS EC2 instances.

```bash
$ scaler_worker_manager_orb tcp://127.0.0.1:2345 --image-id ami-0528819f94f4f5fa5
```

This will start an ORB worker adapter that connects to the Scaler scheduler at `tcp://127.0.0.1:2345`. The scheduler can then request new workers from this adapter, which will be launched as EC2 instances.

### Configuration

The ORB adapter requires `orb-py` and `boto3` to be installed. You can install them with:

```bash
$ pip install "opengris-scaler[orb]"
```

For more details on configuring ORB, including AWS credentials and instance templates, please refer to the [ORB Worker Adapter documentation](https://finos.github.io/opengris-scaler/tutorials/worker_adapters/orb.html).

## Worker Adapter usage

> **Note**: This feature is experimental and may change in future releases.
Expand Down
1 change: 1 addition & 0 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ Content
tutorials/worker_adapters/index
tutorials/worker_adapters/native
tutorials/worker_adapters/fixed_native
tutorials/worker_adapters/orb
tutorials/worker_adapters/aws_hpc/index
tutorials/worker_adapters/common_parameters
tutorials/compatibility/ray
Expand Down
2 changes: 1 addition & 1 deletion docs/source/tutorials/compatibility/ray.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Ray Compatibility Layer
Scaler is a lightweight distributed computation engine similar to Ray. Scaler supports many of the same concepts as Ray including
remote functions (known as tasks in Scaler), futures, cluster object storage, labels (known as capabilities in Scaler), and it comes with comparable monitoring tools.

Unlike Ray, Scaler supports both local clusters and also easily integrates with multiple cloud providers out of the box, including AWS EC2 and IBM Symphony,
Unlike Ray, Scaler supports both local clusters and also easily integrates with multiple cloud providers out of the box, including ORB (AWS EC2) and IBM Symphony,
with more providers planned for the future. You can view our `roadmap on GitHub <https://github.com/finos/opengris-scaler/discussions/333>`_
for details on upcoming cloud integrations.

Expand Down
2 changes: 2 additions & 0 deletions docs/source/tutorials/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,8 @@ The following table maps each Scaler command to its corresponding section name i
- ``[fixed_native_worker_adapter]``
* - ``scaler_worker_adapter_symphony``
- ``[symphony_worker_adapter]``
* - ``scaler_worker_manager_orb``
- ``[orb_worker_adapter]``
* - ``scaler_worker_adapter_ecs``
- ``[ecs_worker_adapter]``
* - ``python -m scaler.entry_points.worker_adapter_aws_hpc``
Expand Down
8 changes: 8 additions & 0 deletions docs/source/tutorials/examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,14 @@ Shows how to send a basic task to scheduler
.. literalinclude:: ../../../examples/simple_client.py
:language: python

Submit Tasks
~~~~~~~~~~~~

Shows various ways to submit tasks (submit, map, starmap)

.. literalinclude:: ../../../examples/submit_tasks.py
:language: python

Client Mapping Tasks
~~~~~~~~~~~~~~~~~~~~

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Common Worker Adapter Parameters
================================

All worker adapters in Scaler share a set of common configuration parameters for connecting to the cluster, configuring the internal web server, and managing worker behavior.
All worker adapters in Scaler share a set of common configuration parameters for connecting to the cluster and managing worker behavior.

.. note::
For more details on how to configure Scaler, see the :doc:`../configuration` section.
Expand Down
6 changes: 6 additions & 0 deletions docs/source/tutorials/worker_adapters/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,11 @@ AWS HPC

The :doc:`AWS HPC <aws_hpc/index>` worker adapter allows Scaler to offload task execution to cloud environments, currently supporting AWS Batch. It is ideal for bursting workloads to the cloud or utilizing specific hardware not available locally.

ORB (AWS EC2)
~~~~~~~~~~~~~

The :doc:`ORB <orb>` worker adapter allows Scaler to dynamically provision workers on AWS EC2 instances. This is ideal for scaling workloads that require significant cloud compute resources or specialized hardware like GPUs.

Common Parameters
~~~~~~~~~~~~~~~~~

Expand All @@ -56,5 +61,6 @@ All worker adapters share a set of :doc:`common configuration parameters <common

native
fixed_native
orb
aws_hpc/index
common_parameters
132 changes: 132 additions & 0 deletions docs/source/tutorials/worker_adapters/orb.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
ORB Worker Adapter
==================

The ORB worker adapter allows Scaler to dynamically provision workers on AWS EC2 instances using the ORB (Open Resource Broker) system. This is particularly useful for scaling workloads that require significant compute resources or specialized hardware available in the cloud.

This tutorial describes the steps required to get up and running with the ORB adapter.

Requirements
------------

Before using the ORB worker adapter, ensure the following requirements are met on the machine that will run the adapter:

1. **orb-py and boto3**: The ``orb-py`` and ``boto3`` packages must be installed. These can be installed using the ``orb`` optional dependency of Scaler:

.. code-block:: bash

pip install "opengris-scaler[orb]"

2. **AWS CLI**: The AWS Command Line Interface must be installed and configured with a default profile that has permissions to launch, describe, and terminate EC2 instances.

3. **Network Connectivity**: The adapter must be able to communicate with AWS APIs and the Scaler scheduler.

Getting Started
---------------

To start the ORB worker adapter, use the ``scaler_worker_manager_orb`` command.

Example command:

.. code-block:: bash

scaler_worker_manager_orb tcp://<SCHEDULER_EXTERNAL_IP>:8516 \
--object-storage-address tcp://<OSS_EXTERNAL_IP>:8517 \
--image-id ami-0528819f94f4f5fa5 \
--instance-type t3.medium \
--aws-region us-east-1 \
--logging-level INFO \
--task-timeout-seconds 60

Equivalent configuration using a TOML file:

.. code-block:: bash

scaler_worker_manager_orb tcp://<SCHEDULER_EXTERNAL_IP>:8516 --config config.toml

.. code-block:: toml

# config.toml

[orb_worker_adapter]
object_storage_address = "tcp://<OSS_EXTERNAL_IP>:8517"
image_id = "ami-0528819f94f4f5fa5"
instance_type = "t3.medium"
aws_region = "us-east-1"
logging_level = "INFO"
task_timeout_seconds = 60

* ``tcp://<SCHEDULER_EXTERNAL_IP>:8516`` is the address workers will use to connect to the scheduler.
* ``tcp://<OSS_EXTERNAL_IP>:8517`` is the address workers will use to connect to the object storage server.
* New workers will be launched using the specified AMI and instance type.

Networking Configuration
------------------------

Workers launched by the ORB adapter are EC2 instances and require an externally-reachable IP address for the scheduler.

* **Internal Communication**: If the machine running the scheduler is another EC2 instance in the same VPC, you can use EC2 private IP addresses.
* **Public Internet**: If communicating over the public internet, it is highly recommended to set up robust security rules and/or a VPN to protect the cluster.

Publicly Available AMIs
-----------------------

We regularly publish publicly available Amazon Machine Images (AMIs) with Python and ``opengris-scaler`` pre-installed.

.. list-table:: Available Public AMIs
:widths: 15 15 20 20 30
:header-rows: 1

* - Scaler Version
- Python Version
- Amazon Linux 2023 Version
- Date (MM/DD/YYYY)
- AMI ID (us-east-1)
* - 1.14.2
- 3.13
- 2023.10.20260120
- 01/30/2026
- ``ami-0528819f94f4f5fa5``

New AMIs will be added to this list as they become available.

Supported Parameters
--------------------

.. note::
For more details on how to configure Scaler, see the :doc:`../configuration` section.

The ORB worker adapter supports ORB-specific configuration parameters as well as common worker adapter parameters.

Orb Template Configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~

* ``--image-id`` (Required): AMI ID for the worker instances.
* ``--instance-type``: EC2 instance type (default: ``t2.micro``).
* ``--aws-region``: AWS region (default: ``us-east-1``).
* ``--key-name``: AWS key pair name for the instances. If not provided, a temporary key pair will be created and deleted on cleanup.
* ``--subnet-id``: AWS subnet ID where the instances will be launched. If not provided, it attempts to discover the default subnet in the default VPC.
* ``--security-group-ids``: Comma-separated list of AWS security group IDs.
* ``--allowed-ip``: IP address to allow in the security group (if created automatically). Defaults to the adapter's external IP.
* ``--orb-config-path``: Path to the ORB root directory (default: ``src/scaler/drivers/orb``).

Common Parameters
~~~~~~~~~~~~~~~~~

For a full list of common parameters including networking, worker configuration, and logging, see :doc:`common_parameters`.

Cleanup
-------

The ORB worker adapter is designed to be self-cleaning, but it is important to be aware of the resources it manages:

* **Key Pairs**: If a ``--key-name`` is not provided, the adapter creates a temporary AWS key pair.
* **Security Groups**: If ``--security-group-ids`` are not provided, the adapter creates a temporary security group to allow communication.
* **Launch Templates**: ORB may additionally create EC2 Launch Templates as part of the machine provisioning process.

The adapter attempts to delete these temporary resources and terminate all launched EC2 instances when it shuts down gracefully. However, in the event of an ungraceful crash or network failure, some resources may persist in your AWS account.

.. tip::
It is recommended to periodically check your AWS console for any orphaned resources (instances, security groups, key pairs, or launch templates) and clean them up manually if necessary to avoid unexpected costs.

.. warning::
**Subnet and Security Groups**: Currently, specifying ``--subnet-id`` or ``--security-group-ids`` via configuration might not have the intended effect as the adapter is designed to auto-discover or create these resources. Specifically, the adapter may still attempt to use default subnets or create its own temporary security groups regardless of these parameters.
2 changes: 2 additions & 0 deletions examples/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ Ensure that the scheduler and cluster are set up before running clients.
Shows how to send a nested task to scheduler
- `simple_client.py`
Shows how to send a basic task to scheduler
- `submit_tasks.py`
Shows various ways to submit tasks (submit, map, starmap)
- `task_capabilities.py`
Shows how to use capabilities to route task to various workers
- `ray_compat/`
Expand Down
Empty file added examples/skip_examples.txt
Empty file.
67 changes: 67 additions & 0 deletions examples/submit_tasks.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
"""
This example demonstrates various ways to submit tasks to a Scaler scheduler.
It shows how to use the Client to:
1. Submit a single task using .submit()
2. Submit multiple tasks using .map()
3. Submit tasks with multiple arguments using .map() and .starmap()
"""

import argparse
import math

from scaler import Client, SchedulerClusterCombo


def square(value: int):
return value * value


def add(x: int, y: int):
return x + y


def main():
parser = argparse.ArgumentParser(description="Submit tasks to a Scaler scheduler.")
parser.add_argument("url", nargs="?", help="The URL of the Scaler scheduler (e.g., tcp://127.0.0.1:2345)")
args = parser.parse_args()

cluster = None
if args.url is None:

print("No scheduler URL provided. Spinning up a local cluster...")
cluster = SchedulerClusterCombo(n_workers=4)
address = cluster.get_address()
else:
address = args.url

try:
print(f"Connecting to scheduler at {address}...")

# Use the Client as a context manager to ensure proper cleanup
with Client(address=address) as client:
print("Submitting a single task using .submit()...")
future = client.submit(square, 4)
print(f"Result of square(4): {future.result()}")

print("\nSubmitting multiple tasks using .map()...")
# client.map() works like Python's built-in map()
results = client.map(math.sqrt, range(1, 6))
print(f"Results of sqrt(1..5): {list(results)}")

print("\nSubmitting tasks with multiple arguments using .map()...")
# You can pass multiple iterables to map() for functions with multiple arguments
results_add = client.map(add, [1, 2, 3], [10, 20, 30])
print(f"Results of add([1,2,3], [10,20,30]): {list(results_add)}")

print("\nSubmitting tasks with multiple arguments using .starmap()...")
# starmap() takes an iterable of argument tuples
results_starmap = client.starmap(add, [(5, 5), (10, 10)])
print(f"Results of starmap(add, [(5,5), (10,10)]): {list(results_starmap)}")
finally:
if cluster:
cluster.shutdown()
print("\nAll tasks completed successfully.")


if __name__ == "__main__":
main()
6 changes: 6 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,10 @@ graphblas = [
aws = [
"boto3",
]
orb = [
"orb-py~=1.1; python_version >= '3.10'",
"boto3; python_version >= '3.10'",
]
all = [
"nicegui[plotly]==2.24.2; python_version == '3.8'",
"nicegui[plotly]==3.6.1; python_version >= '3.9'",
Expand All @@ -63,6 +67,7 @@ all = [
"numpy==2.2.6; python_version >= '3.10'",
"uvloop; platform_system != 'Windows'",
"boto3",
"orb-py~=1.1; python_version >= '3.10'"
]

[dependency-groups]
Expand Down Expand Up @@ -99,6 +104,7 @@ scaler_worker_manager_baremetal_fixed_native = "scaler.entry_points.worker_manag
scaler_worker_manager_symphony = "scaler.entry_points.worker_manager_symphony:main"
scaler_worker_manager_aws_raw_ecs = "scaler.entry_points.worker_manager_aws_raw_ecs:main"
scaler_worker_manager_aws_hpc_batch = "scaler.entry_points.worker_manager_aws_hpc_batch:main"
scaler_worker_manager_orb = "scaler.entry_points.worker_manager_orb:main"

[tool.scikit-build]
cmake.source-dir = "."
Expand Down
5 changes: 5 additions & 0 deletions src/run_worker_manager_orb.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
from scaler.entry_points.worker_manager_orb import main
from scaler.utility.debug import pdb_wrapped

if __name__ == "__main__":
pdb_wrapped(main)()
1 change: 1 addition & 0 deletions src/scaler/cluster/cluster.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ def __init__(self, config: ClusterConfig):
self._logging_paths = config.logging_config.paths
self._logging_config_file = config.logging_config.config_file
self._logging_level = config.logging_config.level
self._deterministic_worker_ids = config.deterministic_worker_ids

# we create the config here, but create the actual adapter in the run method
# to ensure that it's created in the correct process
Expand Down
4 changes: 4 additions & 0 deletions src/scaler/config/section/cluster.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,10 @@ class ClusterConfig(ConfigClass):
)
worker_config: WorkerConfig = dataclasses.field(default_factory=WorkerConfig)
logging_config: LoggingConfig = dataclasses.field(default_factory=LoggingConfig)
deterministic_worker_ids: bool = dataclasses.field(
default=False,
metadata=dict(short="-dwi", action="store_true", help="enable deterministic worker id generation"),
)

def __post_init__(self):
if self.worker_names.names and len(self.worker_names.names) != self.num_of_workers:
Expand Down
Loading
Loading