From 22ccbed1bda69aee9e7270fbd67cbd819b63d1f9 Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 8 Jan 2025 12:15:33 -0800 Subject: [PATCH 01/24] [Orleans] Silo Metadata and Placement Filtering --- .../grains/grain-placement-filtering.md | 110 ++++++++++++++++ docs/orleans/grains/silo-metadata.md | 121 ++++++++++++++++++ 2 files changed, 231 insertions(+) create mode 100644 docs/orleans/grains/grain-placement-filtering.md create mode 100644 docs/orleans/grains/silo-metadata.md diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md new file mode 100644 index 0000000000000..f457be746e5a9 --- /dev/null +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -0,0 +1,110 @@ +--- +title: Grain placement filtering +description: Learn about grain placement filtering in .NET Orleans. +ms.date: 01/08/2025 +--- + +# Grain Placement Filtering + +## Overview + +Placement Filtering in Orleans allows developers additional control over the placement of grains within a cluster. It works in conjunction with placement strategies, adding an additional layer of filtering to determine candidate silos for grain activation. + +This filterting takes place before candidate silos are passed on to the configured placement method allowing for more flexibility and reuse of the filters. +For example, the existing PreferLocal placement strategy is hard coded to fall back to Random placement if the local silo is unable to host the grain type. But by using filters, a `PreferLocalPlacementFilter` could be implemented to filter down to either the local silo or all compatible silos. Then any placement strategy (Random, Resource Optimised Placement, Activation Count, etc.) could be configured for that grain type. This allows for any set of filters and any placement strategy to be configured for a grain type. + +--- + +## How Placement Filtering Works + +Placement Filtering operates as an additional step in the grain placement process. After all compatible silos for the grain type are identified, all Placement Filters configured for that grain type, if any, are applied to allow further refinement of the selection by eliminating silos that do not meet the defined criteria. + +### Ordering + +Filters running in different orders may result in different behavior so explicit ordering is required when two or more filters are defined on a type. This needs to be configured with the `order:` parameter, as the type metadata pulled at runtime may return the attributes on a type in a different order from how they appear in the source code. Ordering must have unique values so an explicit ordering can be determined. + +--- + +## Built-in Filters + +### Silo Metadata + +These filters work with [*Silo Metadata*](../grains/silo-metadata.md) to filter candidate silos. + +#### RequiredMatchSiloMetadata + +Silo Metadata is used to filter candidate silos to only ones that matches all of the specified metadata keys with the calling silo. If there are no compatible silos that match all of the keys then an empty set of silos will be returned and placement will ultimately fail for the grain. + +#### PreferredMatchSiloMetadata + +This filtering will attempt to be filtered to only silos that match all of the configured metadata keys with the calling silo. However, instead of returning an empty set if there are not matches as the above Required filtering this will then fall back to partial matches. The first configured metadata key is dropped and a match is made against the remaining keys. This will continue, dropping the initial keys, until a sufficient number of matches are made. If there are not any compatible silos that match *any* of the metadata keys, then all of the candidate silos are returned. + +The `minCandidates` value configures how many candidates must be found to stop the filtering process. This value is used to prevent a single silo from getting quickly overloaded if it would be the only match. +For example, if filtering on `[PreferredMatchSiloMetadata(["cloud.availability-zone", "cloud.region"], minCandidates:2)]` and there is only one matching silo on both `cloud.availability-zone` and `cloud.region`, then all activations would get placed on that one silo. It is often desirable to not focus activation (or do scheduling in general) on one target. With the above `minCandidates` value of 2, this scenario would fail to match on both keys because only one silo matches both metadata keys. Then it would then fall back to matching only on `cloud.region`. If there were 2 or more silos that match only that key then those would get returned. Otherwise, it would fall back to returning all of the candidates. Note that this config is a minimum value; more candidates could be returned. If you would prefer a most specific matching only then setting this to 1 would only return the best match (the one silo in the above scenario). This could be preferable in specific use cases where there is low activation throughput and where there is a great penalty when moving to a less specific match from a more specific one. In general use, the default value of 2 should be used (and not need to be specified in the attribute). + +--- + +## Implementing Placement Filters + +To implement a custom Placement Filter in Orleans, follow these steps: + +1. **Implementation** + - Create marker Attribute derived from `PlacementFilterAttribute` + - Create Strategy derived from `PlacementFilterStrategy` to manage any configuration values + - Create Director derived from `IPlacementFilterDirector` which contains the filtering logic + - Define the filtering logic in the `Filter` method, which takes a list of candidate silos and returns a filtered list. + +2. **Register the Filter** + - Call `AddPlacementFilter` to register the Strategy and corresponding Director + +3. **Apply the Filter** + - Add the Attribute to a grain class to apply the filter + +Here is an example of a simple custom Placement Filter. It is similar in behavior to using `[PreferLocalPlacement]` without any filter, but this has the advantage of being able to specify any placement method. Whereas `PreferLocalPlacement` falls back to Random placement if the local silo is unable to host a grain, this example has configured `ActivationCountBasedPlacement`. Any other placement could similarly be used with this filter + +```csharp +[AttributeUsage(AttributeTargets.Class, AllowMultiple = false)] +public class ExamplePreferLocalPlacementFilterAttribute(int order) + : PlacementFilterAttribute(new ExamplePreferLocalPlacementFilterStrategy(order)); +``` + +```csharp +public class ExamplePreferLocalPlacementFilterStrategy(int order) : PlacementFilterStrategy(order); +``` + +```csharp +internal class ExamplePreferLocalPlacementFilterDirector(ILocalSiloDetails localSiloDetails) + : IPlacementFilterDirector +{ + public IEnumerable Filter(PlacementFilterStrategy filterStrategy, PlacementTarget target, IEnumerable silos) + { + var siloList = silos.ToList(); + var localSilo = siloList.FirstOrDefault(s => s == localSiloDetails.SiloAddress); + if (localSilo is not null) + { + return [localSilo]; + } + return siloList; + } +} +``` + +After implementing this filter, it can be registered and applied to grains. + +```csharp +builder.ConfigureServices(services => +{ + services.AddPlacementFilter(); +}); +``` + +```csharp +[ExamplePreferLocalPlacementFilter] +[ActivationCountBasedPlacement] +public class MyGrain() : Grain, IMyGrain +{ + ... +} +``` + +--- diff --git a/docs/orleans/grains/silo-metadata.md b/docs/orleans/grains/silo-metadata.md new file mode 100644 index 0000000000000..1e50316b320ce --- /dev/null +++ b/docs/orleans/grains/silo-metadata.md @@ -0,0 +1,121 @@ +--- +title: Silo metadata +description: Learn about silo metadata in .NET Orleans. +ms.date: 01/08/2025 +--- + +# Silo Metadata + +Silo Metadata is a new feature in Orleans that allows developers to assign custom metadata to silos within a cluster. This metadata provides a flexible mechanism for annotating silos with descriptive information or specific capabilities. + +This feature is particularly useful in scenarios where different silos have distinct roles, hardware configurations, or other unique characteristics. For example, silos can be tagged based on their region, compute power, or specialized responsibilities within the system. + +Silo Metadata lays the groundwork for additional Orleans features, such as Placement Filtering. + +## Key Concepts + +Silo Metadata introduces a way to attach key-value pairs to silos within an Orleans cluster. This feature allows developers to configure silo-specific characteristics that can be leveraged by Orleans components. + +Silo Metadata is represented as an **immutable** dictionary of key-value pairs: + +- **Keys**: Strings that identify the metadata (e.g., `"cloud.region"`, `"compute.reservation.type"`). +- **Values**: Strings that describe the corresponding property (e.g., `"us-east1"`, `"spot"`). + +## Configuration + +Silo Metadata in Orleans can be configured using two methods: via .NET Configuration or directly in code. + +### **Configuring Silo Metadata via .NET Configuration** + +Silo Metadata can be defined in the application’s Configuration, such as `appsettings.json`, environment variables, or any other Configuration source. + +#### Example: `appsettings.json` Configuration + +```json +{ + "Orleans": { + "Metadata": { + "cloud.region": "us-east1", + "compute.reservation.type": "spot", + "role": "worker" + } + } +} +``` + +The above configuration defines metadata for a silo, tagging it with: + +- `cloud.region`: `"us-east1"` +- `compute.reservation.type`: `"spot"` +- `role`: `"worker"` + +To apply this configuration, use the following setup in your silo host builder: + +```csharp +var siloBuilder = new SiloHostBuilder() + // Configuration section Orleans:Metadata is used by default + .UseSiloMetadata(); +``` + +Alternatively, an explicit `IConfiguration` or `IConfigurationSection` can be passed in to control where in configuration the metadata is pulled from. + +--- + +### **Configuring Silo Metadata Directly in Code** + +For scenarios requiring programmatic metadata configuration, developers can add metadata directly in the silo host builder. + +#### Example: Direct Code Configuration + +```csharp +var siloBuilder = new SiloHostBuilder() + .UseSiloMetadata(new Dictionary + { + {"cloud.region", "us-east1"}, + {"compute.reservation.type", "spot"}, + {"role", "worker"} + }); +``` + +This example achieves the same result as the JSON configuration but allows metadata values to be computed or loaded dynamically during silo initialization. + +--- + +### **Merging Configurations** + +If both .NET Configuration and direct code configuration are used, the direct configuration overrides any conflicting metadata values from the .NET Configuration. This allows developers to set defaults via configuration files and dynamically adjust specific metadata during runtime. + +## Usage + +Developers can retrieve metadata through the `ISiloMetadataCache` interface. This interface allows for querying metadata for individual silos across the cluster. Metadata will always be returned from a local cache of metadata that gets updated in the background as cluster membership changes. + +### **Accessing Metadata for a Specific Silo** + +The `ISiloMetadataCache` provides a method to retrieve the metadata for a specific silo by its unique identifier (`SiloAddress`). The `ISoloMetadataCache` implementation is registered in the `UseSiloMetadata` method and can be injected as a dependency. + +#### Example: Accessing Metadata for a Silo + +```csharp +var siloMetadata = siloMetadataCache.GetSiloMetadata(siloAddress); + +if (siloMetadata.Metadata.TryGetValue("role", out var role)) +{ + Console.WriteLine($"Silo Role for {siloAddress}: {role}"); + // Execute role-specific logic +} +``` + +In this example: + +- `GetSiloMetadata(siloAddress)` retrieves the metadata for the specified silo. +- Metadata keys like `"role"` can be used to influence application logic. + +--- + +## Internal Implementation + +Internally, the `SiloMetadataCache` monitors changes in cluster membership on `MembershipTableManager` and will keep the local cache of metadata in sync with membership changes. Metadata is immutable for a given Silo so it will be retreived once and cached until that Silo leaves the cluster. Cached metadata for clusters that are `Dead` or have left the membership table will be cleared out of the local cache. + +Each silo hosts a [*GrainService*](../grains/grainservices.md) that provides that silo's metadata. Other silos request a client to a specific silo's `GrainService` to pull a remote silo's metadata to populate its local cache. + +Calls to `SiloMetadataCache : ISiloMetadataCache` then return a result from this local cache. \ No newline at end of file From 94b2fb882d209e02741608701e5390c967518955 Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 08:10:50 -0800 Subject: [PATCH 02/24] Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine --- docs/orleans/grains/grain-placement-filtering.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index f457be746e5a9..43040677db8cc 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -12,9 +12,6 @@ Placement Filtering in Orleans allows developers additional control over the pla This filterting takes place before candidate silos are passed on to the configured placement method allowing for more flexibility and reuse of the filters. For example, the existing PreferLocal placement strategy is hard coded to fall back to Random placement if the local silo is unable to host the grain type. But by using filters, a `PreferLocalPlacementFilter` could be implemented to filter down to either the local silo or all compatible silos. Then any placement strategy (Random, Resource Optimised Placement, Activation Count, etc.) could be configured for that grain type. This allows for any set of filters and any placement strategy to be configured for a grain type. - ---- - ## How Placement Filtering Works Placement Filtering operates as an additional step in the grain placement process. After all compatible silos for the grain type are identified, all Placement Filters configured for that grain type, if any, are applied to allow further refinement of the selection by eliminating silos that do not meet the defined criteria. From bbde6f275cf6a3f38a472a85a150cb19c76c9b04 Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 08:11:00 -0800 Subject: [PATCH 03/24] Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine --- docs/orleans/grains/grain-placement-filtering.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index 43040677db8cc..b2b38d50e4cba 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -12,7 +12,8 @@ Placement Filtering in Orleans allows developers additional control over the pla This filterting takes place before candidate silos are passed on to the configured placement method allowing for more flexibility and reuse of the filters. For example, the existing PreferLocal placement strategy is hard coded to fall back to Random placement if the local silo is unable to host the grain type. But by using filters, a `PreferLocalPlacementFilter` could be implemented to filter down to either the local silo or all compatible silos. Then any placement strategy (Random, Resource Optimised Placement, Activation Count, etc.) could be configured for that grain type. This allows for any set of filters and any placement strategy to be configured for a grain type. -## How Placement Filtering Works +## How placement filtering works + Placement Filtering operates as an additional step in the grain placement process. After all compatible silos for the grain type are identified, all Placement Filters configured for that grain type, if any, are applied to allow further refinement of the selection by eliminating silos that do not meet the defined criteria. From 475b2c9d2ac79a52667828fdbe91c3fde51b747c Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 08:11:11 -0800 Subject: [PATCH 04/24] Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine --- docs/orleans/grains/grain-placement-filtering.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index b2b38d50e4cba..dcaf63dda7c77 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -4,9 +4,7 @@ description: Learn about grain placement filtering in .NET Orleans. ms.date: 01/08/2025 --- -# Grain Placement Filtering - -## Overview +# Grain placement filtering Placement Filtering in Orleans allows developers additional control over the placement of grains within a cluster. It works in conjunction with placement strategies, adding an additional layer of filtering to determine candidate silos for grain activation. From 2ebff2e730a3e5f934d6a23a55824b7dfe44a5e2 Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 08:11:22 -0800 Subject: [PATCH 05/24] Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine --- docs/orleans/grains/grain-placement-filtering.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index dcaf63dda7c77..1e18cb327bb27 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -1,6 +1,7 @@ --- title: Grain placement filtering -description: Learn about grain placement filtering in .NET Orleans. +description: Learn about grain placement filtering in Microsoft Orleans. + ms.date: 01/08/2025 --- From f635aa6ce747798beb9dce8bf6817909bdcd56d1 Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 08:11:35 -0800 Subject: [PATCH 06/24] Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine --- docs/orleans/grains/grain-placement-filtering.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index 1e18cb327bb27..8b0b5b0287ef4 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -7,7 +7,8 @@ ms.date: 01/08/2025 # Grain placement filtering -Placement Filtering in Orleans allows developers additional control over the placement of grains within a cluster. It works in conjunction with placement strategies, adding an additional layer of filtering to determine candidate silos for grain activation. +Glain placement filtering in Orleans allows developers additional control over the placement of grains within a cluster. It works in conjunction with placement strategies, adding an additional layer of filtering to determine candidate silos for grain activation. + This filterting takes place before candidate silos are passed on to the configured placement method allowing for more flexibility and reuse of the filters. For example, the existing PreferLocal placement strategy is hard coded to fall back to Random placement if the local silo is unable to host the grain type. But by using filters, a `PreferLocalPlacementFilter` could be implemented to filter down to either the local silo or all compatible silos. Then any placement strategy (Random, Resource Optimised Placement, Activation Count, etc.) could be configured for that grain type. This allows for any set of filters and any placement strategy to be configured for a grain type. From f3f82cb535cf8ba28316361299a14e260891c2f7 Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 08:12:16 -0800 Subject: [PATCH 07/24] Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine --- docs/orleans/grains/grain-placement-filtering.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index 8b0b5b0287ef4..2d5b797bf715d 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -11,7 +11,9 @@ Glain placement filtering in Orleans allows developers additional control over t This filterting takes place before candidate silos are passed on to the configured placement method allowing for more flexibility and reuse of the filters. -For example, the existing PreferLocal placement strategy is hard coded to fall back to Random placement if the local silo is unable to host the grain type. But by using filters, a `PreferLocalPlacementFilter` could be implemented to filter down to either the local silo or all compatible silos. Then any placement strategy (Random, Resource Optimised Placement, Activation Count, etc.) could be configured for that grain type. This allows for any set of filters and any placement strategy to be configured for a grain type. + +For example, the existing `PreferLocal` placement strategy is hard coded to fall back to `Random` placement if the local silo is unable to host the grain type. But by using filters, a `PreferLocalPlacementFilter` could be implemented to filter down to either the local silo or all compatible silos. Then any placement strategy (`Random`, `ResourceOptimizedPlacement`, `ActivationCount`, etc.) could be configured for that grain type. This allows for any set of filters and any placement strategy to be configured for a grain type. + ## How placement filtering works From ebae17924b90aa321b74ed6a9b6c411835400e24 Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 08:12:31 -0800 Subject: [PATCH 08/24] Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine --- docs/orleans/grains/grain-placement-filtering.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index 2d5b797bf715d..d442c2640eab8 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -17,7 +17,8 @@ For example, the existing `PreferLocal` placement strategy is hard coded to fall ## How placement filtering works -Placement Filtering operates as an additional step in the grain placement process. After all compatible silos for the grain type are identified, all Placement Filters configured for that grain type, if any, are applied to allow further refinement of the selection by eliminating silos that do not meet the defined criteria. +Placement filtering operates as an additional step in the grain placement process. After all compatible silos for the grain type are identified, all placement filters configured for that grain type, if any, are applied to allow further refinement of the selection by eliminating silos that do not meet the defined criteria. + ### Ordering From 10e419704e63b269b1ca43cbd9eecfdc40d0ad32 Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 08:12:44 -0800 Subject: [PATCH 09/24] Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine --- docs/orleans/grains/grain-placement-filtering.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index d442c2640eab8..d0b8f826f23b3 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -45,7 +45,8 @@ For example, if filtering on `[PreferredMatchSiloMetadata(["cloud.availability-z --- -## Implementing Placement Filters +## Implement placement filters + To implement a custom Placement Filter in Orleans, follow these steps: From 8427b82e75a76a3e8fac0ea940ff27b3572a53a2 Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 08:13:19 -0800 Subject: [PATCH 10/24] Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine --- docs/orleans/grains/grain-placement-filtering.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index d0b8f826f23b3..65e4c96503987 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -24,11 +24,11 @@ Placement filtering operates as an additional step in the grain placement proces Filters running in different orders may result in different behavior so explicit ordering is required when two or more filters are defined on a type. This needs to be configured with the `order:` parameter, as the type metadata pulled at runtime may return the attributes on a type in a different order from how they appear in the source code. Ordering must have unique values so an explicit ordering can be determined. ---- +## Built-in filters -## Built-in Filters +Orleans provides various built-in filters for you to choose from. However, if you are unable to find one that suits your needs you can always [implement your own](#implement-placement-filters). -### Silo Metadata +### Silo metadata These filters work with [*Silo Metadata*](../grains/silo-metadata.md) to filter candidate silos. From 8908247305dbf4799cdb622c5ec9e2bf77dc4c3b Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 08:13:38 -0800 Subject: [PATCH 11/24] Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine --- docs/orleans/grains/grain-placement-filtering.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index 65e4c96503987..4e4e0fca1a908 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -32,7 +32,8 @@ Orleans provides various built-in filters for you to choose from. However, if yo These filters work with [*Silo Metadata*](../grains/silo-metadata.md) to filter candidate silos. -#### RequiredMatchSiloMetadata +#### `RequiredMatchSiloMetadata` + Silo Metadata is used to filter candidate silos to only ones that matches all of the specified metadata keys with the calling silo. If there are no compatible silos that match all of the keys then an empty set of silos will be returned and placement will ultimately fail for the grain. From 30180a8be6e57bab368bfd830600f9174ff4ccbe Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 08:13:49 -0800 Subject: [PATCH 12/24] Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine --- docs/orleans/grains/grain-placement-filtering.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index 4e4e0fca1a908..7fc392bc58aec 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -37,7 +37,8 @@ These filters work with [*Silo Metadata*](../grains/silo-metadata.md) to filter Silo Metadata is used to filter candidate silos to only ones that matches all of the specified metadata keys with the calling silo. If there are no compatible silos that match all of the keys then an empty set of silos will be returned and placement will ultimately fail for the grain. -#### PreferredMatchSiloMetadata +#### `PreferredMatchSiloMetadata` + This filtering will attempt to be filtered to only silos that match all of the configured metadata keys with the calling silo. However, instead of returning an empty set if there are not matches as the above Required filtering this will then fall back to partial matches. The first configured metadata key is dropped and a match is made against the remaining keys. This will continue, dropping the initial keys, until a sufficient number of matches are made. If there are not any compatible silos that match *any* of the metadata keys, then all of the candidate silos are returned. From fbb81586482cb1ae42409cdde3b84b0d28fec3ce Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 08:15:34 -0800 Subject: [PATCH 13/24] Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine --- docs/orleans/grains/grain-placement-filtering.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index 7fc392bc58aec..f54f2e08c1127 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -45,8 +45,6 @@ This filtering will attempt to be filtered to only silos that match all of the c The `minCandidates` value configures how many candidates must be found to stop the filtering process. This value is used to prevent a single silo from getting quickly overloaded if it would be the only match. For example, if filtering on `[PreferredMatchSiloMetadata(["cloud.availability-zone", "cloud.region"], minCandidates:2)]` and there is only one matching silo on both `cloud.availability-zone` and `cloud.region`, then all activations would get placed on that one silo. It is often desirable to not focus activation (or do scheduling in general) on one target. With the above `minCandidates` value of 2, this scenario would fail to match on both keys because only one silo matches both metadata keys. Then it would then fall back to matching only on `cloud.region`. If there were 2 or more silos that match only that key then those would get returned. Otherwise, it would fall back to returning all of the candidates. Note that this config is a minimum value; more candidates could be returned. If you would prefer a most specific matching only then setting this to 1 would only return the best match (the one silo in the above scenario). This could be preferable in specific use cases where there is low activation throughput and where there is a great penalty when moving to a less specific match from a more specific one. In general use, the default value of 2 should be used (and not need to be specified in the attribute). ---- - ## Implement placement filters From fc0b782ebe9af4adc3d5330e919052f896364000 Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 08:15:55 -0800 Subject: [PATCH 14/24] Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine --- docs/orleans/grains/grain-placement-filtering.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index f54f2e08c1127..210f485b4c350 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -48,7 +48,8 @@ For example, if filtering on `[PreferredMatchSiloMetadata(["cloud.availability-z ## Implement placement filters -To implement a custom Placement Filter in Orleans, follow these steps: +To implement a custom placement filter in Orleans, follow these steps: + 1. **Implementation** - Create marker Attribute derived from `PlacementFilterAttribute` From 7ed487835105ea09becf94938ff87510c492fcc3 Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 08:16:26 -0800 Subject: [PATCH 15/24] Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine --- docs/orleans/grains/grain-placement-filtering.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index 210f485b4c350..8a1244caa781e 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -66,9 +66,11 @@ To implement a custom placement filter in Orleans, follow these steps: Here is an example of a simple custom Placement Filter. It is similar in behavior to using `[PreferLocalPlacement]` without any filter, but this has the advantage of being able to specify any placement method. Whereas `PreferLocalPlacement` falls back to Random placement if the local silo is unable to host a grain, this example has configured `ActivationCountBasedPlacement`. Any other placement could similarly be used with this filter ```csharp -[AttributeUsage(AttributeTargets.Class, AllowMultiple = false)] +[AttributeUsage( + AttributeTargets.Class, AllowMultiple = false)] public class ExamplePreferLocalPlacementFilterAttribute(int order) - : PlacementFilterAttribute(new ExamplePreferLocalPlacementFilterStrategy(order)); + : PlacementFilterAttribute( + new ExamplePreferLocalPlacementFilterStrategy(order)); ``` ```csharp From 2324adc19d7731d45975bb6cad3e0335226b43db Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 08:16:44 -0800 Subject: [PATCH 16/24] Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine --- docs/orleans/grains/grain-placement-filtering.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index 8a1244caa781e..6177c4d594a53 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -78,7 +78,8 @@ public class ExamplePreferLocalPlacementFilterStrategy(int order) : PlacementFil ``` ```csharp -internal class ExamplePreferLocalPlacementFilterDirector(ILocalSiloDetails localSiloDetails) +internal class ExamplePreferLocalPlacementFilterDirector( + ILocalSiloDetails localSiloDetails) : IPlacementFilterDirector { public IEnumerable Filter(PlacementFilterStrategy filterStrategy, PlacementTarget target, IEnumerable silos) From 956ea79ef01b774ba2e80709025a6071f341d329 Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 08:19:47 -0800 Subject: [PATCH 17/24] Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine --- docs/orleans/grains/grain-placement-filtering.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index 6177c4d594a53..47c73a761b961 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -109,7 +109,8 @@ builder.ConfigureServices(services => [ActivationCountBasedPlacement] public class MyGrain() : Grain, IMyGrain { - ... + // ... + } ``` From e0dd2fe6273b549e366379926563e39756cab4e7 Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 08:19:53 -0800 Subject: [PATCH 18/24] Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine --- docs/orleans/grains/grain-placement-filtering.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index 47c73a761b961..59c91db58f154 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -113,5 +113,3 @@ public class MyGrain() : Grain, IMyGrain } ``` - ---- From 812352cd329d6f3cfbdfb3af2cb3bbe6719e893d Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 08:20:14 -0800 Subject: [PATCH 19/24] Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine --- docs/orleans/grains/grain-placement-filtering.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index 59c91db58f154..fc8e060f2f1e3 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -100,7 +100,10 @@ After implementing this filter, it can be registered and applied to grains. ```csharp builder.ConfigureServices(services => { - services.AddPlacementFilter(); + services.AddPlacementFilter< + ExamplePreferLocalPlacementFilterStrategy, + ExamplePreferLocalPlacementFilterDirector>(); + }); ``` From 5a3dd0308921013c2fa707b7b0910f240129245f Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 08:20:34 -0800 Subject: [PATCH 20/24] Update docs/orleans/grains/grain-placement-filtering.md Co-authored-by: David Pine --- docs/orleans/grains/grain-placement-filtering.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index fc8e060f2f1e3..b98272f7a4e73 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -74,7 +74,8 @@ public class ExamplePreferLocalPlacementFilterAttribute(int order) ``` ```csharp -public class ExamplePreferLocalPlacementFilterStrategy(int order) : PlacementFilterStrategy(order); +public class ExamplePreferLocalPlacementFilterStrategy(int order) + : PlacementFilterStrategy(order); ``` ```csharp From 0be5521d9fcfcf84d74f9b198790cfd15efd33a1 Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 09:33:42 -0800 Subject: [PATCH 21/24] reworded explaination --- docs/orleans/grains/grain-placement-filtering.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index b98272f7a4e73..1fa80517773ea 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -39,11 +39,13 @@ Silo Metadata is used to filter candidate silos to only ones that matches all of #### `PreferredMatchSiloMetadata` +This filtering will attempt to select only silos that match all of the configured metadata keys with the values of the calling silo. However, instead of returning an empty set if there are not matches as the above Required filtering this will then fall back to partial matches. The first configured metadata key is dropped and a match is made against the remaining keys. This will continue, dropping the initial keys, until a sufficient number of matches are made. If there are not any compatible silos that match *any* of the metadata keys, then all of the candidate silos are returned. -This filtering will attempt to be filtered to only silos that match all of the configured metadata keys with the calling silo. However, instead of returning an empty set if there are not matches as the above Required filtering this will then fall back to partial matches. The first configured metadata key is dropped and a match is made against the remaining keys. This will continue, dropping the initial keys, until a sufficient number of matches are made. If there are not any compatible silos that match *any* of the metadata keys, then all of the candidate silos are returned. +The `minCandidates` value configures how many candidates must be found to stop the filtering process. This value is used to prevent a single silo or small number of silos from getting quickly overloaded if the match were that small. -The `minCandidates` value configures how many candidates must be found to stop the filtering process. This value is used to prevent a single silo from getting quickly overloaded if it would be the only match. -For example, if filtering on `[PreferredMatchSiloMetadata(["cloud.availability-zone", "cloud.region"], minCandidates:2)]` and there is only one matching silo on both `cloud.availability-zone` and `cloud.region`, then all activations would get placed on that one silo. It is often desirable to not focus activation (or do scheduling in general) on one target. With the above `minCandidates` value of 2, this scenario would fail to match on both keys because only one silo matches both metadata keys. Then it would then fall back to matching only on `cloud.region`. If there were 2 or more silos that match only that key then those would get returned. Otherwise, it would fall back to returning all of the candidates. Note that this config is a minimum value; more candidates could be returned. If you would prefer a most specific matching only then setting this to 1 would only return the best match (the one silo in the above scenario). This could be preferable in specific use cases where there is low activation throughput and where there is a great penalty when moving to a less specific match from a more specific one. In general use, the default value of 2 should be used (and not need to be specified in the attribute). +If `minCandidates` were not considered, then there could be a scenario where there are a large number of silos but only one silo that best matches the configured metadata keys. All placements would be concentrated on that one silo despite having many more available that could host activations. The purpose of `minCandidates` is to allow for a balance between preferring only best matches and avoiding hot silos. It is often desirable to not focus activation (or do scheduling in general) on one target. Set it to a value larger than 1 to ensure a minimum candidate set size so that future placement decisions are able to avoid concentrating activations on one or a few hot silos. Note that this config is a minimum value; more candidates could be returned. If you would prefer most specific matching only then set `minCandidates: 1` to always prefer best match. This might be preferable in specific use cases where there is low activation throughput and where there is a great penalty when moving to a less specific match from a more specific one. In general use, the default value of 2 should be used (and not need to be specified in the attribute). + +For example, if filtering on `[PreferredMatchSiloMetadata(["cloud.availability-zone", "cloud.region"], minCandidates:2)]` and there is only one matching silo on both `cloud.availability-zone` and `cloud.region`, then this scenario with `minCandidates: 2` would fail to match on both keys because only one silo matches both metadata keys and that's below the minimum configured size of 2. It would then then fall back to matching only on `cloud.region`. If there were 2 or more silos that match only `cloud.region` then those would get returned. Otherwise, it would fall back to returning all of the candidates. ## Implement placement filters From 358a9e2c0e684ebed86c9ef21b8a398a9e927b0b Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Wed, 22 Jan 2025 09:37:38 -0800 Subject: [PATCH 22/24] fixing markdownlint erros for multiple newlines --- docs/orleans/grains/grain-placement-filtering.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index 1fa80517773ea..c71a638b1353f 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -9,17 +9,14 @@ ms.date: 01/08/2025 Glain placement filtering in Orleans allows developers additional control over the placement of grains within a cluster. It works in conjunction with placement strategies, adding an additional layer of filtering to determine candidate silos for grain activation. - This filterting takes place before candidate silos are passed on to the configured placement method allowing for more flexibility and reuse of the filters. For example, the existing `PreferLocal` placement strategy is hard coded to fall back to `Random` placement if the local silo is unable to host the grain type. But by using filters, a `PreferLocalPlacementFilter` could be implemented to filter down to either the local silo or all compatible silos. Then any placement strategy (`Random`, `ResourceOptimizedPlacement`, `ActivationCount`, etc.) could be configured for that grain type. This allows for any set of filters and any placement strategy to be configured for a grain type. ## How placement filtering works - Placement filtering operates as an additional step in the grain placement process. After all compatible silos for the grain type are identified, all placement filters configured for that grain type, if any, are applied to allow further refinement of the selection by eliminating silos that do not meet the defined criteria. - ### Ordering Filters running in different orders may result in different behavior so explicit ordering is required when two or more filters are defined on a type. This needs to be configured with the `order:` parameter, as the type metadata pulled at runtime may return the attributes on a type in a different order from how they appear in the source code. Ordering must have unique values so an explicit ordering can be determined. @@ -34,7 +31,6 @@ These filters work with [*Silo Metadata*](../grains/silo-metadata.md) to filter #### `RequiredMatchSiloMetadata` - Silo Metadata is used to filter candidate silos to only ones that matches all of the specified metadata keys with the calling silo. If there are no compatible silos that match all of the keys then an empty set of silos will be returned and placement will ultimately fail for the grain. #### `PreferredMatchSiloMetadata` @@ -49,10 +45,8 @@ For example, if filtering on `[PreferredMatchSiloMetadata(["cloud.availability-z ## Implement placement filters - To implement a custom placement filter in Orleans, follow these steps: - 1. **Implementation** - Create marker Attribute derived from `PlacementFilterAttribute` - Create Strategy derived from `PlacementFilterStrategy` to manage any configuration values From 7e340f4d758e85612bc29b50d218eccbdb6027c9 Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Thu, 27 Feb 2025 16:02:17 -0800 Subject: [PATCH 23/24] Update silo-metadata.md Aligning with changes to implementation. --- docs/orleans/grains/silo-metadata.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/orleans/grains/silo-metadata.md b/docs/orleans/grains/silo-metadata.md index 1e50316b320ce..955a44be31c75 100644 --- a/docs/orleans/grains/silo-metadata.md +++ b/docs/orleans/grains/silo-metadata.md @@ -116,6 +116,4 @@ In this example: Internally, the `SiloMetadataCache` monitors changes in cluster membership on `MembershipTableManager` and will keep the local cache of metadata in sync with membership changes. Metadata is immutable for a given Silo so it will be retreived once and cached until that Silo leaves the cluster. Cached metadata for clusters that are `Dead` or have left the membership table will be cleared out of the local cache. -Each silo hosts a [*GrainService*](../grains/grainservices.md) that provides that silo's metadata. Other silos request a client to a specific silo's `GrainService` to pull a remote silo's metadata to populate its local cache. - -Calls to `SiloMetadataCache : ISiloMetadataCache` then return a result from this local cache. \ No newline at end of file +Each silo hosts an ISystemTarget that provides that silo's metadata. Calls to `SiloMetadataCache : ISiloMetadataCache` then return a result from this local cache. From ea92aed251472e20c979c13a6f6d6e7723dd7aca Mon Sep 17 00:00:00 2001 From: rkargMsft <164392675+rkargMsft@users.noreply.github.com> Date: Fri, 28 Feb 2025 09:04:22 -0800 Subject: [PATCH 24/24] update orleans toc moved Silo metadata to host configuration section --- docs/orleans/grains/grain-placement-filtering.md | 2 +- .../{grains => host/configuration-guide}/silo-metadata.md | 0 docs/orleans/toc.yml | 4 ++++ 3 files changed, 5 insertions(+), 1 deletion(-) rename docs/orleans/{grains => host/configuration-guide}/silo-metadata.md (100%) diff --git a/docs/orleans/grains/grain-placement-filtering.md b/docs/orleans/grains/grain-placement-filtering.md index c71a638b1353f..12b57a871aff9 100644 --- a/docs/orleans/grains/grain-placement-filtering.md +++ b/docs/orleans/grains/grain-placement-filtering.md @@ -27,7 +27,7 @@ Orleans provides various built-in filters for you to choose from. However, if yo ### Silo metadata -These filters work with [*Silo Metadata*](../grains/silo-metadata.md) to filter candidate silos. +These filters work with [*Silo Metadata*](../host/configuration-guide/silo-metadata.md) to filter candidate silos. #### `RequiredMatchSiloMetadata` diff --git a/docs/orleans/grains/silo-metadata.md b/docs/orleans/host/configuration-guide/silo-metadata.md similarity index 100% rename from docs/orleans/grains/silo-metadata.md rename to docs/orleans/host/configuration-guide/silo-metadata.md diff --git a/docs/orleans/toc.yml b/docs/orleans/toc.yml index 50f31026af6b6..a1a3c0465bb09 100644 --- a/docs/orleans/toc.yml +++ b/docs/orleans/toc.yml @@ -26,6 +26,8 @@ items: href: grains/grain-identity.md - name: Grain placement href: grains/grain-placement.md + - name: Grain placement filtering + href: grains/grain-placement-filtering.md - name: Grain extensions href: grains/grain-extensions.md - name: Timers and reminders @@ -138,6 +140,8 @@ items: href: host/configuration-guide/typical-configurations.md - name: Options classes href: host/configuration-guide/list-of-options-classes.md + - name: Silo metadata + href: host/configuration-guide/silo-metadata.md - name: Activation collection href: host/configuration-guide/activation-collection.md - name: Configure .NET garbage collection