ADX operator maintain named entity-groups

docs/concepts.md

@@ -404,7 +404,7 @@ The **Operator** is the control plane component that manages the lifecycle of al
 2. **Azure Resource Management:** Provisions or connects to ADX clusters/databases as specified in the `ADXCluster` CRD.
 3. **Component Deployment:** Deploys or updates StatefulSets/Deployments for Ingestor, Collector, and Alerter based on their CRDs.
 4. **Status & Health:** Updates CRD status fields and subresource conditions to reflect readiness and errors.
-5. **Federation Support:** Manages federated clusters, heartbeats, and macro-expand KQL functions for cross-cluster querying.
+5. **Federation Support:** Manages federated clusters, heartbeats, named entity groups, and macro-expand KQL functions for cross-cluster querying.
 
 #### Example Operator Workflow
 ```mermaid
@@ -477,8 +477,9 @@ flowchart TD
 
 #### Federation & Multi-Cluster
 - **Partition Clusters:** Each partition cluster is managed by its own operator and contains a subset of the data (e.g., by geo or tenant).
-- **Federated Cluster:** A central operator manages a federated ADX cluster, providing a unified query interface and managing heartbeats and macro-expand KQL functions.
+- **Federated Cluster:** A central operator manages a federated ADX cluster, providing a unified query interface and managing heartbeats, named entity groups, and macro-expand KQL functions.
 - **Heartbeat Table:** Partition clusters send periodic heartbeats to the federated cluster, which uses them to discover topology and liveness.
+- **Named Entity Groups:** The federated operator creates named entity groups (e.g., `MetricsSpoke`, `LogsSpoke`) that reference all partition cluster endpoints for each database, enabling efficient cross-cluster querying via macro-expand.
 
 #### Development & Testing
 - Operator can be run locally or in-cluster.

docs/designs/operator.md

@@ -608,6 +608,17 @@ The operator supports a federated ADX cluster model to enable organizations to p
   - This enables the federated cluster to present a single logical view for each table, while queries are transparently federated across all relevant clusters and databases.
   - The federated operator updates KQL functions and entity groups as the fleet topology changes.
 
+- **Named Entity Groups:**  
+  - The federated operator creates persistent named entity groups in ADX using the naming pattern `{SpokeDatabaseName}Spoke` (e.g., `MetricsSpoke`, `LogsSpoke`).
+  - Each entity group contains references to all partition cluster endpoints that have that database: `cluster('ep1').database('db'), cluster('ep2').database('db'), ...`
+  - Entity groups are replicated across all hub databases, ensuring consistent cross-cluster query capabilities.
+  - Since ADX does not support `.create-or-alter` for entity groups, the operator uses a drop-then-create pattern:
+    ```kql
+    .drop entity_group MetricsSpoke
+    .create entity_group MetricsSpoke (cluster('https://cluster1.kusto.net').database('Metrics'), cluster('https://cluster2.kusto.net').database('Metrics'))
+    ```
+  - Named entity groups provide better manageability than inline definitions: they can be inspected with `.show entity_groups`, updated independently, and referenced by multiple functions.
+
 - **Schema Flexibility:**  
   - The dynamic object for database/table topology allows the federated operator to only federate queries to clusters that actually contain the relevant table, improving efficiency and flexibility.