add information about random vtgate load balancer

Author: nickvanw

content/en/docs/24.0/reference/features/tablet-balancer.md

@@ -26,9 +26,13 @@ In many cases this approach suffices, since if there are a proportional number o
 satisfy the inbound traffic to the vtgates in that cell, then in general the queries will be distributed evenly to
 each tablet.
 
-## Balancer Motivation
+## Balancer Modes
 
 However, in some topologies, a simple affinity algorithm does not effectively balance the load.
+VTGate provides two additional balancer modes to address different topology and traffic patterns:
+**prefer-cell** and **random**.
+
+### When Default Policy is Insufficient
 
 As a simple example:
 
@@ -49,41 +53,90 @@ cell will only receive 1/6 of the queries.
 Other topologies that can cause similar pathologies include cases where there may be cells
 containing replicas but no local vtgates, and/or cells that have only vtgates but no replicas.
 
-For these topologies, the tabletBalancer proportionally assigns the output flow to each tablet,
-preferring the local cell where possible, but only as long as the global query balance is
-maintained.
+### Prefer-Cell Balancer
 
-## Algorithm
+The prefer-cell balancer addresses topologies where tablets and vtgates are unevenly distributed,
+but traffic is relatively balanced across vtgate cells. It proportionally assigns the output
+flow to each tablet, preferring the local cell where possible, but only as long as the global
+query balance is maintained.
 
-To accomplish this goal, the balancer is given:
+To accomplish this goal, the prefer-cell balancer is given:
 
 * The list of cells that receive inbound traffic to vtgates (from configuration)
 * The local cell where the vtgate exists (from configuration)
 * The set of tablets and their cells (learned from discovery)
 
-The model assumes there is an equal probablility of a query coming from each vtgate cell, i.e.
+The model assumes there is an equal probability of a query coming from each vtgate cell, i.e.
 traffic is effectively load balanced between the cells with vtgates.
 
 Given that information, the balancer builds a simple model to determine how much query load
 would go to each tablet if vtgate only routed to its local cell. Then if any tablets are
 unbalanced, it shifts the desired allocation away from the local cell preference in order to
 even out the query load.
 
-Based on this global model, the vtgate then probabalistically picks a destination for each
+Based on this global model, the vtgate then probabilistically picks a destination for each
 query to be sent and uses these weights to order the available tablets accordingly.
 
 Assuming each vtgate is configured with and discovers the same information about the topology,
 and the input flow is balanced across the vtgate cells (as mentioned above), then each vtgate
-should come the the same conclusion about the global flows, and cooperatively should
+should come to the same conclusion about the global flows, and cooperatively should
 converge on the desired balanced query load.
 
+**When to use prefer-cell mode:**
+* Tablets and vtgates are distributed unevenly across cells
+* Traffic is relatively balanced across all vtgate cells
+* You want to maintain cell affinity where possible to minimize latency
+
+### Random Balancer
+
+The random balancer addresses a different scenario: when application traffic is concentrated
+in fewer cells than where database replicas exist. Unlike the prefer-cell balancer, which
+assumes equal traffic distribution across vtgate cells, the random balancer makes no assumptions
+about traffic patterns.
+
+The random balancer selects tablets with uniform probability (1/N for N available tablets),
+completely ignoring cell affinity. This trades off potential latency optimization for a
+guaranteed even load distribution across all tablets, regardless of where traffic originates.
+
+**When to use random mode:**
+* Application traffic is highly concentrated in specific cells (e.g., 90% in one cell, 10% in another)
+* Cross-cell/cross-zone latency is acceptable for your workload
+* Avoiding tablet hotspots is more important than minimizing query latency
+* You have a single-AZ application deployment with multi-AZ database replicas
+
+**Example scenario:**
+
+```
+  Cell A: 90% --> vtgates --> randomly select from all tablets (1/4 each)
+  Cell B: 10% --> vtgates --> randomly select from all tablets (1/4 each)
+
+  Result: All 4 tablets receive ~25% of total load, regardless of cell
+```
+
+With the random balancer, you can optionally use `--balancer-vtgate-cells` to restrict the
+tablet pool to specific cells, but it's not required.
+
 ## Configuration
 
-To enable the balancer requires the following configuration:
+VTGate provides three balancer modes, controlled by the `--vtgate-balancer-mode` flag:
+
+### Balancer Mode Selection
+
+* **`--vtgate-balancer-mode=cell`** (default): Uses local cell affinity random choice (default policy described above)
+* **`--vtgate-balancer-mode=prefer-cell`**: Uses the prefer-cell balancer algorithm
+* **`--vtgate-balancer-mode=random`**: Uses uniform random selection across all tablets
+
+### Configuration Flags
+
+* **`--vtgate-balancer-mode`**: Specifies which balancer mode to use (cell, prefer-cell, or random). Defaults to `cell`.
+
+* **`--balancer-vtgate-cells`**: Comma-separated list of cells that contain vtgates.
+  * **Required** for `prefer-cell` mode
+  * **Optional** for `random` mode (filters tablets to specified cells if provided)
+  * Ignored for `cell` mode
 
-  * `--enable-balancer`:  Enables the balancer.  **Not enabled by default**
-  * `--balancer-vtgate-cells`:  Specifies the set of cells that contain vtgates
+* **`--balancer-keyspaces`**: Comma-separated list of keyspaces for which to use the configured balancer mode. If empty, applies to all keyspaces. This allows gradual rollout of balancer modes.
 
-Optionally this behavior can be restricted only when routing to certain keyspaces as a means of controlling rollout:
+### Deprecated Flag
 
-  * `--balancer-keyspaces`:  Specifies the set of keyspaces for which the balancer should be enabled.
+* **`--enable-balancer`**: **(DEPRECATED)** This flag has been replaced by `--vtgate-balancer-mode=prefer-cell`. While still accepted for backwards compatibility, it will be removed in a future release. If you are currently using `--enable-balancer`, migrate to using `--vtgate-balancer-mode=prefer-cell` instead.

content/en/docs/24.0/reference/programs/vtgate/_index.md

@@ -44,8 +44,8 @@ vtgate \
       --allow-kill-statement                                             Allows the execution of kill statement
       --allowed-tablet-types strings                                     Specifies the tablet types this vtgate is allowed to route queries to. Should be provided as a comma-separated set of tablet types.
       --alsologtostderr                                                  log to standard error as well as files
-      --balancer-keyspaces strings                                       When in balanced mode, a comma-separated list of keyspaces for which to use the balancer (optional)
-      --balancer-vtgate-cells strings                                    When in balanced mode, a comma-separated list of cells that contain vtgates (required)
+      --balancer-keyspaces strings                                       Comma-separated list of keyspaces for which to use the balancer (optional). If empty, applies to all keyspaces.
+      --balancer-vtgate-cells strings                                    Comma-separated list of cells that contain vttablets. For 'prefer-cell' mode, this is required. For 'random' mode, this is optional and filters tablets to those cells.
       --bind-address string                                              Bind address for the server. If empty, the server will listen on all available unicast and anycast IP addresses of the local system.
       --buffer-drain-concurrency int                                     Maximum number of requests retried simultaneously. More concurrency will increase the load on the PRIMARY vttablet when draining the buffer. (default 1)
       --buffer-keyspace-shards string                                    If not empty, limit buffering to these entries (comma separated). Entry format: keyspace or keyspace/shard. Requires --enable-buffer=true.
@@ -73,7 +73,7 @@ vtgate \
       --discovery-high-replication-lag-minimum-serving duration          Threshold above which replication lag is considered too high when applying the min_number_serving_vttablets flag. (default 2h0m0s)
       --discovery-low-replication-lag duration                           Threshold below which replication lag is considered low enough to be healthy. (default 30s)
       --emit-stats                                                       If set, emit stats to push-based monitoring and stats backends
-      --enable-balancer                                                  Enable the tablet balancer to evenly spread query load for a given tablet type
+      --enable-balancer                                                  (DEPRECATED: use --vtgate-balancer-mode instead) Enable the tablet balancer to evenly spread query load for a given tablet type
       --enable-buffer                                                    Enable buffering (stalling) of primary traffic during failovers.
       --enable-buffer-dry-run                                            Detect and log failover events, but do not actually buffer requests.
       --enable-direct-ddl                                                Allow users to submit direct DDL statements (default true)
@@ -263,6 +263,7 @@ vtgate \
   -v, --version                                                          print binary version
       --vmodule vModuleFlag                                              comma-separated list of pattern=N settings for file-filtered logging
       --vschema-ddl-authorized-users string                              List of users authorized to execute vschema ddl operations, or '%' to allow all users.
+      --vtgate-balancer-mode string                                      Tablet balancer mode (options: cell, prefer-cell, random). Defaults to 'cell' which shuffles tablets in the local cell.
       --vtgate-config-terse-errors                                       prevent bind vars from escaping in returned errors
       --warming-reads-concurrency int                                    Number of concurrent warming reads allowed (default 500)
       --warming-reads-percent int                                        Percentage of reads on the primary to forward to replicas. Useful for keeping buffer pools warm