|
| 1 | +# Custom Workload Types with Kruise Rollout |
| 2 | + |
| 3 | +Kruise Rollout supports extending rollout capabilities to custom resources (CRs) by using the `rollouts.kruise.io/workload-type` label. This feature allows you to enable StatefulSet-like, Deployment-like, CloneSet-like, or DaemonSet-like rollout behavior for your custom workloads. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +By default, Kruise Rollout supports the following built-in workload types: |
| 8 | +- Kubernetes native: `Deployment`, `StatefulSet` |
| 9 | +- OpenKruise: `CloneSet`, `Advanced StatefulSet`, `Advanced DaemonSet` |
| 10 | + |
| 11 | +However, you can extend this support to any custom resource by adding the appropriate workload type label. |
| 12 | + |
| 13 | +## Supported Workload Type Labels |
| 14 | + |
| 15 | +The `rollouts.kruise.io/workload-type` label accepts the following values: |
| 16 | + |
| 17 | +| Label value | Behavior | Use case | |
| 18 | +|-------------|----------|----------| |
| 19 | +| `statefulset` | StatefulSet-like rolling updates with partition support | Ordered pod updates, persistent storage workloads | |
| 20 | +| `deployment` | Deployment-like rolling updates | Stateless applications | |
| 21 | +| `cloneset` | CloneSet-like rolling updates | Advanced pod management features | |
| 22 | +| `daemonset` | DaemonSet-like rolling updates | Node-level services | |
| 23 | + |
| 24 | +## Requirements for Custom Resources |
| 25 | + |
| 26 | +For a custom resource to work with Kruise Rollout using the workload-type label, it must have: |
| 27 | + |
| 28 | +### 1. Required Spec Fields |
| 29 | +```yaml |
| 30 | +spec: |
| 31 | + replicas: <integer> |
| 32 | + selector: |
| 33 | + matchLabels: |
| 34 | + <key>: <value> |
| 35 | + template: |
| 36 | + metadata: |
| 37 | + labels: |
| 38 | + <key>: <value> |
| 39 | + spec: |
| 40 | + containers: [...] |
| 41 | +``` |
| 42 | +
|
| 43 | +### 2. Required Status Fields |
| 44 | +```yaml |
| 45 | +status: |
| 46 | + replicas: <integer> |
| 47 | + readyReplicas: <integer> |
| 48 | + updatedReplicas: <integer> |
| 49 | + updatedReadyReplicas: <integer> |
| 50 | + availableReplicas: <integer> |
| 51 | + observedGeneration: <integer> |
| 52 | + updateRevision: <string> |
| 53 | + stableRevision: <string> |
| 54 | +``` |
| 55 | +
|
| 56 | +### 3. Update Strategy (for StatefulSet-like behavior) |
| 57 | +```yaml |
| 58 | +spec: |
| 59 | + updateStrategy: |
| 60 | + type: RollingUpdate |
| 61 | + rollingUpdate: |
| 62 | + partition: <integer> |
| 63 | + paused: <boolean> |
| 64 | +``` |
| 65 | +
|
| 66 | +## Example: StatefulSet-like Custom Resource |
| 67 | +
|
| 68 | +Here's a complete example of how to use a custom resource with StatefulSet-like rollout behavior: |
| 69 | +
|
| 70 | +### Step 1: Define the Custom Resource Definition |
| 71 | +
|
| 72 | +```yaml |
| 73 | +apiVersion: apiextensions.k8s.io/v1 |
| 74 | +kind: CustomResourceDefinition |
| 75 | +metadata: |
| 76 | + name: customstatefulsets.example.com |
| 77 | +spec: |
| 78 | + group: example.com |
| 79 | + versions: |
| 80 | + - name: v1 |
| 81 | + served: true |
| 82 | + storage: true |
| 83 | + schema: |
| 84 | + openAPIV3Schema: |
| 85 | + type: object |
| 86 | + properties: |
| 87 | + spec: |
| 88 | + type: object |
| 89 | + properties: |
| 90 | + replicas: |
| 91 | + type: integer |
| 92 | + selector: |
| 93 | + type: object |
| 94 | + template: |
| 95 | + type: object |
| 96 | + updateStrategy: |
| 97 | + type: object |
| 98 | + properties: |
| 99 | + rollingUpdate: |
| 100 | + type: object |
| 101 | + properties: |
| 102 | + partition: |
| 103 | + type: integer |
| 104 | + paused: |
| 105 | + type: boolean |
| 106 | + status: |
| 107 | + type: object |
| 108 | + properties: |
| 109 | + replicas: |
| 110 | + type: integer |
| 111 | + readyReplicas: |
| 112 | + type: integer |
| 113 | + updatedReplicas: |
| 114 | + type: integer |
| 115 | + # ...other status fields |
| 116 | + scope: Namespaced |
| 117 | + names: |
| 118 | + plural: customstatefulsets |
| 119 | + singular: customstatefulset |
| 120 | + kind: CustomStatefulSet |
| 121 | +``` |
| 122 | +
|
| 123 | +### Step 2: Create the Custom Resource with Workload Type Label |
| 124 | +
|
| 125 | +```yaml |
| 126 | +apiVersion: example.com/v1 |
| 127 | +kind: CustomStatefulSet |
| 128 | +metadata: |
| 129 | + name: my-custom-workload |
| 130 | + labels: |
| 131 | + # this label enables StatefulSet-like rollout behavior |
| 132 | + rollouts.kruise.io/workload-type: statefulset |
| 133 | +spec: |
| 134 | + replicas: 5 |
| 135 | + selector: |
| 136 | + matchLabels: |
| 137 | + app: my-custom-workload |
| 138 | + template: |
| 139 | + metadata: |
| 140 | + labels: |
| 141 | + app: my-custom-workload |
| 142 | + spec: |
| 143 | + containers: |
| 144 | + - name: app |
| 145 | + image: nginx:1.20 |
| 146 | + ports: |
| 147 | + - containerPort: 80 |
| 148 | + updateStrategy: |
| 149 | + type: RollingUpdate |
| 150 | + rollingUpdate: |
| 151 | + partition: 0 |
| 152 | + paused: false |
| 153 | +``` |
| 154 | +
|
| 155 | +### Step 3: Create the Rollout Configuration |
| 156 | +
|
| 157 | +```yaml |
| 158 | +apiVersion: rollouts.kruise.io/v1beta1 |
| 159 | +kind: Rollout |
| 160 | +metadata: |
| 161 | + name: my-custom-workload-rollout |
| 162 | +spec: |
| 163 | + workloadRef: |
| 164 | + apiVersion: example.com/v1 |
| 165 | + kind: CustomStatefulSet |
| 166 | + name: my-custom-workload |
| 167 | + strategy: |
| 168 | + canary: |
| 169 | + steps: |
| 170 | + - replicas: 20% |
| 171 | + pause: {} |
| 172 | + - replicas: 40% |
| 173 | + pause: {} |
| 174 | + - replicas: 60% |
| 175 | + pause: {} |
| 176 | + - replicas: 80% |
| 177 | + pause: {} |
| 178 | + - replicas: 100% |
| 179 | + pause: {} |
| 180 | +``` |
| 181 | +
|
| 182 | +## How It Works? |
| 183 | +
|
| 184 | +1. **Label detection**: Kruise Rollout detects the `rollouts.kruise.io/workload-type` label on your custom resource |
| 185 | +2. **Webhook registration**: The rollout controller automatically registers webhooks for resources with this label |
| 186 | +3. **Behavior mapping**: The controller applies the appropriate rollout behavior based on the label value |
| 187 | +4. **Automatic patching**: If the label is missing, the controller can automatically patch it based on the workload kind |
| 188 | + |
| 189 | +## Automatic Label Patching |
| 190 | + |
| 191 | +Kruise Rollout can automatically patch the workload-type label for known workload types: |
| 192 | + |
| 193 | +- `Deployment` -> `rollouts.kruise.io/workload-type: deployment` |
| 194 | +- `StatefulSet` -> `rollouts.kruise.io/workload-type: statefulset` |
| 195 | +- `CloneSet` -> `rollouts.kruise.io/workload-type: cloneset` |
| 196 | +- `DaemonSet` -> `rollouts.kruise.io/workload-type: daemonset` |
| 197 | + |
| 198 | +## Limitations |
| 199 | + |
| 200 | +1. **Custom controller required**: Your custom resource needs a controller that implements the required spec and status fields |
| 201 | +2. **Webhook compatibility**: The custom resource must be compatible with Kubernetes admission webhooks |
| 202 | +3. **Field structure**: The resource must follow the expected field structure for the chosen workload type |
| 203 | +4. **Update strategy**: For StatefulSet-like behavior, the resource must support partition-based rolling updates |
| 204 | + |
| 205 | +## Troubleshooting |
| 206 | + |
| 207 | +### Common Issues |
| 208 | + |
| 209 | +1. **Webhook not triggered**: Ensure the `rollouts.kruise.io/workload-type` label is present on the resource |
| 210 | +2. **Missing fields**: Verify that all required spec and status fields are implemented |
| 211 | +3. **Controller conflicts**: Make sure your custom controller doesn't conflict with rollout operations |
| 212 | + |
| 213 | +### Debugging |
| 214 | + |
| 215 | +Check the rollout controller logs for detailed information: |
| 216 | +```bash |
| 217 | +kubectl logs -n kruise-rollout deployment/kruise-rollout-controller |
| 218 | +``` |
| 219 | + |
0 commit comments