-
Notifications
You must be signed in to change notification settings - Fork 1k
/
Copy pathkarpenter.k8s.aws_ec2nodeclasses.yaml
570 lines (540 loc) · 27.7 KB
/
karpenter.k8s.aws_ec2nodeclasses.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.14.0
name: ec2nodeclasses.karpenter.k8s.aws
spec:
group: karpenter.k8s.aws
names:
categories:
- karpenter
kind: EC2NodeClass
listKind: EC2NodeClassList
plural: ec2nodeclasses
shortNames:
- ec2nc
- ec2ncs
singular: ec2nodeclass
scope: Cluster
versions:
- name: v1beta1
schema:
openAPIV3Schema:
description: EC2NodeClass is the Schema for the EC2NodeClass API
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: |-
EC2NodeClassSpec is the top level specification for the AWS Karpenter Provider.
This will contain configuration necessary to launch instances in AWS.
properties:
amiFamily:
description: AMIFamily is the AMI family that instances use.
enum:
- AL2
- AL2023
- Bottlerocket
- Ubuntu
- Custom
- Windows2019
- Windows2022
type: string
amiSelectorTerms:
description: AMISelectorTerms is a list of or ami selector terms.
The terms are ORed.
items:
description: |-
AMISelectorTerm defines selection logic for an ami used by Karpenter to launch nodes.
If multiple fields are used for selection, the requirements are ANDed.
properties:
id:
description: ID is the ami id in EC2
pattern: ami-[0-9a-z]+
type: string
name:
description: |-
Name is the ami name in EC2.
This value is the name field, which is different from the name tag.
type: string
owner:
description: |-
Owner is the owner for the ami.
You can specify a combination of AWS account IDs, "self", "amazon", and "aws-marketplace"
type: string
tags:
additionalProperties:
type: string
description: |-
Tags is a map of key/value tags used to select subnets
Specifying '*' for a value selects all values for a given tag key.
maxProperties: 20
type: object
x-kubernetes-validations:
- message: empty tag keys or values aren't supported
rule: self.all(k, k != '' && self[k] != '')
type: object
maxItems: 30
type: array
x-kubernetes-validations:
- message: expected at least one, got none, ['tags', 'id', 'name']
rule: self.all(x, has(x.tags) || has(x.id) || has(x.name))
- message: '''id'' is mutually exclusive, cannot be set with a combination
of other fields in amiSelectorTerms'
rule: '!self.all(x, has(x.id) && (has(x.tags) || has(x.name) ||
has(x.owner)))'
associatePublicIPAddress:
description: AssociatePublicIPAddress controls if public IP addresses
are assigned to instances that are launched with the nodeclass.
type: boolean
blockDeviceMappings:
description: BlockDeviceMappings to be applied to provisioned nodes.
items:
properties:
deviceName:
description: The device name (for example, /dev/sdh or xvdh).
type: string
ebs:
description: EBS contains parameters used to automatically set
up EBS volumes when an instance is launched.
properties:
deleteOnTermination:
description: DeleteOnTermination indicates whether the EBS
volume is deleted on instance termination.
type: boolean
encrypted:
description: |-
Encrypted indicates whether the EBS volume is encrypted. Encrypted volumes can only
be attached to instances that support Amazon EBS encryption. If you are creating
a volume from a snapshot, you can't specify an encryption value.
type: boolean
iops:
description: |-
IOPS is the number of I/O operations per second (IOPS). For gp3, io1, and io2 volumes,
this represents the number of IOPS that are provisioned for the volume. For
gp2 volumes, this represents the baseline performance of the volume and the
rate at which the volume accumulates I/O credits for bursting.
The following are the supported values for each volume type:
* gp3: 3,000-16,000 IOPS
* io1: 100-64,000 IOPS
* io2: 100-64,000 IOPS
For io1 and io2 volumes, we guarantee 64,000 IOPS only for Instances built
on the Nitro System (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-types.html#ec2-nitro-instances).
Other instance families guarantee performance up to 32,000 IOPS.
This parameter is supported for io1, io2, and gp3 volumes only. This parameter
is not supported for gp2, st1, sc1, or standard volumes.
format: int64
type: integer
kmsKeyID:
description: KMSKeyID (ARN) of the symmetric Key Management
Service (KMS) CMK used for encryption.
type: string
snapshotID:
description: SnapshotID is the ID of an EBS snapshot
type: string
throughput:
description: |-
Throughput to provision for a gp3 volume, with a maximum of 1,000 MiB/s.
Valid Range: Minimum value of 125. Maximum value of 1000.
format: int64
type: integer
volumeSize:
allOf:
- pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
- pattern: ^((?:[1-9][0-9]{0,3}|[1-4][0-9]{4}|[5][0-8][0-9]{3}|59000)Gi|(?:[1-9][0-9]{0,3}|[1-5][0-9]{4}|[6][0-3][0-9]{3}|64000)G|([1-9]||[1-5][0-7]|58)Ti|([1-9]||[1-5][0-9]|6[0-3]|64)T)$
anyOf:
- type: integer
- type: string
description: |-
VolumeSize in `Gi`, `G`, `Ti`, or `T`. You must specify either a snapshot ID or
a volume size. The following are the supported volumes sizes for each volume
type:
* gp2 and gp3: 1-16,384
* io1 and io2: 4-16,384
* st1 and sc1: 125-16,384
* standard: 1-1,024
x-kubernetes-int-or-string: true
volumeType:
description: |-
VolumeType of the block device.
For more information, see Amazon EBS volume types (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html)
in the Amazon Elastic Compute Cloud User Guide.
enum:
- standard
- io1
- io2
- gp2
- sc1
- st1
- gp3
type: string
type: object
x-kubernetes-validations:
- message: snapshotID or volumeSize must be defined
rule: has(self.snapshotID) || has(self.volumeSize)
rootVolume:
description: |-
RootVolume is a flag indicating if this device is mounted as kubelet root dir. You can
configure at most one root volume in BlockDeviceMappings.
type: boolean
type: object
maxItems: 50
type: array
x-kubernetes-validations:
- message: must have only one blockDeviceMappings with rootVolume
rule: self.filter(x, has(x.rootVolume)?x.rootVolume==true:false).size()
<= 1
context:
description: |-
Context is a Reserved field in EC2 APIs
https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateFleet.html
type: string
detailedMonitoring:
description: DetailedMonitoring controls if detailed monitoring is
enabled for instances that are launched
type: boolean
instanceProfile:
description: |-
InstanceProfile is the AWS entity that instances use.
This field is mutually exclusive from role.
The instance profile should already have a role assigned to it that Karpenter
has PassRole permission on for instance launch using this instanceProfile to succeed.
type: string
x-kubernetes-validations:
- message: instanceProfile cannot be empty
rule: self != ''
instanceStorePolicy:
description: InstanceStorePolicy specifies how to handle instance-store
disks.
enum:
- RAID0
type: string
metadataOptions:
default:
httpEndpoint: enabled
httpProtocolIPv6: disabled
httpPutResponseHopLimit: 2
httpTokens: required
description: |-
MetadataOptions for the generated launch template of provisioned nodes.
This specifies the exposure of the Instance Metadata Service to
provisioned EC2 nodes. For more information,
see Instance Metadata and User Data
(https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html)
in the Amazon Elastic Compute Cloud User Guide.
Refer to recommended, security best practices
(https://aws.github.io/aws-eks-best-practices/security/docs/iam/#restrict-access-to-the-instance-profile-assigned-to-the-worker-node)
for limiting exposure of Instance Metadata and User Data to pods.
If omitted, defaults to httpEndpoint enabled, with httpProtocolIPv6
disabled, with httpPutResponseLimit of 2, and with httpTokens
required.
properties:
httpEndpoint:
default: enabled
description: |-
HTTPEndpoint enables or disables the HTTP metadata endpoint on provisioned
nodes. If metadata options is non-nil, but this parameter is not specified,
the default state is "enabled".
If you specify a value of "disabled", instance metadata will not be accessible
on the node.
enum:
- enabled
- disabled
type: string
httpProtocolIPv6:
default: disabled
description: |-
HTTPProtocolIPv6 enables or disables the IPv6 endpoint for the instance metadata
service on provisioned nodes. If metadata options is non-nil, but this parameter
is not specified, the default state is "disabled".
enum:
- enabled
- disabled
type: string
httpPutResponseHopLimit:
default: 2
description: |-
HTTPPutResponseHopLimit is the desired HTTP PUT response hop limit for
instance metadata requests. The larger the number, the further instance
metadata requests can travel. Possible values are integers from 1 to 64.
If metadata options is non-nil, but this parameter is not specified, the
default value is 2.
format: int64
maximum: 64
minimum: 1
type: integer
httpTokens:
default: required
description: |-
HTTPTokens determines the state of token usage for instance metadata
requests. If metadata options is non-nil, but this parameter is not
specified, the default state is "required".
If the state is optional, one can choose to retrieve instance metadata with
or without a signed token header on the request. If one retrieves the IAM
role credentials without a token, the version 1.0 role credentials are
returned. If one retrieves the IAM role credentials using a valid signed
token, the version 2.0 role credentials are returned.
If the state is "required", one must send a signed token header with any
instance metadata retrieval requests. In this state, retrieving the IAM
role credentials always returns the version 2.0 credentials; the version
1.0 credentials are not available.
enum:
- required
- optional
type: string
type: object
role:
description: |-
Role is the AWS identity that nodes use. This field is immutable.
This field is mutually exclusive from instanceProfile.
Marking this field as immutable avoids concerns around terminating managed instance profiles from running instances.
This field may be made mutable in the future, assuming the correct garbage collection and drift handling is implemented
for the old instance profiles on an update.
type: string
x-kubernetes-validations:
- message: role cannot be empty
rule: self != ''
- message: immutable field changed
rule: self == oldSelf
securityGroupSelectorTerms:
description: SecurityGroupSelectorTerms is a list of or security group
selector terms. The terms are ORed.
items:
description: |-
SecurityGroupSelectorTerm defines selection logic for a security group used by Karpenter to launch nodes.
If multiple fields are used for selection, the requirements are ANDed.
properties:
id:
description: ID is the security group id in EC2
pattern: sg-[0-9a-z]+
type: string
name:
description: |-
Name is the security group name in EC2.
This value is the name field, which is different from the name tag.
type: string
tags:
additionalProperties:
type: string
description: |-
Tags is a map of key/value tags used to select subnets
Specifying '*' for a value selects all values for a given tag key.
maxProperties: 20
type: object
x-kubernetes-validations:
- message: empty tag keys or values aren't supported
rule: self.all(k, k != '' && self[k] != '')
type: object
maxItems: 30
type: array
x-kubernetes-validations:
- message: securityGroupSelectorTerms cannot be empty
rule: self.size() != 0
- message: expected at least one, got none, ['tags', 'id', 'name']
rule: self.all(x, has(x.tags) || has(x.id) || has(x.name))
- message: '''id'' is mutually exclusive, cannot be set with a combination
of other fields in securityGroupSelectorTerms'
rule: '!self.all(x, has(x.id) && (has(x.tags) || has(x.name)))'
- message: '''name'' is mutually exclusive, cannot be set with a combination
of other fields in securityGroupSelectorTerms'
rule: '!self.all(x, has(x.name) && (has(x.tags) || has(x.id)))'
subnetSelectorTerms:
description: SubnetSelectorTerms is a list of or subnet selector terms.
The terms are ORed.
items:
description: |-
SubnetSelectorTerm defines selection logic for a subnet used by Karpenter to launch nodes.
If multiple fields are used for selection, the requirements are ANDed.
properties:
id:
description: ID is the subnet id in EC2
pattern: subnet-[0-9a-z]+
type: string
tags:
additionalProperties:
type: string
description: |-
Tags is a map of key/value tags used to select subnets
Specifying '*' for a value selects all values for a given tag key.
maxProperties: 20
type: object
x-kubernetes-validations:
- message: empty tag keys or values aren't supported
rule: self.all(k, k != '' && self[k] != '')
type: object
maxItems: 30
type: array
x-kubernetes-validations:
- message: subnetSelectorTerms cannot be empty
rule: self.size() != 0
- message: expected at least one, got none, ['tags', 'id']
rule: self.all(x, has(x.tags) || has(x.id))
- message: '''id'' is mutually exclusive, cannot be set with a combination
of other fields in subnetSelectorTerms'
rule: '!self.all(x, has(x.id) && has(x.tags))'
tags:
additionalProperties:
type: string
description: Tags to be applied on ec2 resources like instances and
launch templates.
type: object
x-kubernetes-validations:
- message: empty tag keys aren't supported
rule: self.all(k, k != '')
- message: tag contains a restricted tag matching kubernetes.io/cluster/
rule: self.all(k, !k.startsWith('kubernetes.io/cluster') )
- message: tag contains a restricted tag matching karpenter.sh/nodepool
rule: self.all(k, k != 'karpenter.sh/nodepool')
- message: tag contains a restricted tag matching karpenter.sh/managed-by
rule: self.all(k, k !='karpenter.sh/managed-by')
- message: tag contains a restricted tag matching karpenter.sh/nodeclaim
rule: self.all(k, k !='karpenter.sh/nodeclaim')
- message: tag contains a restricted tag matching karpenter.k8s.aws/ec2nodeclass
rule: self.all(k, k !='karpenter.k8s.aws/ec2nodeclass')
userData:
description: |-
UserData to be applied to the provisioned nodes.
It must be in the appropriate format based on the AMIFamily in use. Karpenter will merge certain fields into
this UserData to ensure nodes are being provisioned with the correct configuration.
type: string
required:
- amiFamily
- securityGroupSelectorTerms
- subnetSelectorTerms
type: object
x-kubernetes-validations:
- message: amiSelectorTerms is required when amiFamily == 'Custom'
rule: 'self.amiFamily == ''Custom'' ? self.amiSelectorTerms.size() !=
0 : true'
- message: must specify exactly one of ['role', 'instanceProfile']
rule: (has(self.role) && !has(self.instanceProfile)) || (!has(self.role)
&& has(self.instanceProfile))
- message: changing from 'instanceProfile' to 'role' is not supported.
You must delete and recreate this node class if you want to change
this.
rule: (has(oldSelf.role) && has(self.role)) || (has(oldSelf.instanceProfile)
&& has(self.instanceProfile))
status:
description: EC2NodeClassStatus contains the resolved state of the EC2NodeClass
properties:
amis:
description: |-
AMI contains the current AMI values that are available to the
cluster under the AMI selectors.
items:
description: AMI contains resolved AMI selector values utilized
for node launch
properties:
id:
description: ID of the AMI
type: string
name:
description: Name of the AMI
type: string
requirements:
description: Requirements of the AMI to be utilized on an instance
type
items:
description: |-
A node selector requirement with min values is a selector that contains values, a key, an operator that relates the key and values
and minValues that represent the requirement to have at least that many values.
properties:
key:
description: The label key that the selector applies to.
type: string
minValues:
description: |-
This field is ALPHA and can be dropped or replaced at any time
MinValues is the minimum number of unique values required to define the flexibility of the specific requirement.
maximum: 50
minimum: 1
type: integer
operator:
description: |-
Represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt.
type: string
values:
description: |-
An array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. If the operator is Gt or Lt, the values
array must have a single element, which will be interpreted as an integer.
This array is replaced during a strategic merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
required:
- id
- requirements
type: object
type: array
instanceProfile:
description: InstanceProfile contains the resolved instance profile
for the role
type: string
securityGroups:
description: |-
SecurityGroups contains the current Security Groups values that are available to the
cluster under the SecurityGroups selectors.
items:
description: SecurityGroup contains resolved SecurityGroup selector
values utilized for node launch
properties:
id:
description: ID of the security group
type: string
name:
description: Name of the security group
type: string
required:
- id
type: object
type: array
subnets:
description: |-
Subnets contains the current Subnet values that are available to the
cluster under the subnet selectors.
items:
description: Subnet contains resolved Subnet selector values utilized
for node launch
properties:
id:
description: ID of the subnet
type: string
zone:
description: The associated availability zone
type: string
required:
- id
- zone
type: object
type: array
type: object
type: object
served: true
storage: true
subresources:
status: {}