feat: Get Neuron device and core count from EC2 API for all `trn` and `inf` instance types #6510

bryantbiggs · 2024-07-13T14:12:08Z

Fixes #3555

Description

Removes trn* static resource definitions and instead pulls directly from EC2 API to get device count, core count, and total device memory
Converts inf* resource requirement collection from InferenceAcceleratorInfo to NeuronInfo; this aligns with trn* resource information collection for all Neuron related resource information
- You can compare the output of the following to see that NeuronInfo supersedes InferenceAcceleratorInfo:
  - aws ec2 describe-instance-types --query 'InstanceTypes[*].NeuronInfo'
  - aws ec2 describe-instance-types --query 'InstanceTypes[*].InferenceAcceleratorInfo'
Adds support for aws.amazon.com/neuroncore which is used for allocating neuron cores to the container
Adds Neuron device plugin to e2e integration test suite for neuron devices similar to NVIDIA and EFA device plugins
Adds new region us-east-2 to collect instance details for the reference instance types doc page that are not found in us-east-1 nor us-west-2
Auto updates from running make codegen and make docgen

How was this change tested?

make test

Does this change impact docs?

Yes, PR includes docs updates
Yes, issue opened: #
No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

netlify · 2024-07-13T14:12:24Z

✅ Deploy Preview for karpenter-docs-prod ready!

Name	Link
🔨 Latest commit	`30e265d`
🔍 Latest deploy log	https://app.netlify.com/sites/karpenter-docs-prod/deploys/6724036cf5be5a0008448173
😎 Deploy Preview	https://deploy-preview-6510--karpenter-docs-prod.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

github-actions · 2024-07-28T12:04:25Z

This PR has been inactive for 14 days. StaleBot will close this stale PR after 14 more days of inactivity.

github-actions · 2024-08-12T12:06:29Z

This PR has been inactive for 14 days. StaleBot will close this stale PR after 14 more days of inactivity.

engedaam · 2024-08-28T22:08:26Z

Any update here?

coveralls · 2024-09-11T21:33:11Z

Pull Request Test Coverage Report for Build 11620786418

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

23 of 23 (100.0%) changed or added relevant lines in 2 files are covered.
54 unchanged lines in 6 files lost coverage.
Overall coverage decreased (-0.3%) to 82.812%

Files with Coverage Reduction	New Missed Lines	%
pkg/controllers/controllers.go	2	0.0%
pkg/test/environment.go	4	96.0%
pkg/providers/instancetype/types.go	5	96.63%
pkg/operator/operator.go	5	9.15%
pkg/providers/instancetype/instancetype.go	13	93.47%
pkg/providers/instance/instance.go	25	89.12%

Totals
Change from base Build 11407413769:	-0.3%
Covered Lines:	5642
Relevant Lines:	6813

💛 - Coveralls

website/content/en/preview/reference/instance-types.md

github-actions · 2024-09-26T12:05:51Z

This PR has been inactive for 14 days. StaleBot will close this stale PR after 14 more days of inactivity.

test/suites/integration/extended_resources_test.go

pkg/apis/v1/labels.go

pkg/providers/instancetype/types.go

hack/docs/instancetypes_gen/main.go

hack/code/instancetype_testdata_gen/main.go

pkg/providers/instancetype/types.go

examples/workloads/neuron.yaml

pkg/apis/v1/labels.go

website/content/en/preview/reference/settings.md

hack/code/instancetype_testdata_gen/main.go

engedaam

Going to let the integration tests run and we should be go to merge

/karpenter snapshot

github-actions · 2024-10-10T18:40:43Z

Snapshot successfully published to oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter:0-8b93d2d7e5a7d793dc4ba33409059615a6f04020.
To install you must login to the ECR repo with an AWS account:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 021119463062.dkr.ecr.us-east-1.amazonaws.com

helm upgrade --install karpenter oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter --version "0-8b93d2d7e5a7d793dc4ba33409059615a6f04020" --namespace "kube-system" --create-namespace \
  --set "settings.clusterName=${CLUSTER_NAME}" \
  --set "settings.interruptionQueue=${CLUSTER_NAME}" \
  --set controller.resources.requests.cpu=1 \
  --set controller.resources.requests.memory=1Gi \
  --set controller.resources.limits.cpu=1 \
  --set controller.resources.limits.memory=1Gi \
  --wait

test/suites/integration/extended_resources_test.go

…nf*` instance types

…populated correctly

…nded resource test

…erify`

…s or doing

engedaam

/karpenter snapshot

github-actions · 2024-10-22T19:27:10Z

Snapshot successfully published to oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter:0-45c3e77b5dba6fb9fd30249bf0ff0894e0074a82.
To install you must login to the ECR repo with an AWS account:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 021119463062.dkr.ecr.us-east-1.amazonaws.com

helm upgrade --install karpenter oci://021119463062.dkr.ecr.us-east-1.amazonaws.com/karpenter/snapshot/karpenter --version "0-45c3e77b5dba6fb9fd30249bf0ff0894e0074a82" --namespace "kube-system" --create-namespace \
  --set "settings.clusterName=${CLUSTER_NAME}" \
  --set "settings.interruptionQueue=${CLUSTER_NAME}" \
  --set controller.resources.requests.cpu=1 \
  --set controller.resources.requests.memory=1Gi \
  --set controller.resources.limits.cpu=1 \
  --set controller.resources.limits.memory=1Gi \
  --wait

engedaam

LGTM 🚀

website/content/en/preview/reference/instance-types.md

pkg/providers/instancetype/suite_test.go

…euron accelerator name well known label

jmdeal

Sorry for the delay, one suggested update but other than that this looks good to me

website/content/en/preview/concepts/scheduling.md

Co-authored-by: Jason Deal <[email protected]>

jmdeal

LGTM 🚀

engedaam

LGTM 🚀

bryantbiggs force-pushed the feat/neuron-core branch 2 times, most recently from 3d1b8da to e894f8a Compare July 13, 2024 16:16

github-actions bot added lifecycle/stale and removed lifecycle/stale labels Jul 28, 2024

github-actions bot added lifecycle/stale and removed lifecycle/stale labels Aug 12, 2024

jonathan-innis assigned engedaam Aug 21, 2024

bryantbiggs force-pushed the feat/neuron-core branch 3 times, most recently from d48abe9 to f5fdc84 Compare September 11, 2024 21:28

bryantbiggs commented Sep 11, 2024

View reviewed changes

website/content/en/preview/reference/instance-types.md Outdated Show resolved Hide resolved

bryantbiggs marked this pull request as ready for review September 11, 2024 22:36

bryantbiggs requested a review from a team as a code owner September 11, 2024 22:36

bryantbiggs requested a review from jigisha620 September 11, 2024 22:36

github-actions bot added the lifecycle/stale label Sep 26, 2024

engedaam reviewed Sep 26, 2024

View reviewed changes

test/suites/integration/extended_resources_test.go Outdated Show resolved Hide resolved

pkg/apis/v1/labels.go Outdated Show resolved Hide resolved

pkg/providers/instancetype/types.go Show resolved Hide resolved

github-actions bot removed the lifecycle/stale label Sep 27, 2024

engedaam reviewed Sep 27, 2024

View reviewed changes

hack/docs/instancetypes_gen/main.go Show resolved Hide resolved

hack/code/instancetype_testdata_gen/main.go Show resolved Hide resolved

pkg/providers/instancetype/types.go Show resolved Hide resolved

mounchin reviewed Sep 27, 2024

View reviewed changes

examples/workloads/neuron.yaml Outdated Show resolved Hide resolved

bryantbiggs force-pushed the feat/neuron-core branch 2 times, most recently from 06c6bbb to 964090e Compare October 1, 2024 18:08

engedaam reviewed Oct 2, 2024

View reviewed changes

pkg/apis/v1/labels.go Outdated Show resolved Hide resolved

website/content/en/preview/reference/settings.md Outdated Show resolved Hide resolved

hack/code/instancetype_testdata_gen/main.go Outdated Show resolved Hide resolved

bryantbiggs force-pushed the feat/neuron-core branch from 964090e to 3677253 Compare October 2, 2024 16:17

engedaam reviewed Oct 8, 2024

View reviewed changes

engedaam reviewed Oct 14, 2024

View reviewed changes

test/suites/integration/extended_resources_test.go Show resolved Hide resolved

bryantbiggs force-pushed the feat/neuron-core branch 3 times, most recently from 18c3450 to d5b11a9 Compare October 18, 2024 23:07

bryantbiggs added 10 commits October 18, 2024 18:19

feat: Get Neuron device and core count from EC2 API for trn* and `i…

2c072f8

…nf*` instance types

chore: Update instance references doc

981bd4f

fix: Correct logic for calculating number of Neuron cores

656b742

fix: Ensure that instances with accelerators that are not Neuron are …

8e205a6

…populated correctly

fix: Remove support for neurondevice

0e77584

test: Add test case for neuroncore

8b32323

fix: Revert support for /instance-accelerator-memory label

9b69546

fix: Add e2e test requirements replacement for inf/trn family generation

3d294c9

chore: Get make commands happy

c5f3b5b

fix: Add Neuron scheduler and RBAC permissions for Neuron device exte…

dc2aa66

…nded resource test

bryantbiggs force-pushed the feat/neuron-core branch from d5b11a9 to dc2aa66 Compare October 18, 2024 23:19

bryantbiggs added 3 commits October 18, 2024 19:28

fix: Only request x1 neuron* to avoid use of Neuron Scheduler

5b8789f

chore: Add autoformatted changes from make presubmit and/or `make v…

3ad300d

…erify`

chore: Remove auto changes made by make - no clue what this stuff i…

45c3e77

…s or doing

engedaam reviewed Oct 22, 2024

View reviewed changes

engedaam reviewed Oct 23, 2024

View reviewed changes

jmdeal reviewed Oct 23, 2024

View reviewed changes

website/content/en/preview/reference/instance-types.md Show resolved Hide resolved

pkg/providers/instancetype/suite_test.go Show resolved Hide resolved

docs: Update docs to reflect Neuron scheduler impact and changes to N…

309c27d

…euron accelerator name well known label

jmdeal reviewed Oct 31, 2024

View reviewed changes

website/content/en/preview/concepts/scheduling.md Outdated Show resolved Hide resolved

Update scheduling.md

30e265d

Co-authored-by: Jason Deal <[email protected]>

jmdeal approved these changes Oct 31, 2024

View reviewed changes

engedaam approved these changes Oct 31, 2024

View reviewed changes

jmdeal merged commit c2f019d into aws:main Oct 31, 2024
19 checks passed

bryantbiggs deleted the feat/neuron-core branch November 1, 2024 00:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Get Neuron device and core count from EC2 API for all `trn` and `inf` instance types #6510

feat: Get Neuron device and core count from EC2 API for all `trn` and `inf` instance types #6510

bryantbiggs commented Jul 13, 2024 •

edited

Loading

netlify bot commented Jul 13, 2024 •

edited

Loading

github-actions bot commented Jul 28, 2024

github-actions bot commented Aug 12, 2024

engedaam commented Aug 28, 2024

coveralls commented Sep 11, 2024 •

edited

Loading

github-actions bot commented Sep 26, 2024

engedaam left a comment

github-actions bot commented Oct 10, 2024

engedaam left a comment

github-actions bot commented Oct 22, 2024

engedaam left a comment

jmdeal left a comment

jmdeal left a comment

engedaam left a comment

feat: Get Neuron device and core count from EC2 API for all trn* and inf* instance types #6510

feat: Get Neuron device and core count from EC2 API for all trn* and inf* instance types #6510

Conversation

bryantbiggs commented Jul 13, 2024 • edited Loading

netlify bot commented Jul 13, 2024 • edited Loading

✅ Deploy Preview for karpenter-docs-prod ready!

github-actions bot commented Jul 28, 2024

github-actions bot commented Aug 12, 2024

engedaam commented Aug 28, 2024

coveralls commented Sep 11, 2024 • edited Loading

Pull Request Test Coverage Report for Build 11620786418

Warning: This coverage report may be inaccurate.

Details

💛 - Coveralls

github-actions bot commented Sep 26, 2024

engedaam left a comment

Choose a reason for hiding this comment

github-actions bot commented Oct 10, 2024

engedaam left a comment

Choose a reason for hiding this comment

github-actions bot commented Oct 22, 2024

engedaam left a comment

Choose a reason for hiding this comment

jmdeal left a comment

Choose a reason for hiding this comment

jmdeal left a comment

Choose a reason for hiding this comment

engedaam left a comment

Choose a reason for hiding this comment

feat: Get Neuron device and core count from EC2 API for all `trn` and `inf` instance types #6510

feat: Get Neuron device and core count from EC2 API for all `trn` and `inf` instance types #6510

bryantbiggs commented Jul 13, 2024 •

edited

Loading

netlify bot commented Jul 13, 2024 •

edited

Loading

coveralls commented Sep 11, 2024 •

edited

Loading