Skip to content

Add support to configure multiple service ports#316

Merged
shivamerla merged 10 commits intoNVIDIA:mainfrom
shivamerla:support_multiple_service_ports
Mar 14, 2025
Merged

Add support to configure multiple service ports#316
shivamerla merged 10 commits intoNVIDIA:mainfrom
shivamerla:support_multiple_service_ports

Conversation

@shivamerla
Copy link
Copy Markdown
Collaborator

@shivamerla shivamerla commented Feb 7, 2025

  • Default named port of "api" will be added with legacy .spec.port attribute
  • Ability to setup multiple named ports "api", "metrics" etc.
  • auto detection of metrics and api ports for probes and metrics collection

@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented Feb 7, 2025

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@shivamerla shivamerla requested review from shengnuo and varunrsekar and removed request for ArangoGutierrez February 7, 2025 23:13
a-mccarthy
a-mccarthy reviewed Feb 10, 2025
View reviewed changes
Comment thread config/samples/apps_v1alpha1_nemoguardrails.yaml Outdated
varunrsekar
varunrsekar requested changes Feb 10, 2025
View reviewed changes
Comment thread api/apps/v1alpha1/nemo_entitystore_types.go Outdated
Comment thread api/apps/v1alpha1/common_types.go Outdated
Comment thread api/apps/v1alpha1/nemo_entitystore_types.go Outdated
shengnuo
shengnuo reviewed Feb 12, 2025
View reviewed changes
Comment thread api/apps/v1alpha1/nemo_datastore_types.go Outdated
Comment thread api/apps/v1alpha1/nemo_datastore_types.go Outdated
Comment thread api/apps/v1alpha1/common_types.go Outdated
Comment thread api/apps/v1alpha1/nemo_evaluator_types.go
varunrsekar
varunrsekar reviewed Feb 12, 2025
View reviewed changes
Comment thread api/apps/v1alpha1/common_types.go Outdated
Comment thread api/apps/v1alpha1/common_types.go Outdated
shengnuo
shengnuo reviewed Feb 13, 2025
View reviewed changes
Comment thread api/apps/v1alpha1/common_types_test.go Outdated
@shivamerla shivamerla force-pushed the support_multiple_service_ports branch from 11f0e22 to 4d4944e Compare February 14, 2025 04:05
@shivamerla shivamerla force-pushed the support_multiple_service_ports branch from 4d4944e to 41b8ae6 Compare March 7, 2025 07:13
varunrsekar
varunrsekar requested changes Mar 7, 2025
View reviewed changes
Comment thread api/apps/v1alpha1/common_types.go Outdated
Comment thread api/apps/v1alpha1/nemo_customizer_types.go
Comment thread api/apps/v1alpha1/nemo_customizer_types.go Outdated
@shivamerla shivamerla force-pushed the support_multiple_service_ports branch from 24913ab to cca9071 Compare March 11, 2025 17:26
@shivamerla shivamerla requested a review from varunrsekar March 11, 2025 17:27
@shivamerla shivamerla force-pushed the support_multiple_service_ports branch 4 times, most recently from 0eea118 to 18e13fd Compare March 13, 2025 22:50
@shivamerla
Add support to configure multiple service ports
fff31a9 
 * Default named port of "api" will be added with legacy .spec.port attribute
 * Ability to setup multiple named ports "api", "metrics" etc.
 * auto detection of metrics and api ports for probes and metrics collection

Signed-off-by: Shiva Krishna, Merla <smerla@nvidia.com>
@shivamerla
Add ServicePort type
b82da06 
Signed-off-by: Shiva Krishna, Merla <smerla@nvidia.com>
@shivamerla
Update service API to include only necessary parameters
3179563 
Signed-off-by: Shiva Krishna, Merla <smerla@nvidia.com>
@shivamerla
Remote stale changes from the pipeline sample
6b40e90 
Signed-off-by: Shiva Krishna, Merla <smerla@nvidia.com>
@shivamerla
Update probe endpoint for Guardrails service
909e57f 
Signed-off-by: Shiva Krishna, Merla <smerla@nvidia.com>
@shivamerla
Remove metrics port from the API
fa8e309 
Signed-off-by: Shiva Krishna, Merla <smerla@nvidia.com>
@shivamerla
Add default value for service port as 8000
0d8c13f 
Signed-off-by: Shiva Krishna, Merla <smerla@nvidia.com>
@shivamerla
Make service config as optional as well
7db4708 
Signed-off-by: Shiva Krishna, Merla <smerla@nvidia.com>
@shivamerla
Fix tests and add default port value in the CRD
19ffd4a 
Signed-off-by: Shiva Krishna, Merla <smerla@nvidia.com>
@shivamerla shivamerla force-pushed the support_multiple_service_ports branch from 18e13fd to 19ffd4a Compare March 14, 2025 00:16
varunrsekar
varunrsekar requested changes Mar 14, 2025
View reviewed changes
Comment thread api/apps/v1alpha1/common_types.go Outdated
Comment thread api/apps/v1alpha1/nemo_entitystore_types.go Outdated
Comment thread api/apps/v1alpha1/nemo_evaluator_types.go Outdated
Comment thread api/apps/v1alpha1/nimservice_types.go
Comment thread manifests/service.yaml
@shivamerla
Cleanup service params
795f997 
Signed-off-by: Shiva Krishna, Merla <smerla@nvidia.com>
varunrsekar
varunrsekar approved these changes Mar 14, 2025
View reviewed changes
Copy link
Copy Markdown
Collaborator

@varunrsekar varunrsekar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@shivamerla shivamerla merged commit efda5cb into NVIDIA:main Mar 14, 2025
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@varunrsekar varunrsekar varunrsekar approved these changes

@slu2011 slu2011 Awaiting requested review from slu2011

@visheshtanksale visheshtanksale Awaiting requested review from visheshtanksale visheshtanksale is a code owner

@shengnuo shengnuo Awaiting requested review from shengnuo shengnuo is a code owner

+1 more reviewer

@a-mccarthy a-mccarthy a-mccarthy left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@shivamerla @shengnuo @varunrsekar @a-mccarthy