Skip to content

Define SparkApplicationSubmitter interface to allow customizing submitting mechanism #2500

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
May 28, 2025
Merged

Define SparkApplicationSubmitter interface to allow customizing submitting mechanism #2500

merged 2 commits into from
May 28, 2025

Conversation

ChenYi015
Copy link
Contributor

@ChenYi015 ChenYi015 commented Apr 9, 2025 •
edited
Loading

Purpose of this PR

Close #2501

Proposed changes:

  • Define SparkApplicationSubmitter interface to allow customizing submitting mechanism

Change Category

  • Bugfix (non-breaking change which fixes an issue)
  • Feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that could affect existing functionality)
  • Documentation update

Rationale

Checklist

  • I have conducted a self-review of my own code.
  • I have updated documentation accordingly.
  • I have added tests that prove my changes are effective or that my feature works.
  • Existing unit tests pass locally with my changes.

Additional Notes

@ChenYi015 ChenYi015 force-pushed the feature/spark-application-submitter branch from 217877c to cad64d6 Compare April 9, 2025 10:03
@ChenYi015 ChenYi015 force-pushed the feature/spark-application-submitter branch from cad64d6 to 89e9bae Compare April 9, 2025 10:08
@google-oss-prow google-oss-prow bot added size/L and removed size/M labels Apr 9, 2025
@ChenYi015 ChenYi015 marked this pull request as ready for review April 9, 2025 10:51
@ChenYi015 ChenYi015 linked an issue Apr 9, 2025 that may be closed by this pull request
Define an interface to allow customizing Spark application submit mechanism #2501
Closed
@ChenYi015
Copy link
Contributor Author

/assign @jacobsalway @vara-bonthu @ImpSy

vara-bonthu
vara-bonthu approved these changes Apr 15, 2025
View reviewed changes
Copy link
Contributor

@vara-bonthu vara-bonthu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @ChenYi015!!
Introducing the SparkApplicationSubmitter interface is a great step toward decoupling the submission logic. This will pave the way for implementing spark-submit natively in Go.

@google-oss-prow google-oss-prow bot added the lgtm label Apr 15, 2025
nabuskey
nabuskey reviewed May 12, 2025
View reviewed changes
Copy link
Contributor

@nabuskey nabuskey left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a minor comment update to warn users that this is experimental and should be used with caution. Let's merge soon.

@ChenYi015 @nabuskey
Update comments
bc6c8f2 
Co-authored-by: Manabu McCloskey <[email protected]>
Signed-off-by: Yi Chen <[email protected]>
@google-oss-prow google-oss-prow bot removed the lgtm label May 13, 2025
@ChenYi015
Copy link
Contributor Author

Just a minor comment update to warn users that this is experimental and should be used with caution. Let's merge soon.

Have committed your suggestions.

@google-oss-prow google-oss-prow bot added the lgtm label May 28, 2025
@yuchaoran2011
Copy link
Contributor

/approve

@google-oss-prow Google OSS Prow
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: yuchaoran2011

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@google-oss-prow google-oss-prow bot merged commit 8c7a494 into kubeflow:master May 28, 2025
15 checks passed
@ChenYi015 ChenYi015 deleted the feature/spark-application-submitter branch May 28, 2025 05:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nabuskey nabuskey nabuskey left review comments

@yuchaoran2011 yuchaoran2011 yuchaoran2011 approved these changes

@vara-bonthu vara-bonthu vara-bonthu approved these changes

@andreyvelich andreyvelich Awaiting requested review from andreyvelich

@ImpSy ImpSy Awaiting requested review from ImpSy

@jacobsalway jacobsalway Awaiting requested review from jacobsalway

@mwielgus mwielgus Awaiting requested review from mwielgus

Assignees

@yuchaoran2011 yuchaoran2011

@ImpSy ImpSy

@jacobsalway jacobsalway

@vara-bonthu vara-bonthu

Labels
approved lgtm size/L
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Define an interface to allow customizing Spark application submit mechanism
6 participants
@ChenYi015 @yuchaoran2011 @vara-bonthu @nabuskey @ImpSy @jacobsalway