Create ingress for exposing Spark connect server

[ChenYi015]

What feature you would like to be added?

Create ingress for exposing Spark connect server.

Why is this needed?

Ingress can be used to route traffic outside the cluster to the Spark connect service within the cluster.

Describe the solution you would like

Add a new field like .spec.server.ingress to the SparkConnect CRD and implements the corresponding controller logic.

Describe alternatives you have considered

Add a global controller option instead of modifying the SparkConnect CRD.

Additional context

For SparkApplication, the Helm chart provides values like controller.uiIngress.xxx to control the behavior related to Spark web UI ingress.

Love this feature?

Give it a 👍 We prioritize the features with most 👍

[rahul810050]

Hi @ChenYi015 ! I would like to work on this feature.
Here is the proposed design and implementation plan for adding ingress support for Spark Connect:

Proposed Approach

1. Extend the SparkConnect CRD
   Add a new optional field .spec.server.ingress:

spec:
  server:
    ingress:
      enabled: true
      className: ""
      annotations: {}
      hosts:
        - connect.example.com
      tls:
        - secretName: sparkconnect-tls
          hosts:
            - connect.example.com

This follows the same pattern the operator uses for SparkApplication UI ingress and keeps the API declarative and per-resource configurable.

2. Controller Implementation
   Update the SparkConnect controller to:

• Detect when .spec.server.ingress.enabled = true
• Generate a networking.k8s.io/v1 Ingress object that routes to the Spark Connect Service (-server)
• Apply annotations, className, hosts, and TLS from the CRD
• Set OwnerReferences so the ingress is garbage-collected with the SparkConnect object
• Delete the ingress if the user disables it later
  Example reconciliation logic:

if sc.Spec.Server.Ingress != nil && sc.Spec.Server.Ingress.Enabled {
    ingress := buildIngressForSparkConnect(sc)
    reconcileIngress(ctx, ingress)
} else {
    cleanupOldIngress(sc)
}

Ingress reconciliation will run after the Connect Service is created, matching the existing pattern in the SparkApplication controller (UI Service → UI Ingress).

3. Helm Chart Support
   Add Helm values:

sparkConnect:
  ingress:
    enabled: false
    className: ""
    annotations: {}
    hosts: []
    tls: []

This is similar to existing controller.uiIngress.* settings for SparkApplication.
Helm charts will expose these fields but final behavior is owned by the controller.

4. (Optional) Validation
   A validation webhook can ensure hosts/TLS are provided correctly when ingress is enabled.

5. Backward Compatibility
   This change is non-breaking:

• ingress is optional
• enabled defaults to false
• No changes required for existing SparkConnect manifests

6. Alternatives Considered

• A global controller flag (rejected: not flexible for multi-tenant setups)
• Helm-only ingress templates (rejected: controller should manage lifecycle)

If this design is acceptable, I would be happy to take the issue and submit a PR.

[rahul810050]

@ChenYi015 Are there any updates on this issue? I would love to work on it please let me know if it is available.

[rahul810050]

Hi @ChenYi015,
Just checking in — are there any updates on this issue? I'd still love to work on it if it's available. Please let me know!

[ChenYi015]

@rahul810050 Any contributions are welcome!
I prefer embending networking.k8s.io/v1/Ingress as a subfield, just like what we have did to .spec.server.service.

[rahul810050]

@ChenYi015 I made a PR for this issue. Could you please review it??