Using k8s custom resources in EventSource and Sensor

[anencore94]

Hi forks, I just have a simple question.

As far as I understand, If I want to use k8s CR with argo-events(both eventsource and sensor-trigger), for example named pytorchjob. (not a standard k8s object case)

I could watch events for pytorchjob with existing Eventsource by this link

However, since I couldn't trigger pytorchjob with existing k8s object trigger by this link, I should build my own trigger to trigger pytorchjob.

Am I understand right? Or Current EventSource and Sensor supports any k8s custom resources in native way ?
Thanks in advance.

[whynowy]

Event source can watch pytorchjob, also the sensor can create pytorchjob objects.

[whynowy]

Correct.

[whynowy]

Triggering custom resource is defining your own trigger, where you need to implement your own trigger running as a gRPC service, the trigger connect to it and trigger actions.

[anencore94]

I got it :) Thanks!!
I've create a sensor to create Custom Resource (Experiment in kubeflow.org/v1beta1 and It worked well :)

I've tested with following yaml

apiVersion: argoproj.io/v1alpha1
kind: Sensor
metadata:
  name: webhook-katib-2
spec:
  template:
    serviceAccountName: k8s-resource-sa
  dependencies:
    - name: test-dep
      eventSourceName: webhook
      eventName: example
  triggers:
    - template:
        name: webhook-pod-trigger
        k8s:
          operation: create
          source:
            resource:
              apiVersion: kubeflow.org/v1beta1
              kind: Experiment
              metadata:
                namespace: kubeflow
                name: random222
              spec:
                objective:
                  type: maximize
                  goal: 0.99
                  objectiveMetricName: Validation-accuracy
                  additionalMetricNames:
                    - Train-accuracy
                algorithm:
                  algorithmName: random
                parallelTrialCount: 1
                maxTrialCount: 1
                maxFailedTrialCount: 1
                parameters:
                  - name: lr
                    parameterType: double
                    feasibleSpace:
                      min: "0.01"
                      max: "0.03"
                  - name: num-layers
                    parameterType: int
                    feasibleSpace:
                      min: "2"
                      max: "5"
                  - name: optimizer
                    parameterType: categorical
                    feasibleSpace:
                      list:
                        - sgd
                        - adam
                        - ftrl
                trialTemplate:
                  primaryContainerName: training-container
                  trialParameters:
                    - name: learningRate
                      description: Learning rate for the training model
                      reference: lr
                    - name: numberLayers
                      description: Number of training model layers
                      reference: num-layers
                    - name: optimizer
                      description: Training model optimizer (sdg, adam or ftrl)
                      reference: optimizer
                  trialSpec:
                    apiVersion: batch/v1
                    kind: Job
                    spec:
                      template:
                        spec:
                          containers:
                            - name: training-container
                              image: docker.io/kubeflowkatib/mxnet-mnist:v1beta1-45c5727
                              command:
                                - "python3"
                                - "/opt/mxnet-mnist/mnist.py"
                                - "--batch-size=64"
                                - "--lr=${trialParameters.learningRate}"
                                - "--num-layers=${trialParameters.numberLayers}"
                                - "--optimizer=${trialParameters.optimizer}"
                                - "--num-epochs=1"
                          restartPolicy: Never

[anencore94]

From the sensors-triggers link:

Apart from Argo workflow objects, the sensor lets you trigger standard Kubernetes objects such as Pod, Deployment, Job, CronJob, etc

How about change the word "standard" to "any" from the above sentence to make sure ? @whynowy

[whynowy]

Sounds good to me! Go ahead please!