New handler type: common/webhook

[huang195]

Purpose:

• Webhooks are commonly supported and used for integration. Iter8 handler should support this type of integration

I want to discuss if there's a way to define a generic webhook interface, e.g.,

task: common/webhook
with:
  URL: http://
  user: username
  secret: default/secret
  type: [GET|POST]
  body: ...

so that we can re-use the same interface to implement gitops/github_action and notification/slack.

Github Action APIs are defined here: https://docs.github.com/en/rest/reference/actions#create-a-workflow-dispatch-event

@kalantar @sriumcp @fabolive ^^

[sriumcp]

Nice idea... how about a twist as follows?

actions:
- finish:
  - task: common/tekton-trigger-message # creates a slack message and populates the context with this message
    with:
      msgId: my-tekton-msg
  - task: common/github-message # creates a Tekton message and populates the context with this message
    with:
      msgId: my-gh-msg
  - task: common/webhook
    with:
      URL: http://
      user: username
      secret: default/secret
      type: [GET|POST]
      body: my-tekton-msg # creates a GH message and populates the context with this message
  - task: common/webhook
    with:
      URL: http://
      user: username
      secret: default/secret
      type: [GET|POST]
      body: my-gh-msg

[huang195]

AFAIK, a webhook is just a message sent to a particular URL (e.g., https://api.github.com), with a specific path (/iter8-tools/iter8/pulls), with a particular request method (e.g., GET/PUT/POST), and with or without certain parameters (e.g., ?param1=a&param2=b...), with or without payload, and with or without authentication.

The question is do we want to implement this generic webhook interface, so it can be used as a foundation layer for upper layer services. E.g., If we were to use this generic webhook to implement a Github Action task, it would look like:

  - task: common/webhook
    with:
      URL: https://api.github.com/iter8-tools/iter8/workflow-1/dispatches
      user: username
      secret: default/secret
      type: POST
      body: version=v2

but if one were to implement it in a higher-level common/github, it would look like

  - task: common/github
    with:
      Action: Dispatch
      Workflow: workflow-1
      user: username
      secret: default/secret
      body: version=v2

The latter has a bit cleaner interface than the former. And under the cover, it would just convert common/github to a common/webhook call. I think regardless what we decide, the common/webhook is needed in any case. Whether or not upper layer protocols would make use of it is up to the implementer of that handler type.

[sriumcp]

The generic webhook common/webhook is a good idea.

I am wondering if there is a way to compose it with other "body" and "params" builders.. like I mentioned in my example above. By simply prefacing this task with other tasks that build the right params and body out of the experiment, you may be able to get various specific Webhooks implemented?

[huang195]

This would depend on one Iter8 task passing its output as input to another. Is this currently supported? However, as parameters are usually just key-value pairs, I don't know if domain-specific parameter builders would simplify the interface much, but worth to explore

[sriumcp]

It is possible to enable this type functionality in tasks ... using the go context object. It is not a lot of work (minor changes to task implemented will be required).

I have a few other use cases as well (in the KFServing domain, where URLs for applications need to be extracted first, and then used as part of a metrics/collect task), where this pattern might come in handy.

Regardless of these specifics, your common (or Github) webhook idea (in whatever shape it gets realized) is going to be really nice and likely to add a lot of value to GitOps use cases.

[kalantar]

This could be used as a basis the notification/lwebhook task. See #731.

[kalantar]

I believe that we will want to support placeholders in URL, headers and the body as we do for metrics. Values could come from the referenced secret or from the experiment.

[sriumcp]

Perhaps we could workout the idea (in terms of what the user is required to specify / user experience at the experiment yaml level) using a few examples?

The above examples are good, but it would be nice to workout some examples for the params/body as well with specific targets (like GitHub actions or Tekton triggers).

GH actions seems particularly important for GitOps use-cases (will probably have a pre-defined message format).

CloudEvents is another populate messaging format to consider.