Skip to content

[RFC] Flux-CDEvent Provider #4828

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

[RFC] Flux-CDEvent Provider #4828

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
f8d20e5
Create provider RFC
adamkenihan Jun 5, 2024
4ccde7b
add user story image
adamkenihan Jun 5, 2024
8081448
Fixed comments
adamkenihan Jun 14, 2024
42b45b1
Small changes to non-goals and summary
adamkenihan Jun 14, 2024
5de6bed
Removal of user-defined mapping mentions
adamkenihan Jun 28, 2024
bb8448b
Wrapping text on readme for readability
adamkenihan Jul 2, 2024
8678dec
More potential CDEvents for Flux events
adamkenihan Jul 11, 2024
1a5d94b
Update rfcs/NNNN-cdevents-provider/README.md
adamkenihan Jul 18, 2024
987b99c
Changes to mapping
adamkenihan Jul 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
140 changes: 140 additions & 0 deletions rfcs/NNNN-cdevents-provider/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
# Flux CDEvents Provider

<!--
The title must be short and descriptive.
-->

**Status:** provisional

<!--
Status represents the current state of the RFC.
Must be one of `provisional`, `implementable`, `implemented`, `deferred`, `rejected`, `withdrawn`, or `replaced`.
-->

**Creation date:** 2024-01-19

**Last update:** 2024-06-05

## Summary

This RFC proposes to add a `Provider` type to the Flux notification-controller API for the sending of CDEvents.

When `Provider` objects configured to send CDEvents are alerted by a Flux event, they will utilise a user defined mapping of Flux and CDEvents events to send an appropriate CDEvent payload to a defined URL.
<!--
CDEvents Provider for flux. Sends a CDEvent to the specified address based on the flux event that triggered the provider.
-->

## Motivation

CDEvents enables interoperability between supported tools in a workflow, and flux is a very popular continuous delivery tool, as such we have received many questions about implementing CDEvents into the tool. The receiver part of this integration is already implemented in flux 2.3.0
<!--
CDEvents enables interoperability between supported tools in a workflow, and flux is a very popular continuous delivery tool, as such we have received many questions about implementing CDEvents into the tool.
-->

### Goals

Integrate [CDEvents](https://github.com/cdevents) into Flux with a CDEvents Provider that supports user mapping of flux to CDEvent events.

<!--
Integrate CDEvents into Flux with a CDEvents Provider.
-->

### Non-Goals

A CDEvent receiver is already implemented into Flux 2.3.0.

<!--
A CDEvent receiver is already in progress.
-->

## Proposal

Add CDEvents to the list of available `Providers` in Flux Notification controller. The user should be able to create mappings within the configuration of the `Provider` that dictates the CDEvent payload to send depending on the Flux event that triggered the alert. These CDEvent payloads should have meaningful data from the source event. There will be an initial focus on HelmRelease and related events as the format within Flux for those events is much more consistent.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be easier to comment on the individual lines if the paragraph is wrapped, maybe at 80 or 120 characters, whatever you prefer.

These CDEvent payloads should have meaningful data from the source event.

"event source" may fit well here, instead of "source event", indicating that the event source, the controllers, are responsible for meaningful data.

There will be an initial focus on HelmRelease and related events as the format within Flux for those events is much more consistent.

I don't think this needs to be mentioned in proposal. The corresponding github issues can have such implementation details.

<!--
Add CDEvents to the list of available providers in Flux Notification controller. The user should be able to create mappings that tell it which CDEvents to send based on the flux event. It should then send that CDEvent to the specified URL.
-->

### User Stories

Users of multiple CI/CD tools such as Tekton and Flux could use CDEvents as a way to enable interoperability.

For example, a user may want a Tekton `pipeline` to run once a HelmRelease flux resource has succeeded in a Helm install. On successful helm install, Flux will emit an event with reason `InstallSucceeded` which the user will have mapped in their `Provider` configuration to correspond to an `Environment.Modified` CDEvent. The CDEvent `Provider` will then send a payload with that CDEvent, which will also contain data from the Flux event, to a CloudEvents broker that Tekton is subscribed to, and trigger a Pipeline Run within Tekton.

![User Stories Tekton](user-stories-provider.drawio.png)


<!--
Optional if existing discussions and/or issues are linked in the motivation section.
-->

### Alternatives

There are currently no viable alternatives for the CDEvent Provider.
<!--
Certain use cases for CDEvents could be done alternatively using available providers such as the generic webhook.
-->

## Design Details

Adding a Flux `Provider` for CDEvents that will send a CDEvent payload upon receiving a flux event from an alert.

The user will be able to define a Flux `Provider` custom resource and deploy it to their cluster. This provider configuration will allow the user to define a mapping of which Flux events correspond to which CDEvent to send. Once an alert is triggered for this provider, it will send the corresponding CDEvent, based on the Flux event that caused the alert. This CDEvent will be created using the [CDEvents Go SDK](https://github.com/cdevents/sdk-go). As well as the user-defined mapping, the provider will include some default mappings for Flux-Helm events in the case that the user does not provide a custom mapping.

The CDEvents broker is not a part of this design and is left to the users to set up however they wish.

Example Provider YAML:

```yaml
apiVersion: notification.toolkit.fluxcd.io/v1beta2
kind: Provider
metadata:
name: cdevents-provider
namespace: flux-system
spec:
type: cdeventss
secretRef:
name: generic-token
address: http://10.100.0.17:5000
mapping:
ReconciliationFinished: dev.cdevents.pipelinerun.finished
GitOperationFailed: dev.cdevents.incident.reported
Copy link
Contributor

@darkowlzz darkowlzz Jun 13, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to discuss this further about adding a new mapping spec field for CDEvents specifically.
My initial understanding before this proposal was that the working of flux components are well defined which could be mapped to CDEvent event types in a deterministic way. Specific events from source-controller can be a kind of Source Code Control Event, events from kustomize-controller and helm-controller can be a kind of Continuous Deployment Event, etc.

I think the flux events can be categorized accordingly and statically configured to specific CDEvents event types. Any new event introduced in flux should also be categorized as a specific CDEvent. In notification-controller, the event handler can parse the events and forward them to message broker as per the fixed event categories.

I may be unaware of certain use cases of CDEvents that require such custom mapping, as I haven't used it myself.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi sunny, apologies for the slow response, we have decided to in fact switch to an approach that focuses more on a mapping that is defined in the way you described rather than the custom mapping we had considered before. I will make the changes to the RFC proposal to reflect this.


```

Example Default Mappings (Please note these are not final and will be open to discussion):

| Helm Event | CDEvent |
| ------------------------- | ------------------------------- |
| InstallSucceeded | Environment.Modified |
| InstallFailed | Incident.Detected |
|UpgradeSucceeded | TaskRun.Finished (with outcome success) |
|UpgradeFailed | TaskRun.Finished (with outcome failed) |
|TestSucceeded | TestCaseRun.finished (outcome pass) |
|TestFailed | TestCaseRun.finished (outcome fail) |
|RollbackSucceeded | TaskRun.Finished (with outcome success) |
|RollbackFailed | TaskRun.Finished (with outcome failed) |
|UninstallSucceeded | Environment.Modified |
|UninstallFailed | Incident.Detected |
|ReconciliationSucceeded | Environment.Modified |
|ReconciliationFailed | Incident.Detected |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the latest HelmRelease v2 API, I can't find these being used. We had these in the old API.


<!--
This section should contain enough information that the specifics of your
change are understandable. This may include API specs and code snippets.

The design details should address at least the following questions:
- How can this feature be enabled / disabled?
- Does enabling the feature change any default behavior?
- Can the feature be disabled once it has been enabled?
- How can an operator determine if the feature is in use?
- Are there any drawbacks when enabling this feature?
-->

## Implementation History

<!--
Major milestones in the lifecycle of the RFC such as:
- The first Flux release where an initial version of the RFC was available.
- The version of Flux where the RFC graduated to general availability.
- The version of Flux where the RFC was retired or superseded.
-->
Binary file added rfcs/NNNN-cdevents-provider/user-stories-provider.drawio.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading