[Tracking] Context Propagation

[thisthat]

Goal

Propagate the W3C Trace Context and other metadata across promotion stages via KeptnApp CRDs.

Technical Details

I want to Propagate W3C trace-id across KeptnApp.
Propagate context through the deployment.
Any possible refactor: e.g., split KeptnApp release-matrix from pre-post tasks.

DoD

• I have a KeptnApp deployed in prod, I want to trace it back from the version that was deployed in dev, passing through hardening.
• I want to have pre-post tasks added after I started with an auto-discovered KeptnApp
• I want to attach metadata to the deployment of a KeptnApp and also customize that per-workload. This metadata must be:
  • visible in the Spans as span/event attributes.
  • passed to KeptnTasks and KeptnEvaluations at both: pre/post App and Workload levels.

---

Example:

kind: KeptnAppContext
name: "my-awesome-app" # this has to be the same value of part-of label
spec:
  preDeploymentTasks:
      - task1
      - task2
  postDeploymentTasks:
      - task1
      - task2
  preDeploymentEvaluations:
      - eval1
      - eval2
  postDeploymentEvaluations:
      - eval1
      - eval2
  metadata:
      - key: value
      - ...
  traceLinks:
    - "w3c trace id1"
    - "w3c trace id2"
--- 
kind: Deployment
spec:
 template:
   annotations:
     traceparent: "w3c parent trace"
     keptn.sh/metadata: "myMetadataKey1=myValue1,myMetadata2=myvalue2,..."
     k8s.io/part-of: my-awesome-app
     keptn.sh/pre-task: mytask1

Problems:

1. Having a CR with the context could create problems because of:

• apply order: we could start taking care of a Workload before the Ctx CR is applied
• different workloads might have different metadata: e.g. gitSHA could differ from each workload
• Clean-up of old data is tricky since we would need to either edit the same ctx CR or track old versions

2. Precedence of context between App and Workload

Possible solutions:

1. Add metadata information to the Workload directly via annotations

   • metadata is customizable per-workload
   • each workload could get linked to a different parent trace if they are bumped individually across stages
   • we maintain 1 writer per-CR

2. in the case of the same key, workload def wins

KEP

• KEP 87: Lifecycle Toolkit - Context Propagation for GitOps enhancement-proposals#89
• KEP 89: Lifecycle Toolkit - Context Information in Keptn Tasks  enhancement-proposals#91

### Tasks
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2573
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2574
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2575
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2576
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2577
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2578
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2579
- [ ] Think about K8s default labels or used by other CNCF projects for metadata
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2580
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2581
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2582
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2810
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2813
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2814
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2815
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2836
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2869
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2883
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2884
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2895
- [x] Adapt https://keptn.sh/latest/docs/components/lifecycle-operator/keptn-apps/ page according to new epic feature
- [x] add Xrefs between context pages
- [x] Docs: update links to CRD v1beta1
- [x] Create script for converting v1alpha3 KeptnApp to v1beta1 KeptnAppContext/KeptnApp

[heckelmann]

An additional thought on that:

If it would be possible to add an annotation to the deployment manifest like keptn.sh/metadata where I can store additional information (like a commit ID) which is propagated as an environment variable to the tasks, would enable me to use this information to send events to my observability platform and could also be used in the end-2-end trace.