LOG-4445: Forwarder must use hosted cluster_id on HCP cluster

[Clee2691]

Description

This PR updates #2280 to use the hypershift API instead of the unstructured object.

The github.com/openshift/hypershift/api package is added as a dependency.

New permissions are added to the role that CLO's serviceAccount is bound to:

- apiGroups:
  - hypershift.openshift.io
  resources:
  - hostedcontrolplanes
  verbs:
  - list
  - get

Lastly, this PR adds the VRL transform that kubeAPI audit logs should go through as it was missing for http receiver inputs.

/cc @cahartma @vparfonov @alanconway
/assign @jcantrill

Links

• JIRA: https://issues.redhat.com/browse/LOG-4445
• Related PRs
  • WIP: LOG-4445: Forwarder must use hosted cluster_id on HCP cluster. #2274

[openshift-ci-robot]

@Clee2691: This pull request references LOG-4445 which is a valid jira issue.

In response to this:

Description

This PR updates #2280 to use the hypershift API instead of the unstructured object.

The github.com/openshift/hypershift/api package is added as a dependency.

/cc @cahartma @vparfonov @alanconway
/assign @jcantrill

Links

• JIRA: https://issues.redhat.com/browse/LOG-4445
• Related PRs
• WIP: LOG-4445: Forwarder must use hosted cluster_id on HCP cluster. #2274

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[jcantrill]

/approve

[jcantrill]

/hold

pending feedback from QE team

[alanconway]

/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alanconway, Clee2691, jcantrill

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [alanconway,jcantrill]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci-robot]

@Clee2691: This pull request references LOG-4445 which is a valid jira issue.

In response to this:

Description

This PR updates #2280 to use the hypershift API instead of the unstructured object.

The github.com/openshift/hypershift/api package is added as a dependency.

New permissions are added to the role that CLO's serviceAccount is bound to:

- apiGroups:
 - hypershift.openshift.io
 resources:
 - hostedcontrolplanes
 verbs:
 - list
 - get

Lastly, this PR adds the VRL transform that kubeAPI audit logs should go through as it was missing for http receiver inputs.

/cc @cahartma @vparfonov @alanconway
/assign @jcantrill

Links

• JIRA: https://issues.redhat.com/browse/LOG-4445
• Related PRs
• WIP: LOG-4445: Forwarder must use hosted cluster_id on HCP cluster. #2274

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[Clee2691]

/retest

[Clee2691]

/retest

[openshift-ci]

@anpingli: The label(s) /label qe_approved cannot be applied. These labels are supported: acknowledge-critical-fixes-only, platform/aws, platform/azure, platform/baremetal, platform/google, platform/libvirt, platform/openstack, ga, tide/merge-method-merge, tide/merge-method-rebase, tide/merge-method-squash, px-approved, docs-approved, qe-approved, ux-approved, no-qe, downstream-change-needed, rebase/manual, cluster-config-api-changed, approved, backport-risk-assessed, bugzilla/valid-bug, cherry-pick-approved, jira/valid-bug, stability-fix-approved, staff-eng-approved. Is this label configured under labels -> additional_labels or labels -> restricted_labels in plugin.yaml?

In response to this:

/label qe_approved

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[anpingli]

/label qe-approved

[openshift-ci-robot]

@Clee2691: This pull request references LOG-4445 which is a valid jira issue.

In response to this:

Description

This PR updates #2280 to use the hypershift API instead of the unstructured object.

The github.com/openshift/hypershift/api package is added as a dependency.

New permissions are added to the role that CLO's serviceAccount is bound to:

- apiGroups:
 - hypershift.openshift.io
 resources:
 - hostedcontrolplanes
 verbs:
 - list
 - get

Lastly, this PR adds the VRL transform that kubeAPI audit logs should go through as it was missing for http receiver inputs.

/cc @cahartma @vparfonov @alanconway
/assign @jcantrill

Links

• JIRA: https://issues.redhat.com/browse/LOG-4445
• Related PRs
• WIP: LOG-4445: Forwarder must use hosted cluster_id on HCP cluster. #2274

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[anpingli]

QE Test pass

• The hcp cluster id is used when CLF is deployed in the hypershift control panel namespace.
• The management cluster ID is used when CLF is deployed in the other namespaces.

File a bug https://issues.redhat.com/browse/LOG-6921. Please feel free to close it as known issue.
As hcp cluster ID can only be injected in the hypershift control panel namespace, we need a release note about this point.

[Clee2691]

/retest

[Clee2691]

/hold cancel

[openshift-ci]

@Clee2691: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[jcantrill]

@Clee2691 @anpingli have we tested this in a non HCP cluster to ensure the operator does not fail to start because maybe it is missing the CRD for the hypershift apis?

[Clee2691]

Isn't that what the e2e tests basically confirm? CLO is able to be installed and send logs even with the hypershift scheme added. On each reconcile, it will try to fetch the clusterID and version so if it failed, none of those tests would have passed.

[anpingli]

Tested on regular cluster, the clusterID present correctly. No regression issue.

[jcantrill]

/hold

[jcantrill]

/hold cancel
/lgtm

[Clee2691]

/cherry-pick master

[openshift-cherrypick-robot]

@Clee2691: #2990 failed to apply on top of branch "master":

Applying: LOG-4445: Forwarder must use hosted cluster_id on HCP cluster
Using index info to reconstruct a base tree...
M	bundle/manifests/cluster-logging.clusterserviceversion.yaml
M	internal/controller/observability/clusterlogforwarder_controller.go
A	internal/generator/vector/filter/openshift/viaq/filter.go
Falling back to patching base and 3-way merge...
CONFLICT (modify/delete): internal/generator/vector/filter/openshift/viaq/filter.go deleted in HEAD and modified in LOG-4445: Forwarder must use hosted cluster_id on HCP cluster. Version LOG-4445: Forwarder must use hosted cluster_id on HCP cluster of internal/generator/vector/filter/openshift/viaq/filter.go left in tree.
Auto-merging internal/controller/observability/clusterlogforwarder_controller.go
CONFLICT (content): Merge conflict in internal/controller/observability/clusterlogforwarder_controller.go
Auto-merging bundle/manifests/cluster-logging.clusterserviceversion.yaml
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config advice.mergeConflict false"
Patch failed at 0001 LOG-4445: Forwarder must use hosted cluster_id on HCP cluster

In response to this:

/cherry-pick master

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.