Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LOG-6734: Splunk Event Metadata: Enhancements Doc #2979

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

vparfonov
Copy link
Contributor

@vparfonov vparfonov commented Mar 11, 2025

Description

This PR adds detailed documentation for forwarding logs to Splunk using Cluster Log Forwarder.

It covers the use of Spunk metadata keys via Vector configuration parameters: source, indexed_fields, sourceType, and host. Added payloadKey parameter.

  • How source can be dynamically derived from .log_type and .log_source
  • Explanation of indexed_fields, their limitations, and storage implications
  • Description of the extended remap function, allowing nested field support and automatic value conversion
  • payloadKey specifies record field to use as payload
  • sourceType determined automatically based on the type of the final event payload (_json or generic_single_line)
  • host from .hostname, independent of user configuration

/cc
/assign

/cherry-pick

Links

@jcantrill
Copy link
Contributor

/refresh

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 13, 2025
@vparfonov vparfonov changed the title WIP: Splunk event metadata LOG-6734: Splunk event metadata Mar 13, 2025
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Mar 13, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Mar 13, 2025

@vparfonov: This pull request references LOG-6734 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the spike to target the "4.8.0" version, but no target version was set.

In response to this:

Description

This PR adds detailed documentation for forwarding logs to Splunk using Cluster Log Forwarder.

It covers the use of Spunk metadata keys via Vector configuration parameters: source, indexed_fields, sourceType, and host.

  • How source can be dynamically derived from .log_type and .log_source
  • Explanation of indexed_fields, their limitations, and storage implications
  • Description of the extended remap function, allowing nested field support and automatic value conversion
  • Fixed values for sourceType with _json and host from .hostname, independent of user configuration

/cc
/assign

/cherry-pick

Links

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 13, 2025
@jcantrill
Copy link
Contributor

/approve

@jcantrill
Copy link
Contributor

/hold

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 18, 2025
Copy link
Contributor

openshift-ci bot commented Mar 18, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jcantrill, vparfonov

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 18, 2025
@vparfonov vparfonov changed the title LOG-6734: Splunk event metadata LOG-6734: Splunk Event Metadata: Enhancements Doc Mar 18, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Mar 18, 2025

@vparfonov: This pull request references LOG-6734 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the spike to target the "4.8.0" version, but no target version was set.

In response to this:

Description

This PR adds detailed documentation for forwarding logs to Splunk using Cluster Log Forwarder.

It covers the use of Spunk metadata keys via Vector configuration parameters: source, indexed_fields, sourceType, and host. Added payloadKey parameter.

  • How source can be dynamically derived from .log_type and .log_source
  • Explanation of indexed_fields, their limitations, and storage implications
  • Description of the extended remap function, allowing nested field support and automatic value conversion
  • payloadKey specifies record field to use as payload
  • sourceType determined automatically based on the type of the final event payload (_json or generic_single_line)
  • host from .hostname, independent of user configuration

/cc
/assign

/cherry-pick

Links

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@jcantrill
Copy link
Contributor

/override ci/prow/e2e=target
/override ci/prow/unit

Copy link
Contributor

openshift-ci bot commented Mar 18, 2025

@jcantrill: /override requires failed status contexts, check run or a prowjob name to operate on.
The following unknown contexts/checkruns were given:

  • ci/prow/e2e=target

Only the following failed contexts/checkruns were expected:

  • ci/prow/ci-index-cluster-logging-operator-bundle
  • ci/prow/e2e-target
  • ci/prow/functional-target
  • ci/prow/images
  • ci/prow/lint
  • ci/prow/unit
  • ci/prow/upgrade
  • pull-ci-openshift-cluster-logging-operator-master-ci-index-cluster-logging-operator-bundle
  • pull-ci-openshift-cluster-logging-operator-master-e2e-target
  • pull-ci-openshift-cluster-logging-operator-master-functional-target
  • pull-ci-openshift-cluster-logging-operator-master-images
  • pull-ci-openshift-cluster-logging-operator-master-lint
  • pull-ci-openshift-cluster-logging-operator-master-unit
  • pull-ci-openshift-cluster-logging-operator-master-upgrade
  • tide

If you are trying to override a checkrun that has a space in it, you must put a double quote on the context.

In response to this:

/override ci/prow/e2e=target
/override ci/prow/unit

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@jcantrill
Copy link
Contributor

/override ci/prow/e2e-target

Copy link
Contributor

openshift-ci bot commented Mar 18, 2025

@jcantrill: Overrode contexts on behalf of jcantrill: ci/prow/e2e-target

In response to this:

/override ci/prow/e2e-target

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@vparfonov
Copy link
Contributor Author

/test unit

@jcantrill
Copy link
Contributor

/override ci/prow/unit

Copy link
Contributor

openshift-ci bot commented Mar 19, 2025

@jcantrill: Overrode contexts on behalf of jcantrill: ci/prow/unit

In response to this:

/override ci/prow/unit

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Copy link
Contributor

openshift-ci bot commented Mar 19, 2025

@vparfonov: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. release/6.3
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants