Switch container logs input to /var/log/pod/* path

[pchila]

Proposed commit message

This PR changes the input definition for ingesting k8s container logs from /var/log/containers/* to /var/log/pods/* which contains all available logs (including rotated ones) for pods running on the k8s node.

Before this change the integration would configure the elastic-agent input using the currently running container id and locating the corresponding log file under /var/log/containers/* which would point to the current log file, ignoring previously rotated log files for the container.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

• [ ]

How to test this PR locally

Related issues

• Closes [k8s provider] Look up metadata based on file path elastic-agent#5164
• Relates Switch k8s input paths to /var/log/pods/* to ingest rotated container logs elastic-agent#6583

Screenshots

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
50.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: b8d679d

History

• 💚 Build #21103 succeeded 7fa7b6f7d9764199ba2ba8f570bf2b6adc6fcb08

cc @pchila

[botelastic]

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

[botelastic]

Hi! This PR has been stale for a while and we're going to close it as part of our cleanup procedure. We appreciate your contribution and would like to apologize if we have not been able to review it, due to the current heavy load of the team. Feel free to re-open this PR if you think it should stay open and is worth rebasing. Thank you for your contribution!