Skip to content

Conversation

@belimawr
Copy link
Contributor

@belimawr belimawr commented Dec 9, 2025

Proposed commit message

Add support in the journald inpur for using chroot when calling
`journalctl`. In a container environment this allows to mount the host
file system into the container and use its `journalctl`, which
prevents any sort of incompatibility between the `journalctl` in the
container image and the host Journald.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
  • I have added an entry in ./changelog/fragments using the changelog tool.

## Disruptive User Impact

Author's Checklist

  • Finish manual testing steps

How to test this PR locally

Run the tests

cd filebeat
go test -v -count=1 -tags=integration -run=TestNewFactoryChroot ./input/journald/pkg/journalctl
go test -tags=integration -v -count=1 -run=TestJournaldChroot ./tests/integration

Manual testing

1. Create a filebeat.yml

filebeat.yml

filebeat.inputs:
  - type: journald
    id: journald-input-id
    chroot: /host
    journalctl_path: /usr/bin/journalctl

output.discard:
  enabled: true

# output.console:
#   enabled: true
#   pretty: false

logging:
  to_stderr: true
  metrics.period: 5s

2. Build a staicaly linked Filebeat binary

CGO_ENABLED=0 go build -ldflags '-extldflags "-static" -s' -tags timetzdata .

3. Create a Dockerfile and build a test image

Dockerfile

FROM alpine:latest AS prep

FROM scratch

COPY --from=prep /etc/alpine-release /etc/alpine-release
COPY filebeat /
COPY filebeat.yml /
ENTRYPOINT ["/filebeat"]
CMD ["-c", "/filebeat.yml", "--strict.perms=false"]

docker build -t filebeat-chroot-test .

4. Run the container

docker run --rm -v /:/host filebeat-chroot-test

You should see in the logs the journalctl starting:

{
  "@timestamp": "2025-12-10T17:10:04.317Z",
  "ecs.version": "1.6.0",
  "id": "journald-input-id",
  "input_id": "journald-input-id",
  "input_source": "LOCAL_SYSTEM_JOURNAL",
  "log.level": "info",
  "log.logger": "input.journald.reader.journalctl-runner",
  "log.origin": {
    "file.line": 152,
    "file.name": "journalctl/journalctl.go",
    "function": "github.com/elastic/beats/v7/filebeat/input/journald/pkg/journalctl.NewFactory.func1"
  },
  "message": "Journalctl command: journalctl --utc --output=json --no-pager --all --follow --no-tail --boot all",
  "path": "LOCAL_SYSTEM_JOURNAL",
  "service.name": "filebeat"
}
{
  "@timestamp": "2025-12-10T17:10:04.317Z",
  "ecs.version": "1.6.0",
  "id": "journald-input-id",
  "input_id": "journald-input-id",
  "input_source": "LOCAL_SYSTEM_JOURNAL",
  "log.level": "info",
  "log.logger": "input.journald.reader.journalctl-runner",
  "log.origin": {
    "file.line": 158,
    "file.name": "journalctl/journalctl.go",
    "function": "github.com/elastic/beats/v7/filebeat/input/journald/pkg/journalctl.NewFactory.func1"
  },
  "message": "journalctl started with PID 23",
  "path": "LOCAL_SYSTEM_JOURNAL",
  "service.name": "filebeat"
}

You can also see the number events published in the metrics that are logged every 5s.

If you want to see the events, remove the discard output and uncomment
the console output in filebeat.yml, then rebuild the docker image
and run it again.

Related issues

## Use cases
## Screenshots
## Logs

@belimawr belimawr self-assigned this Dec 9, 2025
@belimawr belimawr added backport-skip Skip notification from the automated backport with mergify Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team labels Dec 9, 2025
@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Dec 9, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Dec 9, 2025

🤖 GitHub comments

Just comment with:

  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@belimawr
Copy link
Contributor Author

belimawr commented Dec 9, 2025

The failing test on Windows is unrelated to this PR. The flaky test issue: #48009

@belimawr belimawr changed the title [WIP] Journald chroot [Filebeat/Journald] allow to use a chroot when calling journalctl Dec 9, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Dec 10, 2025

🔍 Preview links for changed docs

@belimawr belimawr marked this pull request as ready for review December 11, 2025 13:01
@belimawr belimawr requested review from a team as code owners December 11, 2025 13:01
@belimawr belimawr requested review from AndersonQ and faec December 11, 2025 13:01
@elasticmachine
Copy link
Contributor

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

Copy link
Member

@AndersonQ AndersonQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a few suggestions

Copy link
Member

@cmacknz cmacknz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of my concerns are security related, executing whatever happens to be called journalctl in the path as root seems like a not so good idea.

# Long description; in case the summary is not enough to describe the change
# this field accommodate a description without length limits.
# NOTE: This field will be rendered only for breaking-change and known-issue kinds at the moment.
#description:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#description:
description: |
Add support in the journald inpur for using chroot when calling
`journalctl`. In a container environment this allows to mount the host
file system into the container and use its `journalctl`, which
prevents any sort of incompatibility between the `journalctl` in the
container image and the host Journald. Allows using the journald input with Wolfi based Docker containers.

Let's include the context from the PR to help people know why they would want to use this.

When using the Journald input from a Docker container, make sure that
either:
- {applies_to}`stack: ga 9.3.0` [`chroot`](#filebeat-input-journald-chroot), [`journalct_path`](#filebeat-input-journald-journalctl-path) are set, or
- The `journalctl` binary in the container is compatible with your
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

List the container variants that include journald by default.

Comment on lines +823 to +824
# The absolute path for the `journalctl` binary. If not set Filebeat
# will look for `journalctl` in `PATH`. When using
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to allow executing whatever thing called journalctl we happen find?

That seems less secure than requiring the absolute path to the binary, as there is only a single file system path that can be exploited now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Especially if we execute journalctl as root, we should narrow the scope of what we will execute as much as possible.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only reason to just look for journalctl is if the path to it isn't consistent all hosts, where finding it then becomes really annoying.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On an ubuntu24 box I see it as the following which is certainly not hard to find.

ubuntu@ubuntu24:~$ which journalctl
/usr/bin/journalctl

If we think using PATH is necessary we could make it optional and default to off.

if !errors.Is(err, io.EOF) {
var logError = false
var pathError *fs.PathError
if errors.As(err, &pathError) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why isn't this also handled for stderr?


jctlCommandMsg := fmt.Sprintf("Journalctl command: %s %s", journalctlPath, strings.Join(args, " "))
if chroot != "" {
jctlCommandMsg = fmt.Sprintf("%s Chroot: %s (command path relative to chroot)", jctlCommandMsg, chroot)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be a bit tidier to use InfoW with command and chroot keys instead of this formatting IMO.

return nil, nil, fmt.Errorf("provided chroot (%s) is not a directory", config.Chroot)
}

fullPath := filepath.Join(config.Chroot, config.JournalctlPath)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may want to verify the ownership of the file, maybe that it is only writable by root so it can't be replaced by something from an arbitrary user, especially if we are going to execute it as root.

Speaking of executing it as a root, if we can execute this as a lower privileged user that would help the security boundary here too.

```
A folder to be used as chroot when calling `journalctl`. This allows
Filebeat to call the host's `journalctl` directly. If using this
option, {{filebeat}} must be run as root and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The wolfi containers are not root by default, you will need examples of how to give the capabilities needed to do this.

You potentially only need CAP_SYS_CHROOT and not all the root capabilities, we should identify what the least privileged way for this to execute is.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we can say that you just need CAP_SYS_CHROOT without actually being uid 0 that would help considerably so we are also not executing potentially arbitrary commands as root.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-skip Skip notification from the automated backport with mergify Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Filebeat/Journald] When in a container, use chroot to run journalctl from the host, thus eliminating version issues

4 participants