[9.4](backport #50980) Fix filebeat test_shutdown_wait_ok

[mergify]

Proposed commit message

With the fix merged from #50625 it results in all events being removed from the pipeline on shutdown. Then this test didn't set the correct settings to prevent file rotation that would cause the test to fail. With both the change in this PR and the race condition fix this test can be re-enabled.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
• [ ] I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

None

How to test this PR locally

Run unit tests.

Related issues

• Closes test_shutdown_wait_ok sometimes fails / filebeat fails to send all queued events on shutdown #14647

  ---

  This is an automatic backport of pull request Fix filebeat test_shutdown_wait_ok #50980 done by Mergify.

[botelastic]

This pull request doesn't have a Team:<team> label.

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)
• /test : Run the Buildkite pipeline.