Memory leak in queued job process() function

[sig-owalker]

Module version(s) affected

5.2.0

Description

There is an issue with the process() function in a queued job. On jobs which have a lot of steps, but each step is quick to process, the job will run out of memory very fast and have to wait and release memory.

This is because each time the process() function runs, approximately 2MB of extra memory is used, even if the process task does nothing. This quickly accumulates and causes the job to go into waiting.

How to reproduce

Reproduction steps:

1. Setup a queued job with around 10000 total steps
2. Create an empty process() function on the job, which only increments $this->currentStep
3. Queue the job and let it run via CRON or by manually running the ProcessJobQueueTask

Possible Solution

No response

Additional Context

No response

Validations

• Check that there isn't already an issue that reports the same bug
• Double check that your reproduction steps work in a fresh installation of silverstripe/installer (with any code examples you've provided)

[GuySartorelli]

This is because each time the process() function runs, approximately 2MB of extra memory is used, even if the process task does nothing.

Were you able to identify where that extra memory usage is coming from?