How to improve Flow workers pickup speed in AIO

[derigs]

All Flow tasks/runs will be triggered as part of the background job system in Nextcloud which only runs jobs every 5 minutes by default. To pick up scheduled jobs faster you can set up background job workers that process the Tasks as soon as they are scheduled. If the PHP code or the Nextcloud settings values are changed while a worker is running, those changes won’t be effective inside the runner. For that reason, the worker needs to be restarted regularly. It is done with a timeout of N seconds which means any changes to the settings or the code will be picked up after N seconds (worst case scenario). This timeout does not, in any way, affect the processing or the timeout of the tasks. Since it is a rework of the AI Worker i will also leave a link here (#5430).

Use these instructions https://docs.nextcloud.com/server/latest/admin_manual/webhook_listeners/index.html#systemd-service but in step 2, use the following script:

#!/bin/bash
echo "Starting Nextcloud Webhook Worker $1"

# Function to attempt starting the worker
start_worker() {
    echo "Attempting to start Nextcloud Webhook Worker $1..."
    
    # Run the worker inside the container (60 seconds for task execution)
    docker exec --user www-data nextcloud-aio-nextcloud php occ background-job:worker -t 60 'OCA\WebhookListeners\BackgroundJobs\WebhookCall'
    
    # Capture the exit status of the docker exec command
    exit_code=$?

    # Check if the worker ran successfully
    if [ $exit_code -eq 0 ]; then
        echo "Worker completed successfully."
        return 0  # Success
    else
        echo "Worker failed to start or complete with exit code $exit_code."
        return 1  # Failure
    fi
}

# Loop to start the worker and retry if it fails
while true; do
    # Check if the container is running
    if ! docker ps --format "{{.Names}}" | grep -q "^nextcloud-aio-nextcloud$"; then
        echo "Nextcloud container is not running. Waiting 60 seconds before next attempt."
        sleep 60  # If the container isn't running, wait for 60 seconds
        continue  # Skip to the next iteration
    fi

    # Start the worker
    start_worker

    # Check if the worker succeeded or failed
    if [ $? -eq 1 ]; then
        # If failed, wait 30 seconds (or adjust as needed) to avoid rapid retries
        echo "Worker failed. Waiting 30 seconds before retrying..."
        sleep 30
    fi
done