Size of workers for parallel execution

[raffaem]

Regarding this chapter of the book, looks like you now need to configure the toal size of objects sent to the worker per backend.

So this line:

options(future.globals.maxSize = 8000 * 1024^2) # 8 GB memory

should be remove, while this line:

plan(multicore, workers = cores)

should become:

plan(multicore, workers = core, maxSizeOfObjects = 10e9) # raise max size to 10 GB

See the changelog for future 1.40:

The maximum total size of objects send to and from the worker can now be configured per backend, e.g. plan(multisession, maxSizeOfObjects = 10e6) will produce an error if the total size of globals exceeds 10 MB.

Backends sequential and multicore no longer has a limit on the maximum size of globals, i.e. they now default to maxSizeOfObjects = +Inf. Backends cluster and multisession also default to maxSizeOfObjects = +Inf, unless R option future.globals.maxSize (sic!) is set.

[vincentarelbundock]

The changelog says that this option CAN be set in plan(), not that it MUST be set in that function. AFAICT, the global option used in the vignette is still supported by future. See for example:

https://github.com/futureverse/future/blob/24815e372420c1001b8a66e144d2d7ff9a3fe170/R/backend_api-Future-class.R#L804