mirai 2.5.3

Updates

• OpenTelemetry: 'daemon connect' spans are now recorded only after a connection has actually been made (#511).
• Fixes a bug which caused mirai to remain unresolved after switching from synchronous daemons to dispatcher daemons in the same session (#509).
• mirai_map() now returns invisibly, consistent with mirai().

mirai 2.5.2

Updates

• The default daemons autoexit = TRUE behaviour has been updated for OpenTelemetry compatibility (#500).
  • Introduces a 200ms grace period for processes to exit normally before a forceful termination.
  • The behavioural changes announced in mirai 2.4.0 are now enforced for all daemon types - eliminating a bug that previously caused this to only be applied to ephemeral daemons.
• OpenTelemetry span names and attributes have been upgraded to be more informative and better follow semantic conventions (#481).
• require_daemons() updates:
  • Function now returns invisibly as intended.
  • Using with .compute as the first argument (which produced a warning) no longer works.
• Requires nanonext >= 1.7.2.

mirai 2.5.1

New Features

• Adds race_mirai(), which accepts a list of mirai and waits efficiently for the next mirai amongst them to resolve (@t-kalinowski, #448).
• New synchronous mode: daemons(sync = TRUE) causes mirai to run synchronously within the current process.
  This facilitates testing and debugging, e.g. via interactive browser() instances (#439, @kentqin-cve #442).

Updates

• everywhere() adds argument .min to specify a minimum number of daemons on which to evaluate the expression (when using dispatcher).
  This creates a synchronization point and can be useful when launching remote daemons to ensure that the expression has run on all daemons to connect (@louisaslett, #330).
• OpenTelemetry span names have been updated.
  Spans for long-running daemons are now split into short spans when they are created and when they end - refer to the updated vignette for more details (#464, #471).
• Removes the following developer features:
  • nextget("pid") is no longer a supported option.
  • Argument id is removed at daemon(). This means that status() no longer reports daemon connection or disconnection events.
• Removes deprecated argument tls at daemon(), launch_local() and launch_remote().
• Removes deprecated dispatcher argument 'none' at daemons(), deprecated in mirai v2.1.0.
• Fixes a phenomenon where stop_mirai() or mirai_map()[.stop] could on occasion cause (dispatcher) daemons to be interrupted and exit on subsequent runs (#459).
• Non-dispatcher daemons now synchronize upon timeout or task-out, ensuring that they exit safely only after all data has been sent (#458).
• Requires nanonext >= 1.7.1.

mirai 2.5.0

Behavioural Changes

• Behavioural changes for daemons():
  • Returns invisibly logical TRUE when creating daemons and FALSE when resetting, for simplicity and consistency (#384).
  • Creating new daemons resets any existing daemons for the compute profile rather than error.
    This means that an explicit daemons(0) is no longer required before applying new settings (thanks @eliocamp, #383).
  • Calling without supplying any arguments now errors rather than return the value of status().

New Features

• Complete observability of mirai requests by emitting OpenTelemetry traces when tracing is enabled by the otelsdk package (#394).
• Adds info() as an alternative to status() for retrieving more succinct information statistics, more convenient for programmatic use (thanks @wlandau, #410).
• Adds with_daemons() and local_daemons() helper functions for using a particular compute profile.
  These work with daemons that are already set up unlike the existing with.miraiDaemons() method, which creates a new scope and tears it down when finished (#360).
• A mirai now has an attribute id, which is a monotonically increasing integer identifier unique to each session.

Updates

• stop_mirai() is more efficient and responsive, especially for 'mirai_map' objects (#417).
• miraiError enhancements:
  • The original condition classes are preserved as $condition.class (thanks @sebffischer, #400).
  • The print method includes the customary additional line break (thanks @sebffischer, #399).
• Fixes daemons(n) failing to launch local daemons if mirai was installed in a custom user library set by an explicit .libPaths() call in '.Rprofile' (thanks @erydit and @dpastoor, #390).
• Improved behaviour for serial_config() custom serialization.
  If the serialization hook function errors or otherwise fails to return a raw vector, this will error out rather than be silently ignored (thanks @dipterix, #378).
• as.promise() method for mirai made robust for high-throughput scenarios (#377).
• mirai_map() now supports Arrow Tables and Polars DataFrames (#366).
• require_daemons() arguments are swapped so that .compute comes before call for ease of use.
  Previous usage will work for the time being, although is deprecated and will be defunct in a future version.
• Enhancements to everywhere():
  • Consecutive everywhere() calls are permissible again when using dispatcher (behaviour update in v2.4.1) (#354).
  • No longer has any effect on the RNG stream when using a reproducible seed value at daemons() (#356).
• A mirai() evaluated on an ephemeral daemon returns invisibly, consistent with other cases (#351).
• daemon() gains a tlscert argument for custom TLS certificates.
  The change in argument name lets this be passed when making a daemons() call (#344).
• The tls argument at daemon(), launch_local() and launch_remote() is deprecated.
• Requires nanonext >= 1.7.0.

mirai 2.4.1

New Features

• Reproducible parallel RNG by setting the seed argument to daemons():
  • The default NULL uses L'Ecuyer-CMRG RNG streams advanced per daemon, the same as base R's parallel package, which produces statistically-sound yet generally non-reproducible results.
  • Setting an integer seed now initializes a L'Ecuyer-CMRG RNG stream for the compute profile, which is advanced for each mirai evaluation, which does provide reproducible results.

Updates

• everywhere() has been updated for robustness and ease of use:
  • Returns a mirai_map object for easier handling (rather than just a list of mirai).
  • When using dispatcher, no longer has the potential to fail if sending large data (#326).
• dispatcher() function signature simplified with rs, tls and pass arguments removed (no user-facing impact).
• Fixes a bug where using non-dispatcher daemons, an unresolvedValue could be returned as the fulfilled value of a promise in extremely rare cases (thanks @James-G-Hill and @olivier7121, #243 and #317).
• Fixes a regression in mirai 2.4.0 where the L'Ecuyer-CMRG seed was not being passed correctly for remote daemons (#333).
• Requires nanonext >= 1.6.2.

mirai 2.4.0

Behavioural Changes

• An ephemeral daemon started by mirai() without setting daemons now exits as soon as the parent process does rather than finish the task.
• Change in daemon() defaults:
  • Argument autoexit default of TRUE now ensures daemons are terminated along with the parent process.
    Set to NA to retain the previous behaviour of having them automatically exit after completing any in-progress tasks.
  • Argument dispatcher now defaults to TRUE.
• Calling daemons() to create local daemons now errors if performed within a mirai_map() call.
  This guards against excessive spawning of local processes on a single machine.

New Features

• Adds cluster_config() to launch remote daemons via HPC resource managers for Slurm, SGE, Torque, PBS and LSF clusters.
• Adds require_daemons() as a developer function that prompts the user to set daemons if not already set, with a clickable function link if the cli package is available.

Updates

• Simplifies launches when using dispatcher - launch_remote() commands are now the same irrespective of the number of launches.
  This is as daemons now retrieve the next RNG stream from dispatcher rather than the rs argument to daemon().
• Deprecated call_mirai_() is now removed.
• Requires nanonext >= 1.6.1.

mirai 2.3.0

Behavioural Changes

• mirai() argument .timeout is upgraded to automatically cancel ongoing mirai upon timeout when using dispatcher (thanks @be-marc, @sebffischer #251).
• serial_config() now accepts vector arguments to register multiple custom serialization configurations. Argument vec is dropped as internal optimizations mean this option no longer needs to be set.

New Features

• host_url() is upgraded to return all local IP addresses (named by network interface), which provides a more comprehensive solution than just using a hostname.
• Adds register_serial() to register serialization configurations for all daemons() calls (may be used by package authors as a convenience).
• Adds on_daemon() which returns a logical value, whether or not evaluation is taking place within a mirai call on a daemon.
• Adds daemons_set() which returns a logical value, whether or not daemons are set for a given compute profile.
• daemons() now supports initial synchronization exceeding 10s (between host/dispatcher/daemons). This is particularly relevant for HPC setups (thanks @sebffischer, #275).

Updates

• For all functions that use .compute, this argument has a new default of NULL, which continues to use the default profile (and hence should not result in any change in behaviour).
• Fixes stop_mirai() failing to interrupt in certain cases on non-Windows platforms, and more robust interruption if tools::SIGINT is supplied or passed through to the autoexit argument of daemon() (thanks @LennardLux, #240).
• daemons() dispatcher argument 'process', deprecated in mirai v2.1.0, is removed.
• Requires nanonext >= 1.6.0.
• Package is re-licensed under the MIT license.

mirai 2.2.0

Behavioural Changes

• Simplified SSH tunnelling for distributed computing:
  • ssh_config() argument 'port' is removed, with the tunnel port now inferred at the time of launch, and no longer set by the configuration.
  • local_url() adds logical argument 'tcp' for easily constructing an automatic local TCP URL when setting daemons() for SSH tunnelling.

New Features

• Adds as.promise() method for 'mirai_map' objects. This will resolve upon completion of the entire map operation.

Updates

• mirai (in R >= 4.5) is now one of the official base R parallel cluster types.
  • register_cluster() is removed as no longer required.
  • Directly use parallel::makeCluster(type = "MIRAI") to create a 'miraiCluster'.
• call_mirai() is now user-interruptible, consistent with all other functions in the package.
  • call_mirai_() is hence redundant and now deprecated.
• with() method for daemons() now propagates ".compute" so that this does not need to be specified in functions such as mirai() within the with() clause.
• mirai() arguments ... and .args now accept environments containing variables beginning with a dot . (#207).
• 'miraiError' stack traces no longer sometimes contain an additional (internal) call (#216).
• 'miraiError' condition $call objects are now stripped of 'srcref' attributes (thanks @lionel-, #218).
• A mirai promise now rejects in exactly the same way whether or not the mirai was already resolved at time of creation.
  This avoids Shiny deep stack trace errors when the mirai had already resolved (#229).
• daemons() calls that error due to the remote launcher no longer leave the compute profile set up (#237).

mirai 2.1.0

Behavioural Changes

• daemons() now requires an explicit reset before providing revised settings for a compute profile, and will error otherwise.
• mirai_map() now errors if daemons have not yet been set (rather than warn and launch one local daemon).
• Removal of mirai v1 compatibility features:
  • saisei() is now removed as no longer required.
  • daemons() dispatcher argument "thread" is removed.
  • daemons() dispatcher arguments "process" and "none" are formally deprecated and will be removed in a future version.

Updates

• 'miraiError' evaluation errors now return the call stack at $stack.trace as a list of calls (with srcrefs removed) without deparsing to character strings.
• mirai_map() improvements:
  • Multiple map on a dataframe or matrix now correctly preserves the row names of the input as the names of the output.
  • Fixes language objects being evaluated before the map function is applied (#194).
  • Fixes classes of objects in a dataframe being dropped during a multiple map (#196).
  • Better cli errors when collecting a 'mirai_map'.
• Fixes daemons(NULL) not causing all daemons started with autoexit = FALSE to quit, regression introduced in mirai v2.0.0.
• Requires nanonext >= 1.5.0.

mirai 2.0.1

Updates

• mirai_map() collection option improvements:
  • The cli package is used, if installed, for richer progress bars and error messages.
  • [.progress_cli] is no longer a separate option.
  • [.stop] now reports the index number that errored.
• collect_mirai() replaces '...' with an 'options' argument, to which collection options should be supplied as a character vector. This avoids non-standard evaluation in this function.
• daemon() now returns an integer exit code to indicate the reason for termination.
• Adds nextcode() to provide a human-readable translation of the exit codes returned by daemon().
• everywhere() now returns a list of at least one mirai regardless of the number of actual connections.