Port Nextflow fix, Bump Viash version, Remove deprecated functionality (#832)

* Fix author names in paper (#644)

* add paper

* update paper

* add more orcids

* move figures

* rename pipeline to workflow

* add section on benefits

* rewrite state of the field

* solve issue with figure 3

* update bibtex

* update bibtex again

* broaden domain

* update figure 2 based on @wilkinson's feedback

* minor changes in paper (#628)

* minor change

* fix refactoring error

* Add JOSS release info

* update readme

* Fix author names

---------

Co-authored-by: Daniel S. Katz <d.katz@ieee.org>

* use reflection to get viash to fix issue in nextflow edge (#812)

* use reflection to get viash to fix issue in nextflow edge

* add entry to changelog

* Backport: Make sure scripts in Nextflow have the right extension (#815)

* Backport: Make sure scripts in Nextflow have the right extension

* fix changelog

* Edit -- I'm going to port it in the different direction

* Pass session as an argument (#818)

* pass session as an argument

* fix import

* Prepare for release

* post release changes, update release guide

* Add entry for 0.8.7

* port of #823

* add pr number

* copied 0.8.8 changelog entry

* update version and changelog

* Update CHANGELOG.md

* Update CHANGELOG.md

* post release changes

* cherry pick 5271a3b

* fix test

* Remove deprecated things (#833)

* wip deprecate things

* fix components

* fix test

* update tests and yamls

* fix test (recent versions of git require a message)

* remove test code for removed functionality

* Update CHANGELOG.md

---------

Co-authored-by: Hendrik Cannoodt <hendrik.cannoodt@gmail.com>
Co-authored-by: Daniel S. Katz <d.katz@ieee.org>

Author: 3 people

CHANGELOG.md

@@ -8,26 +8,40 @@ TODO add summary
 
 ## BREAKING CHANGES
 
-* `Nextflow` runner: remove deprecated `map`, `mapId`, `mapData`, `mapPassthrough` and `renameKeys` (PR #792).
+* `Nextflow` runner: remove deprecated `map`, `mapId`, `mapData`, `mapPassthrough` and `renameKeys` arguments (PR #792).
 
-* `Nextflow` runner: remove helper functions: `setWorkflowArguments`, `getWorkflowArguments`, `strictMap`, `passthroughMap`, `passthroughFlatMap` and `passthroughFilter` (PR #792).
+* `Nextflow` runner: remove helper functions: `setWorkflowArguments`, `getWorkflowArguments`, `strictMap`, `passthroughMap`, `passthroughFlatMap`,  `passthroughFilter`, `channelFromParams`, `runComponents` (PRs #792, #811).
 
-* `Nextflow` runner: `channelFromParams` is now deprecated (PR #792).
+* Remove deprecated functionality `functionality` and `platforms` (PR #832).
 
-* `Nextflow` runner: remove deprecated `runComponents` (PR #811).
+## BUG FIXES
 
-## MINOR CHANGES
+* `NextflowRunner`: Automatically convert integers to doubles when argument type is `double` (port of PR #824, PR #825).
+
+## MINOR FIXES
 
 * `Executable`: Add more info to the --help (PR #802).
 
+# Viash 0.9.4 (2025-04-24): Hotfix for Nextflow edge
 
-# Viash 0.9.3 (yyyy-MM-dd): TODO Add title
+This is a hotfix release for the changes in Nextflow edge's handling of double arguments.
+
+## BUG FIXES
+
+* `NextflowRunner`: Automatically convert integers to doubles when argument type is `double` (port of PR #823, PR #824).
+
+# Viash 0.9.3 (2025-03-31): Support upcoming version of Nextflow
+
+The upcoming release of Nextflow introduces a new class for loading scripts and renamed the old class.
+This release supports this change by using reflection to detect the available class.
 
 ## BUG FIXES
 
 * `NextflowRunner`: ScriptParser was renamed to ScriptLoaderV1 in Nextflow 25.02.0-edge (PR #812). This fix uses reflection
   to detect whether ScriptParser exists -- if not the ScriptLoaderFactory is used instead.
 
+* `NextflowRunner`: Make sure scripts are generated with the right extension (PR #815).
+
 # Viash 0.9.2 (2025-03-04): Bug fix release
 
 This release fixes an edge case where output arguments for subworkflows were malformed.
@@ -40,7 +54,6 @@ This release fixes an edge case where output arguments for subworkflows were mal
 
 * `NextflowRunner`: Fix issue where output arguments for subworkflows were being returned as nested arrays instead of simple arrays (PR #798, PR #800).
 
-
 # Viash 0.9.1 (2024-12-16): Enhanced nextflow support and Scala 3 update
 
 Workflows can now publish results asynchronously by emitting multiple output channels. These results will then be merged into a published output behind the screens.
@@ -227,6 +240,28 @@ Implemented a proper way of caching dependency repositories. The cache is stored
 
 * `viash test`: Fix an issue where the tests would not copy package config settings to determine the docker image name (PR #767).
 
+# Viash 0.8.8 (2025-04-24): Hotfix for Nextflow edge
+
+This is a hotfix release for the changes in Nextflow edge's handling of double arguments.
+
+## BUG FIXES
+
+* `NextflowRunner`: Automatically convert integers to doubles when argument type is `double` (PR #823).
+
+# Viash 0.8.7 (2025-04-01): Backport support upcoming version of Nextflow
+
+The upcoming release of Nextflow introduces a new class for loading scripts and renamed the old class.
+This release supports this change by using reflection to detect the available class.
+
+## BUG FIXES
+
+* `viash build`: Fix error handling of non-generic errors in the build process or while pushing docker containers (PR #696).
+
+* `NextflowRunner`: ScriptParser was renamed to ScriptLoaderV1 in Nextflow 25.02.0-edge (PR #817). Backport from Viash 0.9.3 (PR #812).
+  This fix uses reflection to detect whether ScriptParser exists -- if not the ScriptLoaderFactory is used instead.
+
+* `NextflowRunner`: Backport path handling for `meta.resources_dir` for when symlinks are used (PR #817).
+
 # Viash 0.8.6 (2024-04-26): Bug fixes and improvements for CI
 
 Fix some issues in some edge cases.

RELEASE_GUIDE.md

@@ -15,17 +15,21 @@ e.g. from `version := "0.7.2-dev"` to `version := "0.7.2"`
 
 # Merge PR to main
 
-## Add a tag on main with the new version number
+# Release Viash
 
-# Create GitHub release
+## Run 'Prepare Viash Release' workflow on GitHub
 
-Title: 'Viash 0.7.2'
-Content: Add the full changelog entry of this release
+Select the `main` branch
 
-# Build viash and viash_install and add to the release
+## Release the prepared release
 
-Create the binaries with `make && make tools`.
-Add the files `bin/viash` and `bin/viash_install`.
+Head over releases and select the prepared release.
+
+Double check the tag version.
+
+Add the title from the changelog.md into the release notes.
+
+Select the necessary prerelease or latest release options and hit the 'publish release' button.
 
 # Update the website
 

build.sbt

@@ -1,6 +1,6 @@
 name := "viash"
 
-version := "0.9.3-dev"
+version := "0.10.0-dev"
 
 scalaVersion := "3.3.4"
 

docs/paper/paper.md

@@ -2,29 +2,37 @@
 title: 'Viash: A meta-framework for building reusable workflow modules'
 bibliography: library.bib
 authors:
-- name: Robrecht Cannoodt
+- given-names: Robrecht
+  surname: Cannoodt
   email: robrecht@data-intuitive.com
   orcid: 0000-0003-3641-729X
   corresponding: yes
   affiliation: 1, 2, 3
-- name: Hendrik Cannoodt
+- given-names: Hendrik
+  surname: Cannoodt
   orcid: 0000-0002-0032-6719
   affiliation: '1'
-- name: Dries Schaumont
+- given-names: Dries
+  surname: Schaumont
   orcid: 0000-0002-4389-0440
   affiliation: '1'
-- name: Kai Waldrant
+- given-names: Kai
+  surname: Waldrant
   orcid: 0009-0003-8555-1361
   affiliation: '1'
-- name: Eric Van de Kerckhove
+- given-names: Eric
+  surname: Van de Kerckhove
   affiliation: '1'
-- name: Andy Boschmans
+- given-names: Andy
+  surname: Boschmans
   orcid: 0009-0008-8793-4661
   affiliation: '1'
-- name: Dries De Maeyer
+- given-names: Dries
+  surname: De Maeyer
   orcid: 0000-0002-1314-3348
   affiliation: '4'
-- name: Toni Verbeiren
+- given-names: Toni
+  surname: Verbeiren
   email: toni@data-intuitive.com
   orcid: 0000-0002-7007-6866
   corresponding: yes

src/main/resources/io/viash/runners/nextflow/arguments/_checkArgumentType.nf

@@ -66,64 +66,56 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
       foundClass = "List[${e.foundClass}]"
     }
   } else if (par.type == "string") {
-    // cast to string if need be
+    // cast to string if need be. only cast if the value is a GString
     if (value instanceof GString) {
-      value = value.toString()
+      value = value as String
     }
     expectedClass = value instanceof String ? null : "String"
   } else if (par.type == "integer") {
     // cast to integer if need be
-    if (value instanceof String) {
+    if (value !instanceof Integer) {
       try {
-        value = value.toInteger()
+        value = value as Integer
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Integer"
       }
     }
-    if (value instanceof java.math.BigInteger) {
-      value = value.intValue()
-    }
-    expectedClass = value instanceof Integer ? null : "Integer"
   } else if (par.type == "long") {
     // cast to long if need be
-    if (value instanceof String) {
+    if (value !instanceof Long) {
       try {
-        value = value.toLong()
+        value = value as Long
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Long"
       }
     }
-    if (value instanceof Integer) {
-      value = value.toLong()
-    }
-    expectedClass = value instanceof Long ? null : "Long"
   } else if (par.type == "double") {
     // cast to double if need be
-    if (value instanceof String) {
+    if (value !instanceof Double) {
       try {
-        value = value.toDouble()
+        value = value as Double
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Double"
       }
     }
-    if (value instanceof java.math.BigDecimal) {
-      value = value.doubleValue()
-    }
-    if (value instanceof Float) {
-      value = value.toDouble()
+  } else if (par.type == "float") {
+    // cast to float if need be
+    if (value !instanceof Float) {
+      try {
+        value = value as Float
+      } catch (NumberFormatException e) {
+        expectedClass = "Float"
+      }
     }
-    expectedClass = value instanceof Double ? null : "Double"
   } else if (par.type == "boolean" | par.type == "boolean_true" | par.type == "boolean_false") {
     // cast to boolean if need be
-    if (value instanceof String) {
-      def valueLower = value.toLowerCase()
-      if (valueLower == "true") {
-        value = true
-      } else if (valueLower == "false") {
-        value = false
+    if (value !instanceof Boolean) {
+      try {
+        value = value as Boolean
+      } catch (Exception e) {
+        expectedClass = "Boolean"
       }
     }
-    expectedClass = value instanceof Boolean ? null : "Boolean"
   } else if (par.type == "file" && (par.direction == "input" || stage == "output")) {
     // cast to path if need be
     if (value instanceof String) {
@@ -135,10 +127,13 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
     expectedClass = value instanceof Path ? null : "Path"
   } else if (par.type == "file" && stage == "input" && par.direction == "output") {
     // cast to string if need be
-    if (value instanceof GString) {
-      value = value.toString()
+    if (value !instanceof String) {
+      try {
+        value = value as String
+      } catch (Exception e) {
+        expectedClass = "String"
+      }
     }
-    expectedClass = value instanceof String ? null : "String"
   } else {
     // didn't find a match for par.type
     expectedClass = par.type

…rs/nextflow/channel/channelFromParams.nf …s/nextflow/channel/_channelFromParams.nfsrc/main/resources/io/viash/runners/nextflow/channel/channelFromParams.nf renamed to src/main/resources/io/viash/runners/nextflow/channel/_channelFromParams.nf src/main/resources/io/viash/runners/nextflow/channel/channelFromParams.nf renamed to src/main/resources/io/viash/runners/nextflow/channel/_channelFromParams.nf

@@ -100,27 +100,3 @@ private _channelFromParams(Map params, Map config) {
   def processedParams = _paramsToParamSets(params, config)
   return Channel.fromList(processedParams)
 }
-
-/**
- * Parse nextflow parameters based on settings defined in a viash config 
- * and return a nextflow channel.
- * 
- * @param params Input parameters. Can optionaly contain a 'param_list' key that
- *               provides a list of arguments that can be split up into multiple events
- *               in the output channel possible formats of param_lists are: a csv file, 
- *               json file, a yaml file or a yaml blob. Each parameters set (event) must
- *               have a unique ID.
- * @param config A Map of the Viash configuration. This Map can be generated from the config file
- *               using the readConfig() function.
- * 
- * @return A nextflow Channel with events. Events are formatted as a tuple that contains 
- *         first contains the ID of the event and as second element holds a parameter map.
- *       
- *
- */
-def channelFromParams(Map params, Map config) {
-  log.warn "channelFromParams is deprecated and will be removed in Viash 0.10.0. ",
-    "Nextflow workflows can now be built into standalone components where parsed parameters ",
-    "are automatically provided to the input channel."
-  return _channelFromParams(params, config) 
-}

src/main/scala/io/viash/Main.scala

@@ -198,36 +198,6 @@ object Main extends Logging {
         }
       case _ => 
     }
-
-    // backwards compability for --platform
-    cli.subcommands.lastOption match {
-      case Some(x: ViashCommand) => 
-        if (x.platform.isDefined) {
-          if (x.runner.isDefined || x.engine.isDefined) {
-            throw new IllegalArgumentException("Error: --platform cannot be used together with --runner or --engine.")
-          }
-          warn("Warning: --platform is deprecated in Viash 0.9.0, will be removed in Viash 0.10.0. Use --runner or --engine instead.")
-        }
-      case Some(x: ViashNs) =>
-        if (x.platform.isDefined) {
-          if (x.runner.isDefined || x.engine.isDefined) {
-            throw new IllegalArgumentException("Error: --platform cannot be used together with --runner or --engine.")
-          }
-          warn("Warning: --platform is deprecated in Viash 0.9.0, will be removed in Viash 0.10.0. Use --runner or --engine instead.")
-        }
-      case _ => 
-    }
-    // backwards compability for --apply_platform
-    cli.subcommands match {
-      case List(cli.namespace, cli.namespace.exec) =>
-        if (cli.namespace.exec.applyPlatform()) {
-          if (cli.namespace.exec.applyRunner() || cli.namespace.exec.applyEngine()) {
-            throw new IllegalArgumentException("Error: --platform cannot be used together with --runner or --engine.")
-          }
-          warn("Warning: --apply_platform is deprecated in Viash 0.9.0, will be removed in Viash 0.10.0n. Use --apply_runner or --apply_engine instead.")
-        }
-      case _ =>
-    }
 
     // see if there are package overrides passed to the viash command
     val packSrc = cli.subcommands.lastOption match {
@@ -337,8 +307,8 @@ object Main extends Logging {
           cli.namespace.list,
           packageConfig = pack1,
           addOptMainScript = false, 
-          applyRunner = cli.namespace.list.runner.isDefined || cli.namespace.list.platform.isDefined,
-          applyEngine = cli.namespace.list.engine.isDefined || cli.namespace.list.platform.isDefined
+          applyRunner = cli.namespace.list.runner.isDefined,
+          applyEngine = cli.namespace.list.engine.isDefined
         )
         val configs2 = namespaceDependencies(configs, None, pack1.rootDir)
         ViashNamespace.list(
@@ -351,8 +321,8 @@ object Main extends Logging {
         val configs = readConfigs(
           cli.namespace.exec, 
           packageConfig = pack1, 
-          applyRunner = cli.namespace.exec.applyRunner() || cli.namespace.exec.applyPlatform(),
-          applyEngine = cli.namespace.exec.applyEngine() || cli.namespace.exec.applyPlatform()
+          applyRunner = cli.namespace.exec.applyRunner(),
+          applyEngine = cli.namespace.exec.applyEngine()
         )
         ViashNamespace.exec(
           configs = configs,
@@ -371,7 +341,7 @@ object Main extends Logging {
           cli.config.view,
           packageConfig = pack1,
           addOptMainScript = false,
-          applyRunnerAndEngine = cli.config.view.platform.isDefined || cli.config.view.runner.isDefined || cli.config.view.engine.isDefined
+          applyRunnerAndEngine = cli.config.view.runner.isDefined || cli.config.view.engine.isDefined
         )
         val config2 = DependencyResolver.modifyConfig(config.config, None, pack1.rootDir)
         ViashConfig.view(
@@ -480,11 +450,8 @@ object Main extends Logging {
       viashPackage = Some(packageConfig)
     )
     if (applyRunnerAndEngine) {
-      val runnerStr = subcommand.runner.toOption orElse subcommand.platform.toOption
-      val engineStr = subcommand.engine.toOption orElse subcommand.platform.toOption
-      
-      val runner = config.findRunner(runnerStr)
-      val engines = config.findEngines(engineStr)
+      val runner = config.findRunner(subcommand.runner.toOption)
+      val engines = config.findEngines(subcommand.engine.toOption)
 
       processConfigWithRunnerAndEngine(
         appliedConfig = config,
@@ -509,8 +476,8 @@ object Main extends Logging {
     val queryNamespace = subcommand.query_namespace.toOption
     val queryName = subcommand.query_name.toOption
     val queryConfig = subcommand.query_config.toOption
-    val runnerStr = subcommand.runner.toOption orElse subcommand.platform.toOption
-    val engineStr = subcommand.engine.toOption orElse subcommand.platform.toOption
+    val runnerStr = subcommand.runner.toOption
+    val engineStr = subcommand.engine.toOption
     val configMods = packageConfig.config_mods
 
     val configs0 = Config.readConfigs(