Merge pull request #47 from appraisal-rb/parallelize-appraisal-commands

Parallelize remaining appraisal commands

Author: pboling

CHANGELOG.md

@@ -20,8 +20,11 @@ Please file a bug if you notice a violation of semantic versioning.
 
 ### Added
 
-- `appraisal generate` can now generate appraisal Gemfiles in parallel with
-  `--jobs` / `-j`, or `APPRAISAL_JOBS`.
+- `appraisal generate`, `install`, `update`, `generate-install`, and
+  `generate-update` can now process appraisals in parallel with
+  `--appraisal-jobs` / `-n`, or `APPRAISAL_JOBS`. Appraisal-level
+  parallelism defaults to 2 workers; use `-n 1` to run serially. Bundler older
+  than 2.1 falls back to serial processing.
 
 ### Changed
 
@@ -41,6 +44,17 @@ Please file a bug if you notice a violation of semantic versioning.
 
 ### Fixed
 
+- Appraisal command execution now checks whether the `bundle` executable can
+  boot before falling back to RubyGems spec probing and Bundler installation,
+  preserving TruffleRuby's engine-shipped Bundler in isolated subprocesses.
+- Acceptance fixture Bundler selection now detects TruffleRuby's shipped
+  Bundler without reusing a newer Bundler activated by the appraised suite.
+- Acceptance fixture binstubs now pin the selected Bundler version before
+  loading generated `bin/bundle` handoff code.
+- Bundler-backed appraisal installs now set `BUNDLE_JOBS` explicitly so the
+  configured Appraisal job count also controls Bundler's installer worker
+  count in subprocesses.
+
 ### Security
 
 ## [3.1.4] - 2026-07-01

Gemfile.lock

@@ -443,4 +443,4 @@ DEPENDENCIES
   yard-yaml (~> 0.2, >= 0.2.3)
 
 BUNDLED WITH
-  4.0.14
+  4.0.16

README.md

@@ -259,10 +259,20 @@ Use the command that matches the lifecycle you want:
 | `generate-install` | Yes | Yes, via install | First setup, or after intentional Appraisals changes |
 | `generate-update` | Yes | Yes, via update | Regenerate gemfiles, then update dependency locks |
 
-`generate`, `generate-install`, and `generate-update` can generate appraisal
-gemfiles in parallel with `--jobs` / `-j`, or with `APPRAISAL_JOBS` when no
-CLI value is provided. `generate-install` and `generate-update` parallelize the
-generation phase, then resolve dependencies with the selected gem manager.
+Appraisal2 can process appraisals in parallel with `--appraisal-jobs` / `-n`,
+or with `APPRAISAL_JOBS` when no CLI value is provided. `generate`,
+`generate-install`, and `generate-update` use this for generation. `install`,
+`update`, `generate-install`, and `generate-update` use it to fan out dependency
+resolution across appraisals. External commands run across all appraisals, such
+as `bundle exec appraisal rake test`, use `APPRAISAL_JOBS`. Appraisal-level
+parallelism defaults to 2 workers; use `-n 1` to run serially. When Appraisal2
+is running under Bundler older than 2.1, Appraisal2 falls back to serial
+processing because those Bundler versions do not provide the modern environment
+helpers used to isolate parallel Bundler subprocesses.
+
+Install commands keep `--jobs` / `-j` for the selected gem manager. With
+Bundler, `-j` controls `bundle install --jobs`, while `-n` controls how many
+appraisal gemfiles Appraisal2 processes at the same time.
 
 The deprecated rake task `rake appraisal:install` now delegates to `appraisal generate-install`, preserving its historical generate-and-install behavior while the CLI commands remain explicit.
 
@@ -275,12 +285,27 @@ Built-in dependency commands support the following options:
 | Option | Description |
 |--------|-------------|
 | `--gem-manager`, `-g` | Gem manager to use: `bundler` (default) or `ore`; applies to `install`, `update`, `generate-install`, and `generate-update` |
-| `--jobs`, `-j` | Generate appraisal gemfiles in parallel for `generate`, `generate-install`, and `generate-update`; also installs gems in parallel for `install` and `generate-install` when supported by the selected gem manager |
+| `--appraisal-jobs`, `-n` | Process appraisals in parallel for `generate`, `install`, `update`, `generate-install`, and `generate-update` (default: 2; use `-n 1` for serial execution; Bundler < 2.1 falls back to serial processing) |
+| `--jobs`, `-j` | Pass a parallel job count to the selected gem manager; applies to `install` and `generate-install` |
 | `--retry` | Retry network and git requests that have failed; applies to `install` and `generate-install` (default: 1) |
 | `--without` | A space-separated list of groups to skip during installation; applies to `install` and `generate-install` |
 | `--full-index` | Run bundle install with the full-index argument; applies to `install` and `generate-install` |
 | `--path` | Install gems in the specified directory; applies to `install` and `generate-install` |
 
+```bash
+# Process two appraisals at a time, using the default Appraisal worker count
+bundle exec appraisal generate-install
+
+# Process four appraisals at a time
+bundle exec appraisal generate-install -n 4
+
+# Process two appraisals at a time, and give each Bundler install four jobs
+bundle exec appraisal generate-install -n 2 -j 4
+
+# Run Appraisal serially when parallel appraisal processing is not desired
+bundle exec appraisal generate-install -n 1
+```
+
 ### Using Commands with Named Appraisals
 
 #### Using Appraisal's built-in commands with named appraisals

lib/appraisal/appraisal.rb

@@ -94,7 +94,7 @@ def update(gems = [], options = {})
     end
 
     def gemfile_path
-      gemfile_root.mkdir unless gemfile_root.exist?
+      FileUtils.mkdir_p(gemfile_root)
 
       gemfile_root.join(gemfile_name).to_s
     end

lib/appraisal/cli.rb

@@ -6,14 +6,23 @@
 
 module Appraisal
   class CLI < Thor
+    MINIMUM_PARALLEL_BUNDLER_VERSION = Gem::Version.new("2.1.0")
+
     default_task :install
     map ["-v", "--version"] => "version"
+    map "generate-install" => "generate_install"
+    map "generate-update" => "generate_update"
 
     class_option "gem-manager",
       :aliases => "-g",
       :type => :string,
       :default => "bundler",
       :desc => "Gem manager to use for install/update (bundler or ore)"
+    class_option "appraisal-jobs",
+      :aliases => "-n",
+      :type => :numeric,
+      :banner => "SIZE",
+      :desc => "Process appraisals in parallel using the given number of workers."
 
     class << self
       # Override help command to print out usage
@@ -60,7 +69,7 @@ def strip_heredoc(string)
       :type => :numeric,
       :default => 1,
       :banner => "SIZE",
-      :desc => "Install gems in parallel using the given number of workers."
+      :desc => "Pass the given parallel job count to the selected gem manager."
     method_option "retry",
       :type => :numeric,
       :default => 1,
@@ -79,7 +88,7 @@ def strip_heredoc(string)
         "Bundler will remember this option."
 
     def install
-      install_appraisals(appraisals, install_options)
+      install_appraisals(appraisals, install_options, appraisal_jobs(options))
     end
 
     desc "generate-install", "Generate gemfiles, then resolve and install dependencies for each appraisal"
@@ -88,7 +97,7 @@ def install
       :type => :numeric,
       :default => 1,
       :banner => "SIZE",
-      :desc => "Install gems in parallel using the given number of workers."
+      :desc => "Pass the given parallel job count to the selected gem manager."
     method_option "retry",
       :type => :numeric,
       :default => 1,
@@ -107,18 +116,13 @@ def install
         "Bundler will remember this option."
     def generate_install
       appraisal_list = appraisals
-      generate_appraisals(appraisal_list, generate_jobs(options))
-      install_appraisals(appraisal_list, install_options)
+      generate_appraisals(appraisal_list, appraisal_jobs(options))
+      install_appraisals(appraisal_list, install_options, appraisal_jobs(options))
     end
 
     desc "generate", "Generate a gemfile for each appraisal"
-    method_option "jobs",
-      :aliases => "j",
-      :type => :numeric,
-      :banner => "SIZE",
-      :desc => "Generate appraisal gemfiles in parallel using the given number of workers."
     def generate
-      generate_appraisals(appraisals, generate_jobs(options))
+      generate_appraisals(appraisals, appraisal_jobs(options))
     end
 
     desc "clean", "Remove all generated gemfiles and lockfiles from gemfiles folder"
@@ -128,19 +132,14 @@ def clean
 
     desc "update [LIST_OF_GEMS]", "Update dependencies for each generated appraisal gemfile"
     def update(*gems)
-      update_appraisals(appraisals, gems, update_options)
+      update_appraisals(appraisals, gems, update_options, appraisal_jobs(options))
     end
 
     desc "generate-update [LIST_OF_GEMS]", "Generate gemfiles, then update dependencies for each appraisal"
-    method_option "jobs",
-      :aliases => "j",
-      :type => :numeric,
-      :banner => "SIZE",
-      :desc => "Generate appraisal gemfiles in parallel using the given number of workers."
     def generate_update(*gems)
       appraisal_list = appraisals
-      generate_appraisals(appraisal_list, generate_jobs(options))
-      update_appraisals(appraisal_list, gems, update_options)
+      generate_appraisals(appraisal_list, appraisal_jobs(options))
+      update_appraisals(appraisal_list, gems, update_options, appraisal_jobs(options))
     end
 
     desc "list", "List the names of the defined appraisals"
@@ -165,15 +164,15 @@ def generate_appraisals(appraisal_list, jobs)
       end
     end
 
-    def install_appraisals(appraisal_list, install_options)
-      appraisal_list.each do |appraisal|
+    def install_appraisals(appraisal_list, install_options, jobs)
+      each_appraisal(appraisal_list, jobs) do |appraisal|
         appraisal.install(install_options.dup)
         appraisal.relativize
       end
     end
 
-    def update_appraisals(appraisal_list, gems, update_options)
-      appraisal_list.each do |appraisal|
+    def update_appraisals(appraisal_list, gems, update_options, jobs)
+      each_appraisal(appraisal_list, jobs) do |appraisal|
         appraisal.update(gems, update_options.dup)
       end
     end
@@ -210,17 +209,29 @@ def each_appraisal(appraisal_list, jobs)
     def worker_count(jobs, item_count)
       jobs = jobs.to_i
       jobs = 1 if jobs < 1
+      jobs = 1 unless parallel_appraisals_supported?
       return jobs if item_count.zero?
 
       [jobs, item_count].min
     end
 
-    def generate_jobs(command_options)
-      command_options["jobs"] || command_options[:jobs] || ENV["APPRAISAL_JOBS"] || 1
+    def parallel_appraisals_supported?
+      # Parallel install/update workers depend on Appraisal::Command being able
+      # to build independent Bundler subprocess environments. Bundler 2.1 is the
+      # floor for the modern original_env/unbundled_env API split that replaced
+      # clean_env, so older Bundler versions keep the historical serial path.
+      bundler_version = Gem::Specification.find_all_by_name("bundler").map(&:version).max
+      return false unless bundler_version
+
+      bundler_version >= MINIMUM_PARALLEL_BUNDLER_VERSION
+    end
+
+    def appraisal_jobs(command_options)
+      command_options["appraisal-jobs"] || command_options[:appraisal_jobs] || ENV["APPRAISAL_JOBS"] || 2
     end
 
     def install_options
-      options.to_h
+      options.to_h.reject { |key, _value| key.to_s == "appraisal-jobs" }
     end
 
     def update_options
@@ -229,6 +240,13 @@ def update_options
     end
 
     def method_missing(name, *args)
+      case name.to_s
+      when "generate-install"
+        return generate_install
+      when "generate-update"
+        return generate_update(*args)
+      end
+
       matching_appraisal = AppraisalFile.new.appraisals.detect do |appraisal|
         appraisal.name == name.to_s
       end
@@ -289,7 +307,7 @@ def method_missing(name, *args)
           Command.new(actual_args, :gemfile => matching_appraisal.gemfile_path).run
         end
       else
-        AppraisalFile.each do |appraisal|
+        each_appraisal(AppraisalFile.new.appraisals, appraisal_jobs(options)) do |appraisal|
           Command.new(ARGV, :gemfile => appraisal.gemfile_path).run
         end
       end
@@ -316,6 +334,17 @@ def parse_external_options(args)
           options[:jobs] = Regexp.last_match(1).to_i
         when /^-j(\d+)$/
           options[:jobs] = Regexp.last_match(1).to_i
+        when /^-j$/
+          options[:jobs] = args[index + 1].to_i
+          skip_next = true
+        when /^--appraisal-jobs=\d+$/
+          nil
+        when /^--appraisal-jobs$/
+          skip_next = true
+        when /^-n\d+$/
+          nil
+        when /^-n$/
+          skip_next = true
         when /^--retry=(\d+)$/
           options[:retry] = Regexp.last_match(1).to_i
         when /^--without=(.+)$/
@@ -347,6 +376,10 @@ def extract_gems_and_options(args)
           # Next arg should be the value
           options[:gem_manager] = args[index + 1]
           skip_next = true
+        when /^--appraisal-jobs$/
+          skip_next = true
+        when /^-n$/
+          skip_next = true
         when /^-/
           # Other options are not gems, but we don't handle them all here
           # For now, just skip them to be safe if they start with -