Feature proposal: Summary of which Gems were updated by a `bundle update`

[zofrex]

Summary

Feature proposal: Additional output from bundle update that provides a useful summary of which gems were updated, categorised between major and non-major, and top level and implicit dependencies.

Describe the problem as clearly as you can

When you run bundle update it isn't very obvious from the output which gems updated, because the update messages are mixed in with the install messages:

Using xpath 3.2.0
Using spring-watcher-listen 2.0.1
Using rails-dom-testing 2.0.3
Using loofah 2.18.0
Using activerecord 7.0.3.1 (was 6.1.6)
Using rails-html-sanitizer 1.4.3
Using capybara 3.37.1
Using webdrivers 5.0.0

Scrolling a long list to try to see all the updated ones can be quite tiresome, and doesn't really aid the user in what tasks they are likely trying to do. Possible tasks they may be trying to accomplish:

• See which gems were updated
• Identify which top-level gems were updated (as they may care about these a lot more than implicit dependencies)
• Identify which gems updated past a major version, as this may indicate backwards-compatibility-breaking changes

The other approach is to diff the Gemfile.lock, but that has a lot of repetition:

-    actionmailbox (6.1.6)
-      actionpack (= 6.1.6)
+    actionmailbox (7.0.3.1)
+      actionpack (= 7.0.3.1)
-    actionmailer (6.1.6)
-      actionpack (= 6.1.6)
+    actionmailer (7.0.3.1)
+      actionpack (= 7.0.3.1)

And also doesn't help with extracting higher-level information out of the data such as whether gems are top-level, or highlighting major version bumps.

I think from a flexibility and tooling perspective, a machine-readable output from bundle install will be the most useful thing. But I think for the 90% use-case where users are doing a bundle update themselves, directly on a terminal, a lot of value could be delivered by the relatively simpler feature of adding a human-readable summary to the output.

I think the highest value is to highlight just which gems changed, but I think it's also extremely valuable to separately highlight major version bumps. I think it may also be useful to distinguish between top-level and implicit dependencies.

Here is the output from a PoC I have made:

Bundle updated!
Gems in the group 'production' were not updated.
Major bumps
###########

  Primary dependencies
  ====================
    activerecord-session_store: 1.1.3 => 2.0.0
    i18n-tasks: 0.9.37 => 1.0.11
    puma: 4.3.12 => 5.6.5
    rails: 6.1.6 => 7.0.3.1
    rails-i18n: 6.0.0 => 7.0.5

  Implicit dependencies
  =====================
    actioncable: 6.1.6 => 7.0.3.1
    actionmailbox: 6.1.6 => 7.0.3.1
    actionmailer: 6.1.6 => 7.0.3.1
    actionpack: 6.1.6 => 7.0.3.1
    actiontext: 6.1.6 => 7.0.3.1
    actionview: 6.1.6 => 7.0.3.1
    activejob: 6.1.6 => 7.0.3.1
    activemodel: 6.1.6 => 7.0.3.1
    activerecord: 6.1.6 => 7.0.3.1
    activestorage: 6.1.6 => 7.0.3.1
    activesupport: 6.1.6 => 7.0.3.1
    public_suffix: 4.0.7 => 5.0.0
    rack-test: 1.1.0 => 2.0.2
    railties: 6.1.6 => 7.0.3.1

Minor bumps
###########

  Primary dependencies
  ====================
    bootsnap: 1.12.0 => 1.13.0
    newrelic_rpm: 8.8.0 => 8.10.0
    pg: 1.4.0 => 1.4.3
    selenium-webdriver: 4.2.1 => 4.4.0
    shakapacker: 6.4.1 => 6.5.1
    warning: 1.2.1 => 1.3.0

  Implicit dependencies
  =====================
    addressable: 2.8.0 => 2.8.1
    erubi: 1.10.0 => 1.11.0
    i18n: 1.10.0 => 1.12.0
    minitest: 5.16.1 => 5.16.3
    msgpack: 1.5.2 => 1.5.6
    nokogiri: 1.13.6 => 1.13.8
    parser: 3.1.2.0 => 3.1.2.1
    rack: 2.2.3.1 => 2.2.4
    rb-fsevent: 0.11.1 => 0.11.2
    sprockets: 4.0.3 => 4.1.1
    tilt: 2.0.10 => 2.0.11
    tzinfo: 2.0.4 => 2.0.5
    unicode-display_width: 2.1.0 => 2.2.0

Removed
#######
  mini_portile2: 2.8.0 => ______

Open questions

• How to format the output
• Which way it should be sliced
  • is top level / implicit a useful division?
  • Should it be major/minor first, or primary/implicit?
  • Does the major/minor x primary/implicit combined grouping make sense, or should it be two separate splits of all updated gems by those dimensions?
• Is this the right nomenclature? (primary & implicit in particular)
• Should the feature be added to bundle update straight-up, or behind a new option flag?

[deivid-rodriguez]

In bundle outdated we have a table output, maybe we could use some inspiration from there. An option would be to add the proper columns for dependency type & update type, and sort the table according to those, so we don't need "explicit slicing".

| Gem                        | Previous   | New     | Type     | Bump    |
| -----                      | ---------- | ------  | ------   | ------- |
| activerecord-session_store | 1.1.3      | 2.0.0   | Direct   | Major   |
| i18n-tasks                 | 0.9.37     | 1.0.11  | Direct   | Major   |
| puma                       | 4.3.12     | 5.6.5   | Direct   | Major   |
| rails                      | 6.1.6      | 7.0.3.1 | Direct   | Major   |
| rails-i18n                 | 6.0.0      | 7.0.5   | Direct   | Major   |
| actioncable                | 6.1.6      | 7.0.3.1 | Indirect | Major   |
| actionmailbox              | 6.1.6      | 7.0.3.1 | Indirect | Major   |
| actionmailer               | 6.1.6      | 7.0.3.1 | Indirect | Major   |
| actionpack                 | 6.1.6      | 7.0.3.1 | Indirect | Major   |
| actiontext                 | 6.1.6      | 7.0.3.1 | Indirect | Major   |
| actionview                 | 6.1.6      | 7.0.3.1 | Indirect | Major   |
| activejob                  | 6.1.6      | 7.0.3.1 | Indirect | Major   |
| activemodel                | 6.1.6      | 7.0.3.1 | Indirect | Major   |
| activerecord               | 6.1.6      | 7.0.3.1 | Indirect | Major   |
| activestorage              | 6.1.6      | 7.0.3.1 | Indirect | Major   |
| activesupport              | 6.1.6      | 7.0.3.1 | Indirect | Major   |
| public_suffix              | 4.0.7      | 5.0.0   | Indirect | Major   |
| rack-test                  | 1.1.0      | 2.0.2   | Indirect | Major   |
| railties                   | 6.1.6      | 7.0.3.1 | Indirect | Major   |
| bootsnap                   | 1.12.0     | 1.13.0  | Direct   | Minor   |
| newrelic_rpm               | 8.8.0      | 8.10.0  | Direct   | Minor   |
| pg                         | 1.4.0      | 1.4.3   | Direct   | Minor   |
| selenium-webdriver         | 4.2.1      | 4.4.0   | Direct   | Minor   |
| shakapacker                | 6.4.1      | 6.5.1   | Direct   | Minor   |
| warning                    | 1.2.1      | 1.3.0   | Direct   | Minor   |
| addressable                | 2.8.0      | 2.8.1   | Indirect | Minor   |
| erubi                      | 1.10.0     | 1.11.0  | Indirect | Minor   |
| i18n                       | 1.10.0     | 1.12.0  | Indirect | Minor   |
| minitest                   | 5.16.1     | 5.16.3  | Indirect | Minor   |
| msgpack                    | 1.5.2      | 1.5.6   | Indirect | Minor   |
| nokogiri                   | 1.13.6     | 1.13.8  | Indirect | Minor   |
| parser                     | 3.1.2.0    | 3.1.2.1 | Indirect | Minor   |
| rack                       | 2.2.3.1    | 2.2.4   | Indirect | Minor   |
| rb-fsevent                 | 0.11.1     | 0.11.2  | Indirect | Minor   |
| sprockets                  | 4.0.3      | 4.1.1   | Indirect | Minor   |
| tilt                       | 2.0.10     | 2.0.11  | Indirect | Minor   |
| tzinfo                     | 2.0.4      | 2.0.5   | Indirect | Minor   |
| unicode-display_width      | 2.1.0      | 2.2.0   | Indirect | Minor   |
| mini_portile2              | 2.8.0      | REMOVED | -        | -       |

Regarding primary/implicit terminology, we're definitely not consistent here, so I guess this is an oportunity to unify terminology. Personally I think "direct"/"indirect" is the most clear, but I many times use "transitive" for "indirect/implicit" dependencies. I never use "primary" for direct dependencies, though.

Should the feature be added to bundle update straight-up, or behind a new option flag?

I would add it straight up!

[zofrex]

In bundle outdated we have a table output, maybe we could use some inspiration from there.

Good idea, I think it makes a lot of sense for these two commands to match. outdated produces a table that looks like this, for the record:

Gem                         Current  Latest  Requested  Groups
actioncable                 6.1.6    7.0.4
actionmailbox               6.1.6    7.0.4
activerecord-session_store  1.1.3    2.0.0   >= 0       default
i18n-tasks                  0.9.37   1.0.11  >= 0       development, test

I think we should probably match the formatting between the two commands, so we'd have a table without borders for update as well. Something like:

Gem                        Previous   New     Type     Bump
activerecord-session_store 1.1.3      2.0.0   Direct   Major
i18n-tasks                 0.9.37     1.0.11  Direct   Major
actioncable                6.1.6      7.0.3.1 Indirect Major
actionmailbox              6.1.6      7.0.3.1 Indirect Major
bootsnap                   1.12.0     1.13.0  Direct   Minor
newrelic_rpm               8.8.0      8.10.0  Direct   Minor
addressable                2.8.0      2.8.1   Indirect Minor
erubi                      1.10.0     1.11.0  Indirect Minor
mini_portile2              2.8.0      REMOVED -        -

The outdated command has options to filter by minor/major, but I don't think that filtering the output makes quite as much sense for the update command?

Perhaps there could be an option to change the sorting but I think this is pretty good as-is... unless someone strongly feels like it should be filterable or sortable, I think I will leave it like this, those options could always be added later if people need it.

Regarding primary/implicit terminology, we're definitely not consistent here, so I guess this is an oportunity to unify terminology. Personally I think "direct"/"indirect" is the most clear, but I many times use "transitive" for "indirect/implicit" dependencies. I never use "primary" for direct dependencies, though.

I noticed that the outdated command calls direct dependencies "explicit" dependencies:

       --only-explicit
              Only list gems specified in your Gemfile, not their dependencies.

I agree it would be good to unify the terminology across Bundler but I think that's a much broader scope than this feature. For now I propose using "explicit" and "implicit" here (which matches outdated and the gemfile(5) manual page), and we can revise later if different terms are picked for unifying terminology.

One other thing: I think this summary output should also be displayed by bundle lock, which is essentially doing the same thing as update, at least as far as gem versions goes.