Upgrade Ruby (3.1.4 -> 3.3.8) & Rails (7.1.3 -> 7.2.2.1) #749

fosterfarrell9 · 2025-02-22T20:22:38Z

Note: This PR has a "best before" date: 2025-03-31, see here. The respective Ruby on Rails upgrade guide is here.

This PR upgrades Ruby to 3.3.7 and Rails to 7.2.2.1 (and in particular resolves #611 and goes most of the way for #709), and some gems that needed an upgrade in order to make the Rails upgrade work. So far, it seems to work mostly nicely. Several deprecation warnings were shown; I fixed them all except one (see below),
and some unit tests are failing (see below).

Important

For your local setup to still work after this PR, perform the steps outlined here in the wiki.
Furthermore, since we are changing the Docker base image (FROM), you have to rebuild all MaMpf Docker Containers from scratch.

Notes for the reviewer:

I let the app:update script overwrite our puma.rb file which seemed quite out of date compared to the current standard one. One notable difference is the handling of the pid file. Before, it was

ENV.fetch("PIDFILE") { "tmp/pids/server.pid" }

Now, it is

# Specify the PID file. Defaults to tmp/pids/server.pid in development.
# In other environments, only set the PID file if requested.
pidfile ENV["PIDFILE"] if ENV["PIDFILE"]

On partiulcar, if we need the PID file in production, we should set an environment variable. I am not sure: We have

echo "Ensuring no stale server pid file is present"
rm -f tmp/pids/server.pid

in production/entrypoint_worker.sh.

We still have files related to a "development" environment (and if you search in our codebase, you will find several occurences of "development" referring to a "development" environment. Do we need that? All our development takes place in docker_development .
Rails 7.2 standard production.rb ccontains the line config.force_ssl = true. I added it, but put it in a comment. We should find out what happens if we uncomment it.
It appears that the standard in Rails 7.2 is that in a production console you can onbly see the id of database entries, nothing more. I overrode this in production.rb via

config.active_record.attributes_for_inspect = :all

In docker_development.rb I added the following lines wich come form the current Rails default devlopment file:

  # Raise error when a before_action's only/except options reference missing actions.
  config.action_controller.raise_on_missing_callback_actions = true

  # Apply autocorrection by RuboCop to files generated by `bin/rails generate`.
  # config.generators.apply_rubocop_autocorrect_after_generate!

I think the first one makes sense, and the second one should remain a comment since we have our own rubocop thing going on.

We still get this warning

DEPRECATION WARNING [legacy-js-api]: The legacy JS API is deprecated and will be removed in Dart Sass 2.0.0.
More info: https://sass-lang.com/d/legacy-js-api

See here. It appears we could get rid of it if we upgrade sass-loader to v16 in our yarn modules, but do we want that? It seems that vite offers a better solution, see here.

I am a little confused about the failing unit tests. The error messages say

Failure/Error:
  expect do
    UserCleaner.new.send_additional_warning_mails
  end.not_to have_enqueued_mail(UserCleanerMailer, :pending_deletion_email)

StandardError:
  To use HaveEnqueuedMail matcher set `ActiveJob::Base.queue_adapter = :test`

so this seems to refer to this. But in config/environments/test.rb we already set

config.active_job.queue_adapter = :test

Update: I guess the problem is in config/initializers/active_job.rb where we set (for all environments)

Rails.application.config.active_job.queue_adapter = :sidekiq

which probably overrides the :test setting. Do we have tests (unit or cypress) that rely on sidekiq? If not, we could just do a conditional here.

I checked the things mentioned in Upgrade to Rails 7.2 #611 and did not see any more deprecation warnings. Concerning the active_support.cache_format_version: I think we should be able to safely change this in production now (which happens automatically in this PR since we use the Rails 7.2 framework defaults in application.rb).

…ix deprecation warning

Gemfile

docker/development/Dockerfile

docker/production/Dockerfile

bin/rubocop

Splines · 2025-05-02T16:03:58Z

config/puma.rb

+# Any libraries that use a connection pool or another resource pool should
+# be configured to provide at least as many connections as the number of
+# threads. This includes Active Record's `pool` parameter in `database.yml`.
+threads_count = ENV.fetch("RAILS_MAX_THREADS", 3)


In the database.yml we still use a value of 5 instead of 3 here (in case that the env variable is not set, but we never set it anywhere). See also my comment on the respective Rails issue.

Also see this new guide: Tuning Performance for Deployment

I guess for now let's just keep our value, and if we get a reply we can still adjust the value later on.

Splines · 2025-05-02T16:19:30Z

config/puma.rb

+# Specify the PID file. Defaults to tmp/pids/server.pid in development.
+# In other environments, only set the PID file if requested.
+pidfile ENV["PIDFILE"] if ENV["PIDFILE"]


With regards to the pidfile and your note, see also this issue.

I don't think we need such a file in production at all. In docker/production/entrypoint_worker.sh and docker/entrypoint-dev-test.sh, we manually remove the file.

While with Rails 7.2, this file should no longer be generated (since we don't request it; we never set the PIDFILE env variable), let's leave the rm -f tmp/pids/server.pid statement in the entrypoint to be sure. Anyways, in the dev/test setup we cannot remove that line since in the dev environment, a PID file will always be generated, see this comment.

So this should be fine, feel free to resolve.

…ad of ruby 3.3.7)

Splines · 2025-05-02T17:53:51Z

config/environments/production.rb

+  # config.assume_ssl = true
+
+  # Force all access to the app over SSL, use Strict-Transport-Security, and use secure cookies.
+  # config.force_ssl = true


From the documentation:

Forces all requests to be served over HTTPS, and sets "https://" as the default protocol when generating URLs.

This kind of redirect already happens in apache via this config:

<IfModule mod_rewrite.c> RewriteEngine On RewriteCond %{HTTPS} off RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L] </IfModule>

so why not also use it directly in Rails such that all generated URLs automatically have https:// as default protocol.

Config option activated via commit 19d726b. Feel free to resolve.

Splines

Did you already look into Brakeman?

Brakeman is a great way to prevent common security vulnerabilities in Rails from going into production.

And jemalloc? See also this blog post.

~~Maybe something like Brakeman is something for another PR, but I feel like a small like adding jemalloc could still be incorporated here.~~

As those are external dependencies, I propose to introduce them in future PRs, should we want to include them.

Debian Bookworm (12) only ships postgresql-client-15 and no longer v13 [1]. Therefore, we update postgresl for the local setup. We can then update the postgresql version on the server at a later stage and first try to make sure that locally everything works fine. [1] https://packages.debian.org/bookworm/postgresql-client

Splines · 2025-05-02T19:45:18Z

config/initializers/active_job.rb

@@ -1 +1 @@
-Rails.application.config.active_job.queue_adapter = :sidekiq
+Rails.application.config.active_job.queue_adapter = :sidekiq unless Rails.env.test?


The fix you guessed was exactly what we needed to fix the failing unit tests ✨

See release 4.1.2 of spring where they drop dependency on mutex_m. Therefore, our problem is elegantly solved without the need for a new dependency. https://github.com/rails/spring/blob/main/CHANGELOG.md#412

Splines · 2025-05-02T23:09:51Z

app/models/voucher/voucher.rb

@@ -20,7 +20,7 @@ class Voucher < ApplicationRecord
  DEFAULT_EXPIRATION_DAYS = 3

  ROLE_HASH = { tutor: 0, editor: 1, teacher: 2, speaker: 3 }.freeze
-  enum role: ROLE_HASH
+  enum :role, ROLE_HASH


This violation should have been detected by RuboCop. Opened an issue for it.

This is really weird. I've created a new issue for it #768. For now, let's ignore this in this Rails upgrade that doesn't have to do with it and fix the RuboCop issue afterwards. Feel free to resolve this thread.

Splines · 2025-05-03T10:03:30Z

config/environments/production.rb

@@ -17,7 +17,7 @@
  # Ensures that a master key has been made available in ENV["RAILS_MASTER_KEY"],
  # config/master.key, or an environment key such as config/credentials/production.key.
  # This key is used to decrypt credentials (and other encrypted files).
-  config.require_master_key = true
+  # config.require_master_key = true


Why did you make this a comment? See also the config option.

Splines · 2025-05-03T10:16:02Z

app/models/interaction.rb

@@ -2,7 +2,6 @@ class Interaction < InteractionsRecord
  scope :created_between, lambda { |start_date, end_date|
                            where(created_at: start_date.beginning_of_day..end_date.end_of_day)
                          }
-  require "csv"


I'm not sure why you removed this line (also in other files)? This is not a RuboCop violation. Without this line, this class will throw an error if CSV is not magically loaded before loading this class and we don't want to risk that, right?

On another point: it'd be better to place this at the top of the file to be better visible since that's the usual place where we put the require statements.

"if CSV is not magically loaded before loading this class"

Turns out that's exactly what is done. But there's no magic: instead it is just the line Bundler.require(*Rails.groups) in config/application.rb which is responsible for this behavior. So indeed, removing these require lines as you did works just fine.

As CSV functionality is not used in many places, we might discuss if we instead want to do a gem "csv", "~> 3.3", require: false instead. Then, only require it at the respective places (at the very top of files that use the CSV class). Of course, one single gem doesn't cut startup times, but in total with many gems it might make a difference.

Splines

I've reviewed all changes now.

Splines · 2025-05-03T10:22:47Z

config/environments/test.rb

@@ -1,3 +1,5 @@
+require "active_support/core_ext/integer/time"


You added this require here. Based on this answer, I suspect it coming from new Rails templates themselves. However, you only included this here, while also development.rb and docker_development.rb contain a time-related statement: 2.days.to_i.

fosterfarrell9 added 14 commits February 22, 2025 17:08

Upgrade to Ruby 3.2.7

322c9bc

Update Gemfile.lock

34689b0

Upgrade to Ruby 3.3.7

a2b9c85

Upgrade thredded in order to prepare for Rails 7.2 upgrade

ba786b1

Upgrade Rails gem to Rails 7.2.2.1

15a68eb

Apply app:upgrade

855edea

Flip switches for Rais 7.2 migration

8f0ab9e

Enforce Rubocop rules

75a1a56

Add mutex_m in order to silence Ruby 3.4 warnings

7794930

Use Rails 7.2 defaults

811d874

Replace keyword argument syntax by positional arguments in order to f…

ca1fa1c

…ix deprecation warning

Update schemas to Rails 7.2

90195db

Add csv gem in order to silence deprecation warning

4382a8b

Remove explicit requiring of csv

b7b2433

fosterfarrell9 requested a review from Splines February 22, 2025 20:22

fosterfarrell9 had a problem deploying to testing-review February 22, 2025 20:22 — with GitHub Actions Failure

Add config changes from development to docker_development

77b68fc

fosterfarrell9 had a problem deploying to testing-review February 22, 2025 20:25 — with GitHub Actions Failure

Replace remaining old ruby versions

7f3437b

fosterfarrell9 had a problem deploying to testing-review February 22, 2025 20:30 — with GitHub Actions Failure

fosterfarrell9 changed the title ~~Upgrade/ruby and rails~~ Upgrade ruby and rails Feb 22, 2025

Enforce Rubocop rules

7e14a3c

fosterfarrell9 temporarily deployed to testing-review February 22, 2025 21:02 — with GitHub Actions Inactive

fosterfarrell9 had a problem deploying to testing February 22, 2025 21:08 — with GitHub Actions Failure

fosterfarrell9 temporarily deployed to testing February 22, 2025 21:08 — with GitHub Actions Inactive

Add missing require

cbf17ff

fosterfarrell9 had a problem deploying to testing-review February 22, 2025 22:55 — with GitHub Actions Failure

fosterfarrell9 mentioned this pull request Feb 23, 2025

Upgrade to Rack 3 #751

Open

fosterfarrell9 mentioned this pull request Mar 4, 2025

Remove food routes #754

Open

3 tasks

Splines assigned fosterfarrell9 May 2, 2025

Splines changed the title ~~Upgrade ruby and rails~~ Upgrade Ruby to 3.3.7 and Rails to 7.2.2.1 May 2, 2025

Merge branch 'dev' into upgrade/ruby_and_rails

b3e1181

Splines requested a deployment to testing-review May 2, 2025 15:03 — with GitHub Actions Waiting

Splines reviewed May 2, 2025

View reviewed changes

Gemfile Outdated Show resolved Hide resolved

Gemfile Outdated Show resolved Hide resolved

Splines requested changes May 2, 2025

View reviewed changes

Splines reviewed May 2, 2025

View reviewed changes

Splines added 2 commits May 2, 2025 18:44

Use Debian bookworm instead of bullseye & update to ruby 3.3.8 (inste…

e11da52

…ad of ruby 3.3.7)

Update to latest thredded version

61997f9

Splines requested a deployment to testing-review May 2, 2025 16:46 — with GitHub Actions Waiting

Splines changed the title ~~Upgrade Ruby to 3.3.7 and Rails to 7.2.2.1~~ Upgrade Ruby to 3.3.8 and Rails to 7.2.2.1 May 2, 2025

Force SSL in production

19d726b

Splines requested a deployment to testing-review May 2, 2025 17:54 — with GitHub Actions Waiting

Splines reviewed May 2, 2025

View reviewed changes

Splines added 2 commits May 2, 2025 21:17

Fix rubocop yml file path

7992114

Splines requested a deployment to testing-review May 2, 2025 19:19 — with GitHub Actions Waiting

Fix failing unit tests (don't set queue_adapter to sidekiq)

3d293ab

Splines requested a deployment to testing-review May 2, 2025 19:43 — with GitHub Actions Waiting

Splines reviewed May 2, 2025

View reviewed changes

Splines added 3 commits May 2, 2025 23:55

Temporarily uninstall new gems csv & mutex_m

c4e21e9

Instead of adding mutex_m as dependency, update spring

a12a1ca

See release 4.1.2 of spring where they drop dependency on mutex_m. Therefore, our problem is elegantly solved without the need for a new dependency. https://github.com/rails/spring/blob/main/CHANGELOG.md#412

Add back in csv dependency for Vignettes::CsvHandler

d796bda

Splines requested a deployment to testing-review May 2, 2025 22:05 — with GitHub Actions Waiting

Splines changed the title ~~Upgrade Ruby to 3.3.8 and Rails to 7.2.2.1~~ Upgrade Ruby (3.1.4 -> 3.3.8) & Rails (7.1.3 -> 7.2.2.1) May 2, 2025

Splines reviewed May 2, 2025

View reviewed changes

Specify missing config CLI arg for RuboCop in GitHub Actions

2963988

Splines requested a deployment to testing-review May 3, 2025 09:56 — with GitHub Actions Waiting

Splines requested changes May 3, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade Ruby (3.1.4 -> 3.3.8) & Rails (7.1.3 -> 7.2.2.1) #749

Upgrade Ruby (3.1.4 -> 3.3.8) & Rails (7.1.3 -> 7.2.2.1) #749

fosterfarrell9 commented Feb 22, 2025 •

edited by Splines

Loading

Splines May 2, 2025

Splines May 2, 2025

Splines May 2, 2025

Splines May 2, 2025

Splines May 2, 2025

Splines May 2, 2025

Splines May 2, 2025 •

edited

Loading

Splines left a comment •

edited

Loading

Splines May 2, 2025

Splines May 2, 2025

Splines May 3, 2025

Splines May 3, 2025 •

edited

Loading

Splines May 3, 2025

Splines May 3, 2025 •

edited

Loading

Splines left a comment

Splines May 3, 2025

		@@ -1 +1 @@
		Rails.application.config.active_job.queue_adapter = :sidekiq
		Rails.application.config.active_job.queue_adapter = :sidekiq unless Rails.env.test?

		@@ -1,3 +1,5 @@
		require "active_support/core_ext/integer/time"

Upgrade Ruby (3.1.4 -> 3.3.8) & Rails (7.1.3 -> 7.2.2.1) #749

Are you sure you want to change the base?

Upgrade Ruby (3.1.4 -> 3.3.8) & Rails (7.1.3 -> 7.2.2.1) #749

Conversation

fosterfarrell9 commented Feb 22, 2025 • edited by Splines Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Splines May 2, 2025 • edited Loading

Choose a reason for hiding this comment

Splines left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Splines May 3, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Splines May 3, 2025 • edited Loading

Choose a reason for hiding this comment

Splines left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fosterfarrell9 commented Feb 22, 2025 •

edited by Splines

Loading

Splines May 2, 2025 •

edited

Loading

Splines left a comment •

edited

Loading

Splines May 3, 2025 •

edited

Loading

Splines May 3, 2025 •

edited

Loading