Enable bulk upsert in SeedDo.seed

[willnet]

Add a bulk: true option to the SeedDo.seed method. Normally, SeedDo.seed runs queries for each model, but SeedDo.seed(bulk: true) combines multiple queries into one. This should give a large performance improvement when a project defines a lot of seed data.

Allow the number of records in each query to be set with the batch_size option. The default is 1000.

SeedDo.seed(bulk: { batch_size: 100 })

Requirements

Use Rails upsert_all internally to implement this feature. To keep the behavior close to the existing seed-do behavior, use the :unique_by option, so the related constraints must have unique indexes. Also, :unique_by is supported only by PostgreSQL and SQLite, so this feature is not available on MySQL.

Notes

SeedDo.seed and SeedDo.seed(bulk: true) do not always produce the same results.

For example, with the following seed definition:

User.seed(:name) do |u|
  u.name = 'first'
end

User.create!(name: 'second')

User.seed(:name) do |u|
  u.name = 'third'
end

With SeedDo.seed, records are created in the order first, second, third. However, with SeedDo.seed(bulk: true), the order becomes second, first, third.

Because of this, verify the behavior carefully before using SeedDo.seed(bulk: true) in an existing production environment.

Breaking Change

This change also updates the interface of SeedDo::Seeder#new. In particular, the quiet and insert_only options can no longer be passed to SeedDo::Seeder.new. Since SeedDo::Seeder is an internal class, normal usage through SeedDo.seed is expected to keep working, but any code that calls SeedDo::Seeder.new directly will need to be updated.

[willnet]

Points to consider

• We store which arguments are passed to seed in SeedDo::Runner#buffer, but the model’s seed method cannot access SeedDo::Runner#buffer directly. So we put the SeedDo::Runner instance into the global variable SeedDo.current_runner, which feels ugly. If there is another solution, I want to change the current approach.
• Right now, we simply pass all arguments accumulated in buffer to the upsert_all method. Do we need to do something like sending them in batches of 1000?

[willnet]

Points to consider

For the first one, I kept it as it is because I couldn’t think of a better way. For the second one, I assumed users define a large number of seeds, so I chose a method that sets the batch size.