Skip to content

Conversation

@jrafanie
Copy link
Member

No description provided.

@jrafanie
Copy link
Member Author

jrafanie commented Dec 12, 2025

TODO: loading from schema.rb fails with rails 8

EDIT: This is fixed in #826 for rails 7.2

I think what's happening is db:schema:load is run on the database after migrate so some constraints already exist. I will need to look into it because it looks like the load from schema.rb is trying to do something already done. This is what it looks on rails 7.2:

 % TEST_RAILS_VERSION='7.2' RAILS_ENV=test bundle exec rake app:db:schema:load
rake aborted!
ActiveRecord::StatementInvalid: PG::DuplicateObject: ERROR:  constraint "metric_rollups_01_inheritance_check" for relation "metric_rollups_01" already exists (ActiveRecord::StatementInvalid)
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/connection_adapters/postgresql/database_statements.rb:56:in `exec'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/connection_adapters/postgresql/database_statements.rb:56:in `block (2 levels) in raw_execute'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:1024:in `block in with_raw_connection'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activesupport-7.2.3/lib/active_support/concurrency/null_lock.rb:9:in `synchronize'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:993:in `with_raw_connection'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/connection_adapters/postgresql/database_statements.rb:55:in `block in raw_execute'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activesupport-7.2.3/lib/active_support/notifications/instrumenter.rb:58:in `instrument'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:1141:in `log'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/connection_adapters/postgresql/database_statements.rb:54:in `raw_execute'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:538:in `internal_execute'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:137:in `execute'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/connection_adapters/abstract/query_cache.rb:27:in `execute'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/connection_adapters/postgresql/database_statements.rb:48:in `execute'
/Users/joerafaniello/Code/manageiq-schema/lib/manageiq/schema/schema_statements.rb:36:in `add_miq_metric_table_inheritance'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/migration/default_strategy.rb:10:in `method_missing'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/migration.rb:1059:in `block in method_missing'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/migration.rb:1025:in `block in say_with_time'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/benchmark-0.5.0/lib/benchmark.rb:305:in `measure'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/migration.rb:1025:in `say_with_time'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/migration.rb:1048:in `method_missing'
/Users/joerafaniello/Code/manageiq-schema/spec/dummy/db/schema.rb:7808:in `block in <top (required)>'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/schema.rb:56:in `instance_eval'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/schema.rb:56:in `block in define'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:425:in `with_connection'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/schema.rb:55:in `define'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/schema.rb:50:in `define'
/Users/joerafaniello/Code/manageiq-schema/spec/dummy/db/schema.rb:13:in `<top (required)>'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/tasks/database_tasks.rb:377:in `load'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/tasks/database_tasks.rb:377:in `load_schema'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/tasks/database_tasks.rb:485:in `block (2 levels) in load_schema_current'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:431:in `with_connection'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/tasks/database_tasks.rb:533:in `block in with_temporary_connection'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/tasks/database_tasks.rb:565:in `with_temporary_pool'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/tasks/database_tasks.rb:532:in `with_temporary_connection'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/tasks/database_tasks.rb:484:in `block in load_schema_current'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/tasks/database_tasks.rb:606:in `block (2 levels) in each_current_configuration'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/tasks/database_tasks.rb:603:in `each'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/tasks/database_tasks.rb:603:in `block in each_current_configuration'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/tasks/database_tasks.rb:614:in `each'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/tasks/database_tasks.rb:614:in `each_current_environment'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/tasks/database_tasks.rb:602:in `each_current_configuration'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/tasks/database_tasks.rb:483:in `load_schema_current'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-7.2.3/lib/active_record/railties/databases.rake:477:in `block (3 levels) in <top (required)>'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/rake-13.3.1/exe/rake:27:in `<top (required)>'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/cli/exec.rb:59:in `load'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/cli/exec.rb:59:in `kernel_load'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/cli/exec.rb:23:in `run'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/cli.rb:451:in `exec'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/vendor/thor/lib/thor/command.rb:28:in `run'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/vendor/thor/lib/thor.rb:538:in `dispatch'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/cli.rb:35:in `dispatch'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/vendor/thor/lib/thor/base.rb:584:in `start'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/cli.rb:29:in `start'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/exe/bundle:28:in `block in <top (required)>'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/friendly_errors.rb:118:in `with_friendly_errors'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/exe/bundle:20:in `<top (required)>'
/Users/joerafaniello/.gem/ruby/3.3.9/bin/bundle:25:in `load'
/Users/joerafaniello/.gem/ruby/3.3.9/bin/bundle:25:in `<main>'

On rails 8, it looks like it thinks the schema migration table didn't exist so it tried to create the db and load from schema.rb:

        def initialize_database(db_config)
          with_temporary_pool(db_config) do
            begin
              database_already_initialized = migration_connection_pool.schema_migration.table_exists?
            rescue ActiveRecord::NoDatabaseError
              create(db_config)
              retry
            end

            unless database_already_initialized
              schema_dump_path = schema_dump_path(db_config)
              if schema_dump_path && File.exist?(schema_dump_path)
                load_schema(db_config)
              end
            end

            !database_already_initialized
          end
ActiveRecord::StatementInvalid: PG::DuplicateObject: ERROR:  constraint "metric_rollups_01_inheritance_check" for relation "metric_rollups_01" already exists (ActiveRecord::StatementInvalid)
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/connection_adapters/postgresql/database_statements.rb:167:in `exec'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/connection_adapters/postgresql/database_statements.rb:167:in `perform_query'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/connection_adapters/abstract/database_statements.rb:556:in `block (2 levels) in raw_execute'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/connection_adapters/abstract_adapter.rb:1017:in `block in with_raw_connection'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activesupport-8.0.4/lib/active_support/concurrency/null_lock.rb:9:in `synchronize'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/connection_adapters/abstract_adapter.rb:986:in `with_raw_connection'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/connection_adapters/abstract/database_statements.rb:555:in `block in raw_execute'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activesupport-8.0.4/lib/active_support/notifications/instrumenter.rb:58:in `instrument'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/connection_adapters/abstract_adapter.rb:1137:in `log'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/connection_adapters/abstract/database_statements.rb:554:in `raw_execute'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/connection_adapters/abstract/database_statements.rb:591:in `internal_execute'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/connection_adapters/abstract/database_statements.rb:137:in `execute'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/connection_adapters/abstract/query_cache.rb:27:in `execute'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/connection_adapters/postgresql/database_statements.rb:40:in `execute'
/Users/joerafaniello/Code/manageiq-schema/lib/manageiq/schema/schema_statements.rb:36:in `add_miq_metric_table_inheritance'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/migration/default_strategy.rb:10:in `method_missing'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/migration.rb:1055:in `block in method_missing'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/migration.rb:1021:in `block in say_with_time'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activesupport-8.0.4/lib/active_support/benchmark.rb:17:in `realtime'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/migration.rb:1021:in `say_with_time'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/migration.rb:1044:in `method_missing'
/Users/joerafaniello/Code/manageiq-schema/spec/dummy/db/schema.rb:7808:in `block in <top (required)>'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/schema.rb:56:in `instance_eval'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/schema.rb:56:in `block in define'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:422:in `with_connection'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/schema.rb:55:in `define'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/schema.rb:50:in `define'
/Users/joerafaniello/Code/manageiq-schema/spec/dummy/db/schema.rb:13:in `<top (required)>'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/tasks/database_tasks.rb:386:in `load'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/tasks/database_tasks.rb:386:in `load_schema'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/tasks/database_tasks.rb:672:in `block in initialize_database'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/tasks/database_tasks.rb:554:in `with_temporary_pool'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/tasks/database_tasks.rb:661:in `initialize_database'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/tasks/database_tasks.rb:245:in `block in migrate_all'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/tasks/database_tasks.rb:245:in `each'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/tasks/database_tasks.rb:245:in `migrate_all'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/activerecord-8.0.4/lib/active_record/railties/databases.rake:90:in `block (2 levels) in <top (required)>'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/rake-13.3.1/exe/rake:27:in `<top (required)>'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/cli/exec.rb:59:in `load'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/cli/exec.rb:59:in `kernel_load'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/cli/exec.rb:23:in `run'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/cli.rb:451:in `exec'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/vendor/thor/lib/thor/command.rb:28:in `run'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/vendor/thor/lib/thor.rb:538:in `dispatch'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/cli.rb:35:in `dispatch'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/vendor/thor/lib/thor/base.rb:584:in `start'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/cli.rb:29:in `start'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/exe/bundle:28:in `block in <top (required)>'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/lib/bundler/friendly_errors.rb:118:in `with_friendly_errors'
/Users/joerafaniello/.gem/ruby/3.3.9/gems/bundler-2.7.1/exe/bundle:20:in `<top (required)>'
/Users/joerafaniello/.gem/ruby/3.3.9/bin/bundle:25:in `load'
/Users/joerafaniello/.gem/ruby/3.3.9/bin/bundle:25:in `<main>'

strategy:
matrix:
ruby-version:
- '3.1'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh nice - I missed this one - my rudimentary "is this an MIQ repo" checked exactly 3.1 and 3.3 (which is a weird combo, and why I thought it was a safe heuristic), but this repo apparently also had 3.4

jrafanie added a commit to jrafanie/manageiq-schema that referenced this pull request Dec 16, 2025
…hema.rb

We need to do inheritance in both migrations and from schema dump/load.

For constraints, we don't need to do them from schema load as t.check_constraints
is now included for each metric_* subtable in the dumped schema.rb.

Simplify the interface to add_miq_metric_table_constraint to pass the array of
conditions inline.

Note, this issue was found in ManageIQ#824 and extracted and simplified here.
jrafanie added a commit to jrafanie/manageiq-schema that referenced this pull request Dec 16, 2025
We need to do inheritance in both migrations and from schema dump/load.

For constraints, we don't need to do them from schema load as t.check_constraints
is now included for each metric_* subtable in the dumped schema.rb, so we no
longer pass the conditions from schema.rb dump/load.

Simplify the interface to add_miq_metric_table_constraint to pass the array of
conditions inline.

Note, this issue was found in ManageIQ#824 and extracted and simplified here.
jrafanie added a commit to jrafanie/manageiq-schema that referenced this pull request Dec 18, 2025
We need to do inheritance in both migrations and from schema dump/load.

For constraints, we don't need to do them from schema load as t.check_constraints
is now included for each metric_* subtable in the dumped schema.rb, so we no
longer pass the conditions from schema.rb dump/load.

Simplify the interface to add_miq_metric_table_constraint to pass the array of
conditions inline.

Note, this issue was found in ManageIQ#824 and extracted and simplified here.
@miq-bot
Copy link
Member

miq-bot commented Dec 18, 2025

This pull request is not mergeable. Please rebase and repush.

1 similar comment
@miq-bot
Copy link
Member

miq-bot commented Dec 18, 2025

This pull request is not mergeable. Please rebase and repush.

The migrate interface changed from 7.2 to 8.0:

-      def migrate(version = nil)
+      def migrate(version = nil, skip_initialize: false)
track_miq_metric_table_inheritance(table)
end

def inherited_table_names
Copy link
Member

@Fryguy Fryguy Dec 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we have this method, do we need the track_miq_metric_table_inheritance method (and then inherited_metrics_tables method) anymore? It feels redundant. I see track_miq_metric_table_inheritance also needs the base table, but perhaps we can update inherited_table_names to also include that data as well (or have a shared method like inherited_table_details, and then inherited_tables_names calls that)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, I like your way better - feels more generic.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was debating having a single method that handled 7.2 and 8.0 but thought it would be easier to not touch the 7.2 logic and just delete the conditionals and the code for 7.2 when we're on 8.0.

Copy link
Member

@Fryguy Fryguy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few suggestions

This commit allows rails 7.2 to operate as it previously did.
For 8.0, we have to tell it to ignore our inherits tables so we can dump them
after the base table is dumped.

This is because metric_rollups_base is AFTER metric_rollups_01, etc. when sorted
The same problem happens with metrics_base sorting AFTER metrics_00.

Rails 8.0 added inherits support in schema dumper here:
https://www.github.com/rails/rails/pull/50475

A possibly related issue regarding table order with dumping inherits:
https://www.github.com/rails/rails/issues/54841

A possible solution may come in a future release as part of:
https://www.github.com/rails/rails/pull/54872
end

def inherited_table_names
@inherited_table_names ||= @connection.tables.select { |t| ActiveRecord::Base.connection.inherited_table_names(t).present? }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we sort these?

@Fryguy Fryguy self-assigned this Dec 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants