Skip to content

(MODULES-8700) Autorequire the service in postgresql_psql #1276

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 14, 2021

Conversation

ekohl
Copy link
Collaborator

@ekohl ekohl commented May 27, 2021

In d69ff1e I changed the require to be an autorequire. However, even in the PR I noted1 it may not work. Now I realize this may not work because you can't autorequire on a class. In the same way, you can't collect classes.

I started to see this on my Puppet 7 testing (edited for brevity):

Notice: /Stage[main]/Postgresql::Client/Package[postgresql-client]/ensure: created
Notice: /Stage[main]/Postgresql::Client/File[/usr/local/bin/validate_postgresql_connection.sh]/ensure: defined content as '{sha256}f34c53aa433754c6343e6382cd994e33f2887f9eb54503421eb0db55a55e14a6'
Notice: /Stage[main]/Postgresql::Server::Install/Package[postgresql-server]/ensure: created
Notice: /Stage[main]/Postgresql::Server::Initdb/File[/var/lib/pgsql/data]/ensure: created
Notice: /Stage[main]/Postgresql::Server::Initdb/Exec[postgresql_initdb]/returns: The files belonging to this database system will be owned by user "postgres".
Notice: /Stage[main]/Postgresql::Server::Initdb/Exec[postgresql_initdb]/returns: This user must also own the server process.
Notice: /Stage[main]/Postgresql::Server::Initdb/Exec[postgresql_initdb]/returns:
Notice: /Stage[main]/Postgresql::Server::Initdb/Exec[postgresql_initdb]/returns: initdb: error: invalid locale settings; check LANG and LC_* environment variables
Error: '/usr/bin/initdb --pgdata '/var/lib/pgsql/data'' returned 1 instead of one of [0]
Error: /Stage[main]/Postgresql::Server::Initdb/Exec[postgresql_initdb]/returns: change from 'notrun' to ['0'] failed: '/usr/bin/initdb --pgdata '/var/lib/pgsql/data'' returned 1 instead of one of [0]
Info: Class[Postgresql::Server::Initdb]: Unscheduling all events on Class[Postgresql::Server::Initdb]
Notice: /Stage[main]/Postgresql::Server::Config/File[/var/lib/pgsql/data/postgresql.conf]: Dependency Exec[postgresql_initdb] has failures: true
Warning: /Stage[main]/Postgresql::Server::Config/File[/var/lib/pgsql/data/postgresql.conf]: Skipping because of failed dependencies
Warning: /Stage[main]/Postgresql::Server::Config/File[systemd-conf-dir]: Skipping because of failed dependencies
Warning: /Stage[main]/Postgresql::Server::Config/File[systemd-override]: Skipping because of failed dependencies
Warning: /Stage[main]/Postgresql::Server::Config/File[old-systemd-override]: Skipping because of failed dependencies
Warning: /Stage[main]/Postgresql::Server::Config/Exec[restart-systemd]: Skipping because of failed dependencies
Warning: /Stage[main]/Postgresql::Server::Config/Concat[/var/lib/pgsql/data/pg_hba.conf]/Concat_file[/var/lib/pgsql/data/pg_hba.conf]: Skipping because of failed dependencies
Warning: /Stage[main]/Postgresql::Server::Config/Concat[/var/lib/pgsql/data/pg_hba.conf]/File[/var/lib/pgsql/data/pg_hba.conf]: Skipping because of failed dependencies
Warning: /Stage[main]/Postgresql::Server::Config/Concat[/var/lib/pgsql/data/pg_hba.conf]/Concat_fragment[/var/lib/pgsql/data/pg_hba.conf_header]: Skipping because of failed dependencies
Warning: /Stage[main]/Postgresql::Server::Config/Postgresql::Server::Config_entry[port]/Postgresql_conf[port]: Skipping because of failed dependencies
Warning: /Stage[main]/Postgresql::Server::Config/Postgresql::Server::Config_entry[data_directory]/Postgresql_conf[data_directory]: Skipping because of failed dependencies
Warning: /Stage[main]/Postgresql::Server::Config/Concat[/var/lib/pgsql/data/pg_ident.conf]/Concat_file[/var/lib/pgsql/data/pg_ident.conf]: Skipping because of failed dependencies
Warning: /Stage[main]/Postgresql::Server::Config/Concat[/var/lib/pgsql/data/pg_ident.conf]/File[/var/lib/pgsql/data/pg_ident.conf]: Skipping because of failed dependencies
Warning: /Stage[main]/Postgresql::Server::Config/Concat[/var/lib/pgsql/data/pg_ident.conf]/Concat_fragment[/var/lib/pgsql/data/pg_ident.conf_header]: Skipping because of failed dependencies
Warning: /Stage[main]/Postgresql::Server::Config/Postgresql::Server::Pg_hba_rule[local access as postgres user]/Concat::Fragment[pg_hba_rule_local access as postgres user]/Concat_fragment[pg_hba_rule_local access as postgres user]: Skipping because of failed dependencies
Warning: /Stage[main]/Postgresql::Server::Config/Postgresql::Server::Pg_hba_rule[local access to database with same name]/Concat::Fragment[pg_hba_rule_local access to database with same name]/Concat_fragment[pg_hba_rule_local access to database with same name]: Skipping because of failed dependencies
Warning: /Stage[main]/Postgresql::Server::Config/Postgresql::Server::Pg_hba_rule[allow localhost TCP access to postgresql user]/Concat::Fragment[pg_hba_rule_allow localhost TCP access to postgresql user]/Concat_fragment[pg_hba_rule_allow localhost TCP access to postgresql user]: Skipping because of failed dependencies
Warning: /Stage[main]/Postgresql::Server::Config/Postgresql::Server::Pg_hba_rule[deny access to postgresql user]/Concat::Fragment[pg_hba_rule_deny access to postgresql user]/Concat_fragment[pg_hba_rule_deny access to postgresql user]: Skipping because of failed dependencies
  Warning: /Stage[main]/Postgresql::Server::Config/Postgresql::Server::Pg_hba_rule[allow access to all users]/Concat::Fragment[pg_hba_rule_allow access to all users]/Concat_fragment[pg_hba_rule_allow access to all users]: Skipping because of failed dependencies
Warning: /Stage[main]/Postgresql::Server::Config/Postgresql::Server::Pg_hba_rule[allow access to ipv6 localhost]/Concat::Fragment[pg_hba_rule_allow access to ipv6 localhost]/Concat_fragment[pg_hba_rule_allow access to ipv6 localhost]: Skipping because of failed dependencies
Warning: /Stage[main]/Postgresql::Server::Service/Anchor[postgresql::server::service::begin]: Skipping because of failed dependencies
Warning: /Stage[main]/Postgresql::Server::Service/Service[postgresqld]: Skipping because of failed dependencies
Info: Class[Postgresql::Server]: Unscheduling all events on Class[Postgresql::Server]
Warning: /Stage[main]/Postgresql::Server::Service/Postgresql_conn_validator[validate_service_is_running]: Skipping because of failed dependencies
  Warning: /Stage[main]/Postgresql::Server::Service/Anchor[postgresql::server::service::end]: Skipping because of failed dependencies
  Warning: /Stage[main]/Postgresql::Server::Reload/Exec[postgresql_reload]: Skipping because of failed dependencies
Info: Class[Foreman::Database::Postgresql]: Scheduling refresh of Postgresql::Server::Db[foreman]
Info: Postgresql::Server::Db[foreman]: Scheduling refresh of Postgresql::Server::Database[foreman]
Info: Postgresql::Server::Db[foreman]: Scheduling refresh of Postgresql::Server::Role[foreman]
Info: Postgresql::Server::Db[foreman]: Scheduling refresh of Postgresql::Server::Database_grant[GRANT foreman - ALL - foreman]
Info: Postgresql::Server::Role[foreman]: Scheduling refresh of Postgresql_psql[CREATE ROLE foreman ENCRYPTED PASSWORD ****]
Info: Postgresql::Server::Role[foreman]: Scheduling refresh of Postgresql_psql[ALTER ROLE "foreman" NOSUPERUSER]
  Info: Postgresql::Server::Role[foreman]: Scheduling refresh of Postgresql_psql[ALTER ROLE "foreman" NOCREATEDB]
Info: Postgresql::Server::Role[foreman]: Scheduling refresh of Postgresql_psql[ALTER ROLE "foreman" NOCREATEROLE]
Info: Postgresql::Server::Role[foreman]: Scheduling refresh of Postgresql_psql[ALTER ROLE "foreman" LOGIN]
  Info: Postgresql::Server::Role[foreman]: Scheduling refresh of Postgresql_psql[ALTER ROLE "foreman" INHERIT]
Info: Postgresql::Server::Role[foreman]: Scheduling refresh of Postgresql_psql[ALTER ROLE "foreman" NOREPLICATION]
Info: Postgresql::Server::Role[foreman]: Scheduling refresh of Postgresql_psql[ALTER ROLE "foreman" CONNECTION LIMIT -1]
  Info: Postgresql::Server::Role[foreman]: Scheduling refresh of Postgresql_psql[ALTER ROLE foreman ENCRYPTED PASSWORD ****]
Info: Postgresql::Server::Database_grant[GRANT foreman - ALL - foreman]: Scheduling refresh of Postgresql::Server::Grant[database:GRANT foreman - ALL - foreman]
Error: /Stage[main]/Foreman::Database::Postgresql/Postgresql::Server::Db[foreman]/Postgresql::Server::Role[foreman]/Postgresql_psql[CREATE ROLE foreman ENCRYPTED PASSWORD ****]: Could not evaluate: Error evaluating 'unless' clause, returned pid 1676 exit 2: 'psql: error: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
  '

As can be seen, various resources are correctly skipped due to failed dependencies but Postgresql_psql isn't.

This changes it to an autorequire on the service. This should work since the name of the service resource is static.

There is an edge case where the service isn't managed. For that the anchor is also auto-required. This gives less guarantee, but it's a best-effort for those users.

Fixes: d69ff1e

@ekohl ekohl requested a review from a team as a code owner May 27, 2021 10:19
@puppet-community-rangefinder
Copy link

postgresql_psql is a type

Breaking changes to this file WILL impact these 8 modules (exact match):
Breaking changes to this file MAY impact these 8 modules (near match):

This module is declared in 71 of 576 indexed public Puppetfiles.


These results were generated with Rangefinder, a tool that helps predict the downstream impact of breaking changes to elements used in Puppet modules. You can run this on the command line to get a full report.

Exact matches are those that we can positively identify via namespace and the declaring modules' metadata. Non-namespaced items, such as Puppet 3.x functions, will always be reported as near matches only.

Copy link
Collaborator

@alexjfisher alexjfisher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've hit this before, and this change looks correct to me.

@ekohl ekohl force-pushed the autorequire-service branch from a53328a to c3c1688 Compare May 27, 2021 14:20
@ekohl
Copy link
Collaborator Author

ekohl commented May 27, 2021

I wonder if this is related to rodjek/rspec-puppet#821.

In d69ff1e I changed the require to be
an autorequire. However, even in the PR I noted[1] it may not work. Now
I realize this may not work because you can't autorequire on a class. In
the same way, you can't collect classes.

I started to see this on my Puppet 7 testing (edited for brevity):

    Notice: /Stage[main]/Postgresql::Client/Package[postgresql-client]/ensure: created
    Notice: /Stage[main]/Postgresql::Client/File[/usr/local/bin/validate_postgresql_connection.sh]/ensure: defined content as '{sha256}f34c53aa433754c6343e6382cd994e33f2887f9eb54503421eb0db55a55e14a6'
    Notice: /Stage[main]/Postgresql::Server::Install/Package[postgresql-server]/ensure: created
    Notice: /Stage[main]/Postgresql::Server::Initdb/File[/var/lib/pgsql/data]/ensure: created
    Notice: /Stage[main]/Postgresql::Server::Initdb/Exec[postgresql_initdb]/returns: The files belonging to this database system will be owned by user "postgres".
    Notice: /Stage[main]/Postgresql::Server::Initdb/Exec[postgresql_initdb]/returns: This user must also own the server process.
    Notice: /Stage[main]/Postgresql::Server::Initdb/Exec[postgresql_initdb]/returns:
    Notice: /Stage[main]/Postgresql::Server::Initdb/Exec[postgresql_initdb]/returns: initdb: error: invalid locale settings; check LANG and LC_* environment variables
    Error: '/usr/bin/initdb --pgdata '/var/lib/pgsql/data'' returned 1 instead of one of [0]
    Error: /Stage[main]/Postgresql::Server::Initdb/Exec[postgresql_initdb]/returns: change from 'notrun' to ['0'] failed: '/usr/bin/initdb --pgdata '/var/lib/pgsql/data'' returned 1 instead of one of [0]
    Info: Class[Postgresql::Server::Initdb]: Unscheduling all events on Class[Postgresql::Server::Initdb]
    Notice: /Stage[main]/Postgresql::Server::Config/File[/var/lib/pgsql/data/postgresql.conf]: Dependency Exec[postgresql_initdb] has failures: true
    Warning: /Stage[main]/Postgresql::Server::Config/File[/var/lib/pgsql/data/postgresql.conf]: Skipping because of failed dependencies
    Warning: /Stage[main]/Postgresql::Server::Config/File[systemd-conf-dir]: Skipping because of failed dependencies
    Warning: /Stage[main]/Postgresql::Server::Config/File[systemd-override]: Skipping because of failed dependencies
    Warning: /Stage[main]/Postgresql::Server::Config/File[old-systemd-override]: Skipping because of failed dependencies
    Warning: /Stage[main]/Postgresql::Server::Config/Exec[restart-systemd]: Skipping because of failed dependencies
    Warning: /Stage[main]/Postgresql::Server::Config/Concat[/var/lib/pgsql/data/pg_hba.conf]/Concat_file[/var/lib/pgsql/data/pg_hba.conf]: Skipping because of failed dependencies
    Warning: /Stage[main]/Postgresql::Server::Config/Concat[/var/lib/pgsql/data/pg_hba.conf]/File[/var/lib/pgsql/data/pg_hba.conf]: Skipping because of failed dependencies
    Warning: /Stage[main]/Postgresql::Server::Config/Concat[/var/lib/pgsql/data/pg_hba.conf]/Concat_fragment[/var/lib/pgsql/data/pg_hba.conf_header]: Skipping because of failed dependencies
    Warning: /Stage[main]/Postgresql::Server::Config/Postgresql::Server::Config_entry[port]/Postgresql_conf[port]: Skipping because of failed dependencies
    Warning: /Stage[main]/Postgresql::Server::Config/Postgresql::Server::Config_entry[data_directory]/Postgresql_conf[data_directory]: Skipping because of failed dependencies
    Warning: /Stage[main]/Postgresql::Server::Config/Concat[/var/lib/pgsql/data/pg_ident.conf]/Concat_file[/var/lib/pgsql/data/pg_ident.conf]: Skipping because of failed dependencies
    Warning: /Stage[main]/Postgresql::Server::Config/Concat[/var/lib/pgsql/data/pg_ident.conf]/File[/var/lib/pgsql/data/pg_ident.conf]: Skipping because of failed dependencies
    Warning: /Stage[main]/Postgresql::Server::Config/Concat[/var/lib/pgsql/data/pg_ident.conf]/Concat_fragment[/var/lib/pgsql/data/pg_ident.conf_header]: Skipping because of failed dependencies
    Warning: /Stage[main]/Postgresql::Server::Config/Postgresql::Server::Pg_hba_rule[local access as postgres user]/Concat::Fragment[pg_hba_rule_local access as postgres user]/Concat_fragment[pg_hba_rule_local access as postgres user]: Skipping because of failed dependencies
    Warning: /Stage[main]/Postgresql::Server::Config/Postgresql::Server::Pg_hba_rule[local access to database with same name]/Concat::Fragment[pg_hba_rule_local access to database with same name]/Concat_fragment[pg_hba_rule_local access to database with same name]: Skipping because of failed dependencies
    Warning: /Stage[main]/Postgresql::Server::Config/Postgresql::Server::Pg_hba_rule[allow localhost TCP access to postgresql user]/Concat::Fragment[pg_hba_rule_allow localhost TCP access to postgresql user]/Concat_fragment[pg_hba_rule_allow localhost TCP access to postgresql user]: Skipping because of failed dependencies
    Warning: /Stage[main]/Postgresql::Server::Config/Postgresql::Server::Pg_hba_rule[deny access to postgresql user]/Concat::Fragment[pg_hba_rule_deny access to postgresql user]/Concat_fragment[pg_hba_rule_deny access to postgresql user]: Skipping because of failed dependencies
      Warning: /Stage[main]/Postgresql::Server::Config/Postgresql::Server::Pg_hba_rule[allow access to all users]/Concat::Fragment[pg_hba_rule_allow access to all users]/Concat_fragment[pg_hba_rule_allow access to all users]: Skipping because of failed dependencies
    Warning: /Stage[main]/Postgresql::Server::Config/Postgresql::Server::Pg_hba_rule[allow access to ipv6 localhost]/Concat::Fragment[pg_hba_rule_allow access to ipv6 localhost]/Concat_fragment[pg_hba_rule_allow access to ipv6 localhost]: Skipping because of failed dependencies
    Warning: /Stage[main]/Postgresql::Server::Service/Anchor[postgresql::server::service::begin]: Skipping because of failed dependencies
    Warning: /Stage[main]/Postgresql::Server::Service/Service[postgresqld]: Skipping because of failed dependencies
    Info: Class[Postgresql::Server]: Unscheduling all events on Class[Postgresql::Server]
    Warning: /Stage[main]/Postgresql::Server::Service/Postgresql_conn_validator[validate_service_is_running]: Skipping because of failed dependencies
      Warning: /Stage[main]/Postgresql::Server::Service/Anchor[postgresql::server::service::end]: Skipping because of failed dependencies
      Warning: /Stage[main]/Postgresql::Server::Reload/Exec[postgresql_reload]: Skipping because of failed dependencies
    Info: Class[Foreman::Database::Postgresql]: Scheduling refresh of Postgresql::Server::Db[foreman]
    Info: Postgresql::Server::Db[foreman]: Scheduling refresh of Postgresql::Server::Database[foreman]
    Info: Postgresql::Server::Db[foreman]: Scheduling refresh of Postgresql::Server::Role[foreman]
    Info: Postgresql::Server::Db[foreman]: Scheduling refresh of Postgresql::Server::Database_grant[GRANT foreman - ALL - foreman]
    Info: Postgresql::Server::Role[foreman]: Scheduling refresh of Postgresql_psql[CREATE ROLE foreman ENCRYPTED PASSWORD ****]
    Info: Postgresql::Server::Role[foreman]: Scheduling refresh of Postgresql_psql[ALTER ROLE "foreman" NOSUPERUSER]
      Info: Postgresql::Server::Role[foreman]: Scheduling refresh of Postgresql_psql[ALTER ROLE "foreman" NOCREATEDB]
    Info: Postgresql::Server::Role[foreman]: Scheduling refresh of Postgresql_psql[ALTER ROLE "foreman" NOCREATEROLE]
    Info: Postgresql::Server::Role[foreman]: Scheduling refresh of Postgresql_psql[ALTER ROLE "foreman" LOGIN]
      Info: Postgresql::Server::Role[foreman]: Scheduling refresh of Postgresql_psql[ALTER ROLE "foreman" INHERIT]
    Info: Postgresql::Server::Role[foreman]: Scheduling refresh of Postgresql_psql[ALTER ROLE "foreman" NOREPLICATION]
    Info: Postgresql::Server::Role[foreman]: Scheduling refresh of Postgresql_psql[ALTER ROLE "foreman" CONNECTION LIMIT -1]
      Info: Postgresql::Server::Role[foreman]: Scheduling refresh of Postgresql_psql[ALTER ROLE foreman ENCRYPTED PASSWORD ****]
    Info: Postgresql::Server::Database_grant[GRANT foreman - ALL - foreman]: Scheduling refresh of Postgresql::Server::Grant[database:GRANT foreman - ALL - foreman]
    Error: /Stage[main]/Foreman::Database::Postgresql/Postgresql::Server::Db[foreman]/Postgresql::Server::Role[foreman]/Postgresql_psql[CREATE ROLE foreman ENCRYPTED PASSWORD ****]: Could not evaluate: Error evaluating 'unless' clause, returned pid 1676 exit 2: 'psql: error: could not connect to server: No such file or directory
            Is the server running locally and accepting
            connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
      '

As can be seen, various resources are correctly skipped due to failed
dependencies but Postgresql_psql isn't.

This changes it to an autorequire on the service. This should work since
the name of the service resource is static.

There is an edge case where the service isn't managed. For that the
anchor is also auto-required. This gives less guarantee, but it's a
best-effort for those users.

[1]: puppetlabs#950 (comment)

Fixes: d69ff1e
@ekohl ekohl force-pushed the autorequire-service branch from c3c1688 to 79fa355 Compare May 27, 2021 14:37
@sheenaajay
Copy link
Contributor

Thanks alot @ekohl for your contribution. This gonna fix an old issue too https://tickets.puppetlabs.com/browse/MODULES-8700

@ekohl
Copy link
Collaborator Author

ekohl commented Jun 14, 2021

Do you want me to modify the commit message to include (MODULES-8700)?

@sheenaajay
Copy link
Contributor

Do you want me to modify the commit message to include (MODULES-8700)?

We can modify the PR title to include the ticket number. Will modify now.

@sheenaajay sheenaajay changed the title Autorequire the service in postgresql_psql (MODULES-8700) Autorequire the service in postgresql_psql Jun 14, 2021
@sheenaajay sheenaajay merged commit a46036b into puppetlabs:main Jun 14, 2021
@ekohl ekohl deleted the autorequire-service branch June 14, 2021 18:01
cegeka-jenkins pushed a commit to cegeka/puppet-postgresql that referenced this pull request Feb 3, 2022
(MODULES-8700) Autorequire the service in postgresql_psql
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants