Restarting unit service ondemand-dex hangs

[FLAK-ZOSO]

For some reason if I start the service with systemctl, it hangs.

sudo systemctl restart ondemand-dex

And at the same time it looks as inactive.

● ondemand-dex.service - OnDemand Dex - A federated OpenID Connect provider packaged for OnDemand
   Loaded: loaded (/usr/lib/systemd/system/ondemand-dex.service; enabled; vendor preset: disabled)
   Active: inactive (dead)

This is the service.

# /usr/lib/systemd/system/ondemand-dex.service
[Unit]
Description=OnDemand Dex - A federated OpenID Connect provider packaged for OnDemand
After=network-online.target multi-user.target
Wants=network-online.target

[Service]
SyslogIdentifier=ondemand-dex
WorkingDirectory=/usr/share/ondemand-dex
ExecStart=/usr/sbin/ondemand-dex serve /etc/ood/dex/config.yaml
User=ondemand-dex
Group=ondemand-dex

[Install]
WantedBy=multi-user.target

However, this works seamlessly (without the need to put in the background as I do, indeed).

sudo -u ondemand-dex /usr/sbin/ondemand-dex serve /etc/ood/dex/config.yaml >> /var/log/ondemand/dex.log 2>&1 &

Specifics

• Linux controller0.flak.test 4.18.0-553.58.1.el8_10.x86_64 #1 SMP Thu Jun 26 15:50:39 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
• Rocky Linux release 8.10 (Green Obsidian)

[FLAK-ZOSO]

It was enough to remove the circular dependency, as multi-user.target was waiting behind something else. Nevermind.

# /usr/lib/systemd/system/ondemand-dex.service
[Unit]
Description=OnDemand Dex - A federated OpenID Connect provider packaged for OnDemand
-After=network-online.target multi-user.target
+After=network-online.target
Wants=network-online.target

[Service]
SyslogIdentifier=ondemand-dex
WorkingDirectory=/usr/share/ondemand-dex
ExecStart=/usr/sbin/ondemand-dex serve /etc/ood/dex/config.yaml
User=ondemand-dex
Group=ondemand-dex

[Install]
WantedBy=multi-user.target

[johrstrom]

@treydock is that something we should patch?

[FLAK-ZOSO]

Truth is... either you wait for multi-user.target or you wait for the individual services that you might need, or it will fail.

Since the utilized database is picked up at runtime from the configuration file, the only solution that I can come up with is...

After=network-online.target mysqld.service <whatever-other-daemon-based-database-is-supported>

This wouldn't cause hanging problems since After only waits for some service if that service is enabled, and if that database is not used then it's reasonable to assume it won't be enabled, or worst case it will only slightly prolong the waiting time.

[treydock]

For removing After=multi-user.target, that is something we can patch but if you need to wait on site-specific items like a database you'd have to do a drop-in under /etc/systemd/system/ondemand-dex.service.d/after.conf kind of thing.

[FLAK-ZOSO]

I guess no patches are needed to close the issue then.

Probably the best solution is to point out in the documentation in some troubleshooting section that this can happen. This as in ondemand-dex unit waiting behind multi-user.target and multi-user.target waiting behind it. An addition to the documentation could be the following.

It may happen on your system that systemd does not achieve multi-user.target: this causes ondemand-dex.service to wait behind it, and possibly multi-user.target to wait behind ondemand-dex.service in a loop.
If you decide to remove multi-user.target from the After section, remember to add to it all the dependencies. If you are relying on MySQL, the following might be your service file.

# /usr/lib/systemd/system/ondemand-dex.service
[Unit]
Description=OnDemand Dex - A federated OpenID Connect provider packaged for OnDemand
-After=network-online.target multi-user.target
+After=network-online.target mysqld.service
Wants=network-online.target

[Service]
SyslogIdentifier=ondemand-dex
WorkingDirectory=/usr/share/ondemand-dex
ExecStart=/usr/sbin/ondemand-dex serve /etc/ood/dex/config.yaml
User=ondemand-dex
Group=ondemand-dex

[Install]
WantedBy=multi-user.target