DATABASE_PASSWORD silently overrides Maintenance Mode password

[trevorr]

DATABASE_PASSWORD silently overrides Maintenance Mode password

Description

When running a new instance of FusionAuth using the Docker image with DATABASE_PASSWORD set, the password specified on the Maintenance Mode form is silently ignored. (I was populating DATABASE_PASSWORD from an AWS Secret that accidentally happened to be a JSON blob of RDS credentials instead of just the password.) I was specifying the correct password on the form, but kept getting this error:

The database and schema exist, but for some reason the ordinary user you specified cannot connect to it. Perhaps the user already existed and you provided the wrong password. The error from the database was [Access denied for user 'fusionauth'@'172.30.33.164' (using password: YES)].

The FusionAuth logs didn't contain any mention of DATABASE_PASSWORD being set or the failed login attempts.

Affects versions

1.27.1

Steps to reproduce

Start the FusionAuth Docker image against an empty, existing database with DATABASE_PASSWORD set to an incorrect value. It will be impossible to exit Maintenance Mode.

Expected behavior

One of the following:

1. The password field on the Maintenance Mode form disabled with a note that the password has already been set via the environment variable
2. The password on the Maintenance Mode form to be used to initialize the database (even if FusionAuth can no longer access it on restart)
3. A hint that the password on the form does not match the password in the environment

Platform

• Docker (AWS ECS)
• MySQL 8.0.15

[mooreds]

I'm not sure if this is a documentation oversight, but it sure seems like it'd be a frustrating place to end up in as a user.

[robotdan]

Probably a bug. Any opinion on the solution?

My first though is that if we ended up in an interactive mode with maintenance mode, we should take the values from the form over the environment variable.

[robotdan]

Not sure how to handle this one. I think it is working as designed.

The problem is that we have a defined order of precedence for configuration. The current order is:

• Environment variable
• Java system property
• Config file

So in the Maint. mode form, you enter a username and password, which causes us to update the configuration file. But when we resolve the value we will prefer the environment variable if defined as outlined above in the order of precedence.

This is definitely confusing when using the Maint. mode UI form as described in this issue. However, even if during Maint. mode we preferred the values provided by the configuration file, during normal runtime we would not.

Java cannot set Environment variables, so there is not really a way for us to know that in this case you want to prefer the configuration value over the environment.

Maybe there is another way for us to understand the context of the configuration and change the lookup order for a particular set of configurations - such as db connection (URL, username and password). One option would be to only use environment values for db connection if silent mode is enabled. This may fix this particular issue, but it may also be confusing for other scenarios.

Anyone have a strong opinion on this one? @bguyza @mooreds @voidmain @trevorr ?

[robotdan]

Closing as working as designed for now.

The UI for Maint. mode is really only intended for development and local testing. In production you'll want to be using silent mode w/ environment variables so the Maint. mode screens aren't shown to an end user. Also when using Maint. mode UI - it only updates the config file, which means it is not multi-mode friendly - which is another reason you wouldn't want to use it in prod w/ multi-node.

If anyone feels really strongly in opposition, please outline a use case and we can revisit.

[trevorr]

I'm fine with the precedence, but I'm requesting some indication in the UI that the value can't be set there if the env var is set. It's very easy to forget/overlook the value being set elsewhere, such as docker-compose.yaml, and pull your hair out for an hour over why the password you keep pasting in the form isn't working. If you're only bringing up FA in isolation, this might seem easy to figure out, but often (and in my case) you're bringing up the database as well, so you also have to figure out if the problem was there. In the case of MySQL, there are a surprising number of ways to screw up setting passwords. For instance, they changed the default password encryption in MySQL 8.

I understand the maint UI isn't for production, but since it is the way to bring up the app for development, I think it could be a little more helpful. My suggestion is to disable any UI element that is already set via an env var and say somewhere (e.g. the placeholder text) that it's "Disabled due to ENV_VAR being set".

[robotdan]

Thanks for the feedback @trevorr, that seems like a reasonable solution.

Re-opening.

Internal: Let's review the Maint. mode UI and pre-fill fields as appropriate if environment variables are defined, and disable the fields with some copy on the panel indicating why fields are disabled and how to proceed if you do indeed wish to use Maint. mode UI configuration.