Feature Request: Admin Token Recovery

[peterbarnett03]

Problem

If a user loses their admin-level token, there is currently no way to regenerate it. This is problematic for the implied reasons: data cannot be accessed.

We should not recover the token, but a new token should be able to be generated to overwrite the lost admin-token.

Ideas

Below are some different approaches I've thought of, in the stack ranked order I'd suggest. They are not mutually exclusive though, for example safe mode and recovery code could work well together..

Recovery Code (One-Time Use, Privately Stored)

During install or first token creation, the system generates a one-time recovery code (a long, randomized string) and shows it once to the user, with a clear “store this securely” warning. This code is a recovery code, not a token itself. It can't be used to access any data.

This code is hashed (standard security processes) and stored server-side. If the admin token is lost, a user can issue a token recovery command, passing this recovery code, by flag or by CLI entry. If it matches the stored hash, the system allows a one-time token overwrite, and outputs a new recovery code. Usage of the recovery code is logged.

Safe Mode

We introduce a one-time “recovery mode” triggered via CLI flag or environment variable (e.g., --safe-mode). When the server starts with this flag:

• It starts in isolated mode with no external access, no writing, no querying, no processing, no compacting, no useful mode.
• Only a local CLI call can issue a new admin token.
• Once used, the flag must be removed and the system restarted normally

Secondary Tool for Regeneration

One idea is to have a secondary tool that can only be received from InfluxData. This tool would, in some way, be able to bypass the locked-down security environment, and replace the admin token with a new version.

• Downside: Hard to supply from InfluxData for Core users given amount. Perhaps that's a simple download from our website, but for Enterprise users, there's the additional security of using the licensed email in some way to provide a special download of the tool; e.g., backend we store a random hash tied to licensed email, and we supply them that hash which goes into secondary tool.

Other thoughts

For Enterprise users, we could try leveraging their license email somehow, but unsure how that would play in for all instances, and doesn't solve the problem for Core users. May also be untenable for Home Enterprise users in all areas. Plus, could compromise security in some way if email access is gained as well.

For review by @influxdata/monolith-team

[praveen-influx]

cc @jdstrand

Thanks @peterbarnett03 for writing this up.

Recovery code sounds like a good idea and it's not been suggested before, but what happens if they lose the recovery code too? I think we need to be clear in the message that the tokens and recovery codes are a one time thing and if the user loses it then user cannot access db. The current message is very much exactly what was used in alpha and I think it could do better to warn the user that they need to keep the token safely.

There is one other option I mentioned in this slack convo, it's actually a slight variant to something @jdstrand proposed (although I cannot find the source of that convo now - on a flakey airport wifi). Basically, mount regenerate endpoint itself on the loopback address (original proposal by @jdstrand). This would again have to be something that user opts into when starting the server. We could do few variants of this too, i.e when this is running don't allow any writes (or allow writes) etc. So, it requires restarting the server but regenerate endpoint will be accessible on that interface without requiring a token.

I don't mind which option we choose to implement by the way, I just wanted to capture this other option as well as part of this issue.

[peterbarnett03]

Agree on wording and messaging. PR #26336 for that portion to clearly communicate the importance of storing. When we decide on path forward, we may want to update it again.

[jdstrand]

Recovery Code (One-Time Use, Privately Stored)

This doesn't seem any different than giving out the token in the first place, which already says to keep it safe and store it somewhere secure.

One idea is to have a secondary tool that can only be received from InfluxData. This tool would, in some way, be able to bypass the locked-down security environment, and replace the admin token with a new version.

Let's not do this. We don't want to introduce backdoors into the system.

We introduce a one-time “recovery mode” triggered via CLI flag or environment variable (e.g., --safe-mode)

Something like this is ok. If you are in a position to restart the server, you could do various disruptive things (including tampering with the catalog) so from a security POV, it is 'ok'. @praveen-influx outlined some ideas on how to make this work on a technical level.

[peterbarnett03]

For Recovery Code, I could see us perhaps emailing those to the licensed user; that ensures they're kept somewhere (unless if deleted), while also not granting token-like access unless if input as a serve arg. But doesn't solve for Core use cases, and not sure on best practices in general there; I don't feel compelled to use that approach after thinking through it more.

Researching other approaches systems use, it feels like if you have access to the filesystem itself, then you should be able to reset the token; perhaps by manual edit of a file, or a --safe-mode approach. Don't want to push direct file editing unless if we move it out of the catalog in someway. Curious on @praveen-influx's thoughts on the technical side you mentioned.

[MatthiasWerning]

I would like to add another functionality idea that was introduced to v2 but is still missing as of my knowledge in v3.
When having access to the underlying data source and the actual physical storage it is possible to recover user credentials in v2 using the influxd recovery utility (https://docs.influxdata.com/influxdb/v2/admin/users/recover-credentials/)

As of now there is currently no way to do this v3. This has already been proven to be secure since you have access to the data in its physical representation.

Of course this won't solve the issue if no access to the physical data is present as described by the author of this issue, but would still be a nice addition considering this is already present in earlier influxdb versions.

[jdstrand]

influxd recovery

Note that we are in the process of removing the storage of the raw token data in InfluxDB 2.x. Part of that work will involve the ability to create a new token; this all will be done on the machine running the service. I bring this up because while we don't want to store raw token data in any products, we do want to the ability for people to reset them.