Cleaning configuration backups

[nyarly]

For some time, home-manager has been able to move configuration files out of the way as backups (for example, to a path with a .bak extension added). While this works the first time a managed configuration file replaces an existing file, it has a couple of downsides.

First, it litters the filesystem with .bak files that go stale very quickly. They're likely handy for a week or two at most, if the HM replacement doesn't quite do what you wanted. After that, they're just cruft.

Second, (and perhaps I'm peculiar in this workflow), one might make a "live" copy of a configuration to test out immediate changes, and then update HM accordingly. The subsequent home-manager switch would update the configuration... unless there's already a backup file, in which case it conservatively refuses to clobber the backup.

As the original contributor of the backup feature, I've been thinking about these issues on and off for a while now (and seeing frustration in the issues here as backups became part of the NixOS module).

Would it be possible for home-manager to keep a record of the backups it's made? If for example there were a list of paths and digests, maybe in $XDG_STATE_HOME, then there might be a home-manager clean-backups that goes through the list, deleting any unchanged files. It might also be an optional pre-switch step, which could be enabled by a flag on the command line or an option in the module.

My biggest question is: is it appropriate for home-manager to set up such a state file, or is there an existing location that would be better?

@rycee @khaneliman - this feels like a (small) architectural question, so I'd love your input.

[teto]

nope it's not. There are better tools out there to create backups (cp, restic etc) and better ways to deal with that issue.

When deploying HM warns you about existing file then you have a choise: either you migrate to a hm module and remove/rename the nagging file or you dont.
Automated backups make sense in an automated scenario but then you have control over the machine and likely dont have to worry aobut the backups.

[nyarly]

I guess I didn't explain the context very well, because I'm having trouble understanding your response, I'll try to clarify.

[nyarly]

Home manager doesn't do backups in a disaster-recovery sense (for which, yes, restic would be an appropriate tool.) Instead, it does something like what cp --backup does - instead of failing, it moves a conflicting target file to a new name with a suffix.

Several people (myself included) find this behavior very useful, and it's part of home-manager today. There've been repeated requests for a better UX around its usage, especially when home-manager is a NixOS module. My understanding is that it can be configured to use a backup suffix (a little like cp -b). This only works once per target path though, because on a second collision the suffixed path already exists.

Because a) home-manager manipulates many files at once and b) it fails on the first path collision, moving the offending file is a tedious, repetitive task of the sort that I generally expect a machine to do.

The basic -b flag to home-manager does this well, but sets up "second-order" toil, which is what I'm trying to address here.

[teto]

There've been repeated requests for a better UX around its usage

and there has been repeated requests to provide a scenario for which it would be useful and maybe I missed it but I 've read none. When do you need to backup more than once ? the need to backup arises when you are migrating to a nix config, in which case you shouldn't need to backup anymore ?!

I had hoped the demand was met with home-manager.backupCommand. How is that not enough for you ? Seems like you can achieve what you request with it (record backups in a database etc).

[nyarly]

The first part of my context is that I use home-manager from the command line. I don't see flags that are equivalent to backupCommand or to set HOME_MANAGER_BACKUP_OVERWRITE (which would probably be good enough).

Second, I feel like I've been trying to lay out the scenario in which it would be useful to me: especially when I'm working with an unfamiliar tool, I will sometimes replace the home-manager symlink with a copy from the nix store and edit it in place, which lets me determine the configuration I want to see without going through full environment builds. Once I've found the right settings, I record them in my home-manager configuration and switch. The backup extension configuration lets home-manager help close the loop... once. A subsequent config-record-switch cycle breaks.

There's also the kruft of old .bak files that (once the new configuration is in place and confirmed working) isn't useful. It'd be nice to clean those up.

Maybe what it wants is a "move to .bak and record in my own sqlite db for later cleanup" backupCommand, but I'd need to at least have a place to configure that outside of NixOS. It also seems like a non-trivial task to put together, and if I'm not the only user with this case I'd like to see the work reused.

[teto]

ok now I understand how you get the issue, but I still think your approach is asking for trouble where there is none sry ^^'

I don't see flags that are equivalent to backupCommand or to set HOME_MANAGER_BACKUP_OVERWRITE (which would probably be good enough).

if you dont see them, it's an oversight that should be fixed.

There's also the kruft of old .bak files that (once the new configuration is in place and confirmed working) isn't useful. It'd be nice to clean those up.

why do you keep those around ? as you as you've nixifed a program remove those .bak files and problem solved. You can also use "force" to overwrite an existing file and get rid of the error.

Note that many programs can be configured either as an hybrid mode where the nix generated config includes manual config (true for git, nix.conf) so you get the best of both worlds.
Also many programs accepts a --config flag to load alternative flags.
You can sometimes cheat a bit export XDG_CONFIG_HOME=/tmp program-to-test.
As a last resort you have to remove the HM generated config, write your own but once you are nixifying it, just mv /rm your manual config. What you describe looks very manual and focused so no need for an automated system.

[nyarly]

I still think your approach is asking for trouble where there is none sry ^^'

My approach is different from yours, sure. Halfway through a usual cycle, it works great. If there's trouble, I attribute it to a gap in functionality somewhere, which is why I started this thread. Hard for me to read this as not "you shouldn't want to configure your tools" which seems strange in relationship to a configuration management system.

if you dont see them, it's an oversight that should be fixed.

And maybe that's all that's called for. I wasn't aware of the backupCommand option (missed it in news I guess), but it would require surfacing configuration on home-manager itself, rather than only in the NixOS module.

So, let me restate the proposal here as: can we expose backupCommand as a HM configuration (it should be stable configuration, In my opinion, rather than just a flag)? I'd like to use it to streamline my process, but I'd rather not integrate the Home manager module into my NixOS configuration just yet, and I'd like to be able to use my HM configuration on non-NixOS systems.