Upcoming release: Requirement for Desktop application running

[perry-mitchell]

I've mentioned this several times on social media, but I'm aware that many people probably haven't read such updates. I think this is best handled as a discussion anyway.

What's changing

Due to Chrome's manifest v3 changes, Buttercup can no longer run a daemon process in the background. This process, up until now, was used to keep decrypted credentials in memory while vaults were unlocked. The daemon didn't sleep and its memory was only wiped when the browser closed or the extension updated. Manifest v3 drops support for background (daemon) processes and in doing so will prevent Buttercup from working in the same way.

After toying with running Buttercup restfully in the new service worker model (it didn't work), I've decided to make some drastic changes to how the extension works: namely, that the browser extension will interface directly with the desktop application. All vaults, their statuses and their contents will be retrieved from the desktop application, running in the background, instead of from the extensions memory.

What does this mean?

• You will no longer have vaults stored in the browser extension
• The desktop application must be running to use the extension
• The browser extension's "vaults" will be the same as those connected to the desktop application
• To use a vault in the extension it must be unlocked in the desktop app
• Saved credentials from the browser will be saved via the desktop application
• The local vault connector support in the extension will be removed, as you can natively access file-based vaults in the new model

This is a huge change for the platform, and not one I've decided on without giving it a bit of thought. I definitely preferred having the vaults isolated within the extension, but their were some problems:

• Inefficient - obviously doing it this was wasteful CPU/memory wise as having them stored just once in the desktop application would be better
• More effort - soon you'll just connect a vault once and it'll be available in both places
• Unpolished - the extension has a few quirks, such as updates locking all vaults

I feel like this is genuinely better for Buttercup going forward.

[Pirgosth]

Hey !

I totally understand the new technical constraints due to chrome changes, and we cannot blame you at all for theses upcoming modifications.

However, I'm really sad the hear this as I was really loving the fact that all vaults clients were running independently, especially the chrome extension as it did not required any third software installation.

Because personally I really don't mind installing the software on my laptop at all, but when I'm working from my professional computer I'm not allowed to install such software, thus making using this secret vault almost impossible there.

Is it possible to keep some "legacy" mode, in which we could still be using the previous system, event if that means to unlock our vault every time we want to read any password ?

[perry-mitchell]

However, I'm really sad the hear this as I was really loving the fact that all vaults clients were running independently, especially the chrome extension as it did not required any third software installation.

I know, I originally felt the same. I eat my own dogfood and like to think I have a somewhat common use-case for Buttercup. As mentioned, this change is beyond our control due to Google's stance on extensions - we wouldn't have gone this way had we another option. That being said, it's actually more reliable this way (the extension has always given us problems sync-wise), and the extension will be better for it imo.

Because personally I really don't mind installing the software on my laptop at all, but when I'm working from my professional computer I'm not allowed to install such software, thus making using this secret vault almost impossible there.

Can you not use the portable version of Buttercup desktop? I do appreciate that this might be a challenge where security is tight. I wonder though; if your systems are so locked down that you can't install software, how are you able to install an extension? Normally, so far as I've seen, companies lock down that ability along with the rest of the OS.

Is it possible to keep some "legacy" mode, in which we could still be using the previous system, event if that means to unlock our vault every time we want to read any password ?

I'm afraid not, for now.. I simply don't have the time to embed it again in the new version. It's also, as I just mentioned, a lot of work to maintain. That, and that with Chrome's new manifest v3, we cannot have long-running background processes, so it's impossible to keep a vault unlocked in memory. This would mean that every time you need to check the vault for an entry, it'd have to be unlocked again.. It'd be both insecure and slow.

I'm open to other ideas here.. Don't want to leave anyone in the lurch if there's something obvious we could add here.

[Pirgosth]

Hey ! First of all thanks for the detailed answer !

I know, I originally felt the same. I eat my own dogfood and like to think I have a somewhat common use-case for Buttercup. As mentioned, this change is beyond our control due to Google's stance on extensions - we wouldn't have gone this way had we another option. That being said, it's actually more reliable this way (the extension has always given us problems sync-wise), and the extension will be better for it imo.

Yes I know this pain, it's never that cool nor easy when you're constraint to refactor a lot of your work because external breaking changes are coming in. But you're totally right, I do think too that using less fragmented tools will be better for both usage and maintenance.

Can you not use the portable version of Buttercup desktop? I do appreciate that this might be a challenge where security is tight. I wonder though; if your systems are so locked down that you can't install software, how are you able to install an extension? Normally, so far as I've seen, companies lock down that ability along with the rest of the OS.

Yes definitely ! Sorry, at the time I wrote my first message I was lacking information and I totally forgot about standalone versions of software, so yes I'll be using this one thanks ! Like for most companies I do not have privileged rights on my laptop (we're running Windows) but for that usage it'll be totally fine :)

I'm afraid not, for now.. I simply don't have the time to embed it again in the new version. It's also, as I just mentioned, a lot of work to maintain. That, and that with Chrome's new manifest v3, we cannot have long-running background processes, so it's impossible to keep a vault unlocked in memory. This would mean that every time you need to check the vault for an entry, it'd have to be unlocked again.. It'd be both insecure and slow.

According to what we just say, I realize it won't be useful anymore so don't mind this question haha

I'm open to other ideas here.. Don't want to leave anyone in the lurch if there's something obvious we could add here.

I think you've already done your best, thanks for maintaining this project, I'm really happy overall with it, and with your feedback I'll still use it for sure !

[perry-mitchell]

Thanks for the discussion! It's really motivating to hear all of this, see what the real concerns are. I appreciate it!

[Oxalin]

Any chance to see a new release before this big overhaul? The last update was in January.

On the main topic, thank you for letting us know. This will allow us to prepare for the change. I, for example, will have to test the app+extension combo with our team members before this massive change occurs, where most of them are not even used to have a password manager (we use Buttercup in combination with Google Drive to share a common vault for some accesses).

[perry-mitchell]

Any chance to see a new release before this big overhaul? The last update was in January.

How do you mean? Like a standalone copy you can run locally to test? If so, yes, I could provide that.

I, for example, will have to test the app+extension combo with our team members before this massive change occurs, where most of them are not even used to have a password manager (we use Buttercup in combination with Google Drive to share a common vault for some accesses).

I understand. And props to you for possibly being the one to push the idea!

Regarding the challenge of further complicating your team's workflow, might I ask for more details as to how you're working now?

Is it so that your team is only using the extension, and not the desktop app? If so this is quite interesting to hear as I've felt for a while that the desktop app is the foundation of the suite, being that it seems to have the highest download counts out of all of our apps. It's also handy, imo, as many people store credentials in Buttercup that pertain to services that are not online, such as SSH keys and the like.

If they're already using the desktop app then they'll just need to enable 1 setting for this to work.. otherwise yes, they'd need to do the following:

1. Install the desktop app
2. Connect their Google vault there
3. Update the browser extension (this is automatic)
4. Connect the extension to the desktop app

After that, the state of the vaults in the extension mirrors the state in the desktop app.

[Oxalin]

Exactly, we are not using the desktop app... for now. I remember trying to use the app in combination with the extension (connect the extension to the app), but there was something I was not able to complete at the time and I had put this scenario aside.

Now, since the desktop app will mostly be mandatory, I gave it another try earlier today. So here is the thing: I'm able to connect the extension to the desktop app under Firefox (Chrome is another story, it complains that it failed to fetch "Failed connecting local vault An error occurred when connecting: Failed to fetch"). However, this only allows me to browser the local files on my computer, not the vaults I have added from Google Drive to the desktop app. So now I remember why it was a no go scenario: since we want to share a common vault across our team, we need to have access to the vaults added from Google Drive through the extension.

The only solution I see for now is to have the desktop app with our common vault (added through Google Drive) and go get the credentials directly in the app when needed. The extension would then be useless.

However, from your answer, I'm not sure I'm understanding correctly: should the extension (version 2.25.3) be able to fetch/add any vault already added to the desktop app?

[perry-mitchell]

No, the v2 extension does not have such functionality. I'm currently in the process of rolling out the desktop update first, and once it's been out a while I'll release the browser extension update.

You'll add vaults, after this release, to the desktop app only. If you need the browser extension it will authenticate with the desktop app and you'll have the same vaults and entries available in the browser as you do in the desktop app. If you unlock a vault on desktop its contents will be available to the browser immediately.

The extension certainly won't be useless after this. Collecting logins as you sign in, providing automatic logins when login forms appear etc.. there are a lot of reasons to still use the extension. It's a more complete experience when authenticating. Just depends on what logins you decide to store in Buttercup I suppose.

[julianpoemp]

I don't use the Browser extension, but would it be a good idea to release an update of the extension with a warning banner about the next breaking changes? That way users would have the change to disable auto-update for the extension.

Another option could be to release the extension with a new name, so that the old one still exists, but does not get any updates any more. What do you think?

I'm just thinking about users who don't use social media or Github and could be upset if their vaults do not work any more.

[perry-mitchell]

The warning banner is a good idea.

Unfortunately releasing under a new name is not an option, as I'd want the users to carry over to the new extension. The Google manifest update is not optional so holding back is not a permanent solution for anyone I'm afraid.

There's also a lot invested in the trust and links to the current published extension that I can't easily rebuild.

I'm not sure there's an easier way to do this than to just give everyone fair warning, which I've done so across Twitter and other social media platforms. The banner might help a touch so I'll look into that next.

[perry-mitchell]

I've added the banner, and updated buttercup core, the google drive client and the dropbox client, in the current stable extension. I'll release this tomorrow.

[daveneeley]

Thanks for publishing the update banner in the extension itself! I wasn't aware, but it's a good change for me at least! I could never get web-dav in the browser extension to sync correctly with self-hosted nextcloud. So I've kept the browser extension installed but have never been able to sync it with my main vaults.

[phunehehe]

Hi! First of all thanks for Buttercup and thanks for this discussion, and the banner!

Would you be okay with publishing a new Firefox extension using the old version? That will be very helpful for non-Chrome users who prefer to use the browser extension without the desktop app 🤚 I imagine it will keep working just fine for a long time, and we may be able to figure something else out when e.g. Google changes the API in an incompatible way.

[perry-mitchell]

This is very interesting. I'll consider it.

It's not super likely, unfortunately, as both Chrome and Edge will require manifest v3 and maintaining the current extension code would be a huge pain (it's messy).

I also don't want to confuse users.. if they see multiple extensions in the store that might just create a split in installs that isn't helpful in migrating everyone to the new version, which has to happen at some point anyway.

[ldexterldesign]

👋

All good

The extension has (always had) memory issues for me, specifically in Vivaldi, so be great to not deal with that again - hadn't got around to following up 😖

TBH only use the extension to auto-login and if there's MFA stuff going on too then have to be in the desktop app anyway 😔

Onward!

🫶

[perry-mitchell]

The extension will support MFA (OTP) items in v3, in case that helps anyone's workflow. So they can be entered mostly automatically too.

[pijander]

This happend because you join to a gigant that, at the end, you belong them. In my case i use firefox and probably i'll change my password manager if you join to google in that way. Embrace free software and everything will be fine.

[pijander]

I do NOT want to detract from your software, of course. But if you embrace the idea of leaning on a free group of workers who don't jump through the hoops of software giants, you avoid these stories of changing your software core whenever others want. What are the advantages of following what Google does and says? I think it's pretty debatable. And yes, at the heart of it lies the idea of free software and not dependent on third parties. Thank you so much for your quick feedback.

[ldexterldesign]

👋 @pijander,

Do you have concrete options/solutions rather than simply presenting problems (which sound quite idealogical I hope you don't mind me saying)?: if yes then I would be interested to know else I would invite you to move on to other software that suits you better 🙂

Regards

[julianpoemp]

From the point of view of a developer who maintains a big project (almost) alone I completely understand why Perry tries to optimize the development in that way that he don't have to maintain different code bases for different browsers.

Source: https://gs.statcounter.com/browser-market-share#monthly-202001-202310

If you have a look on the latest browser statistics you can see that > 70% of the internet users use Chromium browsers (Chrome, Opera, Edge). If you manage a project (almost) alone and you want your project to support all browsers it's obvious to follow the limitations of the most relevant browsers (here Chromium) first and the others second (Safari, Firefox).

It's not about "jumping through the hoops of software giants", it's about the best trade-off with the existing resources (time/personal/finance).

Btw. you'll never find a modern software project that is independent on third parties and "software giants": ReactJS is made by facebook and Angular by Google, Typescript by Microsoft a.s.o.

[papiguy]

@perry-mitchell is there a status on when this extension will be released? Is there anything I can do to support the development of this extension?

[perry-mitchell]

@perry-mitchell is there a status on when this extension will be released? Is there anything I can do to support the development of this extension?

There's no hard deadline right now besides my availability. Google has pushed back the forced rollover time to the point where the due date is no longer clear. I think it's better to be safe than sorry and just release it ASAP.

That being said I expect it to perhaps go out this coming week for Edge users, then Chrome and Firefox the following week.

Regarding assisting the development of the extension, we could always use PRs and bug fixes if you find something you want to work on. If you don't want to contribute in that way or feel you're unable to, simply keeping a trusty eye out when we release for bugs is a huge help. 🙏

[perry-mitchell]

Due to the large amount of people commenting about how they thoroughly dislike the requirement to have our desktop app installed, I will no longer be explaining myself nor the direction of the project in each opened issue and comment. I'll simply link to this thread.

There's a lot of moving parts of a project like this, trust me. Do I choose to change such large aspects of this project on a whim? No. This was a calculated change and was done partly because our hand was forced here. That, and me running this project mostly by myself means:

• I can't focus on every aspect of the product how I'd like to.
• I need to make decisions to implement ideas in a way that is possible given the amount of time I have to contribute.
• I get to choose - if you want a say in how this project is run, and where it's going from here - you need to contribute.

To you, reader - I'm sorry if this comes off as bloated or uncaring or unprofessional - but I'm honestly doing what I can with a product I love, and want to be able to love years from now.

[jyelland]

I started using the desktop app for local vault access more than 6 months ago, as a workaround to #452. It was annoying for a day or two, then I got used to it and forgot all about it. I love using Buttercup, I tell others to use it, and I hope that those who are frustrated by the mild nconvenience can be convinced to stay :)

[perry-mitchell]

Very nice to hear.. thanks for taking the time to write this!

[Oxalin]

I've been dealing with the migration of the new version at work this week and I'm glad I was aware it was coming and to have contributed to the development of the "launch in background" feature. When the update began to rollout, I was able to inform my team members of the change. There were some hiccups, the first one being that the extension was updated, but there was no mention of the required desktop application version (the Notifications welcoming us to V3 states that the desktop application is needed, but not the minimal version supported).

That being said, I think we can do better to inform the user about the dependency between the browser extension and the desktop app. I'll open an issue about it.