Replace Whonix in favor of arti in `sd-proxy` by deeplow · Pull Request #2561 · freedomofpress/securedrop-client

deeplow · 2025-07-23T16:00:13Z

Description

Towards freedomofpress/securedrop-workstation#456

Test Plan

Preparation:

make dev with Prepares path for sd-whonix whonix removal securedrop-workstation#1414 (companion workstation PR)
build and install arti in sd-small-bookworm-template under /usr/bin/arti
- either build arti manually (git checkout + cargo build --release --locked --bin arti)
- or use Build a securedrop-arti package #2644
build debs and install in sd-small-bookworm-template (try-client-pr should work here)
shut down sd-small-bookworm-template
restart sd-proxy

Test:

run client application and ensure connections are working

legoktm · 2025-08-20T13:53:58Z

Can we do the Python key/conversion part as a one-time systemd service on VM boot (before arti starts) instead of for every proxy request?

Companion PR for client change [1] with the aim of deprecating whonix, in favor of delegating the tor connectivity aspect to sd-proxy running arti. Changes introduced: 1. sd-proxy connects to 'sys-firewall' directly: since sd-proxy is now handling tor connections, it must connect directly to the internet. It keeps the original goal of preventing the client from being able to connect to arbitrary domains. This is also something that sd-whonix did not guarantee (it could connect to arbitrary domains, albeit via Tor). 2. sd-whonix has access to onion service auth key Access done via qubes feature vm-config.SD_PROXY_ORIGIN_KEY 3. sd-whonix removed 4. Install `securedrop-proxy-config` in sd-proxy template [1]: freedomofpress/securedrop-client#2561 Test in ci: openqa

deeplow · 2025-08-20T20:20:36Z

Can we do the Python key/conversion part as a one-time systemd service on VM boot (before arti starts) instead of for every proxy request?

As mentioned earlier, this is exactly what I was trying to do after I got the base thing working. Even though we're still missing tests (fixing the old and adding new) and checking in on the apparmor situation, I'd say this is ready for initial review. But under the expectation that there will be further changes, if that sounds good @legoktm.

deeplow · 2025-08-20T20:23:39Z

@cfm and @legoktm, what should we do regarding securedrop-qubesdb-tools? Its only use was for whonix-config and the file template mechanism was not expressive enough for what we needed here.

legoktm · 2025-08-20T20:52:15Z

what should we do regarding securedrop-qubesdb-tools

rm -rf IMO if there's no use for it, we can always restore it later on if we do have another use case.

legoktm

Just a first pass review, I will try to test this tomorrow.

cfm · 2025-08-20T22:33:05Z

On Wed, Aug 20, 2025 at 01:52:36PM -0700, Kunal Mehta wrote: > what should we do regarding securedrop-qubesdb-tools `rm -rf` IMO if there's no use for it, we can always restore it later on if we do have another use case.

Not only that: #2032 came about entirely to help configure Tor on the Whonix gateway. If we no longer need to do QubesDB configuration at boot, because everything we need to configure from QubesDB can do it at runtime, that's a Good Thing. :-)

Per suggestion in [1], and following the conclusions in this pull request [2] [1]: #2561 (comment) [2]: freedomofpress/securedrop#7404

Companion PR for client change [1] with the aim of deprecating whonix, in favor of delegating the tor connectivity aspect to sd-proxy running arti. Changes introduced: 1. sd-proxy connects to 'sys-firewall' directly: since sd-proxy is now handling tor connections, it must connect directly to the internet. It keeps the original goal of preventing the client from being able to connect to arbitrary domains. This is also something that sd-whonix did not guarantee (it could connect to arbitrary domains, albeit via Tor). 2. sd-whonix has access to onion service auth key Access done via qubes feature vm-config.SD_PROXY_ORIGIN_KEY 3. sd-whonix removed 4. Install `securedrop-proxy-config` in sd-proxy template [1]: freedomofpress/securedrop-client#2561 Test in ci: openqa

deeplow · 2025-08-21T19:27:49Z

@legoktm FYI, the complementary workstation PR is not yet fully working. But to test this you can manually set the following:

sd-proxy
- netvm: sys-firewall
- add service securedrop-proxy-onion-config

deeplow · 2025-08-21T20:11:43Z

On Wed, Aug 20, 2025 at 01:52:36PM -0700, Kunal Mehta wrote: > what should we do regarding securedrop-qubesdb-tools rm -rf IMO if there's no use for it, we can always restore it later on if we do have another use case.
Not only that: #2032 came about entirely to help configure Tor on the Whonix gateway. If we no longer need to do QubesDB configuration at boot, because everything we need to configure from QubesDB can do it at runtime, that's a Good Thing. :-)

Great! I'll remove it, then. I've added it as a TODO item in the first post.

Companion PR for client change [1] with the aim of deprecating whonix, in favor of delegating the tor connectivity aspect to sd-proxy running arti. Changes introduced: 1. sd-proxy connects to 'sys-firewall' directly: since sd-proxy is now handling tor connections, it must connect directly to the internet. It keeps the original goal of preventing the client from being able to connect to arbitrary domains. This is also something that sd-whonix did not guarantee (it could connect to arbitrary domains, albeit via Tor). 2. sd-proxy has access to onion service auth key Access done via qubes feature vm-config.SD_PROXY_ORIGIN_KEY [1]: freedomofpress/securedrop-client#2561 Test in ci: openqa

legoktm · 2025-08-25T21:00:31Z

I got all of CI passing today; I meant to rebase it into a cleaner git history but didn't get to that today; will do so tomorrow and mark this as formally ready for review. I'll also test integration with #2644.

legoktm · 2025-08-26T17:20:29Z

I got this working today and ran into two things:

Arti is running in all the VMs and template instead of just sd-proxy; I think we need to limit it via qubes-services
We're creating ~/.local/share/arti as world-readable and Arti doesn't like that. We also need to switch it over to use the _arti user, so I can figure out permissions with that.

Companion PR for client change [1] with the aim of deprecating whonix, in favor of delegating the tor connectivity aspect to sd-proxy running arti. Changes introduced: 1. sd-proxy connects to 'sys-firewall' directly: since sd-proxy is now handling tor connections, it must connect directly to the internet. It keeps the original goal of preventing the client from being able to connect to arbitrary domains. This is also something that sd-whonix did not guarantee (it could connect to arbitrary domains, albeit via Tor). 2. sd-proxy has access to onion service auth key Access done via qubes feature vm-config.SD_PROXY_ORIGIN_KEY [1]: freedomofpress/securedrop-client#2561 Test in ci: openqa

legoktm · 2025-08-26T20:20:50Z

I didn't realize that systemd-sysusers doesn't create the home directory of /var/lib/arti (because of course, that's the responsibility for systemd-homed), but there's actually a simpler solution, which is to set StateDirectory=arti (shout out to https://bbs.archlinux.org/viewtopic.php?pid=2016633#p2016633 for mentioning that), which when combined with StateDirectoryMode=0700 enforces permissions too.

legoktm · 2025-08-26T20:46:10Z

I've cleaned up the Git history and verified that it works as expected, should be actually ready for review :)

Run `make build-debs-arti` to build a `securedrop-arti` package. We build it separately since it's pretty slow with high resource requirements and will be changing at a different schedule than our current release schedule (like how we handle Tor on the server). The version is set in arti/debian/changelog, it clones the upstream Git repo, verifies the PGP signature against arti/upstream-keys (fetched from keys.openpgp.org), builds the `arti` binary and includes it in a Debian package. The package creates an `_arti` user with the same home directory as Arti upstream intends to use. No systemd unit is included in this commit, for now it's being shipped in the securedrop-proxy package (see #2561).

cfm

This works great. Just a few questions and suggestions inline.

Assuming running in a qube connected to sys-firewall (will be a workstation PR of its own) and with arti running in proxy mode, this allows the proxy to connect direct to the onion address. This is coupled with workstation PR that: - sets sd-proxy to connect to sys-firewall instead of sys-vpn - installs arti package in sd-proxy's template - starts arti in proxy mode in sd-proxy on startup - sets via qubesdb vm-config.SD_PROXY_ORIGIN value Setting the onion service authentication key is done in a follow-up. Developers can set a `DISABLE_TOR` environment variable to avoid using a SOCKS proxy, and both the client and app do so. Towards freedomofpress/securedrop-workstation#456

@lsd-cat

Onion service key passed as "SD_PROXY_ORIGIN_KEY" via qubesdb (production) or environment variables (testing), following project standards. The "get_env()" function was duplicated from securedrop-qubesdb-tools, since it was not usable directly as a library. On top of that, its only use-case was securedrop-whonix-config, which is now going to be removed. We need to convert the key from the CTor format to Arti, which isn't yet available from upstream. This is based on @lsd-cat's initial conversion script[1]. For more context see [2]. Fixes freedomofpress/securedrop-workstation#1382. [1]: https://gist.github.com/lsd-cat/d3bd05412d83628be7145462081462f9 [2]: freedomofpress/securedrop-workstation#1382 (comment) Co-authored-by: lsd-cat <giulio@gmx.com>

The previous qubesdb-templated file was replaced with a boilerplate python script, since a templated file is not good enough for the key conversions necessary to convert an ctor client key to an arti one. A securedrop-arti systemd service was added, it's based on upstream's [1], but systemd mitigations regarding read/read-write directories were not included due to not working out-of-the box, combined with the fact that the proxy is expected not to keep any state. It runs as a separate `_arti` user. For now we're not protecting it with apparmor, but can take advantage of whatever upstream does later. [1]: https://gitlab.torproject.org/tpo/core/arti/-/blob/084c2dee/debian/systemd/arti.service

* Remove a postinst edge-case since securedrop-workstation-config is no longer installed on any whonix-based VM. * Remove from mermaid diagrams; this was checked against the proxy v3 implementation in [1] and adapted from the previous diagrams that had added a bit more detail to what was originally proposed. [1]: #2453

addressed my own feedback

cfm

All persuasive! Thanks, @deeplow and @legoktm. Merging to unblock freedomofpress/securedrop-workstation#1414 per freedomofpress/securedrop-workstation#1414 (comment).

deeplow added this to SecureDrop Jul 23, 2025

deeplow moved this to In Progress in SecureDrop Jul 23, 2025

cfm mentioned this pull request Jul 24, 2025

Implement v3 proxy architecture #2453

Open

nathandyer moved this from In Progress to Next sprint candidates in SecureDrop Aug 5, 2025

nathandyer moved this from Next sprint candidates to Ready to go in SecureDrop Aug 18, 2025

deeplow moved this from Ready to go to In Progress in SecureDrop Aug 19, 2025

deeplow force-pushed the whonix-deprecation branch from 9a008e1 to e277488 Compare August 19, 2025 16:34

deeplow force-pushed the whonix-deprecation branch 2 times, most recently from 8c25c9d to 9317ef5 Compare August 20, 2025 18:58

deeplow mentioned this pull request Aug 20, 2025

Prepares path for sd-whonix whonix removal freedomofpress/securedrop-workstation#1414

Merged

2 tasks

deeplow marked this pull request as ready for review August 20, 2025 20:08

deeplow requested a review from a team as a code owner August 20, 2025 20:08

deeplow moved this from In Progress to Ready For Review in SecureDrop Aug 20, 2025

legoktm previously requested changes Aug 20, 2025

View reviewed changes

deeplow added a commit that referenced this pull request Aug 21, 2025

Change onion setup systemd service to Type=exec

fc7dfa4

Per suggestion in [1], and following the conclusions in this pull request [2] [1]: #2561 (comment) [2]: freedomofpress/securedrop#7404

nathandyer moved this from Ready For Review to Under Review in SecureDrop Aug 21, 2025

deeplow force-pushed the whonix-deprecation branch from 129f8ec to 5044df3 Compare August 21, 2025 20:06

legoktm force-pushed the whonix-deprecation branch from c0a3ccc to edce2a0 Compare August 26, 2025 20:45

legoktm moved this from In Progress to Ready For Review in SecureDrop Aug 26, 2025

legoktm force-pushed the whonix-deprecation branch from edce2a0 to 1aa0d25 Compare August 27, 2025 16:08

cfm moved this from Ready For Review to Under Review in SecureDrop Aug 27, 2025

cfm requested changes Aug 28, 2025

View reviewed changes

Comment thread app/src/main/proxy.ts

Comment thread proxy/src/main.rs

Comment thread proxy/configure_onion_service.py Outdated

Comment thread proxy/configure_onion_service.py

Comment thread proxy/configure_onion_service.py

cfm assigned legoktm and unassigned cfm Aug 28, 2025

deeplow and others added 5 commits August 28, 2025 16:01

Vet new tokio-socks dependency

cd4df15

legoktm force-pushed the whonix-deprecation branch from 1aa0d25 to d5fdbb4 Compare August 28, 2025 20:03

cfm approved these changes Aug 28, 2025

View reviewed changes

cfm added this pull request to the merge queue Aug 28, 2025

Merged via the queue into main with commit 3495fc8 Aug 28, 2025
61 checks passed

cfm deleted the whonix-deprecation branch August 28, 2025 21:00

github-project-automation bot moved this from Under Review to Done in SecureDrop Aug 28, 2025

nathandyer removed this from SecureDrop Sep 2, 2025

deeplow added this to the 0.17.0 milestone Sep 2, 2025

deeplow mentioned this pull request Sep 5, 2025

Replace Arti with CTor #2676

Merged

6 tasks

Conversation

deeplow commented Jul 23, 2025 • edited by cfm Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Test Plan

Uh oh!

legoktm commented Aug 20, 2025

Uh oh!

deeplow commented Aug 20, 2025

Uh oh!

deeplow commented Aug 20, 2025

Uh oh!

legoktm commented Aug 20, 2025

Uh oh!

legoktm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cfm commented Aug 20, 2025 via email

Uh oh!

deeplow commented Aug 21, 2025

Uh oh!

deeplow commented Aug 21, 2025

Uh oh!

legoktm commented Aug 25, 2025

Uh oh!

legoktm commented Aug 26, 2025

Uh oh!

legoktm commented Aug 26, 2025

Uh oh!

legoktm commented Aug 26, 2025

Uh oh!

cfm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cfm left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

deeplow commented Jul 23, 2025 •

edited by cfm

Loading

cfm left a comment •

edited

Loading