remote file source

[stag-enterprises]

bit of a weird situation here: my storage box provider caps each individual machine to 8tb, and adding more is only possible with another server.

ideally i would be able to access each machine from the same instance. i know i can do this with rclone's docker plugin and using something like a SFTP volume mount, but the storage box's docker interface doesn't allow me to install that. if this isn't possible then i'll probably just move copyparty to an actual server.

unrelated but what is "hovudstraum"? i couldn't find any results by searching lol

[9001]

cool scenario! alright, say you have two servers, ST1 running copyparty and ST2 with additional storage. I think the most surefire way to solve this is with nbd, but it will have several disadvantages:

• you'll need root on both ST1 and ST2
• all traffic going to ST1 for accessing data on ST2 will cause at least twice the amount of network traffic going through ST1; first to grab the data from ST2, and then to ship it back to you
• there's more moving parts, and you'll have to script something for this on your own (unless there's some fancy frontends for this, which there probably is, somewhere)
• you would have ST2 appearing in a subfolder of ST1, so you'd have two separate HDDs, where one would likely be noticeably slower than the other, and you won't have natural "spillover", so if ST1 becomes full it won't automatically start using ST2 for additional space

there is another approach, mergerfs which sounds cool; it merges two filesystems so they look like one, and seamlessly handles copying between the blockdevices, so you won't have that last drawback in that case -- but the only experience I have with mergerfs is from people complaining that copyparty runs into issues on it xD

i think mergerfs should be fine as long as you keep the up2k databases on ST1 and only use it for data, but i'm still wary of recommending it, since I don't know how well it deals with the chunked uploading and stuff... Meanwhile, nbd is way simpler tech, and way easier to reason about and be confident that it will definitely work as intended, so let's continue with the nbd idea...

run the following two commands on ST1, in two separate shells, to share /dev/sdc2 with ST1:

ssh -L 5951:127.0.0.1:5951 root@ST2 'echo 127.0.0.1/32 >/tmp/nbd-allowlist; killall nbd-server; nbd-server -M 1 -l /tmp/nbd-whitelist -d 127.0.0.1:5951 /dev/sdc2'
rmmod nbd; modprobe nbd max_part=8; nbd-client 127.0.0.1 5951 /dev/nbd0 -b 4096; lsblk

and then to explain what we just did --

on ST1, it created an ssh-tunnel for port 5951 to ST2 (so anything connecting to port 5951 on ST1 automagically connects to port 5951 on ST2 instead), and the tunnel performed some setup on ST2 before launching the nbd-server:

• create an allowlist with 127.0.0.1 in it
• kill any other nbd-servers, because if the tunnel goes down, you'll have a zombie nbd-server running on ST2 which prevents using port 5951 again
• launch nbd-server and listen on port 5951

the second command connected to the nbd-server through port 5951 on ST1 (which is actually port 5951 on ST2), which created a virtual blockdevice on ST1 which mirrors the contents of the one on ST2

you'll see nbd0 listed on ST1 as a new blockdevice, with the same size as the physical device on ST2. At this point you can either mount /dev/nbd0 /mnt/st2 to mount the disk as a local filesystem, or you can run luks-open on ST1 if you want data-at-rest encryption. I'd recommend doing the luks-open on ST1, since that way, ST2 will never see the plaintext data, so if ST2 is compromised then it's entirely a useless black box. Either way, the ssh tunnel for the nbd traffic will handle data-in-transit encryption, which is good, because luks alone is not necessarily sufficient for that; you'd have to use the correct ciphers for luks to prevent IV attacks, and I'm not enough of a security expert to say which ones qualify :-)

one super-important thing is that /dev/sdc2 (the example physical device on ST2) must not be in use anywhere else, including on ST2 itself! remember to unmount any filesystems before connecting to it from ST1 with nbd, since if the same blockdevice is mounted in several places at once, you'll definitely hit some corruption! and this is true in general, not just for nbd :>

unrelated but what is "hovudstraum"? i couldn't find any results by searching lol

"primary stream" in old norwegian :D started using it as a joke but my coworkers love it hehe

[stag-enterprises]

thanks for your detailed response. i'll look into nbd, seems promising.

"primary stream" in old norwegian :D started using it as a joke but my coworkers love it hehe

it's a pretty cool name