Slow file uploads (seemingly) due to "attempt to write a readonly database"

[tlemotlemo]

Hey, I'm putting this in Q&A because I assume is is the proper place to put it.

I just installed copyparty on my server and it's been going great! But I'm running into a bottleneck when I try to upload files to the server. It fails at first, showing the error below, waits for a few seconds, fixes the error, then finishes uploading successfully.

server rejected upload (chunk 0 of 1);
will retry later

file: (file)

error 500:

attempt to write a readonly database
URL: (url)

It's not stopping me from uploading entirely, but it's making my file uploads slower.

I'm assuming this is a problem on my end. I'm running copyparty as a systemd service using a file in /etc/systemd/, where the config file I'm using is stored in ~/.config/copyparty (which I linked below). Nothing else is wrong as far as I've looked.

copyparty-main.conf.txt

[9001]

Some context: Copyparty uses a handful of databases; there's a few in ~/.config/copyparty/, but the one that's (almost definitely) related to this would be the one that's at the top-level of the volume you are uploading into. For example, assuming you're uploading to /music then the relevant database would be /home/tlemo/Music/.hist/up2k.db.

The issue is probably one of the following:

• The database is located on a filesystem which does not have all the features that sqlite requires to function; this probably means that you were uploading to / (/ripped) because there are never any such issues with traditional linux filesystems (as used for home partitions)
• The filesystem you have the database on is experiencing corruption, usually due to some sort of hardware issue

So the most likely reason is that you were uploading to / and the underlying filesystem of /ripped is causing issues. Can you share some details about that? How is it mounted, what is it connected to, what is it?

Could also be smart to look for any suspicious messages in the copyparty log, it usually has some interesting details -- but for this specific case it is not so important

[tlemotlemo]

Can you share some details about that? How is it mounted, what is it connected to, what is it?

Sure! /ripped is just a folder in my Linux server's root, it's not a mounted or symlinked file system. The files in my home directory (/music, etc.) are the same, just normal folders and not mountpoints.

I'm not sure if this is true (I'm new to configuring systemd services) but the systemd service I'm using for copyparty is probably running under my user on the server. And so your response made me realize that having /ripped be read-only to my account was probably causing the issue. So I created a new group that owns the directory, and made myself a member of it. I'm still getting the error, though, after logging out, restarting the service, and running systemctl daemon-reload.

Could also be smart to look for any suspicious messages in the copyparty log, it usually has some interesting details -- but for this specific case it is not so important

I did figure out how to check out the log. Nothing's catching my eye but I also have no SQL knowledge whatsoever.

Nov 06 10:32:31 tincan python3[13326]: [OperationalError] attempt to write a readonly database, '/music/yt'
Nov 06 10:32:31 tincan python3[13326]: 10:32:23.966 192.168.1.219 57656    LOG: POST /music/yt/ @tlemo
Nov 06 10:32:31 tincan python3[13326]: 10:32:23.968 192.168.1.219 57656    LOG: writing '/home/tlemo/Music/yt/Nuclear Bomb Scene ｜ TERMINATOR 2 JUDGMENT DAY (1991) Arnold Schwarzenegger, Movie CLIP HD [-Fx-TP5KBZE].webm.PARTIAL' [0]+64199553 #0+62 SMgs9NYCvLLB0LQ bxtMvP6Q8 egs5qXzAp CsUnipdXe BxHHDUrgW pxO4Nz5HV 8zxd1xJay s9TQERPa3 A-hFzCUnb plrdIWTq0 TG-wjdbGj f6lU3_pnp rx0K_EU5Z lcWmASMrt IF7dfNdUE VgOzMQf44 Whf71cE5n AafSAl53p rl4zebKyl gGYJ7csU6 cWzO0skfb HHNXtmxC5 CPVV-jJ0Y cavk_nHUt k25GtFjX1 NfSrdmsJj rnWUyqntk KTpg4aRpV ki0ycd6ts VcFlwlrGI t3JWI7nlp EuyeWm_5d GBBv-Kezh yAHcwX57j b_VodseTN 7Hb48ZKUf 9BvNAOWHi 17BRpI7iH XQ1uEyDlR VYWIBo-Zf cGblhc-e9 zICYyNW4C DXaSlP37C O9R99gH6- bAiFA3V0M kXCAtj1kl 6BrKoKQAN YM6ZbjXZ0 Tr-xZ7mSp bIKaYU-ZF yVD9gPLuV qUmFHwErt --f8jJpw8 vXT_EbeRz ZqtHlVCEG n1Min8JPh 2lM9_2GlI pXtQiO0zt -5QAVYJHO JB3Ys87mp dNLqjsfbS tib2W_x0O
Nov 06 10:32:31 tincan python3[13326]: 10:32:24.358 192.168.1.219 56860    BTW: got 10 more chunks, 23 left
Nov 06 10:32:31 tincan python3[13326]: 10:32:25.384 192.168.1.219 56860    BTW: got 11 more chunks, 12 left
Nov 06 10:32:31 tincan python3[13326]: 10:32:26.074 192.168.1.219 57656    BTW: got 1 more chunks, 61 left
Nov 06 10:32:31 tincan python3[13326]: 10:32:26.410 192.168.1.219 56860    BTW: got 11 more chunks, 1 left
Nov 06 10:32:31 tincan python3[13326]: 10:32:26.453 192.168.1.219 56860    BTW: got 1 more chunks, 0 left
Nov 06 10:32:31 tincan python3[13326]: 10:32:27.133 up2k                   LOG: no more chunks, setting times (1762446747, 1690138377) (59234270) on '/home/tlemo/Music/yt/[Vinesauce] Joel - The Worst Jumpscare In VOTV [bP3Q3jGsd1A].webm'
Nov 06 10:32:31 tincan python3[13326]: 10:32:27.160 192.168.1.219 56860   WARN: http500: httpcli.py:3044 <handle_post_binary>: broker.ask("up2k.finish_upload", ptop, wark, self.u2fh.aps).get()
Nov 06 10:32:31 tincan python3[13326]: broker_thr.py:47 <ask>: return NotExQueue(obj(*args))
Nov 06 10:32:31 tincan python3[13326]: up2k.py:3711 <finish_upload>: self._finish_upload(ptop, wark)
Nov 06 10:32:31 tincan python3[13326]: up2k.py:3759 <_finish_upload>: if self.idx_wark(vflags, *z2):
Nov 06 10:32:31 tincan python3[13326]: up2k.py:3830 <idx_wark>: self.db_add(
Nov 06 10:32:31 tincan python3[13326]: up2k.py:3904 <db_add>: self.db_rm(db, vflags, rd, fn, sz)
Nov 06 10:32:31 tincan python3[13326]: up2k.py:3866 <db_rm>: r = db.execute(sql, s3enc(self.mem_cur, rd, fn))
Nov 06 10:32:31 tincan python3[13326]: [OperationalError] attempt to write a readonly database, '/music/yt'

[9001]

okay either the ownership or permissions of the database have somehow been changed, can you verify that the ownership of /home/tlemo/Music/.hist and all files inside are the same as that of the copyparty process? if not, are there any errors during startup, complaints about corrupted db?

about to board flight so this last update from me for next 15 hours , good luck

[tlemotlemo]

Aaand the problem solved itself?
I don't exactly remember how it went, but I think a couple days ago I noticed the .hist files of my directories were owned by the root user, so I put them under my user instead. And suddenly I can write to all the directories without delay. Even in the ones whose .hist files are still under root's ownership...