Moving files within library loses ratings/playcounts

[Clementine-Issue-Importer]

From Gavin77 on January 04, 2011 17:26:06

What steps will reproduce the problem? 1. I have 2 main folders both included in the library, first is music I need to backup, 2nd is my main archive.
2. Once I've backed up some music I move it to my main archive folder.
3. The ratings and playcounts on those files are then lost. What is the expected output? What do you see instead? Ratings/playcounts should be kept intact. They are not. What version of the product are you using? On what operating system? 0.6 r2562 Kubuntu 10.10

Original issue: http://code.google.com/p/clementine-player/issues/detail?id=1241

[Clementine-Issue-Importer]

From goetzchrist on January 04, 2011 08:55:37

I don't know how does Clementine handle the moving of files, but I remember reading that Amarok did a md5sum of the files when adding them to the database. So when a new file was added, it checked if the md5sum of the new file was already in the database. This way the ratings where not lost when moving files, but not when moving the file and editing the tags at once.

[Clementine-Issue-Importer]

From keirangtp on January 05, 2011 05:14:50

As far as I know Clementine bases it's matching on filenames so moving media breaks stuff.

Sum sounds very reasonable but I'm wondering how fast calculating those really is and how would it impact Clementine's efficiency.

[Clementine-Issue-Importer]

From goetzchrist on January 05, 2011 13:13:36

I have done some tests, this is with a Intel 2160 @ 1.8 GHz:

type | # file | size | Kbps | Play time | time md5sum
-----+--------+---------+------+-----------+-------------
ac3 | 1 | 18 MB | 448 | 5:35 | real 0.093s
mp3 | 1 | 9 MB | 320 | 3:05 | real 0.038s
mp3 | 17 | 175 MB | 320 | 1:16:21 | real 0.746s
mp3 | 238 | 2100 MB | 320 | 17:45:33 | real 40.299s
ogg | 1 | 7 MB | 256 | 3:53 | real 0.034s
ogg | 9 | 83 MB | 256 | 50:20 | real 0.457s
flac | 1 | 45 MB | 620 | 10:11 | real 0.221s
flac | 8 | 250 MB | 600 | 57:44 | real 1.383s
flac | 42 | 1500 MB | 600 | 6:06:33 | real 25.978s
rar | 1 | 700 MB | - | - | real 53.141s
XviD | 1 | 700 MB | 700 | 2:18:00 | real 15.747s

An option to speed this up could be to do a check-sum of the first 100 kB or so.

The ability to rename or move the audio file without loosing the statics is in my opinion important. There are many reasons to rename/move files, like when tagging (and changing file name) with external software or organizing folders.

[Clementine-Issue-Importer]

From john.maguire on January 06, 2011 03:27:14

Labels: Component-MusicLibrary
Blockedon: 1175

[Clementine-Issue-Importer]

From keirangtp on January 06, 2011 03:40:48

Embedded ratings would be a possible solution. The questions is whether we can embed everything using the FMPS spec - rating, playcount, skipcount, score?

[Clementine-Issue-Importer]

From goetzchrist on January 06, 2011 12:23:48

I remembered another problem that (I suppose) could be resolved I the song files had an ID, like the md5sum.

When moving or renaming files, Clementine forgets the manual associated cover art. It has to be set manually again. But when the files are restored as they where before, the cover art is automatically back.

Another problem is when there are tracks in a playlist (but from the library), and they are moved or renamed, the tracks disappear from the playlist, leaving an white empty space (screenshot attached).

Attachment: clementine removed item from path.png

[Clementine-Issue-Importer]

From keirangtp on January 07, 2011 13:44:43

There is another bug about embedding covers which would help the issue just like emebedded ratings would.

[Clementine-Issue-Importer]

From goetzchrist on January 07, 2011 14:59:39

But embedding covers is not always useful, and is not the solution for this problem. Many people have the cover image in the album folder, and won't take the time to emebedded them into the music file. Also, for example, for big size covers, like 1 MB or so, it is bad idea to emebedded the cover in each ~6 MB file from an album. It is better to have a single cover file in the directory.

Instead of trying to emebedded everything into the music file to solve this problem, better to identify the audio files. Using the MD5 hash function to identify the files in the library is what many programs do. Anyway, it has to be done only once every complete scan.

[Clementine-Issue-Importer]

From marcansoft on February 03, 2011 03:20:58

I agree, I think hashing is the best solution. Not necessarily the entire file, just a chunk, but you also need to keep in mind that if the hash covers tags it will change when tags change.

I think it's worth looking at what Amarok 1.4 used to do here, I remember it working pretty well in this regard.

Currently I have to manually edit stuff in the sqlite database when I move files, which is a chore.

[Clementine-Issue-Importer]

From Yizouse on March 19, 2011 00:25:37

I'm quite impressed with this project, but this issue is preventing me from full-time adoption; I simply rely on my files being "mobile". So I did some probing.

Amarok 1.4's collection.db has a "uniqueid" table containing URL/UID mappings. The statistics table uses the UID, allowing for the desired behavior.

This UID is an md5 wrought from:

• the file's TagLib byte vector, if it exists
• the first 8192 bytes or the entire file, whichever is smaller
• file size on disk

I found this in A1.4's src/metabundle.cpp, readUniqueId(). The more involved logic of handling unique ids w/r/t file moving & copying is in src/collectiondb.cpp.

Now to look at the clementine source and see how easily a similar scheme would work with the existing code...

[Clementine-Issue-Importer]

From davidsansome on June 13, 2011 16:24:49

Issue 1988 has been merged into this issue.

[Clementine-Issue-Importer]

From tekdemo on August 27, 2011 14:55:49

Rather than embed things in files, or change the database, a logical behavior would be to add another step into the "organize files" procedure. Currently, it just moves the files, but it should do the following

• move the file to it's new location
• update the database with the new path, ratings, and other metadata
• update the current playlist with the new data, to point to the new location

This seems to be more on par with what Amarok used to do, as it never broke a playlist when organizing files. I frequently had Amarok break metadata if I moved the files manually, but never had an issue by moving things in the application itself.

[Clementine-Issue-Importer]

From dragonfear on September 10, 2011 02:18:43

Amarok's file tracking is shit.

I propose you do this: http://code.google.com/p/clementine-player/issues/detail?id=2225 It's more robust and it incorporates forward-compatible support.

[Clementine-Issue-Importer]

From dragonfear on September 10, 2011 02:27:33

Also, any solution proposed needs to:

• NOT WRITE ANYTHING TO FILES (bad for NFS, VERY DANGEROUS in any case -- Amarok 1.4 RUINED a lot of files that way)
• Take into account moving and retagging files from outside Clementine.

My proposal takes those factors into account

[Clementine-Issue-Importer]

From davidsansome on September 10, 2011 08:53:13

Issue 2225 has been merged into this issue.