Enhancement - filemode that handles original filenames that don't exist

[xaeiougit]

Hi,

I'd like to suggest an enhancement, a filemode called, say, "new", that makes it easier to deal with season packs and/or do cross-seeding (seed the same torrents on multiple sites) when you are adding new torrent files.

Let's say for example there is a currently playing season of 26 episodes which you are downloading every week.

You define a Format Expression, something like "{s00e00} {t.colon(' - ')}" to get rid of any scene tags etc and start saving those files according to this format.

At the end of a season, the individual episode torrents are typically deleted from the server and a season pack/batch torrent is released. You want to continue to seed the whole season without re-downloading everything, so you add this batch torrent in the paused state, and want to point it at your previously downloaded but already re-named episodes.

The logic for "new" filemode would be:

"Update the filename in state/torrents.fastresume according to the Format Expression"

So in this mode FileBotTool would perform the same operation as manually going to the "Files" tab for the season pack/batch torrent in the lower window of deluge and renaming all the files one by one.

Currently if you try to do this with FileBotTool is will fail because it can't find the old filename. I've played around with the current "filemode" and "on conflict" options and can't seem to find a workflow to easily accomplish this without having a lot of symlinks around.

After running filebottool in this mode, the user would simply "recheck" this new batch torrent and un-pause to start seeding.

Not only would this be useful for handling season packs, but also if you want to cross-seed a batch torrent on multiple torrent sites in cases where the uploader has already renamed all the files according to their own taste. The only way to do this currently is to painstakingly rename the files one-by-one when you add the batch torrent. Or you could use a plugin like "batch renamer" but this doesn't have all the nice hooks into TVDB etc.

Sorry if there is a workflow that already accomplishes this, but I did play around quite a bit and couldn't find a simple way to do it.

EDIT - I understand with the way this plugin depends on FileBot, doing this in FileBotTool may not be possible. But I thought I'd ask anyway :)

[Laharah]

Since filebot is in charge of matching files to episodes, there's no easy way to do this. If there aren't any actual files to match against filebot will just throw an error.

I wrote down some brain-storming below with my reasoning. Read on if you're interested but feel free to ignore it.

TL;DR: Good and useful idea, can't work as is. Will try to think of a better way to do it.

---

Looking through the code and playing around a bit with filebot, it looks like you could trick filebot into doing the matching by placing dummy files where the torrent files should be, then letting filebot match those. The problem is that if you tell filebot to move those files, the dummy files will overwrite the original and already named files.

To make this work, the tool would have to be modified to help non-finished torrents get matched and then do the mapping itself. the workflow would look like this:

1. Split torrent files into files that are finished and files that don't exist yet
2. run filebot and let it move the finished files to the correct location
3. store the movements
4. place dummy files for the unfinished files
5. do a dry run on the dummy files (filebot can't be allowed to move them or they will overwrite the existing ones)
6. store the proposed matches/movements
7. remove the dummy files
8. combine the 2 matches into a single rename operation
9. rename and move the torrent locations in deluge
10. force a recheck on the torrent.

Right now I can see 2 problems with this, one is no big deal the other can be a pretty big issue. The first is that this would be a one way operation. Right now, the revert function relies on filebot's database of renames. Since filebot doesn't do the actual moving, there will be no records for the dummy files being moved so you won't be able to undo if something goes wrong. That's not great, but not the worst.

The bigger problem is fragility. Right now filebottool keeps things robust by making all of it's operations atomic. One operation maps to a single filebot call. This would spread a rename over multiple calls which can cause some issues. Lets say you want to rename a season of House of Cards. Lets say you have already downloaded the first 3 episodes and they're renamed and where you want them. The new torrent has those same three episodes and 2 more to finish out the season. You paused the 3 files you already had and only downloaded the 2 you didn't and you go to combine them. Because the rename operation now gets run twice (once for the new files and once for the dummy files), there's a decent chance that the dummy files get matched to the UK version of House of Cards and the new files get matched to the US version of House of Cards. Now you have torrent where half the files are in one folder and the other half are in another. Depending on how you format that it's likely you'd get some monstrosity where you have a file structure like this:

Shows
└── House of Cards (US)/
    ├── Shows/
    │   └── House of Cards (UK)/
    │       ├── s01e04 - Episode 4.mkv
    │       └── s01e05 - Episode 5.mkv
    ├── s01e01 - Episode 1.mkv
    ├── s01e02 - Episode 2.mkv
    └── s01e03 - Episode 3.mkv

This is all made worse by the fact that revert would no longer work.

This could still be just as bad in your simpler scenario, where you have say 12 episodes that were all independently renamed. If any of the renames matched the wrong show, season, or episode; or if the format was slightly different, the merge won't work, and you'll get duplicated files, or swapped files that the force re-check will erase and force a re-download.

I'm not saying no to your idea, but I am saying no to this implementation. As it is, it's too fragile, can delete good data, and can't really be undone.

I'll think about it more and come back to this. Maybe some kind of repack mode that lets you choose a folder and does dry runs on the folder and the new torrent to match using the end-points.

[xaeiougit]

Thanks for your detailed response, what you've said in your brainstorming makes perfect sense. I admit I suspected there may be fragility issues like the one's you've raised, especially when dealing with the more general case of having partially downloaded files etc and the possibility of multiple matches from the database.

When doing this "manually", assisted with the bulk renamer plugin, I have to be very careful about the order in which I do things and move files otherwise things get overwritten and I could lose my originally downloaded files. Generally what I do is:

1. Add the batch torrent paused and stored in a new directory
2. Rename the files with bulk renamer plugin (this changes torrents.fastresume only)
3. Copy the already downloaded files into the new directory (keeping a backup)
4. Run a re-check and if all is as expected then un-pause

Steps 2-4 are very much a manual process where I have to carefully check nothing has gone wrong, and if it has repeat them, sometimes manually renaming individual files using the inbuilt deluge rename rather than the bulk renamer plugin.

Thanks for considering it anyway. If there any hooks into filebot that would make things easier please let me know - I'm a paid user of filebot (mostly motivated by my use of filebottool) and can certainly add a "me too" to any feature requests on the filebot forums :)