Feature Request - Just filter out betas, demos, etc. + older revisions

[SiriusVI87]

I would like to make a feature request:

I want to be able to use your tool to just filter out all the betas, demos and all the other files that would be filtered out with "--no-all", as well as older revisions of a certain rom, but keep duplicates from different regions.

The reason I want a feature like this is that in many cases, games from different regions differ in certain, and sometimes interesting ways. Therefore, I would like to end up with a setup that gives me, e.g.:

SNES (USA)
SNES (EUR)
SNES (JAP)
SNES (OTHER)

So basically I would like to use a command like this:

py generate.py -r --all-regions --no-all -d "SNES.dat" -i "C:\SNES" -o "C:\SNES\out" --no-scan -e zip --move

In my mind, this would produce 1G1R sets for each region. They'd of course be in the same folder, but sorting the files into different regions can be done easily in Windows.

[andrebrait]

This script's goal was to generate a 1 Game = 1 ROM set and it therefore tries to do exactly the opposite of what you want.

It's technically possible to do it by calling the script multiple times, each time filtering by a given region. In the end, you'd have the set you want, but the entire tool was developed around this and it would be quite hard to change that at this point.

DATROMTool would be capable of doing this, but its documentation is severely lacking at the moment and I have not implemented file management other than for the 1g1r mode, so it would only be capable of giving you the DAT with the filtered data, at the moment.

[SiriusVI87]

This script's goal was to generate a 1 Game = 1 ROM set and it therefore tries to do exactly the opposite of what you want.

It's technically possible to do it by calling the script multiple times, each time filtering by a given region. In the end, you'd have the set you want, but the entire tool was developed around this and it would be quite hard to change that at this point.

DATROMTool would be capable of doing this, but its documentation is severely lacking at the moment and I have not implemented file management other than for the 1g1r mode, so it would only be capable of giving you the DAT with the filtered data, at the moment.

thx for replying. I haven't found any tools that are easy to use and capable of this type of sorting, and I don't know how hard it is to implement in your tool, that's why I asked.

I already thought about filtering each set multiple times with different priorities. It's not perfect, because regions that are not USA, EUR and JAP would still be filtered out, but I guess this is the way to go.

Thx!

[andrebrait]

Yeah, it's just an unfortunate thing at the moment but I haven't been able to dedicate time to get DATROMTool out of alpha (what works works well, but it's unpolished) and this script is quite scary to change due to it lacking proper automated testing and Python being hard to refactor safely (as all dynamically-typed interpreted languages are).

I'll see if I can get the time to make DATROMTool good. I think I'll be buying a new retro handheld so I should have a good excuse for that ;)

[SiriusVI87]

Yeah, it's just an unfortunate thing at the moment but I haven't been able to dedicate time to get DATROMTool out of alpha (what works works well, but it's unpolished) and this script is quite scary to change due to it lacking proper automated testing and Python being hard to refactor safely (as all dynamically-typed interpreted languages are).

I'll see if I can get the time to make DATROMTool good. I think I'll be buying a new retro handheld so I should have a good excuse for that ;)

Awersome! Have fun with your new handheld =)

[SiriusVI87]

I've just been checking out DATROMTool and it looks very promissing. I think it's really important that such a tool exists, even more so after unexpectedpanda has sadly decided to not work on Retool anymore.

Will DATROMTool also support Redump dat files?

[andrebrait]

Can't blame him for that. It's hard to make and maintain those tools. He did make some impressive things with his filtering and creation of a while new database for ROMs. He even went as far as putting the database on a separate repo so I could use it as well.

I think he's still maintaining the database, though, which is by far the hardest, most important and valuable part.

Regarding support for Redump, it should be easy to implement on DATROMTool because it has its own internal format for metadata. The current support for No-Intro DATs is merely a layer translating it to the internal metadata.

Output is a similar story, just backwards.

Funnily enough, the hardest part -- file management -- is fully working in DATROMTool (except for CHD). I really just need to finish polishing it and make the test suite more robust / simplify the filtering logic to allow for that.

[andrebrait]

And integrate it with unexpectedpanda's database, I mean.

[SiriusVI87]

This is exciting. I've bee using Retool and 1g1rgenerator in concert to create 1g1r sets for a few years now and it's given me excellent results for bascially all no-intro and redump sets (TOSEC, which I use for older computer systems is still an issue, though).

[andrebrait]

TOSEC IIRC requires a different way of thinking, given they index software, not only games. Many concepts don't translate so well and I think they use some elements not present in the No-Intro XML format.

[SiriusVI87]

Yeah, I'm not very well-versed when it comes to the architecture of these dats, as you might have realized. I'm just an end-user who enjoys these cool tools, haha.

I do know however, that TOSEC groups games, apps and educational content in different folders. I don't know if their dats specify if a file is an app or a game, though. Haven't been able to find anthying that automates the process of trimming down TOSEC sets.

But this is another story. It's just great to see that no-intro and redump will be fine in the future =)