Redump metadats should reference internal serial, not disc serial

[zcal]

The Redump metadat files list serials for each game. As I understand it, when scanning disc-based ROMs with RetroArch, the internal serial is taken from the disc header and compared with the DAT. It appears that the Redump metadats list the disc serials, not the internal serials. This results in failures or misidentified games when there is a mismatch between the two.

For example, Street Fighter Zero 3 in Sega- Saturn.dat has the below entry:

game ( name "Street Fighter Zero 3 (Japan)" region "Japan" serial "T-1247G" rom ( name "Street Fighter Zero 3 (Japan) (Track 1).bin" size 621937008 crc E10DC642 md5 E9088A83E44B0ED2500597328A1EB391 sha1 B1CC45650B526ED88A82EFCD315EE73E89A658D1 serial "T-1247G" ) )

However, Redump shows shows that "T-1247G" is the disc serial, while "T-1246G" is the internal serial from the disc header.

The same problem occurs with Sonic Adventure 2: The Trial for Dreamcast, as described in this other GitHub issue.

[i30817]

As mentioned before for threads of fate and urban chaos, sometimes this happens in redump because of a misprint on the "internal" serial. And sometimes, like in that case, that misprint overlaps with a real game.

Denylisting and a fallback signature/method is the only general way to continue to use serials in this case and not get false positives.

(I use just filename scans).

[RobLoach]

The code for the Saturn detection is over at:
https://github.com/libretro/RetroArch/blob/4b2a318441f10fc6fe38acd54d03d0da58560b7b/tasks/task_database_cue.c#L834

I havne't looked at that in a while, but if you figure out a better byte offset, then please try it out.

[zcal]

@i30817,

While I acknowledge that scanning the disc header for a serial is inferior to running full checksums for validating a ROM, I do think that if that's going to be the approach then it should at least be the internal serials that are included in the DATs. That is, after all, what is pulled directly from the ROM and should correspond more reliably with what RetroArch is scanning.

And please, scanning for filename matches would be a terrible compromise here. There's way more room for error and user frustration than if using internal serials.

Regarding your comments (1,2) on Urban Chaos and Threads of Fate. Redump has a more recent comment on the Urban Chaos internal serial that it is confirmed as matching that of Threads of Fate. Redump's data is therefor "correct" since the project's purpose is to document what is and not what sh/could be. If this results in a mismatch for a RetroArch scan, then I would argue that's a reasonable edge case for RetroArch to either decide to build into their scanner or to ignore.

Regardless, the internal serial is still the one that should be included in the libretro DATs because it aligns directly with what RetroArch can scan for and because there are now more than several identifiable games in which the disc serial does not match.

@RobLoach,

Happy to attempt a look, although C is outside my current skillset. That said, this issue goes beyond Saturn detection. As noted above, we now have examples on Saturn, Dreamcast, PSX, and Sega CD. And I don't think byte offset is the problem here, it's the source data in the DATs.

[i30817]

I'm not saying crcs are the right approach. Indeed, I dont use serials OR checksums exactly because of things like this, speed and hacks.

I'm just saying this can't be solved in general by "just use the right serial". There needs to be a fallback for special cases where the data of the game itself is wrong in a way that is "right" for another game. Its likely the game you want to fix is not this specific case, but I'm just mentioning they exist and have to be blacklisted on the serial scanner.

Also that redump note, it was I that reported it. It was just to be throughout, there being a note in the dat does nothing to the actual bytes in the dump.