Redumper
From Redump Wiki
redumper is an open source optical disc dumping application for Windows and Linux (a macOS version is planned). The application is written by redump moderator superg.
redumper is designed to be very user friendly, a double click of the app will detect the occupied disc drive and account for unique caveat of discs. Additionally, see Dumping Guide (redumper CLI) for a guide on how to dump with redumper via command line and submit discs to redump.org.
For a list of media types supported by redumper see Redumper status.
Contents |
Moderator Guidelines
General
Every new disc / verification submitted by the user must include redumper generated LOG file. This file is a main source of all the information needed to validate or discard a dump. Audio CD LOG file example:
=== 2022-12-03 13:22:34 ========================================================
redumper v2022.12.03 [Dec 3 2022, 13:16:43] (print usage: --help,-h))
command: ./redumper --image-path=cc1_audio --image-name=cc1_audio
*** MODE: dump
drive path: /dev/sg6
drive: PLEXTOR - CD-R PX-W5224A (revision level: 1.04, vendor specific: 04/10/06 17:00)
drive configuration: PLEXTOR (read offset: +30, C2 shift: 294, pre-gap start: -75, read method: D8, sector order: DATA_C2_SUB)
image path: cc1_audio
image name: cc1_audio
disc TOC:
disc type: CD-DA / CD-DATA
track 1 { audio }
index 01 { LBA: 0, MSF: 00:02:00 }
track 2 { audio }
index 01 { LBA: 11212, MSF: 02:31:37 }
track 3 { audio }
index 01 { LBA: 23705, MSF: 05:18:05 }
track 4 { audio }
index 01 { LBA: 38455, MSF: 08:34:55 }
track 5 { audio }
index 01 { LBA: 54435, MSF: 12:07:60 }
track A { audio }
index 01 { LBA: 71212, MSF: 15:51:37 }
PLEXTOR: reading lead-in
PLEXTOR: lead-in found (session index: 0, sectors: 3321)
dump started
dump complete (time: 62s)
media errors:
SCSI: 0
C2: 0
Q: 68
*** MODE: protection
scan started
protection: N/A
scan complete (time: 0s)
*** MODE: split
correcting Q... done
final TOC:
disc type: CD-DA / CD-DATA
track 1 { audio }
index 00 { LBA: -150 .. -1, length: 150, MSF: 00:00:00-02:28:74 }
index 01 { LBA: 0 .. 11024, length: 11025, MSF: 00:02:00-02:28:74 }
track 2 { audio }
index 00 { LBA: 11025 .. 11211, length: 187, MSF: 02:29:00-05:15:41 }
index 01 { LBA: 11212 .. 23516, length: 12305, MSF: 02:31:37-05:15:41 }
track 3 { audio }
index 00 { LBA: 23517 .. 23704, length: 188, MSF: 05:15:42-08:32:16 }
index 01 { LBA: 23705 .. 38266, length: 14562, MSF: 05:18:05-08:32:16 }
track 4 { audio }
index 00 { LBA: 38267 .. 38454, length: 188, MSF: 08:32:17-12:05:21 }
index 01 { LBA: 38455 .. 54246, length: 15792, MSF: 08:34:55-12:05:21 }
track 5 { audio }
index 00 { LBA: 54247 .. 54434, length: 188, MSF: 12:05:22-15:51:36 }
index 01 { LBA: 54435 .. 71211, length: 16777, MSF: 12:07:60-15:51:36 }
track A { audio }
index 01 { LBA: 71212 .. 71309, length: 98, MSF: 15:51:37-15:52:59 }
non-zero TOC sample range: [ -88200 .. 41930280]
non-zero data sample range: [ 42352 .. 41864267]
Universal Hash (SHA-1, offset: +42352): 9545d7d9ea95210cac221a8ebaabb8f9d0976534
detecting offset
audio silence detection... done
Perfect Audio Offset (silence level: 0): [-6475 .. +20287]
warning: fallback offset 0 applied
disc write offset: +0
detection complete (time: 43s)
checking tracks
track 1... passed
track 2... passed
track 3... passed
track 4... passed
track 5... passed
track A... passed
check complete (time: 0s)
splitting tracks
writing "cc1_audio (Track 1).bin"
writing "cc1_audio (Track 2).bin"
writing "cc1_audio (Track 3).bin"
writing "cc1_audio (Track 4).bin"
writing "cc1_audio (Track 5).bin"
split complete (time: 3s)
writing CUE-sheet
cc1_audio.cue... done
dat:
<rom name="cc1_audio (Track 1).bin" size="25930800" crc="a332bc5d" md5="aef38166e73b465180de6a7d76af6f49" sha1="675fdac8920c35d2a233db733b072b298c5bb129" />
<rom name="cc1_audio (Track 2).bin" size="29381184" crc="b4979243" md5="2a069dc4b8d380e3b9bd608cc3cf97b7" sha1="d25750df6fd9e261c784af5d6b7680570a8cf217" />
<rom name="cc1_audio (Track 3).bin" size="34692000" crc="adadc51e" md5="12349fd04699688543b9b860c9b1c2fa" sha1="6c50664d3c419d6c3b46997fe55e5f02f13b19c8" />
<rom name="cc1_audio (Track 4).bin" size="37584960" crc="b700d33a" md5="f7dc5bb594112b5c4c6c4de3b4f34fb6" sha1="978ea8a8acd58576ecc85f3e1c27ed761a7f0214" />
<rom name="cc1_audio (Track 5).bin" size="39901680" crc="c3c24318" md5="031b571e4a25c861e22e251612b93552" sha1="dc9df3a8feba5402e5e11921994d4f9cb2992505" />
CUE [cc1_audio.cue]:
FILE "cc1_audio (Track 1).bin" BINARY
TRACK 01 AUDIO
INDEX 01 00:00:00
FILE "cc1_audio (Track 2).bin" BINARY
TRACK 02 AUDIO
INDEX 00 00:00:00
INDEX 01 00:02:37
FILE "cc1_audio (Track 3).bin" BINARY
TRACK 03 AUDIO
INDEX 00 00:00:00
INDEX 01 00:02:38
FILE "cc1_audio (Track 4).bin" BINARY
TRACK 04 AUDIO
INDEX 00 00:00:00
INDEX 01 00:02:38
FILE "cc1_audio (Track 5).bin" BINARY
TRACK 05 AUDIO
INDEX 00 00:00:00
INDEX 01 00:02:38
*** MODE: info
The usual workflow would be to open the file with any text editor and search for a "warning" keyword. Each warning represents important information about the dump. Warning messages are pretty self explanatory but some will be explained in a greater detail. Following warnings are considered as show stoppers and new disc submissions with such warnings should be discarded:
warning: drive doesn't support reading of subchannel data warning: drive doesn't support C2 Error pointers warning: unsupported drive read method warning: subchannel data is not available, generating TOC index 0 entries
Warnings that are safe to ignore:
warning: fake TOC detected, using default 74min disc size (invalid TOC entries, the only known disc that generates such a warning is [PSX] Breaker Pro NTSC PAL)
Warnings that require attention or further investigation:
warning: unable to read CD-TEXT, SCSI ({}) (drive failed to read CD-TEXT, questionable and this shouldn't pop up on compatible drives)
warning: descramble failed (LBA: {} .. {}) (an indication of CD-ROM standard deviation and potential mastering issues, including, but not limited to, disc write offset shift)
warning: TOC / QTOC mismatch ... (TOC and subchannel desync that might affect track split or CUE-sheet flags and track type, redump.org favors TOC based split but the decision has to be made which variant is preferable for a given disc)
warning: incomplete dump detected, using available dump size (dump is incomplete with regard to TOC lead-out entry, safe to ignore for dumps with TOC lead-out entry that spans beyond physical media size, such as unlicensed early [PS2] Datel discs, such discs can be dumped either with --stop-lba=<value> or Ctrl+C redumper interruption and split has to be performed with --iso9660-trim option, this is advanced use and will be covered later)
warning: split is performed by QTOC (indicates the use of --force-qtoc option which instructs redumper to perform a subchannel based track split, such dumps should be accepted only if agreed beforehand for a given disc)
Output Files
- *.toc: Table of Contents (TOC) in RAW TOC format (legacy command)
- *.fulltoc: Table of Contents (TOC) in RAW FULL_TOC format (multisession supported), might not get generated if drive doesn't support it
- *.subcode: Subchannel dump in RAW format. It is not demultiplexed and thus incompatible with DIC
- *.state: redumper state file which stores the status of dumped data with CD sample (32-bit) granularity. Each byte of this file corresponds to 32-bit value in the .scram file under the same offset
- *.scram: Data channel RAW (scrambled) dump file with extended lead-in / lead-out areas that is drive read offset corrected. This is equivalent to DIC *.scm but incompatible to it as *.scm is combined offset corrected and doesn't carry lead-in / lead-out information.
- *.scrap: *.scram file variation for the dumps with data tracks dumped using BE command. This format variation contains unscrambled data track stored at drive read offset. Such format is used to logically separate dumps created using scrambled mode commands vs generic drive dumps. *.scrap and *.scram are mutually exclusive and only one variation can exist for a named dump
- *.bin: redump.org compatible track split binary files
- *.cue: redump.org compatible CUE-sheet
- *.asus: LG/ASUS/LITE-ON full cache dump that is used to get extra leadout sectors, useful for debug
- *.log: redumper LOG file with all the necessary information needed to validate a submission
Audio CD
Problem
Some Audio CD discs contain non-zero data in lead-in/lead-out areas of a CD. There are multiple reasons for this, such as disc write offset spillover or non-compliant mastering.
Hardware
CD lead-in consists of RAW TOC and 2 seconds (150 sectors) of pre-gap. Good PLEXTOR drives can read last second of pre-gap the usual way and TOC with first second of pre-gap by utilizing PLEXTOR negative LBA range reading. PLEXTOR is the only drive that is currently known to provide such a functionality. On the other hand, some good known LG/ASUS/LITE-ON drives are able to read a little bit less than 2 seconds of pre-gap (~135 out of 150 sectors) the usual way. However, what LG/ASUS/LITE-ON drives can't do, is to read RAW TOC and a small portion (~15 first sectors) of pre-gap. When it comes to lead-out, good PLEXTOR can read ~100 lead-out sectors the usual way, good known LG/ASUS/LITE-ON drives are unable to read lead-out the usual way, but some amount of lead-out sectors can be extracted directly from the drive cache using vendor commands. The other promising development is Rib's modified ASUS firmware that fully unlocks lead-out reading (and he discovered ASUS cache read command in the first place). Compatibility with the modified firmware is enabled as of redumper build 221.
PLEXTOR Lead-In Notes
As described in the previous section, good PLEXTOR drives are able to read lead-in area using negative LBA addressing. Regardless of a starting point, such reads are "virtualized" by the drive and will always start somewhere in the lead-in TOC area (no positional information in subchannel Q as TOC is stored there so drive isn't able to seek precisely). Consecutively reading sectors by using negative LBA range will read all TOC sectors from the starting point and pre-gap sectors will follow. However, any external pause between sequential reads (Debugger pause, sleep() call etc.) will lead to drive counter resetting and starting reading in the lead-in TOC area again. That said, the procedure is not flawless. Rarely, depending on some internal state, drive returns garbage and rebooting the drive usually helps. The other problem is that some particular PLEXTOR models return less data than expected and minimum required is at least 75 sectors (1 second). Getting first 75 sectors is enough as the other 75 sectors will be read later by the usual reading process thus full 2 seconds of pre-gap will be extracted.
Solution
redumper always tries to capture maximum data depending on drive capabilities. Another thing that redumper is doing is intelligent audio offset detection which in most cases helps to get rid of non-zero data spillover if such spillover was a result of disc write offset, more technical details here: audio offset If for a given disc / detected write offset combination, non-zero data still exists in lead-in/lead-out areas, such data is stored in Track 0 and Track A/AA respectively.
The most important thing to keep in mind from the above explanations is that some rare discs with non-zero data early in lead-in (RAW TOC or early pre-gap sectors) can be dumped only by using compatible PLEXTOR drive. But it doesn't mean that most of the discs cannot be dumped by known LG/ASUS/LITE-ON drives, or even a GENERIC drive, if caution is exercised. redumper includes additional diagnostics for all the corner cases that help to detect if a given submission might be incomplete. As it was explained earlier, such diagnostics comes in a form of audio cd specific warning messages:
warning: fallback offset 0 applied (safe to ignore, basically this tells that redumper didn't manage to detect an offset and consequence of this is that Track 1 - Track N will match DIC output)
warning: incomplete pre-gap (session: ..., unavailable: {}/{}) (this is an indication that drive didn't manage to get full 2 seconds of pre-gap, at the time of writing 12/20/2022, this is expected on any non good PLEXTOR and can be ignored, see the following two warnings)
warning: lead-in starts with unavailable sector (session: {}) (there might be more non-zero data in the lead-in, in general this means that there is a risk of more uncaptured data, request a good PLEXTOR dump if possible)
warning: lead-out ends with unavailable sector (session: {}) (there might be more non-zero data in the lead-out, same as the previous warning, there is a risk of more uncaptured data, request a good PLEXTOR dump if possible)
warning: lead-in contains non-zero data (session: {}, sectors: {}/{}) (informational, guarantees that Track 0 will be generated)
warning: lead-out contains non-zero data (session: {}, sectors: {}/{}) (informational, guarantees that Track A/AA will be generated)
If Track 0 and/or Track A/AA are generated, for pure Audio CDs only, the hashes should be added to the comments field, example: Oyaji Hunter Mahjong. Track 0 should be named as "leadin.bin" and Track A/AA should be named "leadout.bin" to keep things uniform. Note that, at present, Track 0/A are not known to be useful for anything other than Audio CDs.
The original Track 0 and Track A/AA naming scheme is used by redumper just to sort files in a natural order, that is lead-in before Track 1 and lead-out after the last track. In case if multisession CD with non-zero data in lead-in/lead-out, the redumper naming scheme is Track 0.<session number> and Track A/AA.<session number> respectively yet no such disc was discovered at the time of writing (11/20/2022). The multisession uniform naming for this case is proposed to be "leadin<session number>.bin" and "leadout<session number>.bin" respectively. xmp tags have to be used in order to keep the entries tidy in comments, for the Oyaji Hunter Mahjong that would be:
<xmp><rom name="leadin.bin" size="28981344" crc="34173ffb" md5="d4a1d46d6721ab93b3da1a130d3a5274" sha1="958b5a41456c90cb5c6df8d676c3d2d90a940609" /></xmp><xmp><rom name="leadout.bin" size="159936" crc="b4e470a3" md5="1f5dc334f2679331c2f6b3b26c198f01" sha1="545033f1a4ef0c9b7d98ff55e9127b6f4188121f"/></xmp>
Universal Hash
By default, redumper outputs SHA-1 hash of the maximal non-zero span of the disc. Such hash is especially useful to match same audio discs with different write offset as the hashes for such cases would be the same, example: Chrono Cross: Music Selection. Each matching ringcode submission requires an additional non-zero start value entry in order to calculate disc write offset in relation to that value. It is recommended to use the earliest ringcode as a base (offset: 0) and all the following ringcodes with the relative offset to the base. Using the Chrono Cross: Music Selection as an example, the workflow would be next:
First disc redumper output:
non-zero data sample range: [ +42352 .. +41864267] Universal Hash (SHA-1): 9545d7d9ea95210cac221a8ebaabb8f9d0976534
Second disc redumper output:
non-zero data sample range: [ +42246 .. +41864161] Universal Hash (SHA-1): 9545d7d9ea95210cac221a8ebaabb8f9d0976534
As the both universal hashes match, for the first disc we store:
Ring #1 non-zero start: +42352
and for the second disc we store:
Ring #2 non-zero start: +42246
At the same time, in the ringcode, for the first disc we store offset 0 and for the second disc we store offset -106 (the difference between second disc non-zero data sample start and first disc non-zero data sample start 42246-42352=-106). No other information has to be stored in the comments.
