parse_and_convert_rws_file.py - parses a rws audio file (xbox 360) and converts it to pcm, then runs it through sox to produce a WAV

parse_and_convert_rws_file.py

#!/usr/bin/env python3
# 
# parse rws file format and extract it, then run it through sox to produce a wav
#
#
# written by Mark Grandi - May 9th 2014
#

import argparse, sys, traceback, os, os.path, subprocess, tempfile, shutil


def main(args):
    '''
    @param args - the namespace object given to us by argparse'''

    # recurse and find .rws files
    filesToProcess = list()

    for dirpath, dirnames, filenames in os.walk(args.rwsFolder):

        for iterFileName in filenames:

            if os.path.splitext(iterFileName)[1].lower() == ".rws":
                filesToProcess.append(os.path.join(dirpath, iterFileName))



    # go through each rws file and convert it

    tempdir = tempfile.TemporaryDirectory()

    print("Temporary directory is {}".format(tempdir.name))

    for iterFile in filesToProcess:

        filename = os.path.split(iterFile)[1]

        pcmFilePath = os.path.join(tempdir.name, os.path.splitext(filename)[0] + ".pcm")

        testBytes = (0).to_bytes(100, "little")

        with open(iterFile, "rb") as rwsFile:

            with open(pcmFilePath, "wb") as pcmFile:

                # read the header of the file, 2047 bytes
                rwsFile.read(2047)

                # now loop and read the audio data, and the null bytes
                counter = 0
                while True:
                    counter += 1

                    
                    # test to see if we need to break
                    oldPos = rwsFile.tell()
                    testData = rwsFile.read(100)

                    # see if we are at the end of the file? cause at the end of the file we have more then
                    # 2020 NULL bytes so if we get 100 consecutive bytes of NULL then its pretty safe to say
                    # that we are at the end of the file
                    if testData == testBytes or len(testData) == 0:
                        # at the end of the file
                        break

                    # put back at old position
                    rwsFile.seek(oldPos)

                    # read audio data
                    iterAudioData = rwsFile.read(32796)

                    # null bytes
                    rwsFile.read(2020)

                    # write to pcm file
                    pcmFile.write(iterAudioData)

                if args.justDumpRaw:

                    # we are not converting, just dumping the raw pcm file 
                    outputPcm = os.path.join(args.wavFolder, os.path.splitext(filename)[0] + ".pcm")

                    shutil.copyfile(pcmFilePath, outputPcm)
                    print("finished {}: raw pcm copied to {}".format(filename, outputPcm))

                else:
                    # convert as normal
                    outputWav = os.path.join(args.wavFolder, os.path.splitext(filename)[0] + ".wav")

                    # sox -t raw -r 44100 -e signed-integer -b 16  --endian little -c 2 <input file> <output file>
                    subprocess.call(["sox", "-t", "raw", "-r", "44100", "-e", "signed-integer", "-b", "16", 
                        "--endian", "little", "-c", "2", pcmFilePath, outputWav])
                    print("finished {}: converted and saved to {}".format(filename, outputWav))

    # delete temporary directory
    tempdir.cleanup()


def printTraceback():
    '''prints the traceback'''

    # get variables for the method we are about to call
    exc_type, exc_value, exc_traceback = sys.exc_info()

    # print exception
    traceback.print_exception(exc_type, exc_value, exc_traceback)


def isDirectoryType(stringArg):
    ''' helper method for argparse to see if the argument is a directory
    @param stringArg - the argument we get from argparse
    @return the path if it is indeed a directory, or raises ArgumentTypeError if its not.'''

    path = os.path.realpath(os.path.normpath(stringArg))

    if not os.path.isdir(path):

        raise argparse.ArgumentTypeError("{} is not a directory!".format(path))

    return path


if __name__ == "__main__":
    # if we are being run as a real program

    parser = argparse.ArgumentParser(description="parses a rws audio file (xbox 360) and converts it to pcm, then run it through sox to produce a WAV", 
    epilog="Copyright Mark Grandi, May 9, 2014")

    parser.add_argument("rwsFolder", type=isDirectoryType, help="the folder containing .RWS files")
    parser.add_argument("wavFolder", type=isDirectoryType, help="the folder where we output the .wav files")
    parser.add_argument("--justDumpRaw", action="store_true", help="if set then we will just dump the raw .pcm files to "
        "wavFolder and not run them through sox")

    try:
        main(parser.parse_args())
    except Exception as e: 
        print("Something went wrong...error: {}".format(e))
        print("##################")
        printTraceback()
        print("##################")
        sys.exit(1)

Hey, so I used this for 'The Legend Of Spyro - Dawn Of The Dragon' to convert to WAV and whilst I get the audio data I also get a load of hissing in the background.

I did find your topic here - https://forum.xentax.com/viewtopic.php?f=17&t=11528

But I'm trying to establish where you put "so I just added a 0x00 byte to the end of the raw PCM data i extracted, and now it works perfectly! Audio data is....picky >.>" that null byte in?

Hey, so I used this for 'The Legend Of Spyro - Dawn Of The Dragon' to convert to WAV and whilst I get the audio data I also get a load of hissing in the background.

I did find your topic here - https://forum.xentax.com/viewtopic.php?f=17&t=11528

But I'm trying to establish where you put "so I just added a 0x00 byte to the end of the raw PCM data i extracted, and now it works perfectly! Audio data is....picky >.>" that null byte in?

Hi!

I'll have to find my notes to ensure, but it might have just been that I literally put a null byte at the end of the PCM audio that my script extracts, before it sends it to Sox , so maybe use the --justDumpRaw option, add a null byte at the end using a hex editor and then putting it through sox manually? I'll comment again once I find my notes

Hey, just tried that and I still get hissing back, will await for your further response when you've confirmed from your notes.

I think I've gotten somewhere.

This bit in your code:

rwsFile.read(2047)

I changed 2047 to 2048 and when importing the PCM file into Audacity it sounds fine with no hissing, but when using SOX to convert to wav it becomes garbled.

I will continue to play around with the conversion and see if I get the desired effect.

---

Nevermind I just exported it using Audacity and now it's all good with no hissing :)

Thank you very much for creating this!

I'm noticing that the start of the audio file still has hissing problems, I think what happened is that this isn't my final code, i think i wrote it to be more intelligent about the file format, and I only created this gist to post my 'current at the time' code when i made the forum post

i'll see if i can find my final code for you

@QueryMan1992 i actually found my code and made a github repo out of it, see https://github.com/mgrandi/parse_rws_format

specifically, this file: https://github.com/mgrandi/parse_rws_format/blob/master/parse_rws_ver2.py

it uses ffmpeg this time, but if you don't have that you can do py -3 parse_rws_ver2.py <rws folder> <output folder> --justDumpRaw and import it into audacity (signed 16 bit, big endian, 2 channel and export it to whatever you want

i checked myself and the start of the file is free of hissing :)

Thanks for this, it's perfect! 👍