ue4_save_game_extractor_recompressor.py

#!/usr/bin/env python3

"""
Python 3 code that can decompress (to a .gvas file), or recompress (to a .savegame file)
the UE4 savegame file that Astroneer uses.

Though I wrote this for tinkering with Astroneer games saves, it's probably
generic to the Unreal Engine 4 compressed saved game format.

Examples:

ue4_save_game_extractor_recompressor.py --extract  --file z2.savegame  # Creates z2.gvas
ue4_save_game_extractor_recompressor.py --compress --file z2.gvas      # Creates z2.NEW.savegame
ue4_save_game_extractor_recompressor.py --test     --file z2.savegame  # Creates *.test files

---

Copyright (c) 2016-2020 Martin Falatic

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
"""

import argparse
import os
import sys
import zlib

HEADER_FIXED_HEX = "BE 40 37 4A EE 0B 74 A3 01 00 00 00"
HEADER_FIXED_BYTES = bytes.fromhex(HEADER_FIXED_HEX)
HEADER_FIXED_LEN = len(HEADER_FIXED_BYTES)
HEADER_RAW_SIZE_LEN = 4
HEADER_GVAS_MAGIC = b'GVAS'
COMPRESSED_EXT = 'savegame'
EXTRACTED_EXT = 'gvas'


def extract_data(filename_in, filename_gvas):
    data_gvas = bytes()
    with open(filename_in, 'rb') as compressed:
        header_fixed = compressed.read(HEADER_FIXED_LEN)
        header_raw_size = compressed.read(HEADER_RAW_SIZE_LEN)
        gvas_size = int.from_bytes(header_raw_size, byteorder='little')
        header_hex = ''.join('{:02X} '.format(x) for x in header_fixed)
        if HEADER_FIXED_BYTES != header_fixed:
            print(f"Header bytes do not match: Expected '{HEADER_FIXED_HEX}' got '{header_hex}'")
            sys.exit(1)
        data_compressed = compressed.read()
        data_gvas = zlib.decompress(data_compressed)
        sz_in = len(data_compressed)
        sz_out = len(data_gvas)
        if gvas_size != sz_out:
            print(f"gvas size does not match: Expected {gvas_size} got {sz_out}")
            sys.exit(1)
        with open(filename_gvas, 'wb') as gvas:
            gvas.write(data_gvas)
        header_magic = data_gvas[0:4]
        if HEADER_GVAS_MAGIC != header_magic:
            print(f"Warning: Raw save data magic: Expected {HEADER_GVAS_MAGIC} got {header_magic}")
        print(f"Inflated from {sz_in:d} (0x{sz_in:0x}) to {sz_out:d} (0x{sz_out:0x}) bytes as {filename_gvas}")
    return data_gvas


def compress_data(filename_gvas, filename_out):
    data_gvas = None
    data_compressed = bytes()
    with open(filename_gvas, 'rb') as gvas:
        data_gvas = gvas.read()
    header_magic = data_gvas[0:4]
    if HEADER_GVAS_MAGIC != header_magic:
        print(f"Warning: Raw save data magic: Expected {HEADER_GVAS_MAGIC} got {header_magic}")
    with open(filename_out, 'wb') as compressed:
        compress = zlib.compressobj(
            level=zlib.Z_DEFAULT_COMPRESSION,
            method=zlib.DEFLATED,
            wbits=4+8,  # zlib.MAX_WBITS,
            memLevel=zlib.DEF_MEM_LEVEL,
            strategy=zlib.Z_DEFAULT_STRATEGY,
        )
        data_compressed += compress.compress(data_gvas)
        data_compressed += compress.flush()
        compressed.write(HEADER_FIXED_BYTES)
        compressed.write(len(data_gvas).to_bytes(HEADER_RAW_SIZE_LEN, byteorder='little'))
        compressed.write(data_compressed)
        sz_in = len(data_gvas)
        sz_out = len(data_compressed)
        print(f"Deflated from {sz_in:d} (0x{sz_in:0x}) to {sz_out:d} (0x{sz_out:0x}) bytes as {filename_out}")
    return data_compressed


if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="UE4 Savegame Extractor/Compressor")
    parser.add_argument('--filename')
    parser.add_argument('--extract', action='store_true')
    parser.add_argument('--compress', action='store_true')
    parser.add_argument('--test', action='store_true')
    args = parser.parse_args()

    argerrors = False
    if not args.filename:
        print("Error: No filename specified")
        argerrors = True
    if (args.extract and args.compress):
        print("Error: Choose only one of --extract or --compress")
        argerrors = True
    if (args.extract or args.compress) and args.test:
        print("Error: --test switch stands alone")
        argerrors = True
    if argerrors:
        sys.exit(1)

    filename = args.filename
    dirname, basename = os.path.split(filename)
    rootname, extname = os.path.splitext(basename)

    if args.extract:
        filename_in = filename
        filename_gvas = os.path.join(dirname, f'{rootname}.{EXTRACTED_EXT}')
        data_gvas = extract_data(filename_in=filename_in, filename_gvas=filename_gvas)
    elif args.compress:
        filename_gvas = filename
        filename_out = os.path.join(dirname, f'{rootname}.NEW.{COMPRESSED_EXT}')
        data_compressed = compress_data(filename_gvas=filename, filename_out=filename_out)
    elif args.test:
        filename_in = filename
        filename_gvas_1 = os.path.join(dirname, f'{rootname}.{EXTRACTED_EXT}.1.test')
        filename_out = os.path.join(dirname, f'{rootname}.NEW.{COMPRESSED_EXT}.test')
        filename_gvas_2 = os.path.join(dirname, f'{rootname}.{EXTRACTED_EXT}.2.test')
        data_gvas = extract_data(filename_in=filename_in, filename_gvas=filename_gvas_1)
        data_compressed = compress_data(filename_gvas=filename_gvas_1, filename_out=filename_out)
        data_check = extract_data(filename_in=filename_out, filename_gvas=filename_gvas_2)
        status = "Passed" if data_gvas == data_check else "Failed"
        print()
        print(f"{status}: Tested decompress-compress-decompress")

This script has been very useful in investigating Astroneer save files - great work!

I haven't reverse-engineered too much of the format yet, but https://github.com/oberien/gvas-rs looks like a good starting point to figure out how the decompressed file is encoded.

I thought nobody noticed this... I'm grateful that it came in handy! I later found an in-game trainer that was fun to dabble with (I've not played in a while though.)

As for why I didn't know anyone commented, I've learned that gist has a major old bug wherein you don't get notifications for comments to gists.

If you leave a comment, it would be appreciated if you ping me elsewhere so I know to find it!

Im new to this sorta stuff how do i use it

@MartyMacGyver

How do I recompress the file? I can decompress them but what is the command to recompress?

Is this project still active?

I haven't done anything with this in years. I'm not sure it works with current versions of Astroneeer.

Its still decompresses the file just fine. When I edit the -raw and rerun the program it overwrites the -raw instead of recompressing it.

I am not super familiar with python, so I was wondering if you could point me in the right direction. If not, that is fine too!

@ChunkySpaceman, @unnamedDE, et al...

This was meant to be a proof of concept, not a final utility.

Given a savegame file (e.g., abc123.savegame), this will open and extract it to the "raw" uncompressed file (abc123.savegame-raw), then it immediately re-compresses it to abc123.savegame-z. It checks along the way to make sure that, from the perspective of the raw data within the input and output compressed files, it's all the same data inside.

But I figured why not just make it a proper utility, with arguments and everything? So I did... it should also be easier to read what each block of code is doing now.

See the top of the file for usage instructions. Python >=3.6 is best (I used 3.8 to develop this).

@MartyMacGyver, if you have any interest in save editing Astroneer still, I run a Discord server where we've made some pretty good strides, thanks to your code. Add me on Discord if you're interested @Spyci#0001.

how can we edit the .gvas code ? because i can't compile this : https://github.com/oberien/gvas-rs

Heya, @MartyMacGyver , sorry to bother you with such an old topic.

I have used your code to decompile a savegame, and i'm trying to change anything that indicates that i used creative mode, so that the missions are available again. I turned on creative without thinking too much about it, just to take a look around, and didnt realize that missions would be locked from then on.

I don't even know where to begin searching for something to edit. The decompiler worked, but i dont know...

Thanks for the code, anyway!

@ReDJstone I haven't worked on this in years.... there's a comment above describing an active effort regarding save editing that might be useful.

I'm new to all of this coding and things, where I have to put the namefile in?, I can't figure it out

I'm new to all of this coding and things, where I have to put the namefile in?, I can't figure it out

pls guys... U_U

I haven't worked on this in over 5 years - the instructions are in the code comments, but I don't think this works anymore and is now obsolete.