Serial_Version_Edit.md

Serial Version Edit

Converts BL3 item serial codes into equivalents at other versions.

Background

If you've ever tried downpatching your game you've probably noticed it makes all your items disappear. This tool helps avoid that.

Your save file contains all items stored as serial codes. Each serial code has a version embedded in it, and when the game loads your character it completely ignores any serial codes with a greater version than what it can handle. Unfortuantly, whenever the game saves your character, it saves the serial codes at the latest version, even if it could fit into an earlier one. This is why items disappear - they're all at the latest vesion, so older game versions cannot handle them.

This tool lets you convert the serial codes to other versions, so hopefully you can find a version of the code that an older patch of the game will load.

Installation

This tool requires you to install Python 3, and Apocalyptech's Python Commandline Save Editor. Apoc has a nice comprehensive guide on how to install them here.

Once you have those two installed, to install this tool simply download the attached serial_version_edit.py from here and save it somewhere.

Usage

To start off open a command window in the same folder as the serial_version_edit.py (in windows explorer shift+rmb the background -> Open command/powershell window here).

To convert serial codes, simply add them as arguments when running the program. You can add as many as you want. As an example:

python serial_version_edit.py BL3(AwAAAABxeoC5/ZGAkjsIhNoYGNw0ggYA)

This will output something like the following:

# Hex (level 57)
# Was version 57, converted to version 41
BL3(AwAAAAB1LoCp/UlA5Q4CoTYGBjeNoAEA)

You can then import this serial code into your save file using any other save editor as you please.

This program output is formatted so that you can feed it directly into Apoc's editor to import the items into a save. You just need to redirect output to a file.

python serial_version_edit.py BL3(AwAAAABxeoC5/ZGAkjsIhNoYGNw0ggYA) > items.txt
bl3-save-edit old.sav new.sav -i items.txt

Optional Arguments

-a/--create-all

This will create one serial code at every supported version for each item, useful for if you don't want to try narrow down what version exactly you need.

-v VERSION/--version VERSION

By default the program converts serial codes into the lowest version supporting them. By using this argument you can provide a specifc version instead. Ignored if --create-all is specified.

serial_version_edit.py

import argparse
import base64
import math as maths
from sys import stderr

from bl3save import datalib

DW = datalib.DataWrapper()


def get_min_bits_for_value(value: int) -> int:
    """ Returns the minimum amount of bits required to store `value`. """

    return maths.ceil(maths.log(value + 1) / maths.log(2))


def load_serial(serial: str) -> datalib.BL3Serial:
    """ Helper that takes a `BL3(...)` style serial string and creates a `BL3Serial` object. """

    item = datalib.BL3Serial(datalib.BL3Serial.decode_serial_base64(serial), DW)
    item._parse_serial()
    if not item.parsed or not item.parts_parsed:
        raise ValueError(f"Unable to fully parse serial {serial}")
    return item


def get_min_version_for_bits(category: str, bits: int) -> int:
    """
    Returns the lowest serial version that supports at least the specified amount of bits in the
     specified category.
    """

    if DW.serial_db.get_num_bits(category, DW.serial_db.max_version) < bits:
        raise ValueError(f"No known version supports {bits} bits for category {category}")

    all_versions: list[dict[str, int]] = DW.serial_db.db[category]["versions"]
    min_version: int = DW.serial_db.max_version

    for version in all_versions:
        if version["bits"] >= bits and version["version"] < min_version:
            min_version = version["version"]
    return min_version


def get_min_item_version(item: datalib.BL3Serial) -> int:
    """
    Returns the lowest serial version the specified item is able to be serialized to.

    This may be lower than the serial version on the first game version supporting the item.
    """

    min_version = 1

    single_values: dict[str, int] = {
        "InventoryBalanceData": item._balance_idx,
        "InventoryData": item._invdata_idx,
        "ManufacturerData": item._manufacturer_idx
    }

    repeated_values: dict[str, list[tuple[str, int]]] = {
        item._part_invkey: item._parts,
        "InventoryGenericPartData": item._generic_parts
    }

    for category, value in single_values.items():
        bits = get_min_bits_for_value(value)
        version = get_min_version_for_bits(category, bits)
        if version > min_version:
            min_version = version

    for category, array in repeated_values.items():
        for part in array:
            bits = get_min_bits_for_value(part[1])
            version = get_min_version_for_bits(category, bits)
            if version > min_version:
                min_version = version

    return min_version


def deparse_serial_at_version(item: datalib.BL3Serial, version: int) -> str:
    """
    Returns a BL3(...) style serial string for the specified item at the specified version.

    Assumes that the item is able to be properly serialized at this version.

    Unfortuantly the `BL3Serial._deparse_serial()` method either copies the old part data, or forces
     the version to max, so I have to recreate most of it.
    """

    balance_bits = DW.serial_db.get_num_bits("InventoryBalanceData", version)
    invdata_bits = DW.serial_db.get_num_bits("InventoryData", version)
    manufacturer_bits = DW.serial_db.get_num_bits("ManufacturerData", version)
    part_bits = DW.serial_db.get_num_bits(item._part_invkey, version)
    generic_bits = DW.serial_db.get_num_bits("InventoryGenericPartData", version)

    bits = datalib.ArbitraryBits()
    bits.append_value(128, 8)
    bits.append_value(version, 7)
    bits.append_value(item._balance_idx, balance_bits)
    bits.append_value(item._invdata_idx, invdata_bits)
    bits.append_value(item._manufacturer_idx, manufacturer_bits)
    bits.append_value(item._level, 7)

    bits.append_value(len(item._parts), 6)
    for _, part_idx in item._parts:
        bits.append_value(part_idx, part_bits)

    bits.append_value(len(item._generic_parts), 4)
    for (_, part_idx) in item._generic_parts:
        bits.append_value(part_idx, generic_bits)

    bits.append_value(len(item._additional_data), 8)
    for value in item._additional_data:
        bits.append_value(value, 8)

    bits.append_value(item._num_customs, 4)

    # We don't care about the extra reroll data stored in version 4, so just force it back to 3
    new_serial = datalib.BL3Serial._encrypt_serial(bits.get_data(), 3, item.orig_seed)

    return "BL3(" + base64.b64encode(new_serial).decode("latin1") + ")"


if __name__ == "__main__":
    parser = argparse.ArgumentParser(description=(
        "Edits the version of `BL3(...)` style serial codes, allowing older versions of the game to"
        " load them."
    ))
    parser.add_argument("-v", "--version", type=int, help=(
        "Creates the serial code at a specific version rather than just the minimum."
        " Ignored if `--create-all` is specified."
    ))
    parser.add_argument("-a", "--create-all", action="store_true", help=(
        "Creates a serial code at every version supporting the item."
    ))
    parser.add_argument("serial", nargs="+", help="The serial code(s) to edit.")
    args = parser.parse_args()

    if args.create_all:
        for s in args.serial:
            try:
                item = load_serial(s)
            except Exception:  # Apoc just throws bare exceptions grr
                stderr.write(f"ERROR: Unable to parse serial code '{s}'\n\n")
                continue

            print(f"# {item.eng_name} ({item.get_level_eng()})")

            min_version = get_min_item_version(item)
            max_version = DW.serial_db.max_version
            print(
                f"# Was version {item._version}, converted to versions {min_version}-{max_version}"
            )

            for version in range(min_version, max_version + 1):
                print(deparse_serial_at_version(item, version))

            print()
        exit()

    if args.version is not None and args.version > DW.serial_db.max_version:
        stderr.write((
            f"ERROR: Requested version {args.version} is greater than the maximum known version"
            f" {DW.serial_db.max_version}.\n"
            f"       You may have to update bl3-cli-saveedit.\n"
        ))
        exit()

    for s in args.serial:
        try:
            item = load_serial(s)
        except Exception:
            stderr.write(f"ERROR: Unable to parse serial code '{s}'\n\n")
            continue

        min_version = get_min_item_version(item)
        output_version = min_version
        if args.version is not None:
            if args.version < min_version:
                stderr.write((
                    f"ERROR: Item '{item.eng_name} ({item.get_level_eng()})' cannot be converted to"
                    f" version {args.version}, minimum supported version is {min_version}.\n\n"
                ))
                continue
            output_version = args.version

        print(f"# {item.eng_name} ({item.get_level_eng()})")
        print(f"# Was version {item._version}, converted to version {output_version}")
        print(deparse_serial_at_version(item, output_version))
        print()