PS3 Linux / NetBSD 6.0 HDD Partition Expansion

ps3_hdd_expansion_notes.md

Cloning (and expanding) a 60GB PS3 HDD to a 320GB HDD

These are the steps that were taken to clone a 60GB PlayStation 3 (PS3) HDD to a 320GB HDD, and then expand the USERDATA partition to the full size of the new disk using a NetBSD VM.

Clone the Drive

The original 60GB HDD was cloned to an image using ddrescue.

Original disk: /dev/loop0 (mounted 60GB HDD image file) New Disk: /dev/sdd (320GB HDD)

    sudo ddrescue -fn /dev/loop0 /dev/sdd diskcopy.log

Output:

    andrew@mint:~/ps3$ sudo ddrescue -fn /dev/loop0 /dev/sdd diskcopy.log
    GNU ddrescue 1.23
    Press Ctrl-C to interrupt
        ipos:   60011 MB, non-trimmed:        0 B,  current rate:  61538 kB/s
        opos:   60011 MB, non-scraped:        0 B,  average rate:  68195 kB/s
    non-tried:        0 B,  bad-sector:        0 B,    error rate:       0 B/s
    rescued:   60011 MB,   bad areas:        0,        run time:     14m 39s
    pct rescued:  100.00%, read errors:        0,  remaining time:         n/a
                                time since last successful read:         n/a
    Finished

Decrypt and Mount the 320GB HDD

Running lsblk should yeld output like this.

    andrew@mint:~/ps3$ sudo lsblk -p -o MODEL,SIZE,NAME /dev/sdd
    MODEL              SIZE NAME
    VMware Virtual S 298.1G /dev/sdd
                     298.1G └─/dev/mapper/ps3hdd-bs
                     298.1G   └─/dev/mapper/ps3hdd
                       256M     ├─/dev/mapper/ps3hdd1
                       256M     │ └─/dev/mapper/ps3vflash
                      14.7M     │   ├─/dev/mapper/ps3vflash1
                     199.8M     │   ├─/dev/mapper/ps3vflash2
                        16M     │   ├─/dev/mapper/ps3vflash3
                       512K     │   ├─/dev/mapper/ps3vflash4
                         4M     │   ├─/dev/mapper/ps3vflash5
                       256K     │   └─/dev/mapper/ps3vflash6
                      53.6G     ├─/dev/mapper/ps3hdd2
                         2G     └─/dev/mapper/ps3hdd3

Backup Partition Table

    sudo dd if=/dev/mapper/ps3hdd of=ps3pt.img bs=512 count=2

Output:

    andrew@mint:~/ps3$ sudo dd if=/dev/mapper/ps3hdd of=ps3pt.img bs=512 count=2
    2+0 records in
    2+0 records out
    1024 bytes (1.0 kB, 1.0 KiB) copied, 0.000294504 s, 3.5 MB/s

    andrew@mint:~/ps3$ hexdump -C ps3pt.img 
    00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    00000010  00 00 00 00 0f ac e0 ff  00 00 00 00 de ad fa ce  |................|
    00000020  00 00 00 00 00 00 00 03  00 00 00 00 00 00 00 02  |................|
    00000030  00 00 00 00 00 00 00 08  00 00 00 00 00 08 00 00  |................|
    00000040  10 70 00 00 01 00 00 01  00 00 00 00 00 00 00 0b  |.p..............|
    00000050  10 70 00 00 02 00 00 01  00 00 00 00 00 00 00 03  |.p..............|
    00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    *
    000000c0  00 00 00 00 00 08 00 20  00 00 00 00 06 b4 7c 60  |....... ......|`|
    000000d0  10 70 00 00 01 00 00 01  00 00 00 00 00 00 00 03  |.p..............|
    000000e0  10 70 00 00 02 00 00 01  00 00 00 00 00 00 00 03  |.p..............|
    000000f0  10 20 00 00 03 00 00 01  00 00 00 00 00 00 00 03  |. ..............|
    00000100  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    *
    00000150  00 00 00 00 06 bc 7c 80  00 00 00 00 00 3f ff f8  |......|......?..|
    00000160  10 70 00 00 01 00 00 01  00 00 00 00 00 00 00 03  |.p..............|
    00000170  10 70 00 00 02 00 00 01  00 00 00 00 00 00 00 03  |.p..............|
    00000180  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    *
    00000400

Backup CACHE partition

This partition needs to be relocated to allow us to extend USERDATA. Create an image of it that we will later restore.

    sudo dd if=/dev/mapper/ps3hdd3 of=ps3hdd3.img bs=64k status=progress

Output:

    andrew@mint:~/ps3$ sudo dd if=/dev/mapper/ps3hdd3 of=ps3hdd3.img bs=64k status=progress
    2123563008 bytes (2.1 GB, 2.0 GiB) copied, 35 s, 60.7 MB/s
    32767+1 records in
    32767+1 records out
    2147479552 bytes (2.1 GB, 2.0 GiB) copied, 35.7471 s, 60.1 MB/s

    andrew@mint:~/ps3$ hexdump -C ps3hdd3.img | head
    00000000  e9 00 00 20 20 20 20 20  20 20 20 00 02 40 20 00  |...        ..@ .|
    00000010  02 00 00 00 00 f8 00 00  00 00 00 00 00 00 00 00  |................|
    00000020  e0 ff 3f 00 00 02 00 00  00 00 00 00 02 00 00 00  |..?.............|
    00000030  01 00 06 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    00000040  00 00 29 00 00 00 00 4e  4f 20 4e 41 4d 45 20 20  |..)....NO NAME  |
    00000050  20 20 46 41 54 33 32 20  20 20 00 00 00 00 00 00  |  FAT32   ......|
    00000060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    *
    000001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa  |..............U.|
    00000200  52 52 61 41 00 00 00 00  00 00 00 00 00 00 00 00  |RRaA............|

Update Partition Table

Run PS3 HDD Expander.sh to extend USERDATA and relocate CACHE.

Expand the maximum size the partition can be (317656989696 bytes).

    ./'PS3 HDD Expander.sh'

Output:

    /dev/sdd total sector count: 625142448
        320072933376 bytes, 298 GiB

    1) Safe max bytes for USERDATA:
        316753838080 bytes (295.000 GiB)
        618659840 sectors
        This size has proven to most reliable in testing

    2) Warning USERDATA would round up:
        317656989696 bytes (295.841 GiB)
        620423808 sectors
        This is the maximum size the partition can be

    3) Max bytes without round up for USERDATA:
        317290708480 bytes (295.499 GiB)
        619708415 sectors

    4) Custom Size
    Select USERDATA partition size: 2
    No changes made to LBA_USERDATA_LENGTH

Running lsblk should now yeld output like this:

    andrew@mint:~/ps3$ sudo lsblk -p -o MODEL,SIZE,NAME /dev/sdd
    MODEL              SIZE NAME
    VMware Virtual S 298.1G /dev/sdd
                     298.1G └─/dev/mapper/ps3hdd-bs
                     298.1G   └─/dev/mapper/ps3hdd
                       256M     ├─/dev/mapper/ps3hdd1
                       256M     │ └─/dev/mapper/ps3vflash
                      14.7M     │   ├─/dev/mapper/ps3vflash1
                     199.8M     │   ├─/dev/mapper/ps3vflash2
                        16M     │   ├─/dev/mapper/ps3vflash3
                       512K     │   ├─/dev/mapper/ps3vflash4
                         4M     │   ├─/dev/mapper/ps3vflash5
                       256K     │   └─/dev/mapper/ps3vflash6
                     295.8G     ├─/dev/mapper/ps3hdd2
                         2G     └─/dev/mapper/ps3hdd3

Restore CACHE partition

    sudo dd if=ps3hdd3.img of=/dev/mapper/ps3hdd3 bs=64k status=progress

Output:

    andrew@mint:~/ps3$ sudo dd if=ps3hdd3.img of=/dev/mapper/ps3hdd3 bs=64k status=progress
    2116157440 bytes (2.1 GB, 2.0 GiB) copied, 48 s, 44.1 MB/s 
    32767+1 records in
    32767+1 records out
    2147479552 bytes (2.1 GB, 2.0 GiB) copied, 62.555 s, 34.3 MB/s

Extend USERDATA partition UFS file system using NetBSD 6.0

See here for details of configuring a NetBSD 6.0 VM

    sudo qemu-system-x86_64 -smp 2 -m 2g \
                -drive file="$HOME/Downloads/NetBSD-6.0-amd64.qcow2",format=qcow2,if=virtio \
                -drive file=/dev/mapper/ps3hdd2,format=raw,if=virtio \
                -nographic

The PS3 partition should be /dev/ld1a. You can list all drives with sysctl hw.disknames.

Note NetBSD refers to UFS as FFS. This version appears unable to mount the PS3 partition, however all utility programs seem to work fine.

    # mount -v /dev/ld1a /cell_mw_cfs/ 
    exec: mount_ffs /dev/ld1a /cell_mw_cfs
    mount_ffs: /dev/ld1a on /cell_mw_cfs: incorrect super block

View disk summary

    fdisk /dev/ld1a

Output:

    # fdisk /dev/ld1a
    fdisk: primary partition table invalid, no magic in sector 0
    Disk: /dev/ld1a
    NetBSD disklabel disk geometry:
    cylinders: 16383, heads: 16, sectors/track: 63 (1008 sectors/cylinder)
    total sectors: 620423808

    BIOS disk geometry:
    cylinders: 1023, heads: 255, sectors/track: 63 (16065 sectors/cylinder)
    total sectors: 620423808

    Partitions aligned to 2048 sector boundaries, offset 2048

    Partition table:
    0: <UNUSED>
    1: <UNUSED>
    2: <UNUSED>
    3: <UNUSED>
    Bootselector disabled.
    No active partition.

Pre-extend fsck

    fsck_ffs -yf /dev/ld1a

Output:

    # fsck_ffs -yf /dev/ld1a
    ** /dev/rld1a
    ** Swapped byte order
    ** File system is already clean
    ** Last Mounted on /cell_mw_cfs
    ** Phase 1 - Check Blocks and Sizes
    ** Phase 2 - Check Pathnames
    ** Phase 3 - Check Connectivity
    ** Phase 4 - Check Reference Counts
    ** Phase 5 - Check Cyl groups
    66 files, 128564 used, 13489747 free (31 frags, 3372429 blocks, 0.0% fragmentation)

Extend file system with resize_ffs

    resize_ffs /dev/ld1a

Output:

    # resize_ffs /dev/ld1a                                                         
    It's required to manually run fsck on file system before you can resize it

    Did you run fsck on your disk (Yes/No) ? yes
    # 

Post-extend fsck

    fsck_ffs -yf /dev/ld1a

This produced a ton of output on the first run.

On the second run the output was:

    # fsck_ffs -yf /dev/ld1a 
    ** /dev/rld1a
    ** Swapped byte order
    ** File system is already clean
    ** Last Mounted on /cell_mw_cfs
    ** Phase 1 - Check Blocks and Sizes
    ** Phase 2 - Check Pathnames
    ** Phase 3 - Check Connectivity
    ** Phase 4 - Check Reference Counts
    ** Phase 5 - Check Cyl groups
    130 files, 217004 used, 74899947 free (31 frags, 18724979 blocks, 0.0% fragmentation)

Shutdown the NetBSD VM.

Unmount Drives

Use PS3 HDD Umounter.sh option 1 to unmount everything.

    ./'PS3 HDD Umounter.sh'

Return drive to PS3

Successfully boots.

PS3 reports 263 GB / 298 GB free

Bonus: change normal user free space reserve from 8% to 1%

In NetBSD run:

    tunefs -m 1 /dev/ld1a

Output: # tunefs -m 1 /dev/ld1a tunefs: /dev/ld1a: swapping byte order tunefs: tuning /dev/ld1a tunefs: minimum percentage of free space changes from 8% to 1% tunefs: should optimize for space with minfree < 5%

The PS3 now reports 283 GB / 298 GB free

ps3_netbsd_6.0_qemu_vm.md

NetBSD 6.0 amd64 QEMU VM

These steps were tested on a Linux Mint system:

    Linux mint 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 
    x86_64 x86_64 x86_64 GNU/Linux

qemu-system-x86_64 version:

    QEMU emulator version 6.2.0 (Debian 1:6.2+dfsg-2ubuntu6.15)
    Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers

You can append -enable-kvm to the qemu-system-x86_64 commands below to enable full virtualisation, if your OS installation and hardware allow it. Test by running kvm-ok.

Creating a NetBSD 6.0 QEMU VM

A pre-built disk image based on these steps can downloaded here:
https://archive.org/details/NetBSD_6.0_amd64_qemu_vm

1. Download the installation ISO from the NetBSD archive.

   https://archive.netbsd.org/pub/NetBSD-archive/NetBSD-6.0/iso/NetBSD-6.0-amd64.iso
   

2. Create a virtual disk.

   qemu-img create -f qcow2 "$HOME/Downloads/NetBSD-6.0-amd64.qcow2" 16G  
   

   Command output:

   andrew@mint:~$ qemu-img create -f qcow2 "$HOME/Downloads/NetBSD-6.0-amd64.qcow2" 16G
   Formatting '/home/andrew/Downloads/NetBSD-6.0-amd64.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=17179869184 lazy_refcounts=off refcount_bits=16
   

3. Start the QEMU VM for installation.

   sudo qemu-system-x86_64 -smp 2 -m 2g \
       -cdrom "$HOME/Downloads/NetBSD-6.0-amd64.iso" \
       -drive file="$HOME/Downloads/NetBSD-6.0-amd64.qcow2",format=qcow2,if=virtio
   

4. Proceed through the Install NetBSD to hard disk option, using the defaults except for:

   • Installation without X11
   • Bootblocks - use serial port com0 9600 baud (this makes it easier to output to the terminal when using -nographic)
   • Root shell - /bin/ksh

5. Once the installation is complete select Exit Install System and shutdown the VM. Once the system has been halted close the QEMU window to end the process.

Running the QEMU VM

    sudo qemu-system-x86_64 -smp 2 -m 2g \
        -drive file="$HOME/Downloads/NetBSD-6.0-amd64.qcow2",format=qcow2,if=virtio \
        -nographic

All output will be in the terminal you run the process from.

Log in with root and no password.

How to change keyboard layout

Quick reference for disk commands

These are current version docs so may not be 100% accurate to version 6.0

After shutting down the VM you may have to kill the process, otherwise it sits at a system halted prompted.

Passing a PS3 partition through to the VM

    sudo qemu-system-x86_64 -smp 2 -m 2g \
        -drive file="$HOME/Downloads/NetBSD-6.0-amd64.qcow2",format=qcow2,if=virtio \
        -drive file=/dev/mapper/ps3hdd2,format=raw,if=virtio \
        -nographic

The PS3 drive should be seen as /dev/ld1a.

@dnwe No problem. The author of those scripts is extremely knowledgeable and can be contacted via the forums that hosts them if you need further help.

Also may be of interest; I shared a few more details on this web site: https://ps3hdd.pages.dev/ (I was going to move all the gists over to that too but never got around to it).