This is a record of the steps which I took to migrate the data on my PS3s hard disk drive to a new larger drive, without having to use the slow and often unreliable built in backup tool.
This procedure is only possible on a PS3 where Custom Firmware has first been installed to obtain the consoles eid_root_key, used for the per-console disk drive encryption. Once this key was obtained, the PS3 was reverted to Official Firmware and all steps below were subsequently performed after the PS3 has been returned to Official Firmware.
This procedure was done as part of a data recovery process on my original disk drive. The goal was to repair that drive and, if successful, migrate the data from the old mechanical drive to a new SSD drive. As any kind of data recovery requires Custom Firmware in order to be able to access the drive encryption key, an additional goal was to do this in such a way that minimal evidence was left on the system that might indicate Custom Firmware was installed, such than when the system was returned to Official Firmware it would not be at risk of either a hardware or account ban.
The PS3 was reporting that the System Storage was corrupted and that it could not be repaired. In troubleshooting this I had cloned the original mechanical 640GB disk drive to a new 1.5TB SSD disk drive. The corruption that the PS3 was reporting was repairable via a simple fsck on the dev_hdd0 partition of the drive, performed while the drive was mounted within a PowerPC64 FreeBSD VM (after having been decrypted by a Linux Mint system).
With the drive working again I was left with the issue of having a system that could only address 640GB of the 1.5TB drive it has been cloned to. As there is no native facility for an Official Firmware PS3 to resize the file system, and as I was not comfortable in my ability to attempt to manually resize it by manipulating the partition table, I opted to copy the files to an intermediary disk. I would then allow the PS3 to format the 1.5TB drive, and then copy the files back. The procedure took ~11 hours in total to relocate the ~530GB of data between drives.
If you were copying from an old drive to a brand new drive these steps could be adjusted to avoid copying to the intermediary drive, and instead copying directly from the old and to the new drive. This should reduce the time taken to complete the procedure in half, but this would require careful adjustment of the PS3 mounting commands to enable two drives to be decrypted and accessed at once.
At a high level, the process of migrating the data between two disk drives (via an intermediary drive) from a PS3 Slim system involves these steps:
-
Use PS3 KO Manager.sh option 1 to load modules
Only swap16-ecb.ko is loaded
-
Decrypt the PS3 drive
I did this by running commands from PS3 HDD Mounter.sh manually so as to avoid automatically mounting the decrypted partitions in the Linux system
The PS3 drive is on /dev/sdc
PS3_DEV_NAME=/dev/sdc
sudo cryptsetup create -c bswap16-ecb -d /dev/zero ps3hdd-bs ${PS3_DEV_NAME}
sudo cryptsetup create -c aes-xts-plain64 -d ${HOME}/ps3/keys/ata_key.bin -s 256 ps3hdd /dev/mapper/ps3hdd-bs
sudo kpartx -a /dev/mapper/ps3hdd
sudo cryptsetup create -c aes-xts-plain64 -d ${HOME}/ps3/keys/vflash_key.bin -s 256 -p 8 ps3vflash /dev/mapper/ps3hdd1
sudo kpartx -a /dev/mapper/ps3vflash
-
Start the ppc64 FreeBSD VM and pass the decrypted PS3 partition, and the intermediary backup drive through to it
See here for more detail on installing and preparing qemu
/home/andrew/Downloads/disk-pseries.qcow2 is the FreeBSD boot and system drive
/dev/sdb is the device I will use to backup the PS3 data. These steps destroy all existing data on the device
/dev/mapper/ps3hdd2 is the PS3 dev_hdd0 partition
sudo qemu-system-ppc64 -smp 2 -m 2g \
-drive file=/dev/mapper/ps3hdd2,format=raw,if=virtio \
-drive file=/dev/sdb,format=raw,if=virtio \
-drive file=/home/andrew/Downloads/disk-pseries.qcow2,format=qcow2,if=virtio \
-net nic,model=virtio \
-machine pseries,cap-cfpc=broken,cap-sbbc=broken,cap-ibs=broken \
-vga none -nographic
-
Prepare the backup drive. Note these steps will destroy any data on /dev/sdb
See this useful guide for FreeBSD disk setup commands
I did try to use a Linux ext4 partition for this, but I found FreeBSD was not able to reliably write to the drive, so instead opted to use a native disk format
/dev/sdb is mapped to /dev/vtbd1 within the FreeBSD VM
1. Create a new GPT partition table
2. Create a UFS partition occupying all space available on the device
gpart destroy -F vtbd1
gpart create -s gpt vtbd1
gpart add -t freebsd-ufs -l ps3backup -b 1M vtbd1
newfs -U /dev/gpt/ps3backup
Terminal output from the commands
root@:~ # gpart destroy -F vtbd1
vtbd1 destroyed
root@:~ # gpart create -s gpt vtbd1
vtbd1 created
root@:~ # gpart add -t freebsd-ufs -l ps3backup -b 1M vtbd1
vtbd1p1 added
root@:~ # newfs -U /dev/gpt/ps3backup
/dev/gpt/ps3backup: 1023999.0MB (2097149912 sectors) block size 32768, fragment size 4096
using 1638 cylinder groups of 625.22MB, 20007 blks, 80128 inodes.
with soft updates
super-block backups (for fsck_ffs -b #) at:
192, 1280640, 2561088, 3841536, 5121984, 6402432, 7682880, 8963328, 10243776, 11524224, 12804672, 14085120, 15365568, 16646016, 17926464, 19206912,
...
...
2070484608, 2071765056, 2073045504, 2074325952, 2075606400, 2076886848, 2078167296, 2079447744, 2080728192, 2082008640, 2083289088, 2084569536,
2085849984, 2087130432, 2088410880, 2089691328, 2090971776, 2092252224, 2093532672, 2094813120, 2096093568
-
Mount the backup drive to the backup directory
mkdir backup
mount /dev/gpt/ps3backup backup
-
Mount the PS3 dev_hdd0
/dev/mapper/ps3hdd2 is mapped to /dev/vtbd2 within the FreeBSD VM
Use file -s /dev/vtbd2 to confirm this is the partition
root@:~ # file -s /dev/vtbd2
/dev/vtbd2: Unix Fast File system [v2] (big-endian) last mounted on /cell_mw_cfs, last written at Thu Nov 9 22:18:53 2023, clean flag 1, readonly flag 0, number of blocks 311386280, number of data blocks 301585654, number of cylinder groups 3311, block size 16384, fragment size 2048, average file size 16384, average number of files in dir 64, pending blocks to free 0, pending inodes to free 0, system-wide uuid 0, minimum percentage of free blocks 8, TIME optimization
Note that the PS3 seemingly mounts the drive at /cell_mw_cfs, so for consistency with OFW we should mount it there too on this system (while it likely doesn't matter, the last mount path could potentially be logged by the PS3)
mkdir /cell_mw_cfs
mount /dev/vtbd2 /cell_mw_cfs
-
Check if any symlinks exist on the PS3 drive
I was unsure how symlinks may copy if they existed, so would want to check manually. As it happens, there were none
find /cell_mw_cfs/. -type l -ls
-
Copy all files from the PS3 dev_hdd0 to the backup drive
-
Count files and directories on PS3 drive for comparison after the copy
find /cell_mw_cfs/ -type f -print | wc -l
find /cell_mw_cfs/ -type d -print | wc -l
root@:~ # find /cell_mw_cfs/ -type f -print | wc -l
370846
root@:~ # find /cell_mw_cfs/ -type d -print | wc -l
23237
-
Make backup directory and begin copy
mkdir backup/ps3
cp -Rp /cell_mw_cfs/ backup/ps3/
This step will take a number of hours, depending on the size of your data and speed of the drives... but it will still be several times faster than the PS3 builtin backup utility!
root@:~ # cp -Rp /cell_mw_cfs/ backup/ps3/
root@:~ #
-
Check file and directory count on backup drive to compare with the original numbers
find backup/ps3/ -type f -print | wc -l
find backup/ps3/ -type d -print | wc -l
root@:~ # find backup/ps3/ -type f -print | wc -l
370846
root@:~ # find backup/ps3/ -type d -print | wc -l
23237
-
Unmount the drives
umount backup
umount /cell_mw_cfs
-
Shutdown the FreeBSD VM
-
Create an image of the dev_flash2 partition from the PS3 drive
sudo dd if=/dev/mapper/ps3vflash3 of=dev_flash2.img status=progress
andrew@mint:~/ps3$ sudo dd if=/dev/mapper/ps3vflash3 of=dev_flash2.img status=progress
32768+0 records in
32768+0 records out
16777216 bytes (17 MB, 16 MiB) copied, 0.842484 s, 19.9 MB/s
-
Use PS3 HDD Umounter.sh to remove PS3 drive mapper
-
Shutdown Linux Mint or "safely" disconnect the PS3 drive from the system
-
Connect the new drive to the PS3, and format it via Safe Mode
Option 5 Restore PS3 System
Turn the system off during the initial setup process (prompt to select language), and disconnect from PS3
-
Connect the new drive to Linux Mint, repeat steps 1, 2, 3, 5 and 6 (do not repeate step 4 otherwise you will erase the backup data!) to decrypt the drive, and prepare the FreeBSD VM
Double check device names are the same as before
With the newly formatted PS3 drive mounted, running ls you can see it doesn't even have a home directory created yet (as it was turned off during the initial setup)
root@:~ # ls -la /cell_mw_cfs/
total 32
drwxr-xr-x 8 root wheel 512 Jan 1 2012 .
drwxr-xr-x 19 root wheel 512 Nov 10 07:10 ..
drwx-----x 5 root wheel 512 Jan 1 2012 crash_report
drwx------ 3 root wheel 512 Jan 1 2012 drm
drwx------ 3 root wheel 512 Jan 1 2012 mms
drwx------ 4 root wheel 512 Jan 1 2012 tmp
drwx------ 2 root wheel 512 Jan 1 2012 vm
drwx------ 4 root wheel 512 Jan 1 2012 vsh
df confirms the full space is available on the partition
root@:~ # df /dev/vtbd2
Filesystem 1K-blocks Used Avail Capacity Mounted on
/dev/vtbd2 1547482120 510360 1423173192 0% /cell_mw_cfs
file -s displays some file system info
root@:~ # file -s /dev/vtbd2
/dev/vtbd2: Unix Fast File system [v2] (big-endian) last mounted on /cell_mw_cfs,
last written at Fri Nov 10 07:13:16 2023, clean flag 0,
readonly flag 0, number of blocks 399410172,
number of data blocks 386870530, number of cylinder groups 5405,
block size 16384, fragment size 4096, average file size 16384,
average number of files in dir 64, pending blocks to free 0,
pending inodes to free 0, system-wide uuid 0,
minimum percentage of free blocks 8, TIME optimization
-
Delete all files and folders on the new PS3 dev_hdd0
rm -Rf /cell_mw_cfs/
root@:~ # rm -Rf /cell_mw_cfs/
rm: /cell_mw_cfs/: Device busy
root@:~ # ls -la /cell_mw_cfs/
total 8
drwxr-xr-x 2 root wheel 512 Nov 10 07:22 .
drwxr-xr-x 19 root wheel 512 Nov 10 07:10 ..
-
Copy the backed up files to the new PS3 drive
cp -Rp backup/ps3/ /cell_mw_cfs/
root@:~ # cp -Rp backup/ps3/ /cell_mw_cfs/
root@:~ #
-
Confirm the file and directory counts match previous numbers
find /cell_mw_cfs/ -type f -print | wc -l
find /cell_mw_cfs/ -type d -print | wc -l
root@:~ # find /cell_mw_cfs/ -type f -print | wc -l
370846
root@:~ # find /cell_mw_cfs/ -type d -print | wc -l
23237
ls output
root@:~ # ls -la /cell_mw_cfs/
total 76
drwxr-xr-x 17 root wheel 512 Mar 8 2023 .
drwxr-xr-x 19 root wheel 512 Nov 10 07:10 ..
drwx-----x 5 root wheel 512 Jan 13 2011 crash_report
drwx------ 2 root wheel 512 Jul 17 2021 data
drwx------ 5 root wheel 512 Jun 25 2011 drm
drwxr-xr-x 321 root wheel 7168 Nov 3 16:31 game
drwx------ 7 root wheel 512 Oct 1 2011 home
drwx------ 9 root wheel 512 Jan 13 2011 mms
drwx------ 48 root wheel 1024 Jul 3 2013 music
drwx------ 10 root wheel 512 Nov 27 2014 photo
drwx------ 3 root wheel 512 Apr 27 2008 savedata
drw-rw-rw- 2 root wheel 4608 Dec 19 2013 theme
drwx------ 18 root wheel 512 Nov 8 19:27 tmp
drwx---rwx 3 root wheel 512 Jul 5 2012 updater
drwx------ 5 root wheel 512 Oct 17 2013 video
drwx------ 2 root wheel 512 Mar 8 2023 vm
drwx------ 7 root wheel 512 Mar 9 2023 vsh
df output
root@:~ # df /dev/vtbd2
Filesystem 1K-blocks Used Avail Capacity Mounted on
/dev/vtbd2 1547482120 518723412 904960140 36% /cell_mw_cfs
-
Unmount drives
umount backup
umount /cell_mw_cfs
-
Run an fsck on the PS3 partition
This isn't really required, but I noted that the man page for fsck_ufs suggested that the -E option may perform a TRIM on the SSD for this partition, however it's unclear if such instructions would even be passed back through to the physical controller
Given that the PS3 does not support this feature, it may be beneficial to be able to periodically run this via mounting the drive in a FreeBSD system - if it works in this configuration
fsck_ufs -Ey /dev/vtbd2
root@:~ # fsck_ufs -Ey /dev/vtbd2
** /dev/vtbd2
** Last Mounted on /cell_mw_cfs
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
394083 files, 129680853 used, 257189677 free (11109 frags, 64294642 blocks, 0.0% fragmentation)
***** FILE SYSTEM IS CLEAN *****
file -s output
root@:~ # file -s /dev/vtbd2
/dev/vtbd2: Unix Fast File system [v2] (big-endian) last mounted on /cell_mw_cfs, last written at Fri Nov 10 12:07:42 2023, clean flag 1, readonly flag 0, number of blocks 399410172, number of data blocks 386870530, number of cylinder groups 5405, block size 16384, fragment size 4096, average file size 16384, average number of files in dir 64, pending blocks to free 0, pending inodes to free 0, system-wide uuid 0, minimum percentage of free blocks 8, TIME optimization
-
Shutdown the FreeBSD VM
-
Restore the dev_flash2 image to the new drive
sudo dd if=dev_flash2.img of=/dev/mapper/ps3vflash3 status=progress
andrew@mint:~/ps3$ sudo dd if=dev_flash2.img of=/dev/mapper/ps3vflash3 status=progress
14692864 bytes (15 MB, 14 MiB) copied, 1 s, 14.7 MB/s
32768+0 records in
32768+0 records out
16777216 bytes (17 MB, 16 MiB) copied, 1.20792 s, 13.9 MB/s
-
Use PS3 HDD Umounter.sh to remove PS3 drive mapper
-
Shutdown Linux Mint or "safely" disconnect the PS3 drive from the system
-
Connect the PS3 drive to PS3
-
Procedure complete
The PS3 booted up and was immediately usable, with all data available.
I did have to reactivate the console in order to be able to play any installed games (installed game information listed the buyer as blank).
It is unclear if this was caused solely by the disk migration, or if it is a consequence of having reinstalled firmware several times in my data recovery efforts.
Either way, the content was reactivated via:
PlayStation Network -> Account Management -> System Activation -> PS3 System -> Game
Repeat for Video if required, and for each PSN account if multiple are used on the system
