For a .xlsm or .dotm file type you need to do it a slightly different way.
- Change the extension of the .xlsm file to .zip.
- Open the .zip file (with WinZip or WinRar etc) and go to the xl folder.
- Extract the vbaProject.bin file and open it in a Hex Editor (I use HxD, its completely free and lightweight.)
- Search for DPB and replace with DPx and save the file.
- Replace the old vbaProject.bin file with this new on in the zipped file.
- Change the file extension back to .xlsm.
- Open workbook skip through the warning messages.
- Open up Visual Basic inside Excel.
- Go to Tools > VBAProject Properties > Protection Tab.
- Put in a new password and save the .xlsm file.
- Close and re open and your new password will work.