Skip to content

Instantly share code, notes, and snippets.

@dragon788
Last active November 1, 2024 21:59
Show Gist options
  • Save dragon788/26921410d8de054366188c5c5435ae01 to your computer and use it in GitHub Desktop.
Save dragon788/26921410d8de054366188c5c5435ae01 to your computer and use it in GitHub Desktop.
Making the Windows 10 "chubby" install.wim compatible with a FAT32 USB so a UEFI bootable USB can be created from Linux/macOS/ChromeOS

MAGIC aka Making Anything Gruelingly "Impossible" Coherent

Whatever operating system you are using to create the USB, you will need to have a Windows 10 ISO, either from Microsoft or your system manufacturer and have a USB drive 8GB or larger (or one with at least 5GB of free space and using the FAT32 filesystem, but using a fresh and empty one is best).

TL;DR

#macOS/Linux
# First try the `bootiso` program, it has options for splitting the WIM for you!
# https://jsamr.github.io/bootiso/
# You need to already have 7zip aka `p7zip` on macOS and Linux, and `wimlib` macOS via `brew` or `wimtools` on Linux

### !!!!!!!!!!!!!!!!!!! ###
#!
#! THE DRIVE PREP STEPS ARE DESTRUCTIVE, THEY WILL WIPE YOUR USB DRIVE, be CONFIDENT IT IS THE RIGHT ONE!!!
#!
### !!!!!!!!!!!!!!!!!!! ###


## macOS USB drive prep
# if [ "$(uname -o)" == "Darwin"]; then 
diskutil list # or fdisk -l
##!!!! WARNING, THIS STEP WIPES THE DRIVE!
diskutil eraseDisk MS-DOS "WIN10" MBR /dev/disk2 # use the drive that is the USB you inserted and want to use
# The drive should get "mounted" automatically so we can use it to set the target drive
export USB_DRIVE=/Volumes/WIN10
## End macOS USB drive prep


## Linux USB drive prep
# if [ "$(uname -o)" == "Linux" ]; then
lsblk # this should show all drives and indicate the removable ones
fdisk -l
##!!!! WARNING, THESE fdisk AND mkfs STEPS WIPE THE DRIVE!

sudo fdisk /dev/sdX # use the letter of the USB you inserted and want to use
fdisk> o # gives you a clean partition table of type MBR see https://wiki.archlinux.org/title/Fdisk#Create_new_table
fdisk> n # create new partition
fdisk> p # primary partition type
fdisk> <enter> # start at beginning of disk
fdisk> <enter> # use whole disk
fdisk> l # prints filesystem type list
fdisk> t # change filesystem type
fdisk> b # select option for FAT32
fdisk> a # set partition to "active" so it is bootable
fdisk> w # write the changes
fdisk> q # quit fdisk

mkfs.fat -F 32 /dev/sdX1 # use the same letter and we are specifying the first partition we just created 

# You might not need sudo, but it avoids permissions issues
sudo mkdir /media/WIN10
sudo mkdir mount /dev/sdX1 /media/WIN10
export USB_DRIVE=/media/WIN10 
## End Linux USB drive prep

# Common steps macOS/Linux
export ISO_FILE="$HOME/Downloads/Win10_*"
set -a; : ${TEMP:=/tmp}; set +a;

## if you might want to create multiple installer USB drives without performing the extractions and splits each time,  you should copy the first set of files from the ISO as well and then just copy the contents of eg "$TEMP/usb/" to the drive with `rsync` for easy resuming if something happens
# mkdir -p "$TEMP/usb"; 7z x '-xr!install.wim' "$ISO_FILE" -o"$TEMP/usb/"

7z x '-xr!install.wim' "$ISO_FILE" -o"$USB_DRIVE"
7z x '-ir!install.wim' "$ISO_FILE" -o"$TEMP"
wimsplit $TEMP/sources/install.wim $USB_DRIVE/sources/install.swm 3000

## The wimsplit might run faster if you have the free space space available to split it within $TEMP and then copy the separate swm files over as that should result in a continuous write instead of thrashing the USB with a bunch of small writes, doing this before the other files might also result in a bit of a speed-up as flash drives tend to slow down as they get fuller due to the wear leveling trying to find unused blocks or having to clear used blocks (they don't have TRIM like SSDs)
# wimsplit $TEMP/sources/install.wim $TEMP/install.swm 3000; cp -f $TEMP/install*.swm $USB_DRIVE/sources/
## If you are making multiple drives
# wimsplit $TEMP/sources/install.wim $TEMP/usb/install.swm 3000; rsync -aVP $TEMP/usb/ $USB_DRIVE # untested


cd $USB_DRIVE
sync # this requests the OS flush any pending writes to disk, it can take a while or return immediately depending on your system

cd $HOME

# macOS
diskutil eject /dev/disk2

# Linux
eject /dev/disk2 # might need sudo?

It isn't a huge speed increase, but if you were writing several USBs it would add up significantly

$ time wimsplit $TEMP/sources/install.wim $USB_DRIVE/sources/install.swm 3000
Splitting WIM: 5057 MiB of 5057 MiB (100%) written, part 2 of 2
Finished splitting "/tmp/sources/install.wim"

real	20m43.110s
user	0m43.198s
sys	0m6.442s

$ time { wimsplit $TEMP/sources/install.wim $TEMP/install.swm 3000; cp -f $TEMP/install*.swm $USB_DRIVE/sources/; }
Splitting WIM: 5057 MiB of 5057 MiB (100%) written, part 2 of 2
Finished splitting "/tmp/sources/install.wim"

real	18m45.550s
user	0m41.271s
sys	0m10.547s

Problem statement (Your biggest problem is you need Windows... but who am I to judge?)

You probably stumbled across this post because you got an error trying to use Unetbootin or 7zip or hdiutil and diskutil or Disk Utility when trying to make a bootable Windows USB. This might have been E_FAIL from p7zip/7zip, or a hard to spot error in the logs when you cp or 7z x the entire contents of the ISO over to a FAT32 USB. Since FAT32 can only handle files up to 4GB (it truncates anything larger), the "fluffy" install.wim that exceeds this limit gets corrupted and results in a USB that you can boot from, but fails partway through attempting to install Windows, luckily it usually stops before it deletes the partitions that currently exist on the destination hard drive, so you may still have a bootable system that you can follow these directions to salvage or properly prepare the USB drive.

Assumptions (yes I know what assume does to do U and Me)

  1. You have an ISO downloaded to $HOME/Downloads/Win10_1903_V2_English_x64.iso or your native language equivalent.
  2. You have a USB drive you have formatted with FAT32, either using "Files" in ChromeOS or Disk Utility or Gparted or the command line equivalents (fdisk, mkfs.fat32, and/or parted) in Linux or the Disk Utility or the command line diskutil in macOS or just formatting the drive by right clicking and doing "Format" in Windows... but why aren't you using the Microsoft tool if you have Windows available (maybe you like doing it the "hard" way to learn more about how things work?).
  3. You have a little patience, internet available and have the right permissions to install a couple packages required to achieve our goal of a bootable Windows 10 USB.

You came here with a USB drive that seemed to work but complained about a missing install.wim

If you have a drive that boots but gives an error message, you can skip a few of the steps below and save yourself a lot of time. The main bits you'll care about are installing the required packages below, getting setup (aka having the ISO and USB drive mounted at known paths), and then you should delete the sources/install.wim from your USB, and then skip to the extricating portion below.

Packages required

For ChromeOS (with Linux apps), or Linux or macOS we will need the wimtools package and the 7zip CLI aka p7zip package. You can sudo apt update && sudo apt install wimtools p7zip in your Linux terminal or on macOS once you have Homebrew from https://brew.sh you can run brew install wimlib p7zip. On Windows we only need 7zip, you can install that by downloading from their site, or better yet get the fantastic Chocolatey tool from https://chocolatey.org/install and it includes a basic version of 7zip as part of its install, so we don't have to find and install the GUI version.

For Linux and macOS (and on Windows 8.x+) we could technically "mount" the ISO to make the files available instead of extracting them to the USB using 7zip, but I wanted to make the instructions consistent across the platforms as well as easier to copy and paste for fewer mistakes.

Getting setup

Once you have plugged in and formatted your USB drive to FAT32 (use a name like Win10-1903 to make it easier to identify), then make note of the path where it is "mounted".

  • On ChromeOS you will need to right click the drive in the "Files" app and select "Share with Linux", this will create a link at /mnt/chromeos/removable/<drivename>, eg /mnt/chromeos/removable/Win10-1903. You should right click the Downloads folder and "Share with Linux" as well so we can access the ISO file.
  • On Linux depending on your system it may be at /media/Win10-1903 or $HOME/.gvfs/Win10-1903 or /media/$USER/Win10-1903.
  • On macOS the USB will be under a path like /Volumes/Win10-1903.
  • On Windows the USB drive will have a letter viewable in Windows Explorer or in Powershell with Get-PSDrive or something like that.

If you are lazy like me, you don't want to type out the path several times, so you can use export ISO_FILE=/path/to/your/Win10.iso ie export ISO_FILE=$HOME/Downloads/Win10_SOME_VERSION.iso or on Windows in Powershell $env:ISO_FILE=C:\Users\YourUserName\Downloads\Win10.iso.

Then you can do the same for the destination drive, export USB_DRIVE=/path/to/your/USB_mount or $env:USB_DRIVE=D:\.

Extracting the easy bits

We need to do this work in a couple stages, mainly because there is a "plump" install.wim file that is ~4.3GB which isn't compatible with the FAT32 file system which is limited to 4GB files.

The x argument to 7zip stands for eXtract, the -x[r]!wildcard tells it to exclude recursively any file named install.wim (if we don't tell it recursive, we have to specify the precise sources/install.wim and if it ever moved the command would fail). The -i[r]!wildcard means include recursively any file named install.wim which also ends up resolving the whole path of the file sources/install.wim like the exclude above.

First we'll extract everything BUT that file to the USB drive. 7z x '-xr!install.wim' "$ISO_FILE" -o"$USB_DRIVE" If you are updating an existing drive to the same version, you may want to use -aos to skip existing files, or -aoa to overwrite the files without being prompted. 7z x '-xr!install.wim' "$ISO_FILE" -o"$USB_DRIVE" -aoa

or for Windows since we are doing this a couple times, let's save our fingers. Set-Alias 7z C:\ProgramData\chocolatey\tools\7z So now we can just 7z x '-xr!install.wim' "$env:ISO_FILE" -o"$env:USB_DRIVE"

This can take quite a while, take a bio break or grab a soda from the fridge.

Extricating the fat man from the little suit

In Bash or other Linux shells this should set TEMP if it isn't already. set -a; : ${TEMP:=/tmp}; set +a or export TEMP=/tmp if the next command throws an error We need to extract the "voluptuous" install.wim to a temporary location so we can give it a makeover (the lack of a space between -o and "$TEMP" is actually REQUIRED by some weirdness in 7zip). 7z x '-ir!install.wim' "$ISO_FILE" -o"$TEMP" or for Windows (assuming you used the Set-Alias above 7z x '-ir!install.wim' "$env:ISO_FILE" -o "$env:TEMP"

Now we need to split the file into smaller pieces so it fits within the constraints of FAT32, sadly Spanx don't quite cut it here. The final argument (the number) is in megabytes, the lowest you might want to go is 100, but anywhere between 100-4000 should work, I use 2000 or 3000 to give FAT32 some breathing room, sadly the larger you go the longer it takes before it prints any progress (on non-Windows systems), so if you need immediate and constant feedback, you can try 100-500. NOTE: The file extension of .swm on the second argument is REQUIRED, the Windows installer only looks for install.wim or install.swm, if you typo it as .wsm as I did the first time, it will boot into the installation only to complain that it can't find the files it needs to proceed.

wimsplit $TEMP/sources/install.wim $USB_DRIVE/sources/install.swm 3000 or for Windows Dism /Split-Image /ImageFile:"$env:TEMP/sources/install.wim" /SWMFile:"$env:USB_DRIVE\sources\install.swm" /FileSize:3000

Once the split is complete, you can eject your USB drive and WAIT patiently for it to safely eject, since I'm guessing you don't want to corrupt your installer and have to start over from the formatting step again. You can also delete the $TEMP/sources/install.wim if you want, but since we used the system's temporary directory, the file should get deleted the next time you restart.

Successful conclusion?

Now you should be able to put the USB into the computer you want to install Windows on and turn it on and once the logo shows press F12 or F9 or F10 or whatever key your BIOS directs you to press to get into the boot selection menu. Pick the Legacy or UEFI option corresponding to your USB drive (it may just show the manufacturer and not the Win10-1903 name we gave it).

@bonellia
Copy link

I love you big time, when I have spare time, I'll add a pull request with a script to trivialize this for some others :)

@dragon788
Copy link
Author

@boneila I added a TL;DR at the top since I just had to deal with this again myself.

@CubedPapayas
Copy link

CubedPapayas commented Sep 13, 2021

How long is it supposed to take to extract the files from the ISO to the USB drive? I had mine running overnight and it only reads at 3%.

EDIT: fixed my issue, was extracting from the disk image for some reason. Everything is working now, thank you for the guide!

@mfconsolani
Copy link

It worked like a charm. I owe you!

@jbiskofski
Copy link

Thanks for sharing all this. Theste steps worked for me, but next time Im just going to create the USB on a windows computer.

@milahu
Copy link

milahu commented Apr 10, 2022

i had to convert install.esd to install.wim

error:

wimsplit mnt/sources/install.esd split/install.wim 4000 
# [ERROR] Splitting of WIM containing solid resources is not supported.
#         Export it in non-solid format first.
#
# Finished splitting "mnt/sources/install.esd"
# ERROR: Exiting with error code 68:
#        The requested operation is unsupported.

fix:

du -h mnt/sources/install.esd 
# 4.2G mnt/sources/install.esd

# convert to non-solid format per `man wimexport`
# compression level 1 -> 6GB install.wim
# i dont care about file size, just make it quick
wimexport mnt/sources/install.esd all install.wim --compress=LZX:1

mkdir split
wimsplit install.wim split/install.wim 4000 

du -h split/*
# 2.0G split/install2.wim
# 4.0G split/install.wim

@rbgtk
Copy link

rbgtk commented Nov 24, 2022

Thanks a bunch! From me, the poor Linux admin, and my friend who's Windows shat itself to death

@CyrisPC
Copy link

CyrisPC commented Jan 6, 2023

there's a lot of unnecessary rambling in this tutorial and some commands and thing said that aren't true or make much sense lol. this need a rewrite although thanks for the info we could extract

@DSixtoAlonso
Copy link

I had to prepare the destination Hard Drive as GPT then reprepare the Windows flashUSB as MBR instead of GPT for it to install on the destination GPT Hard Drive, this was my code:

  1. Prepare usb (on macOS Mojave using Terminal):
    diskutil list
    Your usb should be under /dev/disk2

diskutil eraseDisk MS-DOS "WIN10" GPT /dev/disk2
Or
diskutil eraseDisk MS-DOS "WIN10" MBR /dev/disk2

hdiutil mount ~/Downloads/win10x64.iso

rsync -vha --exclude=sources/install.wim /Volumes/CCCOMA_X64FRE_EN-GB_DV9/* /Volumes/WIN10

This will load all iso files onto the USB, will take 1 minute...

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Enter password, it will update Homebrew...
Then use Homebrew to install a tool called wimlib

brew install wimlib

Then go ahead and create the directory that you're going to write the files into:

mkdir /Volumes/WIN10/sources

Then run this command. Note that this process may take several hours, you may see 0% progress until it finishes. Don't abort it. It will use wimlib to split the install.wim file into 2 files less than 4 GB each (I use 3.8 GB in the following command), then copy them over to your USB:

wimlib-imagex split /Volumes/CCCOMA_X64FRE_EN-GB_DV9/sources/install.wim /Volumes/WIN10/sources/install.swm 3800

Once that's done, you can eject your USB from your Mac inside Finder. Note that Windows will automatically rejoin these files later when you're installing.

Step 7: Put your USB into your new PC and start loading Windows
Congratulations - your computer now should boot directly from your USB drive. If it doesn't, you may need to check your new PC's BIOS and change the boot order to boot from your USB drive.

  1. Insert Windows flash Usb to destination PC, then prepare that destination Hard Drive as GPT using the command prompt (ctrl+F10 in Installation) on the Windows flashUSB:
    diskpart
    list disk
    select disk 0 (the destination Hard Drive)
    clean
    convert gpt
    Closedown command prompt and continue with Windows installation for 30 minutes.

Thank you for re-highlighting the issue of preparing the Windows flashUSB as MBR if GPT didn't work, Quincy Larson (https://www.freecodecamp.org/news/how-make-a-windows-10-usb-using-your-mac-build-a-bootable-iso-from-your-macs-terminal/#:~:text=Quincy%20Larson.%20Most%20new%20PCs%20don't%20come) said the same thing yet I missed that part in error!

Praise Jehovah - Hallelujah!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment