Simple Python Script to Edit an Ubuntu ISO to Add Automated Server Install Capability

edit_ubuntu_autoinstall.py

import io
import pathlib

import pycdlib

ubuntu = pathlib.Path('ubuntu-22.04.1-live-server-amd64.iso')
new_iso_path = pathlib.Path('ubuntu-22.04.1-live-server-amd64-auto.iso')

iso = pycdlib.PyCdlib()
iso.open(ubuntu)

extracted = io.BytesIO()
iso.get_file_from_iso_fp(extracted, iso_path='/BOOT/GRUB/GRUB.CFG;1')
extracted.seek(0)
data = extracted.read()
print(data.decode())

new = data.replace(b' ---', b'quiet autoinstall ds=nocloud\;s=/cdrom/nocloud/ ---')
print(new.decode())

iso.rm_file(iso_path='/BOOT/GRUB/GRUB.CFG;1', rr_name='grub.cfg')
iso.add_fp(io.BytesIO(new), len(new), '/BOOT/GRUB/GRUB.CFG;1', rr_name='grub.cfg')

iso.add_directory('/NOCLOUD', rr_name='nocloud')

user_data = b"""#cloud-config
autoinstall:
  version: 1
  identity:
    hostname: ubuntu-server
    password: "$6$exDY1mhS4KUYCE/2$zmn9ToZwTKLhCw.b4/b.ZRTIZM30JZ4QrOQ2aOXJ8yk96xpcCof0kxKwuX1kqLG/ygbJ1f8wxED22bTL4F46P0"
    username: ubuntu
"""

iso.add_fp(io.BytesIO(user_data), len(user_data), '/NOCLOUD/USER_DATA;1', rr_name='user-data')
iso.add_fp(io.BytesIO(b''), len(b''), '/NOCLOUD/META_DATA;1', rr_name='meta-data')
iso.write(new_iso_path)
iso.close()

edit_ubuntu_autoinstall_simple.py

import io
import pathlib

import pycdlib

ubuntu = pathlib.Path('ubuntu-22.04.1-live-server-amd64.iso')
new_iso = pathlib.Path('ubuntu-22.04.1-live-server-amd64-auto.iso')

iso = pycdlib.PyCdlib()
iso.open(ubuntu)

extracted = io.BytesIO()
iso.get_file_from_iso_fp(extracted, iso_path='/BOOT/GRUB/GRUB.CFG;1')
extracted.seek(0)
data = extracted.read()
print(data.decode())

new = data.replace(b' ---', b'quiet autoinstall ---').replace(b'timeout=30', b'timeout=1')
print(new.decode())

iso.rm_file(iso_path='/BOOT/GRUB/GRUB.CFG;1', rr_name='grub.cfg')
iso.add_fp(io.BytesIO(new), len(new), '/BOOT/GRUB/GRUB.CFG;1', rr_name='grub.cfg')

iso.write(new_iso)
iso.close()

make_cloud_init_data_iso.py

import io
import pathlib

import pycdlib

new_iso = pathlib.Path('ubuntu-22.04-auto.iso')

iso = pycdlib.PyCdlib()
iso.new(rock_ridge='1.09', vol_ident='CIDATA')

user_data = b"""#cloud-config
autoinstall:
  version: 1
  identity:
    hostname: ubuntu-server
    password: "$6$exDY1mhS4KUYCE/2$zmn9ToZwTKLhCw.b4/b.ZRTIZM30JZ4QrOQ2aOXJ8yk96xpcCof0kxKwuX1kqLG/ygbJ1f8wxED22bTL4F46P0"
    username: ubuntu
"""

iso.add_fp(io.BytesIO(user_data), len(user_data), '/USERDATA;1', rr_name='user-data')
iso.add_fp(io.BytesIO(b''), len(b''), '/METADATA;1', rr_name='meta-data')

iso.write(new_iso)
iso.close()

The username/password here is ubuntu/ubuntu

There are two options above. The script edit_ubuntu_autoinstall.py does everything in one step and produces a new installer ISO that is used alone. The other two scripts separate the steps into an editor for the installer ISO that only changes the grub.cfg just enough to have it search for the other ISO if it is attached and has the volume label CIDATA. The third script then creates an ISO with volume label CIDATA that then contains the proper cloud-init files needed for automatic install.

Thanks a lot for the scripts. It's so simple to use :-)
By the way, Let me know how to remove the "Try or Install Ubuntu Server" or "Test memory" being asked at the beginning. Is it possible to boot directly to installation?

how to remove the "Try or Install Ubuntu Server" or "Test memory" being asked at the beginning. Is it possible to boot directly to installation?

I'm not sure, but I will revisit these scripts soon to update them to the more recent official ISOs. I will see if there is a way to change that. My goal was to make an absolute minimum number of changes to the ISO and still have it work without user interaction.

will the generated ISO do a fully automated installation ? including the storage part ?

@changchichung This will perform an absolute minimum automated install. I have tested this in Proxmox (which uses QEMU under the hood). My recommendation to you is to test it and see if it does what you need. If it doesn't do what you need, I recommend starting with the automated install documentation (this is what I used to create the script above).

https://ubuntu.com/server/docs/install/autoinstall

thank you , it works like a charm in Proxmox , I will test it with a physical machine later , thanks for the wonderful script !

You're very welcome. One of my goals with this was to have a starting point where the absolute minimum of changes from Ubuntu's defaults are used. This is to reduce the number of variables to a tiny set. From that point, any types of changes and special configurations can be added and tested one by one. But any further special configuration beyond what this script does right now is out of scope.

I have updated the scripts to the most recent LTS. I have also just tested the scripts using 22.10, ubuntu-22.10-live-server-amd64.iso, and it all works fine with that as well.

@porala Sorry, I haven't had a achance to look into how to skip forward past the menu that you're referring to. It doesn't stop at that point, one just needs to wait a bit longer. I don't need to skip past this for my use case, so I just don't have time to look into how to do that.

@porala you can try to edit boot/grub/grub.cfg , update the timeout parameter

For changing the grub menu, just do a complete replacement of it, you can delete menu points, and edit timeout parameters if needed.
Example untested:

import io
import pathlib

import pycdlib

ubuntu = pathlib.Path('ubuntu-22.04.1-live-server-amd64.iso')
new_iso = pathlib.Path('ubuntu-22.04.1-live-server-amd64-auto.iso')

iso = pycdlib.PyCdlib()
iso.open(ubuntu)

extracted = io.BytesIO()
iso.get_file_from_iso_fp(extracted, iso_path='/BOOT/GRUB/GRUB.CFG;1')
extracted.seek(0)
data = """YOUR GRUB CFG FILE HERE
"""

iso.rm_file(iso_path='/BOOT/GRUB/GRUB.CFG;1', rr_name='grub.cfg')
iso.add_fp(io.BytesIO(data), len(data), '/BOOT/GRUB/GRUB.CFG;1', rr_name='grub.cfg')

iso.write(new_iso)
iso.close()

Great work, any chance you can add in the md5 process to remove the verification issue from the edit of grub and to just plain remove the enforced check?

Clear out the md5sum.txt file to cancel install verification of files.

iso.rm_file(iso_path='/md5sum.txt;1', rr_name='md5sum.txt')
iso.add_fp(io.BytesIO("", len(""), '/md5sum.txt;1', rr_name='md5sum.txt')

You could edit the md5 sum of grub in that file if you felt like it for real verification..

Thanks, I personally don't require a verification check. I just happened to notice during the process it was failing validation from the change to the grub.cfg. Thanks for posting the additional content to disable the check and thanks for your work! Made my life easier that's for sure :)

Made some modifications to disable the verifications for future reference.

import io
import pathlib

import pycdlib

ubuntu = pathlib.Path('ubuntu-20.04.5-live-server-amd64.iso')
new_iso = pathlib.Path('ubuntu-20.04.5-live-server-amd64-auto.iso')

iso = pycdlib.PyCdlib()
iso.open(ubuntu)

extracted = io.BytesIO()
iso.get_file_from_iso_fp(extracted, iso_path='/BOOT/GRUB/GRUB.CFG;1')
extracted.seek(0)
data = extracted.read()
print(data.decode())

new = data.replace(b'Install Ubuntu Server', b'Autoinstall Ubuntu Server - Disabled Verfification').replace(b' ---', b'quiet autoinstall fsck.mode=skip ---').replace(b'timeout=30', b'timeout=1')
print(new.decode())

iso.rm_file(iso_path='/BOOT/GRUB/GRUB.CFG;1', rr_name='grub.cfg')
iso.add_fp(io.BytesIO(new), len(new), '/BOOT/GRUB/GRUB.CFG;1', rr_name='grub.cfg')

iso.write(new_iso)
iso.close()