Skip to content

Instantly share code, notes, and snippets.

@exocode
Last active August 16, 2024 06:48
Show Gist options
  • Save exocode/a7e12b063f23a1ef899b23bcbfc7d123 to your computer and use it in GitHub Desktop.
Save exocode/a7e12b063f23a1ef899b23bcbfc7d123 to your computer and use it in GitHub Desktop.
Create xfs partitions on Hetzner via cloud-init. It keeps root disk available again after rebooting. Simply change your desired sizes and filesystem to use it for your needs.
#cloud-config
resize_rootfs: false
disk_setup:
/dev/sda:
table_type: 'mbr'
layout:
- 25
- 75
overwrite: true
fs_setup:
- label: root_fs
filesystem: 'ext4'
device: /dev/sda
partition: sda1
overwrite: true
- label: data_disk
filesystem: 'xfs'
device: /dev/sda
partition: sda2
overwrite: true
runcmd:
- [ partx, --update, /dev/sda ]
- [ mkfs.xfs, /dev/sda2 ]
- [ partprobe ]
- parted /dev/sda set 1 boot on p
mounts:
- ["/dev/sda1", "/"]
- ["/dev/sda2", "/data_disk"]
@cookbook87
Copy link

is this used as a customization script?

@chris2k20
Copy link

When using this file as cloud-config I got the following error and the server doesn't start anymore:
image

I use terraform with option user_data = "${file("cloud-init.yaml")}" where cloud-init.yaml is this file (tested with debian-10 and ubuntu-20.04).

@exocode
Copy link
Author

exocode commented Oct 20, 2020

When using this file as cloud-config I got the following error and the server doesn't start anymore:
image

I use terraform with option user_data = "${file("cloud-init.yaml")}" where cloud-init.yaml is this file (tested with debian-10 and ubuntu-20.04).

I think I can't help you. Its explicitly made for use in "Hetzner Cloud", and the use of Mongoid-database. I am not aware of Terraform, sorry

@chris2k20
Copy link

I think I can't help you. Its explicitly made for use in "Hetzner Cloud", and the use of Mongoid-database. I am not aware of Terraform, sorry

Hi exocode, thanks for your fast reply.

I think that's no terraform problem, because I created a VM manually in the Hetzner-Cloud Interface (with the user data above) and the same problem occurred. Same screen after first reboot.

image

@bogomips
Copy link

bogomips commented Apr 6, 2021

I have exactly the same issue, did you guys solve it?
Likely, grup is broken as "parted /dev/sda set 1 boot on p" cannot be executed as parted is not installed

@exocode
Copy link
Author

exocode commented Apr 6, 2021

In my case it's years ago, that I needed that, may you need to attach a volume?
Sorry if I can't help you...
In a few days/weeks I have to build that again. I will share my progress then. (Maybe you too)

@bogomips
Copy link

bogomips commented Apr 6, 2021

I've also contacted the Hetzner support, hoping they could help (I doubt though)
I can attach a volume pretty easily and I don't care for the extra cost, but volumes are slower, like 3 times slower, so I don't want to put my db data on a Volume.
I will surely share my progress here, Thanks!

@exocode
Copy link
Author

exocode commented Apr 6, 2021

I've also contacted the Hetzner support, hoping they could help (I doubt though)

Noramlly they will help you. I got parts of this script from them too.. I could take a few days, but they definitely solve these issues

@bogomips
Copy link

bogomips commented Apr 7, 2021

As I expected: "[... ]Packages for our dedicated root servers and virtual servers do not include software support. The packages also do not include support for configuring these servers. [... ]"
I've open a thread in the Hetzner forum, hoping someone could shed some light.
Meanwhile, had to partition my disk starting from a rescue img.

@ilyangru
Copy link

ilyangru commented Apr 14, 2021

As I expected: "[... ]Packages for our dedicated root servers and virtual servers do not include software support. The packages also do not include support for configuring these servers. [... ]"
I've open a thread in the Hetzner forum, hoping someone could shed some light.
Meanwhile, had to partition my disk starting from a rescue img.
I found the following solution:

#cloud-config
# Ubuntu 18.04+
resize_rootfs: false
write_files:
  - content: |
    # Any text
    path: /etc/growroot-disabled

runcmd:
  - [ sgdisk, -e, /dev/sda ]
  - [ partprobe ]
  - [ parted, -s, /dev/sda, mkpart, primary, xfs, "25%", "100%" ]
  - [ mkfs.xfs, /dev/sda2 ]
  - [ growpart, /dev/sda, 1 ] 
  - [ resize2fs, /dev/sda1 ]

mounts:
 - [ /dev/sda2, /var/lib/mongodb ]

@bogomips
Copy link

hey @ilyangru
Thanks for your contribution, I will test your solution as soon as I have some time and will share my feedback!

@ZeroTricks
Copy link

@ilyangru's works for me, thanks!

@literakl
Copy link

Thank you for helpful information. I already have a replica with three nodes and I wonder how to convert the existing ext4 root fs to XFS without loosing data, programs and configuration. I was thinking of the following procedure:

  1. stop all services except sshd
  2. copy all files to a volume (except special directories /dev, /proc)
  3. somehow convert the root fs - would your script work for the existing VPS?
  4. copy all data (except fstab) back
  5. remove the volume

Is this viable approach? Thanks

@exocode
Copy link
Author

exocode commented Aug 31, 2021

I am quite sure, you can't change filesystems while data in on that same disk. You'll need another already correct formatted disk, where you then copy your data to

@bogomips
Copy link

bogomips commented Nov 20, 2021

hey @ilyangru I finally had time to test it, it does not work for me.
I also had to install parted as it was not present.

#cloud-config
package_update: true
package_upgrade: true
packages:
  - parted
[....]

I believe it has to do with the distro, I am using debian 10.
Are you guys using ubuntu?

My cloud-init-output.log looks like

Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.
Error: Unable to satisfy all constraints on the partition.
Error accessing specified device /dev/sda2: No such file or directory
Usage: mkfs.xfs
/* blocksize */         [-b size=num]
/* metadata */          [-m crc=0|1,finobt=0|1,uuid=xxx,rmapbt=0|1,reflink=0|1]
/* data subvol */       [-d agcount=n,agsize=n,file,name=xxx,size=num,
                            (sunit=value,swidth=value|su=num,sw=num|noalign),
                            sectsize=num
/* force overwrite */   [-f]
/* inode size */        [-i log=n|perblock=n|size=num,maxpct=n,attr=0|1|2,
                            projid32bit=0|1,sparse=0|1]
/* no discard */        [-K]
/* log subvol */        [-l agnum=n,internal,size=num,logdev=xxx,version=n
                            sunit=value|su=num,sectsize=num,lazy-count=0|1]
/* label */             [-L label (maximum 12 characters)]
/* naming */            [-n size=num,version=2|ci,ftype=0|1]
/* no-op info only */   [-N]
/* prototype file */    [-p fname]
/* quiet */             [-q]
/* realtime subvol */   [-r extsize=num,size=num,rtdev=xxx]
/* sectorsize */        [-s size=num]
/* version */           [-V]
                        devicename
<devicename> is required unless -d name=xxx is given.
<num> is xxx (bytes), xxxs (sectors), xxxb (fs blocks), xxxk (xxx KiB),
      xxxm (xxx MiB), xxxg (xxx GiB), xxxt (xxx TiB) or xxxp (xxx PiB).
<value> is xxx (512 byte blocks).
NOCHANGE: partition 1 is size 39747551. it cannot be grown
resize2fs 1.44.5 (15-Dec-2018)
Filesystem at /dev/sda1 is mounted on /; on-line resizing required
old_desc_blocks = 1, new_desc_blocks = 3
The filesystem on /dev/sda1 is now 4968443 (4k) blocks long.

EDITED
I can confirm it perfectly worked with ubuntu-20.01 , the partition appeared right after the reboot
It did not work on Debian 11.

Does somebody know what makes the difference?

It could be https://bugs.launchpad.net/cloud-init/+bug/1920939

@laurecs
Copy link

laurecs commented Feb 19, 2022

Thanks for the example, I've followed but I can't get it to save mounts in the /etc/fstab. Here is what I've tried.

#cloud-config
package_update: true
package_upgrade: true
packages:
  - parted

resize_rootfs: false

disk_setup:
  /dev/xvdb:
    table_type: 'gpt'
    layout:
      - 10
      - 30
      - 10
    overwrite: true

fs_setup:
  - label: elastic
    filesystem: 'ext4'
    device: '/dev/xvdb'
    partition: 'xvdb1'
    overwrite: true
  - label: elastic_data
    filesystem: 'ext4'
    device: '/dev/xvdb'
    partition: 'xvdb2'
    overwrite: true
  - label: elastic_logs
    filesystem: 'ext4'
    device: '/dev/xvdb'
    partition: 'xvdb3'
    overwrite: true

runcmd:
  - [ partx, --update, /dev/xvdb ]
  - [ mkfs.ext4, /dev/xvdb1 ]
  - [ mkfs.ext4, /dev/xvdb2 ]
  - [ mkfs.ext4, /dev/xvdb3 ]
  - [ partprobe ]
  - mkdir /opt/elastic && mkdir /opt/elastic/data && mkdir /opt/elastic/logs

mounts:
  - [ "/dev/xvdb1", "/opt/elastic" ]
  - [ "/dev/xvdb2", "/opt/elastic/data" ]
  - [ "/dev/xvdb3", "/opt/elastic/logs" ]

@mauriziomarini
Copy link

there is no /boot ext3 partition in all these suggestions
why?

@mrwormhole
Copy link

mrwormhole commented Mar 11, 2023

@ilyangru you are the king, if they made a statue, they would run out of concrete for your balls

@tabascoterrier
Copy link

The @ilyangru example very nearly worked for me as is with an Ubuntu 20.04 box, I just had to quote the "1" in the 5th runcmd. It took me longer than I'd like to figure out so hopefully that'll help someone else.

#cloud-config
# Ubuntu 18.04+
resize_rootfs: false
write_files:
  - content: |
    # Any text
    path: /etc/growroot-disabled

runcmd:
  - [ sgdisk, -e, /dev/sda ]
  - [ partprobe ]
  - [ parted, -s, /dev/sda, mkpart, primary, xfs, "25%", "100%" ]
  - [ mkfs.xfs, /dev/sda2 ]
  - [ growpart, /dev/sda, "1" ] 
  - [ resize2fs, /dev/sda1 ]

mounts:
 - [ /dev/sda2, /var/lib/mongodb ]

@Zverik
Copy link

Zverik commented Aug 16, 2024

Thank you folks, the very last cloud-config worked for me too. I didn't need to mount or format new partitions, but I needed two of them, so I used this line for parted:

  - [ parted, -a, opt, -s, /dev/sda, mkpart, primary, "15%", "60%", mkpart, primary, "60%", "100%"]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment