Skip to content

Instantly share code, notes, and snippets.

@dm0-

dm0-/coreos-workstation.md

Created July 18, 2016 23:58
Show Gist options
  • Save dm0-/9eba453a310c1594314d7cfe386c2788 to your computer and use it in GitHub Desktop.
Save dm0-/9eba453a310c1594314d7cfe386c2788 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
coreos-workstation.md

CoreOS Workstation Notes

This is a summary of the system setup used for the demonstration on 2016-07-18.

Goals / Requirements

This was put together with the following intent:

  • It must build upon an unmodified CoreOS user space image.

  • All installed operating systems must be able to be booted both in a container/VM and natively via EFI interchangeably.

  • No files or directories required by the container/VM environments may be left behind when the operating systems are booted natively.

  • The guest systems must approximate a complete boot; i.e. they need to run their init systems.

  • Full graphical displays from guest operating systems must be available.

  • Containers must write to disk immediately without snapshots etc. In other words, if the power is cut after writing a file in a container, the file must be available to the same OS when booted natively.

Disk Layout

  • The first partition is the EFI system partition. It started with only Windows, and then I added the signed Fedora bootloader files to it, and later a custom Gentoo kernel.

  • The second, third, and fourth partitions are preinstalled Windows nonsense. I shrank the primary one during the Fedora installation to make room for the distros.

  • The fifth partition is the Fedora root file system from a regular install.

  • The sixth partition is the Gentoo root file system from a regular install.

  • The seventh (and final) parition is a VFAT file system to be used for shared space between the operating systems, and just for random extra storage. This file system has the label config-2 so it is also used as the CoreOS configdrive. The "extra" CoreOS stuff is in here.

CoreOS Setup

Booting

CoreOS is run from its PXE initrd image. GRUB has a menu entry that loads the kernel and initrd as with any typical distro. For a sample menu entry, write this snippet to /etc/grub.d/09_coreos on Fedora and make it executable.

The important arguments are coreos.autologin=tty1 to get a shell prompt without needing a password, and coreos.configdrive=1 to mount the system's configdrive even though it is running natively.

Cloud Config

Write the following cloud-config file to /openstack/latest/user_data on the configdrive partition to configure CoreOS on boot.

Kernel Drivers

The CoreOS kernel does not build many drivers useful for workstation systems with its normal configuration. Use the following instructions for building a custom kernel that is based on the normal CoreOS kernel. (If by chance all of your hardware is supported by the regular CoreOS kernel, skip this section.)

After completing the build and install, move the kernel to where it was configured in the GRUB menu item above. Copy the kernel modules directory into /coreos/modules on the configdrive so it will be mounted into place by the cloud-config file. Also copy any required firmware into /coreos/firmware.

Additional Programs

The default configuration expects to have two missing programs to run on the CoreOS host. The first is wpa_supplicant to support wireless networking with WPA2 security. The second is a QEMU binary that can draw directly to the Linux kernel framebuffer. Both were basically copied from some of my other build scripts.

If you don't want to build wpa_supplicant, it is possible to delegate this work to a container. Run it with host networking, and wpa_supplicant in the container will configure wireless for the host interface.

If you don't want to build QEMU, then you can't run virtual machines without a container with a display server. The expected QEMU binary uses [a large patch] (https://github.com/dm0-/gnuxc/blob/master/patches/qemu-2.6.0-fbdev.patch) to render the display on the console.

Problems

Where to begin...

  • SELinux has no chance of running in enforcing mode, with the differences between the CoreOS policy and Fedora labels. CoreOS runs in permissive mode by default, so this won't actually prevent things from executing.

  • Due to the above, booting Fedora natively after using it in a container will trigger a file system relabel.

  • Windows still expects the system clock to use local time by default. Run the following command to make it use UTC like everything else to avoid blatantly wrong time zone changes.

    https://gist.github.com/dm0-/7b5d2f73fb31ea786bebd50156ad16bd

  • Note that the Windows VM directly boots the primary hard drive. A sloppy boot configuration could try to boot the same system twice, causing all sorts of fun corruption. It should be safe with the default configuration, as Fedora should have set itself as the EFI boot manager's first choice on the physical machine, but the default /EFI/BOOT/BOOTX64.EFI is a copy of the Windows boot manager, which will be used in the clean QEMU EFI BIOS.

  • You're on your own for configuring things in Gentoo.

  • The containers are currently started with Docker, using busybox to start in an initrd type of setting. This should be changed to systemd-nspawn to address many shortcomings.

  • A thousand other things...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment