lima_cheatsheet.md

Lima Cheatsheet for macOS

I stumbled on an incredible open source solution, Lima. In some ways, it has some of Hashicorp's Vagrant's resemblance, but in my opinion, it's so much superior in many ways as it doesn't use external dependencies such as Oracle's VirtualBox.

If you're used to docker on the command line but wanted to play with VMs rather than containers, then this is for you! No need to fuss with cloud-init as a user account (using your existing username in whoami) with a proper home directory would be auto-generated for you with full passwordless sudo privilege. SSH access to the VMs from the host is seamlessly integrated as well.

This quick guide is for those who need to set up virtual machines with modern Linux OSes (only few selected OSes are best supported) from scratch using only the command line. You can have a VM up and running within minutes from absolutely nothing.

NOTE: This was last tested using:

• limactl version 0.23.2
• socket_vmnet v1.1.4
• macOS Sonoma v14.6.1
• M3 (Apple Silicon) MacBook Pro

I spent a lot of time grappling with limactl to get the desired virtual machine configurations I wanted, which led to this write-up, hoping this will help others. Their documentation is great, but could be a bit more verbose.

Setting up Lima on macOS

Note: The default macOS configuration (e.g. zsh as default shell) is presumed here.

Install via Homebrew and set up limactl shell completion:

brew install lima
echo -e "\n# limactl autocompletion script\nautoload -U compinit\ncompinit\nsource <(limactl completion zsh)\n" >> ~/.zshrc && source ~/.zshrc

Test by starting a VM instance named default with default configuration to ensure the installation is successful. The default name would be automatically used by default if no name is specified. It'd take less than 5 minutes for the entire process to complete. This test is needed to autopopulate the ~/.lima subdirectories:

limactl start --tty=false

Stop and delete the default named instance after successful VM creation:

limactl stop default; limactl delete default

Install the Needed socket_vmnet Binaries

This is inevitable if you want all of the following:

• Direct network access to the VMs using their IP addresses from the host (macOS)
• VMs can communicate with each other in the same network
• Assign custom MAC addresses to the VMs

To install and configure socket_vmnet in one shot:

brew install socket_vmnet yq
sudo brew services start socket_vmnet
echo -e '\n# Use the latest socket_vmnet binary when possible\nif [ -f "$HOME/.lima/_config/networks.yaml" ]; then\n  new_path=$(realpath /opt/homebrew/opt/socket_vmnet/bin/socket_vmnet)\n  yq eval ".paths.socketVMNet = \"$new_path\"" \\\n  "$HOME/.lima/_config/networks.yaml" -i\nfi\n' >> ~/.zshrc && source ~/.zshrc
limactl sudoers >etc_sudoers.d_lima && sudo install -o root etc_sudoers.d_lima "/private/etc/sudoers.d/lima"

Test to ensure that the VM can use the VMNet network. The creation process would halt immediately if there is an issue and the error message is usually self-explanatory. You may only need to reboot the macOS host once if you persistently run into issues.

limactl start --plain --network=lima:shared --tty=false

# When done testing, run:
limactl stop default; limactl delete default

Note: Even though vmType is set to vz by default as indicated in the template://default YAML for macOS users, qemu type/driver is used instead when using lima:shared (which is VMnet) network.

Note: In an event that socket_vmnet or limactl is updated, you may need to run the following once each update:

limactl sudoers >etc_sudoers.d_lima && sudo install -o root etc_sudoers.d_lima "/private/etc/sudoers.d/lima"

More on VMnet Network (optional read):

You can view the VMnet network from the ifconfig -a command output. You may see similar information as shown below as your VMs would be in the 192.168.105.0/24 subnet:

vmenet0: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        ether 5a:57:f5:2a:ef:19
        media: autoselect
        status: active
bridge100: flags=8a63<UP,BROADCAST,SMART,RUNNING,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
        options=3<RXCSUM,TXCSUM>
        ether 82:a9:97:34:2b:64
        inet 192.168.105.1 netmask 0xffffff00 broadcast 192.168.105.255
        inet6 fe80::80a9:97ff:fe34:2b64%bridge100 prefixlen 64 scopeid 0x15 
        inet6 fd75:e234:5f93:1be2:18ec:dd56:5550:ee57 prefixlen 64 autoconf secured 
        Configuration:
                id 0:0:0:0:0:0 priority 0 hellotime 0 fwddelay 0
                maxage 0 holdcnt 0 proto stp maxaddr 100 timeout 1200
                root id 0:0:0:0:0:0 priority 0 ifcost 0 port 0
                ipfilter disabled flags 0x0
        member: vmenet0 flags=3<LEARNING,DISCOVER>
                ifmaxaddr 0 port 20 priority 0 path cost 0
        nd6 options=201<PERFORMNUD,DAD>
        media: autoselect
        status: active

The interface is managed by macOS's bootpd. The existing configuration lives in /private/etc/bootpd.plist file path. The file is not to be modified directly and ideally should be left alone. By default, the DHCP lease is set to 86400 seconds (24 hours). As far as I know, there isn't a convenient way to manage DHCP reservation.

limactl Command Basics

At this point, you should have limactl command working with autocompletion enabled. You can review all the subcommands by running limactl <TAB> and even view the manpage of the subcommands such as man limactl-create for limactl create command.

The official Lima documentation has good starting examples, but I'll only cover what I believe is needed by most Linux enthusiasts as VirtualBox and UTM usually have "shared" networking enabled by default for VMs to talk to each other. Although, no instruction on configuring video display would be shown here as headless VMs are desirable in most situations.

Here is the basic limectl start command (the limectl create command is also interchangeable except VM won't boot) command you can use in most situations below:

limactl start \
  --containerd="none" \
  --cpus=1 \
  --disk=10 \
  --memory=0.5 \
  --network=lima:shared \
  --plain \
  --set \
  ".upgradePackages = true | \
  .networks[0].macAddress = \"52:55:55:aa:bb:cc\"" \
  --tty=false \
  --name=your-new-vm \
  template://ubuntu-lts

After you've created and started the VM, run limactl list to view your created VM. Here's the output you should see below:

NAME           STATUS     SSH                VMTYPE    ARCH       CPUS    MEMORY    DISK     DIR
your-new-vm    Running    127.0.0.1:59342    qemu      aarch64    1       512MiB    10GiB    ~/.lima/your-new-vm

Accessing the VM:

You can access the VM's shell directly rather than using ssh command by running limactl shell your-new-vm using your macOS's current username as shown in whoami command on the host.

There are more than one way to access the VM:

• ssh -p 59342 localhost or ssh -p 59342 127.0.0.1
• ssh $(limactl shell your-new-vm ip -4 addr show dev lima0 | grep -o 'inet [0-9]\{1,3\}\(\.[0-9]\{1,3\}\)\{3\}' | awk '{print $2}') or ssh 192.168.105.10
• ssh -F "$HOME/.lima/your-new-vm/ssh.config" lima-your-new-vm

Breakdown of the limactl command:

While most of the command options are explained in the official documentation, the nuances, which may not be apparent to some users, are covered here.

• --containerd: When using the user flag which is set by default, containerd user-level executable would be installed in /usr/local/bin/containerd directory. This is needed if you want to run containers using the nerdctl.lima command as described in the doc. Although, the usage of --plain option overrides --containerd option if used.

• --cpus: The amount of host's CPU core you want to allocate to the VM. You can only use a whole number as the decimal value isn't acceptable.

• --disk: The size of the VM's main disk storage. Value can be a decimal. However, it was observed that for some reason VM won't be created successfully if size is less than 4 GiB.

• --memory: The size of the VM's memory you want to allocate to the VM. You can use a decimal here. The unit size is in GiB. For example, the value 0.5 means 512 MiB, which implies 0.5 GiB, of course.

• --network: The type of network configuration that you want to define. By default, the VM isn't accessible by the host OS or other VMs, and the guest OS would have the IP address, 192.168.5.15. Although, to make the VMs only accessible to each other and not the host, one can use --network=lima:user-v2 option. At the time of this writing, user-v2 network is experimental and custom MAC addresses cannot be assigned to the VMs with this network type.

  To make the VM accessible to both host OS and other VMs, one must use the VMNet network as indicated in the document. To configure it, use the --network=lima:shared option. However, to configure the MAC address on the VM, one must use the --set option, which is explained later. In the VMNet network, the 192.168.105.0/24 subnet is used for the guest VMs as they would receive a secondary network interface with lima0 as its default name to be used for internetworking communication with other VMs and the host.

  Although, the MAC address cannot be completely arbitrary as it has to follow the proper assignment standard (please see more information on Locally Administered Addresses here). Otherwise, an IP address won't be assigned. Thus, rendering the interface in a DOWN state. This would impact the VM creation and start time. By default, Lima uses the first three octets (52:55:55) for all limaX interfaces in the guests.

• --plain: As described in the documentation, mounts aren't enabled, automatic port-forwardings aren't set, containerd is disabled, and so on. This is useful if a vanilla VM configuration is desired. Obviously, the VM creation and initialization would be much quicker with --plain flag.

• --set: With this option, you would be able to override the configurations as shown in the VM's Lima configuration file generated by a template. Although, you're to use the yq syntax format to do so. This is great if you need to make changes that aren't available as limactl create command option. For example, to enable support to automatically upgrade all the guest OS's packages to the latest version upon VM initialization, you'd need to add ".upgradePackages = true" string to the --set flag. To add another configuration string such as ".networks[0].macAddress = \"52:55:55:aa:bb:cc\"", you'd need to use either a comma (,) or pipe (|) character to use as a delimiter to append it to an existing --set flag string. Please note that you'd need to use backslash (\) to properly escape the needed double quotation marks as appeared in the Lima configuration file.

• --tty: Without this option, it's implied the value is true as it'd allow user to interactively modify the VM's Lima configuration file, which would result from the template that's used, in a text editor immediately before the VM creation and initialization. If automation is desired, then the false value should be set for this option.

• --name: The value set here would become the name for the VM instance as shown in limactl list command output. Although, the name must meet the following regular expression rule: ^[A-Za-z0-9]+(?:[._-](?:[A-Za-z0-9]+))*$. Otherwise, you'd get an error message.

  It's also worth mentioning that it's highly encouraged not to use the dot (.) and underscore (_) characters in the VM instance name as the naming consistency would break as the VM's guest OS hostname would not properly inherit the instance name for some reason. For example, if --name=random-vm-name-1 flag was set, the VM's instance name would be random-vm-name-1, and the configuration subdirectory would be named ~/.lima/random-vm-name-1. Although, the guest OS's hostname would be lima-random-vm-name-1 as you can see the lima- prefix would be used in conjunction with the VM instance's name.

• template://: While this is not a required flag, the implied flag would be template://default as the default template would be used, of course. It's currently using Ubuntu with containerd. Please see a list of supported templates as it's strongly recommended using tier-1 templates for predictable behaviors with the VMs. limactl create --list-templates is another way to view the available templates. You can also view the template files directly as the file paths are listed in the limactl info JSON output. Although, it's not recommended modifying those files directly.

More Useful limactl Commands:

Here are other useful commands that are worth considering. The INSTANCE would be your VM instance name as shown in limactl list output.

• limactl edit INSTANCE:

  • You can interactively make changes to the VM's Lima (YAML) configuration file this way. For example, you can increase the VM's memory if needed. The VM must be in a Stopped state before you can make the change, or you'd get a warning.

• limactl factory-reset INSTANCE:

  • In case you need to revert the VM to its declarative state as described in the VM's Lima configuration file. You can expect to lose all data inside the VM's disk storage.

• limactl snapshot subcommands:

  • You can make a "backup" of the VM's current state, including the disk data. You'd need to use the --tag flag to specify or name the snapshot. The commands are rather intuitive, which won't be covered here. At the time of this writing, limactl snapshot is still experimental.

• limactl protect INSTANCE:

  • This command helps prevent accidental deletion as our muscle memory may not be cautious enough to avoid specifying VMs we do not want to delete via limactl delete command. To "unprotect" the VMs to prepare it for intentional deletion, you can use the limactl unprotect INSTANCE command.