nix vm using flake.md

Setting up qemu VM using nix flakes

Did you know that it is rather easy to setup a VM to test your NixOs configuration?

Create simple flake:

# flake.nix
{
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";

  outputs = { self, nixpkgs, ... }:
    let
      system = "x86_64-linux";
    in
    {
      # test is a hostname for our machine
      nixosConfigurations.test = nixpkgs.lib.nixosSystem {
        inherit system;
        modules = [
          ./configuration.nix
        ];
      };
    };
}

Add your configuration

# configuration.nix
{ config, lib, pkgs, ... }: {
  # customize kernel version
  boot.kernelPackages = pkgs.linuxPackages_5_15;
  
  users.groups.admin = {};
  users.users = {
    admin = {
      isNormalUser = true;
      extraGroups = [ "wheel" ];
      password = "admin";
      group = "admin";
    };
  };

  virtualisation.vmVariant = {
    # following configuration is added only when building VM with build-vm
    virtualisation = {
      memorySize = 2048; # Use 2048MiB memory.
      cores = 3;
      graphics = false;
    };
  };

  services.openssh = {
    enable = true;
    settings.PasswordAuthentication = true;
  };

  networking.firewall.allowedTCPPorts = [ 22 ];
  environment.systemPackages = with pkgs; [
    htop
  ];

  system.stateVersion = "23.05";
}

Run a VM

git init # skip this step if you are inside already tracked repository
git add . # flakes requires at least tracking the files
nixos-rebuild build-vm --flake .#test
# expose port 22 from guest
export QEMU_NET_OPTS="hostfwd=tcp::2221-:22"
result/bin/run-nixos-vm

Profit!

# ssh onto the machine
ssh -oUserKnownHostsFile=/dev/null -oStrictHostKeyChecking=no admin@localhost -p 2221

To use without nixos-rebuild:

nix build .#nixosConfigurations.test.config.system.build.vm

I got this error:

       error:
       Failed assertions:
       - The following users have a primary group that is undefined: admin
       Hint: Add this to your NixOS configuration:
         users.groups.admin = {};

which is solved as suggested, if you want to update the gist.

You are correct, fixed it. Thanks for the suggestion!

I ended up here from Google, and thought I'd share a little snippet that might be helpful. You can make building the VM easier by exposing the config.system.build.vm value directly:

# flake.nix
{
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";

  outputs = { self, nixpkgs, ... }:
    let
      system = "x86_64-linux";
      test-vm = nixpkgs.lib.nixosSystem {
        inherit system;
        modules = [
          ./configuration.nix
        ];
      };
    in
    {
      # test is a hostname for our machine. This is optional if you don't need
      # to also expose the NixOS configuration for other purposes.
      nixosConfigurations.test = test-vm;

      # expose the build attribute directly
      vms.test = test-vm.config.system.build.vm;
    };
}

This allows you to run something like this to build the system:

nix build .#vms.test
./result/bin/run-nixos-vim

That name could be anything, though. This is by no means necessary, but is very nice if you are going to iterate on a VM at all.

There is an easier way to run the VM using the flake's apps output:

apps = rec {
  default = test;
  test = {
    type = "app";
    program = "${nixosConfigurations.test.config.system.build.vm}/bin/run-nixos-vm";
  };
};

With this, executing nix run will run the default app, which in this case is the test VM. If you have more than one VM, you can specify which one you'd like to run with nix run .#<name>. For example, I can run test with nix run .#test.

Also, it's optional to add flake-utils, but it's quite useful for supporting multiple systems.

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";

    flake-utils = {
      url = "github:numtide/flake-utils";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs =
    { self, nixpkgs, ... }@inputs:
    inputs.flake-utils.lib.eachDefaultSystem (system: rec {
      nixosConfigurations.test = nixpkgs.lib.nixosSystem {
        inherit system;
        specialArgs.inputs = inputs;
        modules = [
          ./configuration.nix
        ];
      };
      apps = {
        default = {
          type = "app";
          program = "${nixosConfigurations.test.config.system.build.vm}/bin/run-nixos-vm";
        };
      };
    });
}

@eljamm thanks for the tip!

If you have more than one VM, you can specify which one you'd like to run with nix run .#<name>. For example, I can run test with nix run .#test.

Just to be clear, do I need to create an app for each VM I want, right?
I believe this might be obvious for seasoned nixers, but for newcomers not so much 😉

I've resorted to task (which I was already using at work as a make alternative) to simplify my commands and avoid repetitions

tasks:
  vm:
    internal: true
    cmds:
      - nix build .#nixosConfigurations.{{.TARGET}}.config.system.build.vm
      - ./result/bin/run-{{.TARGET}}-vm
    env:
      QEMU_NET_OPTS: hostfwd=tcp::2221-:22

  vm:test:
    desc: Deploy test in a virtual machine
    cmds:
      - task: vm
        vars:
          TARGET: test

  vm:other:
    desc: Deploy other in a virtual machine
    cmds:
      - task: vm
        vars:
          TARGET: other

Task is included in my default devShell, so I can nix develop and then use task vm:test and task vm:other to build and run my VMs.

Just to be clear, do I need to create an app for each VM I want, right?

@sanzoghenzo Yup, but this can easily be simplified with a function if you'd like to avoid repetition. For example:

let
  # Create a VM app from a nixos config
  mkAppVM = name: {
    type = "app";
    program = "${self.nixosConfigurations.${name}.config.system.build.vm}/bin/run-nixos-vm";
  };
in
{
  apps = rec {
    default = vm-test;

    # VM tests
    vm-test = mkAppVM "test"; # start with `nix run .#apps.vm-test`
    vm-other = mkAppVM "other"; # start with `nix run .#apps.vm-other`
  };
};

Which can also be done for the system config:

let
  mkSystem =
    modules:
    nixpkgs.lib.nixosSystem {
      specialArgs.inputs = inputs;
      system = "x86_64-linux";
      modules = [
        ./hosts/common.nix
      ] ++ modules;
    };
in
{
  nixosConfigurations = {
    test = mkSystem [
      ./hosts/vm/test.nix
      ./modules/desktops/gnome
    ];

    other = mkSystem [
      ./hosts/vm/other.nix
      ./modules/desktops/kde
    ];
  };
};

Note: The rec keyword allows us to refer to vm-test from within the same attrset (see Nix language basics — nix.dev).

Also, you can include the QEMU_NET_OPTS env var into your dev shell:

let
  pkgs = nixpkgs.legacyPackages.${system};
  system = "x86_64-linux";
in
{
  devShells = {
    default = pkgs.mkShellNoCC {
      QEMU_NET_OPTS = "hostfwd=tcp::2221-:22";
    };
  };
}

Just like with apps, you can write more devShells according to your needs and activate them with nix develop .#devShells.<name>.

Alternatively, you can forward the port in the system config with:

virtualisation.forwardPorts = [
  {
    from = "host";
    host.port = 2221;
    guest.port = 22;
  }
];

Notes:

• You can set up flake-utils so you can directly use nix run .#<name> and nix develop .#<name> instead of nix run .#apps.<name> and nix develop .#devShells.<name>
• You can use direnv to auto-activate the devShell when you navigate to the directory.

Now, we can refactor things a bit and import mkAppVM and mkSystem from a separate file, which might make things cleaner:

utils.nix

{ inputs, system }:
{
  # Create a VM app from a nixos config
  mkAppVM = name: {
    type = "app";
    program = "${inputs.self.nixosConfigurations.${name}.config.system.build.vm}/bin/run-nixos-vm";
  };

  mkSystem =
    modules:
    inputs.nixpkgs.lib.nixosSystem {
      specialArgs.inputs = inputs;
      inherit system;
      modules = [
        ./hosts/common.nix
      ] ++ modules;
    };
}

Which leaves us with this final configuration:

flake.nix

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
  };

  outputs =
    { self, nixpkgs, ... }@inputs:
    (
      let
        system = "x86_64-linux";
        pkgs = nixpkgs.legacyPackages.${system};

        # utility functions
        inherit (import ./utils.nix { inherit inputs system; }) mkAppVM mkSystem;
      in
      {
        nixosConfigurations = {
          test = mkSystem [
            ./hosts/vm/test.nix
            ./modules/desktop/gnome
          ];

          other = mkSystem [
            ./hosts/vm/other.nix
            ./modules/desktop/kde
          ];
        };

        apps = rec {
          default = vm-test;

          # VM tests: `nix run .#apps.<name>`
          vm-test = mkAppVM "test";
          vm-other = mkAppVM "other";
        };

        devShells = {
          default = pkgs.mkShellNoCC {
            QEMU_NET_OPTS = "hostfwd=tcp::2221-:22";
          };
        };
      }
    );
}

I've resorted to task (which I was already using at work as a make alternative) to simplify my commands and avoid repetitions

Thanks for suggesting this as it looks like an interesting and flexible tool, but for this usecase it makes more sense to me to use apps because it integrates directly with Nix, thus leveraging its full capabilites. Also, after removing the boilerplate, the Nix code looks cleaner and more readable to me.

That said, I'm curious if task and similar tools would be more suitable for more complex cases.

Thanks @eljamm for the exaustive explanation!

program = "${inputs.self.nixosConfigurations.${name}.config.system.build.vm}/bin/run-nixos-vm";

inherit (import ./utils.nix { inherit inputs system; }) mkAppVM mkSystem;

Not what I'd call "cleaner" and "more readable" if you ask me 😅
Of course this is me trying to leverage the wonders of NixOS (declarativeness, reproducibility) but being too lazy to learn yet another programming language 🤣

But seriously, thanks again, I'll make treasure of your lesson!

I just wanted to leave a comment that this gist helped me a lot so I wanted to thank you for writing this, the comments on here are also very insightful and gives me plenty to dive in.

Thank you all!!

To use without nixos-rebuild:

nix build .#nixosConfigurations.test.config.system.build.vm

Or just nix run .#nixosConfigurations.test.config.system.build.vm

Hate to Necropost but huge thanks to all of you! This was exactly what i was searching for.

May you always have the right amount of milk for your cereal and coffee :)

thank you guys, this helped me a lot!

Note: The rec keyword allows us to refer to vm-test from within the same attrset

this was the best explanation about rec I ever read! not overcomplicated, to the point. 😄

I found that the VM executable is named after the hostname. so if the hostname is different from the config, like setting it explicitly in config.networking.hostName this should work:

  mkVM = name: {
    type = "app";
    program = "${inputs.self.nixosConfigurations.${name}.config.system.build.vm}/bin/run-${
        inputs.self.nixosConfigurations.${name}.config.networking.hostName
      }-vm";
  };

(linebreaks don't matter in the .nix file, but better for readability)

You could also use lib.getExe instead of typing out self.nixosConfigurations.${name}... twice and lib.genAttrs instead of flake-utils (see nixpkgs' flake.nix).

Mostly pet peeves, since these functions do exactly what everything else before does.

{
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";

  outputs =
    inputs@{
      self,
      nixpkgs,
      ...
    }:
    let
      inherit (nixpkgs) lib;
      forSystems = lib.genAttrs lib.systems.flakeExposed;

      mkVM = name: {
        type = "app";
        program = lib.getExe self.nixosConfigurations.${name}.config.system.build.vm;
      };
    in
    {
      nixosConfigurations.host1 = lib.nixosSystem {
        # You don't need to specify `system` here, since the `nixpkgs.hostPlatform` option does already.
        specialArgs = { inherit inputs; };
        modules = [ ./hosts/host1 ];
      };

      apps = forSystems (system: {
        default = self.apps.${system}.vm-host1;
        vm-host1 = mkVM "host1";
      });
    };
}

Does anyone know how which config to add to pass through an usb device?