Skip to content

Instantly share code, notes, and snippets.

@apstndb

apstndb/0_Firecracker_on_GCE.md

Last active March 15, 2020 12:50
Show Gist options
  • Save apstndb/42365f72eea8ebec96d1f220f29df006 to your computer and use it in GitHub Desktop.
Save apstndb/42365f72eea8ebec96d1f220f29df006 to your computer and use it in GitHub Desktop.
Download ZIP
Run Firecracker on GCE
Raw
0_Firecracker_on_GCE.md

https://cloud.google.com/compute/docs/instances/enable-nested-virtualization-vm-instances?hl=en を参考に Compute Engine で nested VM を構築。

gcloud config set compute/zone us-central1-b
gcloud compute disks create disk1 --image-project debian-cloud --image-family debian-9
gcloud compute images create nested-vm-image --source-disk disk1 --licenses "https://www.googleapis.com/compute/v1/projects/vm-options/global/licenses/enable-vmx"
gcloud compute instances create example-nested-vm --image nested-vm-image --min-cpu-platform="Intel Skylake"

https://github.com/firecracker-microvm/firecracker/blob/7f29bca9ca197283275eab62fddc1c10ab580794/docs/getting-started.md getting-started 通りにやる

gcloud compute ssh example-nested-vm
curl -fsSL https://github.com/firecracker-microvm/firecracker/releases/download/v0.11.0/firecracker-v0.11.0 -o firecracker
curl -fsSL https://github.com/firecracker-microvm/firecracker/releases/download/v0.11.0/jailer-v0.11.0 -o jailer
chmod +x firecracker jailer

sudo rm -f /tmp/firecracker.sock
sudo ./firecracker --api-sock /tmp/firecracker.sock

Firecracker は入力を受け取るため別のセッションにて

gcloud compute ssh example-nested-vm
curl -fsSL -o hello-vmlinux.bin https://s3.amazonaws.com/spec.ccfc.min/img/hello/kernel/hello-vmlinux.bin
curl -fsSL -o hello-rootfs.ext4 https://s3.amazonaws.com/spec.ccfc.min/img/hello/fsfiles/hello-rootfs.ext4
sudo curl --unix-socket /tmp/firecracker.sock -i \
    -X PUT 'http://localhost/boot-source'   \
    -H 'Accept: application/json'           \
    -H 'Content-Type: application/json'     \
    -d '{
        "kernel_image_path": "./hello-vmlinux.bin",
        "boot_args": "console=ttyS0 reboot=k panic=1 pci=off"
    }'
sudo curl --unix-socket /tmp/firecracker.sock -i \
    -X PUT 'http://localhost/drives/rootfs' \
    -H 'Accept: application/json'           \
    -H 'Content-Type: application/json'     \
    -d '{
        "drive_id": "rootfs",
        "path_on_host": "./hello-rootfs.ext4",
        "is_root_device": true,
        "is_read_only": false
    }'
sudo curl --unix-socket /tmp/firecracker.sock -i \
    -X PUT 'http://localhost/actions'       \
    -H  'Accept: application/json'          \
    -H  'Content-Type: application/json'    \
    -d '{
        "action_type": "InstanceStart"
     }'

firecracker の側でログインコンソールが入力待ちになるので、 ユーザ名 root, パスワード root でログイン

Raw
1_dmesg.txt
[ 0.000000] Linux version 4.14.55-84.37.amzn2.x86_64 (mockbuild@ip-10-0-1-79) (gcc version 7.3.1 20180303 (Red Hat 7.3.1-5) (GCC)) #1 SMP Wed Jul 25 18:47:15 UTC 2018
[ 0.000000] Command line: console=ttyS0 reboot=k panic=1 pci=off root=/dev/vda virtio_mmio.device=4K@0xd0000000:5
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'
[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[ 0.000000] x86/fpu: xstate_offset[5]: 1088, xstate_sizes[5]: 64
[ 0.000000] x86/fpu: xstate_offset[6]: 1152, xstate_sizes[6]: 512
[ 0.000000] x86/fpu: xstate_offset[7]: 1664, xstate_sizes[7]: 1024
[ 0.000000] x86/fpu: Enabled xstate features 0xe7, context size is 2688 bytes, using 'standard' format.
[ 0.000000] e820: BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x0000000007ffffff] usable
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] DMI not present or invalid.
[ 0.000000] Hypervisor detected: KVM
[ 0.000000] tsc: Fast TSC calibration failed
[ 0.000000] tsc: Unable to calibrate against PIT
[ 0.000000] tsc: No reference (HPET/PMTIMER) available
[ 0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[ 0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable
[ 0.000000] e820: last_pfn = 0x8000 max_arch_pfn = 0x400000000
[ 0.000000] MTRR default type: uncachable
[ 0.000000] MTRR fixed ranges disabled:
[ 0.000000] 00000-FFFFF uncachable
[ 0.000000] MTRR variable ranges disabled:
[ 0.000000] 0 disabled
[ 0.000000] 1 disabled
[ 0.000000] 2 disabled
[ 0.000000] 3 disabled
[ 0.000000] 4 disabled
[ 0.000000] 5 disabled
[ 0.000000] 6 disabled
[ 0.000000] 7 disabled
[ 0.000000] MTRR: Disabled
[ 0.000000] x86/PAT: MTRRs disabled, skipping PAT initialization too.
[ 0.000000] CPU MTRRs all blank - virtualized system.
[ 0.000000] x86/PAT: Configuration [0-7]: WB WT UC- UC WB WT UC- UC
[ 0.000000] found SMP MP-table at [mem 0x0009fc00-0x0009fc0f] mapped at [ffffffffff200c00]
[ 0.000000] Scanning 1 areas for low memory corruption
[ 0.000000] Base memory trampoline at [ffff880000099000] 99000 size 24576
[ 0.000000] BRK [0x020b0000, 0x020b0fff] PGTABLE
[ 0.000000] BRK [0x020b1000, 0x020b1fff] PGTABLE
[ 0.000000] BRK [0x020b2000, 0x020b2fff] PGTABLE
[ 0.000000] No NUMA configuration found
[ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000007ffffff]
[ 0.000000] NODE_DATA(0) allocated [mem 0x07fde000-0x07ffffff]
[ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[ 0.000000] kvm-clock: cpu 0, msr 0:7fdc001, primary cpu clock
[ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.000000] DMA32 [mem 0x0000000001000000-0x0000000007ffffff]
[ 0.000000] Normal empty
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.000000] node 0: [mem 0x0000000000100000-0x0000000007ffffff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000007ffffff]
[ 0.000000] On node 0 totalpages: 32670
[ 0.000000] DMA zone: 64 pages used for memmap
[ 0.000000] DMA zone: 21 pages reserved
[ 0.000000] DMA zone: 3998 pages, LIFO batch:0
[ 0.000000] DMA32 zone: 448 pages used for memmap
[ 0.000000] DMA32 zone: 28672 pages, LIFO batch:7
[ 0.000000] Intel MultiProcessor Specification v1.4
[ 0.000000] MPTABLE: OEM ID: FC
[ 0.000000] MPTABLE: Product ID: 000000000000
[ 0.000000] MPTABLE: APIC at: 0xFEE00000
[ 0.000000] Processor #0 (Bootup-CPU)
[ 0.000000] IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
[ 0.000000] Processors: 1
[ 0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
[ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
[ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x000fffff]
[ 0.000000] e820: [mem 0x08000000-0xffffffff] available for PCI devices
[ 0.000000] Booting paravirtualized kernel on KVM
[ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[ 0.000000] random: get_random_bytes called from start_kernel+0x94/0x486 with crng_init=0
[ 0.000000] setup_percpu: NR_CPUS:128 nr_cpumask_bits:128 nr_cpu_ids:1 nr_node_ids:1
[ 0.000000] percpu: Embedded 41 pages/cpu @ffff880007c00000 s128728 r8192 d31016 u2097152
[ 0.000000] pcpu-alloc: s128728 r8192 d31016 u2097152 alloc=1*2097152
[ 0.000000] pcpu-alloc: [0] 0
[ 0.000000] KVM setup async PF for cpu 0
[ 0.000000] kvm-stealtime: cpu 0, msr 7c15040
[ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes)
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 32137
[ 0.000000] Policy zone: DMA32
[ 0.000000] Kernel command line: console=ttyS0 reboot=k panic=1 pci=off root=/dev/vda virtio_mmio.device=4K@0xd0000000:5
[ 0.000000] PID hash table entries: 512 (order: 0, 4096 bytes)
[ 0.000000] Memory: 111064K/130680K available (8204K kernel code, 622K rwdata, 1464K rodata, 1268K init, 2820K bss, 19616K reserved, 0K cma-reserved)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.000000] Kernel/User page tables isolation: enabled
[ 0.004000] Hierarchical RCU implementation.
[ 0.004000] RCU restricting CPUs from NR_CPUS=128 to nr_cpu_ids=1.
[ 0.004000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[ 0.004000] NR_IRQS: 4352, nr_irqs: 48, preallocated irqs: 16
[ 0.004000] Console: colour dummy device 80x25
[ 0.004000] console [ttyS0] enabled
[ 0.004000] tsc: Detected 1999.999 MHz processor
[ 0.004397] Calibrating delay loop (skipped) preset value.. 3999.99 BogoMIPS (lpj=7999996)
[ 0.012093] pid_max: default: 32768 minimum: 301
[ 0.016617] Security Framework initialized
[ 0.024097] SELinux: Initializing.
[ 0.028176] SELinux: Starting in permissive mode
[ 0.032459] Dentry cache hash table entries: 16384 (order: 5, 131072 bytes)
[ 0.041083] Inode-cache hash table entries: 8192 (order: 4, 65536 bytes)
[ 0.052326] Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
[ 0.060117] Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes)
[ 0.084428] Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8
[ 0.092031] Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4
[ 0.100060] Spectre V2 : Mitigation: Full generic retpoline
[ 0.108025] Spectre V2 : Spectre v2 mitigation: Filling RSB on context switch
[ 0.116025] Spectre V2 : Spectre v2 mitigation: Enabling Indirect Branch Prediction Barrier
[ 0.124025] Spectre V2 : Enabling Restricted Speculation for firmware calls
[ 0.132034] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl and seccomp
[ 0.200302] Freeing SMP alternatives memory: 28K
[ 0.220167] smpboot: Max logical packages: 1
[ 0.228771] x2apic enabled
[ 0.232037] Switched APIC routing to physical x2apic.
[ 0.253386] ..TIMER: vector=0x30 apic1=0 pin1=0 apic2=-1 pin2=-1
[ 0.256000] TSC deadline timer enabled
[ 0.256000] smpboot: CPU0: Intel(R) Xeon(R) Processor (family: 0x6, model: 0x55, stepping: 0x3)
[ 0.257451] Performance Events: unsupported p6 CPU model 85 no PMU driver, software events only.
[ 0.261332] Hierarchical SRCU implementation.
[ 0.268961] smp: Bringing up secondary CPUs ...
[ 0.272038] smp: Brought up 1 node, 1 CPU
[ 0.276049] smpboot: Total of 1 processors activated (3999.99 BogoMIPS)
[ 0.284076] devtmpfs: initialized
[ 0.288823] x86/mm: Memory block size: 128MB
[ 0.295051] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.296077] futex hash table entries: 256 (order: 2, 16384 bytes)
[ 0.307197] NET: Registered protocol family 16
[ 0.310429] cpuidle: using governor ladder
[ 0.312053] cpuidle: using governor menu
[ 0.368944] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[ 0.374163] dmi: Firmware registration failed.
[ 0.377215] NetLabel: Initializing
[ 0.380037] NetLabel: domain hash size = 128
[ 0.384031] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO
[ 0.388404] NetLabel: unlabeled traffic allowed by default
[ 0.393573] clocksource: Switched to clocksource kvm-clock
[ 0.400690] VFS: Disk quotas dquot_6.6.0
[ 0.406038] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.426620] NET: Registered protocol family 2
[ 0.435008] TCP established hash table entries: 1024 (order: 1, 8192 bytes)
[ 0.444679] TCP bind hash table entries: 1024 (order: 2, 16384 bytes)
[ 0.453431] TCP: Hash tables configured (established 1024 bind 1024)
[ 0.462694] UDP hash table entries: 256 (order: 1, 8192 bytes)
[ 0.470507] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
[ 0.478902] NET: Registered protocol family 1
[ 0.486382] virtio-mmio: Registering device virtio-mmio.0 at 0xd0000000-0xd0000fff, IRQ 5.
[ 0.513639] platform rtc_cmos: registered platform RTC device (no PNP device found)
[ 0.526671] Scanning for low memory corruption every 60 seconds
[ 0.537105] audit: initializing netlink subsys (disabled)
[ 0.546999] Initialise system trusted keyrings
[ 0.553175] Key type blacklist registered
[ 0.558292] audit: type=2000 audit(1575450745.398:1): state=initialized audit_enabled=0 res=1
[ 0.569449] workingset: timestamp_bits=36 max_order=15 bucket_order=0
[ 0.587940] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[ 0.596366] SELinux: Registering netfilter hooks
[ 0.615171] Key type asymmetric registered
[ 0.620839] Asymmetric key parser 'x509' registered
[ 0.627477] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 254)
[ 0.642225] io scheduler noop registered (default)
[ 0.653096] io scheduler cfq registered
[ 0.658833] intel_idle: does not run on family 6 model 85
[ 0.659231] virtio-mmio virtio-mmio.0: Failed to enable 64-bit or 32-bit DMA. Trying to continue, but this might not work.
[ 0.675286] Serial: 8250/16550 driver, 1 ports, IRQ sharing disabled
[ 0.720723] serial8250: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a U6_16550A
[ 0.742976] loop: module loaded
[ 0.750892] tun: Universal TUN/TAP device driver, 1.6
[ 0.757903] hidraw: raw HID events driver (C) Jiri Kosina
[ 0.765610] nf_conntrack version 0.5.0 (1024 buckets, 4096 max)
[ 0.774961] ip_tables: (C) 2000-2006 Netfilter Core Team
[ 0.782232] Initializing XFRM netlink socket
[ 0.789289] NET: Registered protocol family 10
[ 0.799543] Segment Routing with IPv6
[ 0.805043] NET: Registered protocol family 17
[ 0.811429] Bridge firewalling registered
[ 0.818741] registered taskstats version 1
[ 0.824479] Loading compiled-in X.509 certificates
[ 0.832822] Loaded X.509 cert 'Build time autogenerated kernel key: 3472798b31ba23b86c1c5c7236c9c91723ae5ee9'
[ 0.845452] zswap: default zpool zbud not available
[ 0.851625] zswap: pool creation failed
[ 0.857440] Key type encrypted registered
[ 0.870380] EXT4-fs (vda): mounted filesystem with ordered data mode. Opts: (null)
[ 0.880994] VFS: Mounted root (ext4 filesystem) on device 254:0.
[ 0.890019] devtmpfs: mounted
[ 0.905244] Freeing unused kernel memory: 1268K
[ 0.916183] Write protecting the kernel read-only data: 12288k
[ 0.946759] Freeing unused kernel memory: 2016K
[ 0.961427] Freeing unused kernel memory: 584K
[ 1.171059] random: fast init done
[ 1.536261] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x39a85afc727, max_idle_ns: 881590685098 ns
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment