-
-
Save ilovezfs/db1b3de775582713aca8 to your computer and use it in GitHub Desktop.
diff --git a/module/zfs/vdev_disk.c b/module/zfs/vdev_disk.c | |
index f7fcaad..dab2b65 100644 | |
--- a/module/zfs/vdev_disk.c | |
+++ b/module/zfs/vdev_disk.c | |
@@ -103,7 +103,9 @@ vdev_disk_open(vdev_t *vd, uint64_t *psize, uint64_t *max_psize, uint64_t *ashif | |
context = vfs_context_create((vfs_context_t)0); | |
/* Obtain an opened/referenced vnode for the device. */ | |
+ printf("+vnode_open %s\n", vd->vdev_path); | |
error = vnode_open(vd->vdev_path, spa_mode(vd->vdev_spa), 0, 0, &devvp, context); | |
+ printf("-vnode_open %s\n", vd->vdev_path); | |
if (error) { | |
goto out; | |
} |
What a pity fbt dtraces don't work the same in OS X, this is exactly where you'd want to have the full freedom to munge the stack.
Clean shutdown/restart -> everything's OK, same overlapping device names though. The +vnode_open/-vnode_open printfs are not too interesting in this case. Back momentarily with the results after a forced panic.
First run through after 'sudo dtrace -w -n "BEGIN{ panic();}"'
$ zpool list
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
Donkey - - - - - FAULTED -
which is boring; name change on all three halves of the mirror. I think one of them has to be valid so that the pool imports DEGRADED in order to trigger the hanging zpool list problem.
Ok, one of the pools (naturally, the fastest...) has come up DEGRADED and "zpool list" takes its time.
I'll grab a spindump of it, but it only runs a few seconds. Here's the relevant printf output:
Jun 17 13:11:00 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:11:00 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:11:00 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:11:00 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:11:00 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:11:00 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:11:00 cla. zed[2742]: eid=337 class=statechange
Jun 17 13:11:00 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:11:00 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:11:00 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:11:00 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:11:00 cla. zed[2744]: eid=338 class=statechange
Jun 17 13:11:00 cla. zed[2746]: eid=339 class=statechange
Jun 17 13:11:01 cla. zed[2748]: eid=340 class=statechange
Jun 17 13:11:01 cla. zed[2750]: eid=341 class=statechange
Jun 17 13:11:03 cla. WDQuickView[1078]: Updating service information
Jun 17 13:11:09 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:11:09 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:11:09 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:11:09 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:11:09 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:11:09 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:11:09 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:11:09 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:11:09 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:11:09 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:11:09 cla. zed[2789]: eid=342 class=statechange
Jun 17 13:11:09 cla. zed[2791]: eid=343 class=statechange
Jun 17 13:11:09 cla. zed[2797]: eid=344 class=statechange
Jun 17 13:11:09 cla. zed[2799]: eid=345 class=statechange
Jun 17 13:11:09 cla. zed[2801]: eid=346 class=statechange
Jun 17 13:11:10 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:11:10 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:11:10 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:11:10 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:11:10 cla. zed[2803]: eid=347 class=statechange
Jun 17 13:11:10 cla. zed[2805]: eid=348 class=statechange
Jun 17 13:11:10 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:11:10 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:11:10 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:11:10 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:11:10 cla. zed[2807]: eid=349 class=statechange
Jun 17 13:11:10 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:11:10 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:11:10 cla. zed[2813]: eid=350 class=statechange
Jun 17 13:11:10 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:11:10 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:11:10 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:11:10 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:11:10 cla. zed[2815]: eid=351 class=statechange
Jun 17 13:11:10 cla. zed[2817]: eid=352 class=statechange
Jun 17 13:11:10 cla. zed[2819]: eid=353 class=statechange
Jun 17 13:11:10 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:11:10 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:11:10 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:11:10 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:11:10 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:11:10 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:11:10 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:11:10 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:11:10 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:11:10 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:11:10 cla. zed[2821]: eid=354 class=statechange
Jun 17 13:11:10 cla. zed[2823]: eid=355 class=statechange
Jun 17 13:11:10 cla. zed[2825]: eid=356 class=statechange
Jun 17 13:11:10 cla. zed[2827]: eid=357 class=statechange
Jun 17 13:11:11 cla. zed[2829]: eid=358 class=statechange
Jun 17 13:11:11 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:11:11 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:11:11 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:11:11 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:11:11 cla. zed[2835]: eid=359 class=statechange
Jun 17 13:11:11 cla. zed[2837]: eid=360 class=statechange
Jun 17 13:11:11 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:11:11 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:11:11 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:11:11 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:11:11 cla. zed[2839]: eid=361 class=statechange
Jun 17 13:11:11 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:11:11 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:11:11 cla. zed[2841]: eid=362 class=statechange
Jun 17 13:11:11 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:11:11 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:11:11 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:11:11 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:11:11 cla. zed[2843]: eid=363 class=statechange
Jun 17 13:11:12 cla. zed[2845]: eid=364 class=statechange
Jun 17 13:11:12 cla. zed[2847]: eid=365 class=statechange
Jun 17 13:11:12 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:11:12 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:11:12 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:11:12 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:11:12 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:11:12 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:11:12 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:11:12 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:11:12 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:11:12 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:11:12 cla. zed[2849]: eid=366 class=statechange
Jun 17 13:11:12 cla. zed[2851]: eid=367 class=statechange
Jun 17 13:11:12 cla. zed[2853]: eid=368 class=statechange
Jun 17 13:11:12 cla. zed[2858]: eid=369 class=statechange
Jun 17 13:11:12 cla. zed[2861]: eid=370 class=statechange
Jun 17 13:11:13 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:11:13 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:11:13 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:11:13 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:11:13 cla. zed[2863]: eid=371 class=statechange
Jun 17 13:11:13 cla. zed[2865]: eid=372 class=statechange
Jun 17 13:11:13 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:11:13 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:11:13 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:11:13 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:11:13 cla. zed[2867]: eid=373 class=statechange
Jun 17 13:11:13 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:11:13 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:11:13 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:11:13 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:11:13 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:11:13 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:11:13 cla. zed[2869]: eid=374 class=statechange
Jun 17 13:11:13 cla. zed[2872]: eid=375 class=statechange
Jun 17 13:11:13 cla. zed[2877]: eid=376 class=statechange
Jun 17 13:11:13 cla. zed[2879]: eid=377 class=statechange
Jun 17 13:11:13 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:11:13 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:11:13 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:11:13 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:11:13 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:11:13 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:11:13 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:11:13 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:11:13 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:11:13 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:11:13 cla. zed[2881]: eid=378 class=statechange
Jun 17 13:11:13 cla. zed[2883]: eid=379 class=statechange
Jun 17 13:11:14 cla. zed[2885]: eid=380 class=statechange
Jun 17 13:11:14 cla. zed[2887]: eid=381 class=statechange
Jun 17 13:11:14 cla. zed[2889]: eid=382 class=statechange
Jun 17 13:11:14 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:11:14 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:11:14 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:11:14 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:11:14 cla. zed[2896]: eid=383 class=statechange
Jun 17 13:11:14 cla. zed[2898]: eid=384 class=statechange
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk27s1
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk27s1
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk28s2
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk28s2
Jun 17 13:11:15 cla. zed[2900]: eid=385 class=statechange
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk24s2
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk24s2
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk29s2
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk29s2
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk26s2
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk26s2
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk30s2
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk30s2
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk31s2
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk31s2
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk25s2
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk25s2
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk6s1
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk6s1
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk17s1
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk17s1
Jun 17 13:11:15 cla. zed[2902]: eid=386 class=statechange
Jun 17 13:11:15 cla. zed[2904]: eid=387 class=statechange
Jun 17 13:11:15 cla. zed[2906]: eid=388 class=statechange
Jun 17 13:11:15 cla. zed[2908]: eid=389 class=statechange
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk27s1
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk27s1
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk28s2
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk28s2
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk24s2
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk24s2
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk29s2
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk29s2
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk26s2
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk26s2
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk30s2
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk30s2
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk31s2
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk31s2
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk25s2
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk25s2
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk6s1
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk6s1
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk17s1
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk17s1
Jun 17 13:11:15 cla. zed[2913]: eid=390 class=statechange
Jun 17 13:11:15 cla. zed[2915]: eid=391 class=statechange
Jun 17 13:11:15 cla. zed[2917]: eid=392 class=statechange
Jun 17 13:11:15 cla. zed[2919]: eid=393 class=statechange
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:11:15 cla. zed[2925]: eid=394 class=statechange
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:11:15 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:11:15 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:11:15 cla. zed[2927]: eid=395 class=statechange
Jun 17 13:11:15 cla. zed[2929]: eid=396 class=statechange
Jun 17 13:11:15 cla. zed[2931]: eid=397 class=statechange
Jun 17 13:11:15 cla. zed[2933]: eid=398 class=statechange
Jun 17 13:11:15 cla. zed[2935]: eid=399 class=statechange
Jun 17 13:11:15 cla. zed[2937]: eid=400 class=statechange
Jun 17 13:11:16 cla. zed[2939]: eid=401 class=statechange
Jun 17 13:11:16 cla. zed[2941]: eid=402 class=statechange
Jun 17 13:11:16 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:11:16 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:11:16 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:11:16 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:11:16 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:11:16 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:11:16 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:11:16 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:11:16 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:11:16 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:11:16 cla. zed[2943]: eid=403 class=statechange
Jun 17 13:11:16 cla. zed[2945]: eid=404 class=statechange
Jun 17 13:11:16 cla. zed[2947]: eid=405 class=statechange
Jun 17 13:11:16 cla. zed[2952]: eid=406 class=statechange
Jun 17 13:11:16 cla. zed[2954]: eid=407 class=statechange
Jun 17 13:11:16 cla. zed[2956]: eid=408 class=statechange
Jun 17 13:11:16 cla. zed[2958]: eid=409 class=statechange
Jun 17 13:11:16 cla. zed[2960]: eid=410 class=statechange
Jun 17 13:11:17 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:11:17 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:11:17 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:11:17 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:11:17 cla. zed[2967]: eid=411 class=statechange
Jun 17 13:11:17 cla. zed[2969]: eid=412 class=statechange
Jun 17 13:11:17 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:11:17 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:11:17 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:11:17 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:11:17 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:11:17 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:11:17 cla. zed[2971]: eid=413 class=statechange
Jun 17 13:11:17 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:11:17 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:11:17 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:11:17 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:11:17 cla. zed[2973]: eid=414 class=statechange
Jun 17 13:11:17 cla. zed[2975]: eid=415 class=statechange
Jun 17 13:11:17 cla. zed[2977]: eid=416 class=statechange
Jun 17 13:11:17 cla. zed[2979]: eid=417 class=statechange
Jun 17 13:11:17 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:11:17 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:11:17 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:11:17 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:11:17 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:11:17 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:11:17 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:11:17 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:11:17 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:11:17 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:11:17 cla. zed[2985]: eid=418 class=statechange
Jun 17 13:11:17 cla. zed[2987]: eid=419 class=statechange
Jun 17 13:11:17 cla. zed[2989]: eid=420 class=statechange
Jun 17 13:11:17 cla. zed[2991]: eid=421 class=statechange
Jun 17 13:11:17 cla. zed[2993]: eid=422 class=statechange
Jun 17 13:11:18 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:11:18 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:11:18 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:11:18 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:11:18 cla. zed[2995]: eid=423 class=statechange
Jun 17 13:11:18 cla. zed[2997]: eid=424 class=statechange
and the output of "zpool status -v"
$ zpool status -v
pool: Donkey
state: FAULTED
status: An intent log record could not be read.
Waiting for adminstrator intervention to fix the faulted pool.
action: Either restore the affected device(s) and run 'zpool online',
or ignore the intent log records by running 'zpool clear'.
see: http://zfsonlinux.org/msg/ZFS-8000-K4
scan: resilvered 4.33M in 0h0m with 0 errors on Tue Jun 17 09:43:50 2014
config:
NAME STATE READ WRITE CKSUM
Donkey FAULTED 0 0 0 bad intent log
mirror-0 ONLINE 0 0 0
disk19s2 ONLINE 0 0 0
disk20s2 ONLINE 0 0 0
disk23s2 ONLINE 0 0 0
logs
mirror-1 UNAVAIL 0 0 0 insufficient replicas
disk13s1 FAULTED 0 0 0 corrupted data
disk7s1 UNAVAIL 0 0 0 corrupted data
pool: Trinity
state: UNAVAIL
status: One or more devices could not be opened. There are insufficient
replicas for the pool to continue functioning.
action: Attach the missing device and online it using 'zpool online'.
see: http://zfsonlinux.org/msg/ZFS-8000-3C
scan: none requested
config:
NAME STATE READ WRITE CKSUM
Trinity UNAVAIL 0 0 0 insufficient replicas
mirror-0 UNAVAIL 0 0 0 insufficient replicas
disk27 UNAVAIL 0 0 0 corrupted data
disk28s2 FAULTED 0 0 0 corrupted data
mirror-1 DEGRADED 0 0 0
disk24s2 ONLINE 0 0 0
disk29s2 UNAVAIL 0 0 0 cannot open
mirror-2 UNAVAIL 0 0 0 insufficient replicas
disk26s2 UNAVAIL 0 0 0 cannot open
disk30s2 FAULTED 0 0 0 corrupted data
mirror-3 DEGRADED 0 0 0
disk31s2 FAULTED 0 0 0 corrupted data
disk25s2 ONLINE 0 0 0
logs
mirror-4 UNAVAIL 0 0 0 insufficient replicas
disk6s1 FAULTED 0 0 0 corrupted data
disk17s1 FAULTED 0 0 0 corrupted data
pool: ssdpool
state: DEGRADED
status: One or more devices could not be used because the label is missing or
invalid. Sufficient replicas exist for the pool to continue
functioning in a degraded state.
action: Replace the device using 'zpool replace'.
see: http://zfsonlinux.org/msg/ZFS-8000-4J
scan: scrub canceled on Tue Jun 17 12:07:41 2014
config:
NAME STATE READ WRITE CKSUM
ssdpool DEGRADED 0 0 0
mirror-0 DEGRADED 0 0 0
disk18 UNAVAIL 0 0 0 corrupted data
disk4 ONLINE 0 0 0
logs
mirror-1 DEGRADED 0 0 0
disk16 ONLINE 0 0 0
disk9 UNAVAIL 0 0 0 corrupted data
errors: No known data errors
$ time zpool list & sudo spindump zpool
[2] 5280
Sampling all processes for 10 seconds with 10 milliseconds of run time between samples
Focusing on zpool [5282]
Sampling completed, processing symbols...
Spindump analysis written to file /tmp/zpool_5282.spindump.txt
[1] Done time zpool list
cla:~ aguestpunk$ NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
Donkey - - - - - FAULTED -
Trinity - - - - - FAULTED -
ssdpool 258G 172G 86.1G 66% 1.17x DEGRADED -
real 0m19.034s
user 0m0.010s
sys 0m1.001s
[2]+ Done time zpool list
I'll attach the spindump as a gist.
And the vnode printfs for that run.
It revisits a bunch of devices a lot in about 30 seconds, but only the not-imported pools in /etc/zfs/zpool.cache (it does not examine the imported but degraded pool, nor of course the pool that is never imported and so is never in zpool.cache anyway)) :
revisits device
72 /dev/disk7s1 [Donkey log]
72 /dev/disk13s1 [Donkey log]
48 /dev/disk23s2 [Donkey storage]
48 /dev/disk20s2 [Donkey storage]
48 /dev/disk19s2 [Donkey storage]
8 /dev/disk6s1 [Trinity]
8 /dev/disk31s2
8 /dev/disk30s2
8 /dev/disk29s2
8 /dev/disk28s2
8 /dev/disk27s1
8 /dev/disk26s2
8 /dev/disk25s2
8 /dev/disk24s2
8 /dev/disk17s1
It doesn't visit the devices in the imported (DEGRADED) ssdpool at all.
Jun 17 13:16:28 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:28 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:28 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:28 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:28 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:28 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:28 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:28 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:28 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:28 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:29 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:29 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:29 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:29 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:29 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:29 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:29 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:29 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:29 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:29 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:30 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:30 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:30 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:30 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:30 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:30 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:30 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:30 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:30 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:30 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:30 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:30 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:30 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:30 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:30 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:30 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:30 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:30 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:30 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:30 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:30 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:30 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:30 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:30 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:31 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:31 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:31 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:31 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:32 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:32 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:32 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:32 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:32 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:32 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:32 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:32 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:32 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:32 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:32 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:32 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:32 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:32 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:32 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:32 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:32 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:32 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:32 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:32 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:33 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:33 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:33 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:33 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:33 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:33 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:33 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:33 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:33 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:33 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:34 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:34 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:34 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:34 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:34 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:34 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:34 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:34 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:34 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:34 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:34 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:34 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:34 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:34 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk27s1
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk27s1
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk28s2
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk28s2
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk24s2
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk24s2
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk29s2
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk29s2
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk26s2
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk26s2
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk30s2
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk30s2
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk31s2
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk31s2
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk25s2
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk25s2
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk6s1
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk6s1
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk17s1
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk17s1
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk27s1
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk27s1
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk28s2
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk28s2
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk24s2
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk24s2
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk29s2
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk29s2
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk26s2
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk26s2
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk30s2
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk30s2
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk31s2
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk31s2
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk25s2
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk25s2
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk6s1
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk6s1
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk17s1
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk17s1
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:35 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:35 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:36 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:36 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:36 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:36 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:36 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:36 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:36 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:36 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:36 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:36 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:37 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:37 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:37 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:37 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:37 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:37 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:37 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:37 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:37 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:37 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:37 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:37 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:37 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:37 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:38 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:38 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:38 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:38 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:38 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:38 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:38 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:38 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:38 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:38 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:39 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:39 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:39 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:39 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:42 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:42 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:42 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:42 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:42 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:42 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:42 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:42 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:42 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:42 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:43 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:43 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:43 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:43 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:43 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:43 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:43 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:43 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:43 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:43 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:44 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:44 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:44 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:44 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:45 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:45 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:45 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:45 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:45 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:45 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:45 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:45 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:45 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:45 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:45 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:45 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:45 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:45 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:45 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:45 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:45 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:45 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:45 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:45 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:47 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:47 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:47 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:47 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:47 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:47 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:47 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:47 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:47 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:47 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:47 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:47 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:47 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:47 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:48 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:48 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:48 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:48 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:48 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:48 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:48 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:48 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:48 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:48 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:49 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:49 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:49 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:49 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:49 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:49 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:49 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:49 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:49 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:49 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:49 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:49 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:49 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:49 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:50 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:50 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:50 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:50 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:50 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:50 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:50 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:50 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:50 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:50 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:51 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:51 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:51 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:51 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:52 cla kernel[0]: +vnode_open /dev/disk27s1
Jun 17 13:16:52 cla kernel[0]: -vnode_open /dev/disk27s1
Jun 17 13:16:52 cla kernel[0]: +vnode_open /dev/disk28s2
Jun 17 13:16:52 cla kernel[0]: -vnode_open /dev/disk28s2
Jun 17 13:16:52 cla kernel[0]: +vnode_open /dev/disk24s2
Jun 17 13:16:52 cla kernel[0]: -vnode_open /dev/disk24s2
Jun 17 13:16:52 cla kernel[0]: +vnode_open /dev/disk29s2
Jun 17 13:16:52 cla kernel[0]: -vnode_open /dev/disk29s2
Jun 17 13:16:52 cla kernel[0]: +vnode_open /dev/disk26s2
Jun 17 13:16:52 cla kernel[0]: -vnode_open /dev/disk26s2
Jun 17 13:16:52 cla kernel[0]: +vnode_open /dev/disk30s2
Jun 17 13:16:52 cla kernel[0]: -vnode_open /dev/disk30s2
Jun 17 13:16:52 cla kernel[0]: +vnode_open /dev/disk31s2
Jun 17 13:16:52 cla kernel[0]: -vnode_open /dev/disk31s2
Jun 17 13:16:54 cla kernel[0]: +vnode_open /dev/disk25s2
Jun 17 13:16:54 cla kernel[0]: -vnode_open /dev/disk25s2
Jun 17 13:16:58 cla kernel[0]: +vnode_open /dev/disk6s1
Jun 17 13:16:58 cla kernel[0]: -vnode_open /dev/disk6s1
Jun 17 13:16:58 cla kernel[0]: +vnode_open /dev/disk17s1
Jun 17 13:16:58 cla kernel[0]: -vnode_open /dev/disk17s1
Jun 17 13:16:58 cla kernel[0]: +vnode_open /dev/disk27s1
Jun 17 13:16:58 cla kernel[0]: -vnode_open /dev/disk27s1
Jun 17 13:16:58 cla kernel[0]: +vnode_open /dev/disk28s2
Jun 17 13:16:58 cla kernel[0]: -vnode_open /dev/disk28s2
Jun 17 13:16:58 cla kernel[0]: +vnode_open /dev/disk24s2
Jun 17 13:16:58 cla kernel[0]: -vnode_open /dev/disk24s2
Jun 17 13:16:58 cla kernel[0]: +vnode_open /dev/disk29s2
Jun 17 13:16:58 cla kernel[0]: -vnode_open /dev/disk29s2
Jun 17 13:16:58 cla kernel[0]: +vnode_open /dev/disk26s2
Jun 17 13:16:58 cla kernel[0]: -vnode_open /dev/disk26s2
Jun 17 13:16:58 cla kernel[0]: +vnode_open /dev/disk30s2
Jun 17 13:16:58 cla kernel[0]: -vnode_open /dev/disk30s2
Jun 17 13:16:58 cla kernel[0]: +vnode_open /dev/disk31s2
Jun 17 13:16:58 cla kernel[0]: -vnode_open /dev/disk31s2
Jun 17 13:16:58 cla kernel[0]: +vnode_open /dev/disk25s2
Jun 17 13:16:58 cla kernel[0]: -vnode_open /dev/disk25s2
Jun 17 13:16:58 cla kernel[0]: +vnode_open /dev/disk6s1
Jun 17 13:16:58 cla kernel[0]: -vnode_open /dev/disk6s1
Jun 17 13:16:58 cla kernel[0]: +vnode_open /dev/disk17s1
Jun 17 13:16:58 cla kernel[0]: -vnode_open /dev/disk17s1
Jun 17 13:16:58 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:58 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:58 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:58 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:58 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:58 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:58 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:58 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:58 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:58 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:16:59 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:16:59 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:16:59 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:16:59 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:16:59 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:16:59 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:16:59 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:16:59 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:16:59 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:16:59 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:17:00 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:17:00 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:17:00 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:17:00 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:17:00 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:17:00 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:17:00 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:17:00 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:17:00 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:17:00 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:17:00 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:17:00 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:17:00 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:17:00 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:17:00 cla kernel[0]: +vnode_open /dev/disk19s2
Jun 17 13:17:00 cla kernel[0]: -vnode_open /dev/disk19s2
Jun 17 13:17:00 cla kernel[0]: +vnode_open /dev/disk20s2
Jun 17 13:17:00 cla kernel[0]: -vnode_open /dev/disk20s2
Jun 17 13:17:00 cla kernel[0]: +vnode_open /dev/disk23s2
Jun 17 13:17:00 cla kernel[0]: -vnode_open /dev/disk23s2
Jun 17 13:17:00 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:17:00 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:17:00 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:17:00 cla kernel[0]: -vnode_open /dev/disk7s1
Jun 17 13:17:01 cla kernel[0]: +vnode_open /dev/disk13s1
Jun 17 13:17:01 cla kernel[0]: -vnode_open /dev/disk13s1
Jun 17 13:17:01 cla kernel[0]: +vnode_open /dev/disk7s1
Jun 17 13:17:01 cla kernel[0]: -vnode_open /dev/disk7s1
Yes, easy enough to apply and I can engineer an unclean shutdown. I want to wait on the resilver, and who knows when that will end.
estimates on resilvering is a cross-platform bear because of wildly inaccurate estimates of how long the metadata scan phase will take. Fortunately all the storage devices are kept busy reading during a resilver because of the scattering of the extra metadata copies when there are multiple top level vdevs.
Oh, and it just finished, yay.