Skip to content

Instantly share code, notes, and snippets.

@lachie83

lachie83/azure-k8s-1-6-pv-error.md

Last active May 9, 2017 20:45
Show Gist options
  • Save lachie83/c1d39326ab8e904cc7a89b190a1c4c1b to your computer and use it in GitHub Desktop.
Save lachie83/c1d39326ab8e904cc7a89b190a1c4c1b to your computer and use it in GitHub Desktop.
Download ZIP
Raw
azure-k8s-1-6-pv-error.md

Error Message

MountVolume.SetUp failed for volume "kubernetes.io/azure-disk/889cb7d1-34f6-11e7-bda0-000d3a19697d-pvc-88904331-34f6-11e7-bda0-000d3a19697d" (spec.Name: "pvc-88904331-34f6-11e7-bda0-000d3a19697d") pod "889cb7d1-34f6-11e7-bda0-000d3a19697d" (UID: "889cb7d1-34f6-11e7-bda0-000d3a19697d") with: mount failed: exit status 32

Details

kubectl describe po/<podname>
<snip>
Events:
  FirstSeen	LastSeen	Count	From					SubObjectPath	Type		Reason			Message
  ---------	--------	-----	----					-------------	--------	------			-------
  6m		6m		2	default-scheduler					Warning		FailedScheduling	[SchedulerPredicates failed due to PersistentVolumeClaim is not bound: "iced-fish-jenkins", which is unexpected., SchedulerPredicates failed due to PersistentVolumeClaim is not bound: "iced-fish-jenkins", which is unexpected., SchedulerPredicates failed due to PersistentVolumeClaim is not bound: "iced-fish-jenkins", which is unexpected.]
  6m		6m		1	default-scheduler					Normal		Scheduled		Successfully assigned iced-fish-jenkins-1799419808-b1g5s to k8s-levoc02ap01-20728491-2
  4m		26s		3	kubelet, k8s-levoc02ap01-20728491-2			Warning		FailedMount		Unable to mount volumes for pod "iced-fish-jenkins-1799419808-b1g5s_default(889cb7d1-34f6-11e7-bda0-000d3a19697d)": timeout expired waiting for volumes to attach/mount for pod "default"/"iced-fish-jenkins-1799419808-b1g5s". list of unattached/unmounted volumes=[jenkins-home]
  4m		26s		3	kubelet, k8s-levoc02ap01-20728491-2			Warning		FailedSync		Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod "default"/"iced-fish-jenkins-1799419808-b1g5s". list of unattached/unmounted volumes=[jenkins-home]
  6m		14s		11	kubelet, k8s-levoc02ap01-20728491-2			Warning		FailedMount		MountVolume.SetUp failed for volume "kubernetes.io/azure-disk/889cb7d1-34f6-11e7-bda0-000d3a19697d-pvc-88904331-34f6-11e7-bda0-000d3a19697d" (spec.Name: "pvc-88904331-34f6-11e7-bda0-000d3a19697d") pod "889cb7d1-34f6-11e7-bda0-000d3a19697d" (UID: "889cb7d1-34f6-11e7-bda0-000d3a19697d") with: mount failed: exit status 32
Mounting command: mount
Mounting arguments: /var/lib/kubelet/plugins/kubernetes.io/azure-disk/mounts/levoc02-dynamic-pvc-88904331-34f6-11e7-bda0-000d3a19697d.vhd /var/lib/kubelet/pods/889cb7d1-34f6-11e7-bda0-000d3a19697d/volumes/kubernetes.io~azure-disk/pvc-88904331-34f6-11e7-bda0-000d3a19697d  [bind]
Output: mount: special device /var/lib/kubelet/plugins/kubernetes.io/azure-disk/mounts/levoc02-dynamic-pvc-88904331-34f6-11e7-bda0-000d3a19697d.vhd does not exist

Showing PVC as bound

$ kubectl get pv,pvc
NAME                                          CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS    CLAIM                       STORAGECLASS   REASON    AGE
pv/pvc-88904331-34f6-11e7-bda0-000d3a19697d   8Gi        RWO           Delete          Bound     default/iced-fish-jenkins   default                  9m

NAME                    STATUS    VOLUME                                     CAPACITY   ACCESSMODES   STORAGECLASS   AGE
pvc/iced-fish-jenkins   Bound     pvc-88904331-34f6-11e7-bda0-000d3a19697d   8Gi        RWO           default        9m

PV deets

$ kubectl describe pv pvc-88904331-34f6-11e7-bda0-000d3a19697d
Name:		pvc-88904331-34f6-11e7-bda0-000d3a19697d
Labels:		<none>
Annotations:	kubernetes.io/createdby=azure-disk-dynamic-provisioner
		pv.kubernetes.io/bound-by-controller=yes
		pv.kubernetes.io/provisioned-by=kubernetes.io/azure-disk
StorageClass:	default
Status:		Bound
Claim:		default/iced-fish-jenkins
Reclaim Policy:	Delete
Access Modes:	RWO
Capacity:	8Gi
Message:
Source:
    Type:		AzureDisk (an Azure Data Disk mount on the host and bind mount to the pod)
    DiskName:		levoc02-dynamic-pvc-88904331-34f6-11e7-bda0-000d3a19697d.vhd
    DiskURI:		https://f4hrn6tx5suoymstr0.blob.core.windows.net/vhds/levoc02-dynamic-pvc-88904331-34f6-11e7-bda0-000d3a19697d.vhd
    FSType:		ext4
    CachingMode:	None
    ReadOnly:		false
Events:			<none>

I see the following blob storage device in the AZ ui.

https://f4hrn6tx5suoymstr0.blob.core.windows.net/vhds/levoc02-dynamic-pvc-88904331-34f6-11e7-bda0-000d3a19697d.vhd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment