JuiceFS csi-v0 logs example

juicefs-csi-controller.md

$ stern -n kube-system juicefs-csi-controller

juicefs-plugin I1021 06:37:52.136819       1 mount_linux.go:174 Cannot run systemd-run, assuming non-systemd OS
juicefs-plugin I1021 06:37:52.137071       1 mount_linux.go:175 systemd-run failed with: exit status 1
juicefs-plugin I1021 06:37:52.137110       1 mount_linux.go:176 systemd-run output: Failed to create bus connection: No such file or directory
juicefs-plugin I1021 06:37:52.137356       1 driver.go:112 Listening for connections on address: &net.UnixAddr{Name:"/var/lib/csi/sockets/pluginproxy/csi.sock", Net:"unix"}
csi-attacher I1021 06:38:09.108893       1 main.go:77 Version: v0.4.2-2-g2a08048
csi-attacher I1021 06:38:09.111942       1 connection.go:89 Connecting to /var/lib/csi/sockets/pluginproxy/csi.sock
csi-attacher I1021 06:38:09.112396       1 connection.go:116 Still trying, connection is CONNECTING
csi-attacher I1021 06:38:09.112966       1 connection.go:113 Connected
csi-attacher I1021 06:38:09.113209       1 connection.go:236 GRPC call: /csi.v0.Identity/Probe
csi-attacher I1021 06:38:09.113250       1 connection.go:237 GRPC request: {}
csi-attacher I1021 06:38:09.116194       1 connection.go:239 GRPC response: {}
csi-attacher I1021 06:38:09.122886       1 connection.go:240 GRPC error: <nil>
csi-attacher I1021 06:38:09.122894       1 main.go:215 Probe succeeded
csi-attacher I1021 06:38:09.123146       1 connection.go:236 GRPC call: /csi.v0.Identity/GetPluginInfo
csi-attacher I1021 06:38:09.123164       1 connection.go:237 GRPC request: {}
csi-attacher I1021 06:38:09.125620       1 connection.go:239 GRPC response: {"name":"csi.juicefs.com","vendor_version":"0.1.0"}
csi-attacher I1021 06:38:09.126232       1 connection.go:240 GRPC error: <nil>
csi-attacher I1021 06:38:09.126239       1 main.go:129 CSI driver name: "csi.juicefs.com"
csi-attacher I1021 06:38:09.126245       1 connection.go:236 GRPC call: /csi.v0.Identity/GetPluginCapabilities
csi-attacher I1021 06:38:09.126248       1 connection.go:237 GRPC request: {}
csi-attacher I1021 06:38:09.128936       1 connection.go:239 GRPC response: {}
csi-attacher I1021 06:38:09.129945       1 connection.go:240 GRPC error: <nil>
csi-attacher I1021 06:38:09.130001       1 main.go:138 CSI driver does not support Plugin Controller Service, using trivial handler
csi-attacher I1021 06:38:09.130925       1 reflector.go:131 Starting reflector *v1beta1.VolumeAttachment (10m0s) from k8s.io/client-go/informers/factory.go:131
csi-attacher I1021 06:38:09.131060       1 reflector.go:169 Listing and watching *v1beta1.VolumeAttachment from k8s.io/client-go/informers/factory.go:131
csi-attacher I1021 06:38:09.131710       1 reflector.go:131 Starting reflector *v1.PersistentVolume (10m0s) from k8s.io/client-go/informers/factory.go:131
csi-attacher I1021 06:38:09.131787       1 reflector.go:169 Listing and watching *v1.PersistentVolume from k8s.io/client-go/informers/factory.go:131
csi-attacher I1021 06:38:09.132227       1 controller.go:110 Starting CSI attacher
csi-attacher I1021 06:38:09.232627       1 shared_informer.go:123 caches populated
...
csi-attacher I1021 06:46:10.147039       1 reflector.go:357 k8s.io/client-go/informers/factory.go:131: Watch close - *v1.PersistentVolume total 0 items received
csi-attacher I1021 06:47:51.148839       1 reflector.go:357 k8s.io/client-go/informers/factory.go:131: Watch close - *v1beta1.VolumeAttachment total 0 items received
csi-attacher I1021 06:48:09.145984       1 reflector.go:215 k8s.io/client-go/informers/factory.go:131: forcing resync
csi-attacher I1021 06:48:09.148203       1 reflector.go:215 k8s.io/client-go/informers/factory.go:131: forcing resync
csi-attacher I1021 06:54:29.153065       1 reflector.go:357 k8s.io/client-go/informers/factory.go:131: Watch close - *v1.PersistentVolume total 0 items received
csi-attacher I1021 06:55:02.156051       1 reflector.go:357 k8s.io/client-go/informers/factory.go:131: Watch close - *v1beta1.VolumeAttachment total 0 items received
csi-attacher I1021 06:57:33.720971       1 controller.go:197 Started PV processing "juicefs"
csi-attacher I1021 06:57:33.759829       1 controller.go:197 Started PV processing "juicefs"
csi-attacher I1021 06:57:33.771447       1 controller.go:197 Started PV processing "juicefs"
csi-attacher I1021 06:57:33.890683       1 controller.go:167 Started VA processing "csi-e099e59a67fe3e478ca1a0a9e096351db18db43bad88dbc68518ce59c3ff99f7"
csi-attacher I1021 06:57:33.894384       1 trivial_handler.go:48 Trivial sync[csi-e099e59a67fe3e478ca1a0a9e096351db18db43bad88dbc68518ce59c3ff99f7 started
csi-attacher I1021 06:57:33.895381       1 util.go:33 Marking as attached "csi-e099e59a67fe3e478ca1a0a9e096351db18db43bad88dbc68518ce59c3ff99f7"
csi-attacher I1021 06:57:33.919664       1 util.go:43 Marked as attached "csi-e099e59a67fe3e478ca1a0a9e096351db18db43bad88dbc68518ce59c3ff99f7"
csi-attacher I1021 06:57:33.919702       1 trivial_handler.go:56 Marked VolumeAttachment csi-e099e59a67fe3e478ca1a0a9e096351db18db43bad88dbc68518ce59c3ff99f7 as attached
csi-attacher I1021 06:57:33.919729       1 controller.go:167 Started VA processing "csi-e099e59a67fe3e478ca1a0a9e096351db18db43bad88dbc68518ce59c3ff99f7"
csi-attacher I1021 06:57:33.920063       1 trivial_handler.go:48 Trivial sync[csi-e099e59a67fe3e478ca1a0a9e096351db18db43bad88dbc68518ce59c3ff99f7 started
csi-attacher I1021 06:58:09.147118       1 reflector.go:215 k8s.io/client-go/informers/factory.go:131: forcing resync
csi-attacher I1021 06:58:09.147402       1 controller.go:197 Started PV processing "juicefs"
csi-attacher I1021 06:58:09.148416       1 reflector.go:215 k8s.io/client-go/informers/factory.go:131: forcing resync
csi-attacher I1021 06:58:09.148701       1 controller.go:167 Started VA processing "csi-e099e59a67fe3e478ca1a0a9e096351db18db43bad88dbc68518ce59c3ff99f7"
csi-attacher I1021 06:58:09.148761       1 trivial_handler.go:48 Trivial sync[csi-e099e59a67fe3e478ca1a0a9e096351db18db43bad88dbc68518ce59c3ff99f7 started

juicefs-csi-node.md

$ stern -n kube-system juicefs-csi-node

juicefs-plugin I1021 06:37:48.975958       1 mount_linux.go:174 Cannot run systemd-run, assuming non-systemd OS
juicefs-plugin I1021 06:37:48.999577       1 mount_linux.go:175 systemd-run failed with: exit status 1
juicefs-plugin I1021 06:37:48.999589       1 mount_linux.go:176 systemd-run output: Failed to create bus connection: No such file or directory
juicefs-plugin I1021 06:37:49.004144       1 driver.go:112 Listening for connections on address: &net.UnixAddr{Name:"/csi/csi.sock", Net:"unix"}
juicefs-plugin I1021 06:38:00.965705       1 node.go:194 NodeGetId: called with args
juicefs-plugin I1021 06:38:00.983417       1 node.go:185 NodeGetInfo: called with args
csi-driver-registrar I1021 06:38:00.960439       1 main.go:138 Version: v0.4.2-2-ga8fd229d
csi-driver-registrar I1021 06:38:00.960513       1 main.go:145 Attempting to open a gRPC connection with: "/csi/csi.sock"
csi-driver-registrar I1021 06:38:00.960523       1 connection.go:69 Connecting to /csi/csi.sock
csi-driver-registrar I1021 06:38:00.960819       1 connection.go:96 Still trying, connection is CONNECTING
csi-driver-registrar I1021 06:38:00.961067       1 connection.go:93 Connected
csi-driver-registrar I1021 06:38:00.961086       1 main.go:153 Calling CSI driver to discover driver name.
csi-driver-registrar I1021 06:38:00.961094       1 connection.go:137 GRPC call: /csi.v0.Identity/GetPluginInfo
csi-driver-registrar I1021 06:38:00.961098       1 connection.go:138 GRPC request: {}
csi-driver-registrar I1021 06:38:00.962597       1 connection.go:140 GRPC response: {"name":"csi.juicefs.com","vendor_version":"0.1.0"}
csi-driver-registrar I1021 06:38:00.963148       1 connection.go:141 GRPC error: <nil>
csi-driver-registrar I1021 06:38:00.963154       1 main.go:161 CSI driver name: "csi.juicefs.com"
csi-driver-registrar I1021 06:38:00.963158       1 main.go:165 Loading kubeconfig.
csi-driver-registrar I1021 06:38:00.963309       1 node_register.go:55 Calling CSI driver to discover node ID.
csi-driver-registrar I1021 06:38:00.963317       1 connection.go:137 GRPC call: /csi.v0.Node/NodeGetId
csi-driver-registrar I1021 06:38:00.963321       1 connection.go:138 GRPC request: {}
csi-driver-registrar I1021 06:38:00.966070       1 connection.go:140 GRPC response: {"node_id":"minikube"}
csi-driver-registrar I1021 06:38:00.973204       1 connection.go:141 GRPC error: <nil>
csi-driver-registrar I1021 06:38:00.973213       1 node_register.go:63 CSI driver node ID: "minikube"
csi-driver-registrar I1021 06:38:00.973240       1 node_register.go:86 Starting Registration Server at: /registration/csi.juicefs.com-reg.sock
csi-driver-registrar I1021 06:38:00.973311       1 node_register.go:93 Registration Server started at: /registration/csi.juicefs.com-reg.sock
csi-driver-registrar I1021 06:38:00.975355       1 main.go:111 Received GetInfo call: &InfoRequest{}
csi-driver-registrar I1021 06:38:00.997952       1 main.go:121 Received NotifyRegistrationStatus call: &RegistrationStatus{PluginRegistered:true,Error:,}
...
juicefs-plugin I1021 06:57:41.533048       1 node.go:168 NodeGetCapabilities: called with args
juicefs-plugin I1021 06:57:41.534329       1 node.go:168 NodeGetCapabilities: called with args
juicefs-plugin I1021 06:57:41.540064       1 node.go:62 NodePublishVolume: volume_id is tencentcloud-ap-hongkong
juicefs-plugin I1021 06:57:41.540082       1 node.go:73 NodePublishVolume: volume_capability is mount:<fs_type:"juicefs" > access_mode:<mode:MULTI_NODE_MULTI_WRITER >
juicefs-plugin I1021 06:57:41.540177       1 node.go:79 NodePublishVolume: creating dir /var/lib/kubelet/pods/0eb0dc02-f3d0-11e9-aca1-080027f086c5/volumes/kubernetes.io~csi/juicefs/mount
juicefs-plugin I1021 06:57:41.540197       1 node.go:85 NodePublishVolume: NodePublishSecret contains keys [accesskey bucket name secretkey token
juicefs-plugin I1021 06:57:45.937532       1 node.go:92 NodePublishVolume: authentication output is
juicefs-plugin I1021 06:57:45.937730       1 node.go:125 NodePublishVolume: mounting "tencentcloud-ap-hongkong" at "/jfs/tencentcloud-ap-hongkong" with options
juicefs-plugin I1021 06:57:45.937823       1 mount_linux.go:146 Mounting cmd (mount) with arguments ([-t juicefs tencentcloud-ap-hongkong /jfs/tencentcloud-ap-hongkong])
juicefs-plugin I1021 06:57:48.391061       1 node.go:138 NodePublishVolume: binding tencentcloud-ap-hongkong at /jfs/tencentcloud-ap-hongkong with options
juicefs-plugin I1021 06:57:48.391239       1 mount_linux.go:146 Mounting cmd (mount) with arguments ([-t none -o bind /jfs/tencentcloud-ap-hongkong /var/lib/kubelet/pods/0eb0dc02-f3d0-11e9-aca1-080027f086c5/volumes/kubernetes.io~csi/juicefs/mount])
juicefs-plugin I1021 06:57:48.393203       1 mount_linux.go:146 Mounting cmd (mount) with arguments ([-t none -o bind,remount /jfs/tencentcloud-ap-hongkong /var/lib/kubelet/pods/0eb0dc02-f3d0-11e9-aca1-080027f086c5/volumes/kubernetes.io~csi/juicefs/mount])
juicefs-plugin I1021 06:57:48.397508       1 node.go:144 NodePublishVolume: mounted tencentcloud-ap-hongkong at /var/lib/kubelet/pods/0eb0dc02-f3d0-11e9-aca1-080027f086c5/volumes/kubernetes.io~csi/juicefs/mount with options

kube-controller-manager.md

$ stern -n kube-system kube-controller-manager

...
I1021 06:37:25.885418       1 event.go:221] Event(v1.ObjectReference{Kind:"StatefulSet", Namespace:"kube-system", Name:"juicefs-csi-controller", UID:"3ec37b11-f3cd-11e9-aca1-080027f086c5", APIVersion:"apps/v1", ResourceVersion:"588", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' create Pod juicefs-csi-controller-0 in StatefulSet juicefs-csi-controller successful
I1021 06:37:25.934860       1 event.go:221] Event(v1.ObjectReference{Kind:"DaemonSet", Namespace:"kube-system", Name:"juicefs-csi-node", UID:"3ec5daac-f3cd-11e9-aca1-080027f086c5", APIVersion:"apps/v1", ResourceVersion:"589", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: juicefs-csi-node-kjwk8
W1021 06:47:44.153193       1 reflector.go:270] k8s.io/client-go/informers/factory.go:131: watch of *v1beta1.Event ended with: The resourceVersion for the provided watch is too old.
E1021 06:57:33.665953       1 pv_protection_controller.go:116] PV juicefs failed with : Operation cannot be fulfilled on persistentvolumes "juicefs": the object has been modified; please apply your changes to the latest version and try again
E1021 06:57:33.677713       1 pv_protection_controller.go:116] PV juicefs failed with : Operation cannot be fulfilled on persistentvolumes "juicefs": the object has been modified; please apply your changes to the latest version and try again
I1021 06:57:33.876601       1 reconciler.go:289] attacherDetacher.AttachVolume started for volume "juicefs" (UniqueName: "kubernetes.io/csi/csi.juicefs.com^tencentcloud-ap-hongkong") from node "minikube"
I1021 06:57:33.916979       1 operation_generator.go:336] AttachVolume.Attach succeeded for volume "juicefs" (UniqueName: "kubernetes.io/csi/csi.juicefs.com^tencentcloud-ap-hongkong") from node "minikube"
I1021 06:57:33.917248       1 event.go:221] Event(v1.ObjectReference{Kind:"Pod", Namespace:"default", Name:"juicefs-app", UID:"0eb0dc02-f3d0-11e9-aca1-080027f086c5", APIVersion:"v1", ResourceVersion:"2090", FieldPath:""}): type: 'Normal' reason: 'SuccessfulAttachVolume' AttachVolume.Attach succeeded for volume "juicefs"
W1021 07:11:18.247071       1 reflector.go:270] k8s.io/client-go/informers/factory.go:131: watch of *v1beta1.Event ended with: The resourceVersion for the provided watch is too old.

kubelet.md

$ sudo journalctl -u kubelet

Oct 21 06:37:50 minikube kubelet[3395]: E1021 06:37:50.002416    3395 plugin_watcher.go:115] error failed to get plugin info using RPC GetInfo at socket /var/lib/kubelet/plugins/csi.juicefs.com/csi.sock, err: rpc error: code = Unimplement
ed desc = unknown service pluginregistration.Registration when handling create event: "/var/lib/kubelet/plugins/csi.juicefs.com/csi.sock": CREATE
Oct 21 06:38:00 minikube kubelet[3395]: I1021 06:38:00.982104    3395 csi_plugin.go:111] kubernetes.io/csi: Trying to register a new plugin with name: csi.juicefs.com endpoint: /var/lib/kubelet/plugins/csi.juicefs.com/csi.sock versions: 0
.2.0,0.3.0
Oct 21 06:38:00 minikube kubelet[3395]: I1021 06:38:00.982142    3395 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: csi.juicefs.com at endpoint: /var/lib/kubelet/plugins/csi.juicefs.com/csi.sock
...
Oct 21 06:57:33 minikube kubelet[3395]: I1021 06:57:33.883203    3395 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "juicefs" (UniqueName: "kubernetes.io/csi/csi.juicefs.com^tencentcloud-ap-hongkon
g") pod "juicefs-app" (UID: "0eb0dc02-f3d0-11e9-aca1-080027f086c5")
Oct 21 06:57:33 minikube kubelet[3395]: I1021 06:57:33.883262    3395 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "default-token-h8jsr" (UniqueName: "kubernetes.io/secret/0eb0dc02-f3d0-11e9-aca1-
080027f086c5-default-token-h8jsr") pod "juicefs-app" (UID: "0eb0dc02-f3d0-11e9-aca1-080027f086c5")
Oct 21 06:57:33 minikube kubelet[3395]: E1021 06:57:33.883361    3395 nestedpendingoperations.go:267] Operation for "\"kubernetes.io/csi/csi.juicefs.com^tencentcloud-ap-hongkong\"" failed. No retries permitted until 2019-10-21 06:57:34.38
3332284 +0000 UTC m=+1397.696900367 (durationBeforeRetry 500ms). Error: "Volume has not been added to the list of VolumesInUse in the node's volume status for volume \"juicefs\" (UniqueName: \"kubernetes.io/csi/csi.juicefs.com^tencentclou
d-ap-hongkong\") pod \"juicefs-app\" (UID: \"0eb0dc02-f3d0-11e9-aca1-080027f086c5\") "
Oct 21 06:57:34 minikube kubelet[3395]: I1021 06:57:34.385643    3395 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "juicefs" (UniqueName: "kubernetes.io/csi/csi.juicefs.com^tencentcloud-ap-hongkon
g") pod "juicefs-app" (UID: "0eb0dc02-f3d0-11e9-aca1-080027f086c5")
Oct 21 06:57:34 minikube kubelet[3395]: E1021 06:57:34.385785    3395 nestedpendingoperations.go:267] Operation for "\"kubernetes.io/csi/csi.juicefs.com^tencentcloud-ap-hongkong\"" failed. No retries permitted until 2019-10-21 06:57:35.38
5763263 +0000 UTC m=+1398.699331347 (durationBeforeRetry 1s). Error: "Volume has not been added to the list of VolumesInUse in the node's volume status for volume \"juicefs\" (UniqueName: \"kubernetes.io/csi/csi.juicefs.com^tencentcloud-a
p-hongkong\") pod \"juicefs-app\" (UID: \"0eb0dc02-f3d0-11e9-aca1-080027f086c5\") "
Oct 21 06:57:35 minikube kubelet[3395]: I1021 06:57:35.391534    3395 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "juicefs" (UniqueName: "kubernetes.io/csi/csi.juicefs.com^tencentcloud-ap-hongkon
g") pod "juicefs-app" (UID: "0eb0dc02-f3d0-11e9-aca1-080027f086c5")
Oct 21 06:57:35 minikube kubelet[3395]: E1021 06:57:35.392050    3395 nestedpendingoperations.go:267] Operation for "\"kubernetes.io/csi/csi.juicefs.com^tencentcloud-ap-hongkong\"" failed. No retries permitted until 2019-10-21 06:57:37.39
2023408 +0000 UTC m=+1400.705591486 (durationBeforeRetry 2s). Error: "Volume has not been added to the list of VolumesInUse in the node's volume status for volume \"juicefs\" (UniqueName: \"kubernetes.io/csi/csi.juicefs.com^tencentcloud-a
p-hongkong\") pod \"juicefs-app\" (UID: \"0eb0dc02-f3d0-11e9-aca1-080027f086c5\") "
Oct 21 06:57:37 minikube kubelet[3395]: I1021 06:57:37.401045    3395 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "juicefs" (UniqueName: "kubernetes.io/csi/csi.juicefs.com^tencentcloud-ap-hongkon
g") pod "juicefs-app" (UID: "0eb0dc02-f3d0-11e9-aca1-080027f086c5")
Oct 21 06:57:37 minikube kubelet[3395]: E1021 06:57:37.401202    3395 nestedpendingoperations.go:267] Operation for "\"kubernetes.io/csi/csi.juicefs.com^tencentcloud-ap-hongkong\"" failed. No retries permitted until 2019-10-21 06:57:41.40
118077 +0000 UTC m=+1404.714748853 (durationBeforeRetry 4s). Error: "Volume has not been added to the list of VolumesInUse in the node's volume status for volume \"juicefs\" (UniqueName: \"kubernetes.io/csi/csi.juicefs.com^tencentcloud-ap
-hongkong\") pod \"juicefs-app\" (UID: \"0eb0dc02-f3d0-11e9-aca1-080027f086c5\") "
Oct 21 06:57:41 minikube kubelet[3395]: I1021 06:57:41.427038    3395 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "juicefs" (UniqueName: "kubernetes.io/csi/csi.juicefs.com^tencentcloud-ap-hongkon
g") pod "juicefs-app" (UID: "0eb0dc02-f3d0-11e9-aca1-080027f086c5")
Oct 21 06:57:41 minikube kubelet[3395]: I1021 06:57:41.429819    3395 operation_generator.go:1205] Controller attach succeeded for volume "juicefs" (UniqueName: "kubernetes.io/csi/csi.juicefs.com^tencentcloud-ap-hongkong") pod "juicefs-ap
p" (UID: "0eb0dc02-f3d0-11e9-aca1-080027f086c5") device path: "csi-e099e59a67fe3e478ca1a0a9e096351db18db43bad88dbc68518ce59c3ff99f7"
Oct 21 06:57:41 minikube kubelet[3395]: I1021 06:57:41.528613    3395 operation_generator.go:498] MountVolume.WaitForAttach entering for volume "juicefs" (UniqueName: "kubernetes.io/csi/csi.juicefs.com^tencentcloud-ap-hongkong") pod "juic
efs-app" (UID: "0eb0dc02-f3d0-11e9-aca1-080027f086c5") DevicePath "csi-e099e59a67fe3e478ca1a0a9e096351db18db43bad88dbc68518ce59c3ff99f7"
Oct 21 06:57:41 minikube kubelet[3395]: I1021 06:57:41.530902    3395 operation_generator.go:507] MountVolume.WaitForAttach succeeded for volume "juicefs" (UniqueName: "kubernetes.io/csi/csi.juicefs.com^tencentcloud-ap-hongkong") pod "jui
cefs-app" (UID: "0eb0dc02-f3d0-11e9-aca1-080027f086c5") DevicePath "csi-e099e59a67fe3e478ca1a0a9e096351db18db43bad88dbc68518ce59c3ff99f7"
Oct 21 06:57:41 minikube kubelet[3395]: I1021 06:57:41.533453    3395 csi_attacher.go:352] kubernetes.io/csi: attacher.MountDevice STAGE_UNSTAGE_VOLUME capability not set. Skipping MountDevice...
Oct 21 06:57:41 minikube kubelet[3395]: I1021 06:57:41.533500    3395 operation_generator.go:528] MountVolume.MountDevice succeeded for volume "juicefs" (UniqueName: "kubernetes.io/csi/csi.juicefs.com^tencentcloud-ap-hongkong") pod "juice
fs-app" (UID: "0eb0dc02-f3d0-11e9-aca1-080027f086c5") device mount path "/var/lib/kubelet/plugins/kubernetes.io/csi/pv/juicefs/globalmount"