Skip to content

Instantly share code, notes, and snippets.

@misterdjules
Created July 20, 2017 00:39
Show Gist options
  • Select an option

  • Save misterdjules/3399631742564df81aa963a29d5c03cf to your computer and use it in GitHub Desktop.

Select an option

Save misterdjules/3399631742564df81aa963a29d5c03cf to your computer and use it in GitHub Desktop.
mount -F nfs 192.168.128.5:/exports/data /var/tmp/data/

From the container mounting the NFS filesystem:

[root@6823ef52-00a3-4c12-990e-90b08923f110 ~]# pstack 86456
86456:  vi /var/tmp/data/bar.txt
[root@6823ef52-00a3-4c12-990e-90b08923f110 ~]#
[root@S1155632YM00197 (nightly-1) ~]# mdb -k
Loading modules: [ unix genunix specfs dtrace mac cpu.generic uppc apix scsi_vhci ufs ip hook neti sockfs arp usba stmf_sbd stmf zfs mm sd lofs idm sata mpt_sas crypto random cpc logindmux ptm kvm sppp nsmb smbsrv nfs ipc ]
> 0t86456::pid2proc |::walk thread |::findstack -v
stack pointer for thread ffffff39a6abb760: ffffff00f847a920
[ ffffff00f847a920 _resume_from_idle+0x112() ]
  ffffff00f847a950 swtch+0x141()
  ffffff00f847a990 cv_wait+0x70(ffffff0072ad83d6, fffffffffbc447d8)
  ffffff00f847a9d0 page_io_wait+0x53(ffffff0072ad8390)
  ffffff00f847aaa0 pvn_vplist_dirty+0x3ca(ffffff3d7b327b40, 0, fffffffff830ae90, 0, ffffff33ae8e8a30)
  ffffff00f847ab50 nfs_putpages+0x2a0(ffffff3d7b327b40, 0, 0, 0, ffffff33ae8e8a30)
  ffffff00f847abd0 nfs3_putpage+0xe5(ffffff3d7b327b40, 0, 0, 0, ffffff33ae8e8a30, 0)
  ffffff00f847ac40 nfs3_putpage_commit+0xb8(ffffff3d7b327b40, 0, 0, ffffff33ae8e8a30)
  ffffff00f847ad60 nfs3_close+0x1e6(ffffff3d7b327b40, 2503, 1, 1000, ffffff33ae8e8a30, 0)
  ffffff00f847ade0 fop_close+0x61(ffffff3d7b327b40, 2503, 1, 1000, ffffff33ae8e8a30, 0)
  ffffff00f847ae20 closef+0x5e(ffffff33a59d3e88)
  ffffff00f847ae90 closeandsetf+0x398(4, 0)
  ffffff00f847aeb0 close+0x13(4)
  ffffff00f847af10 _sys_sysenter_post_swapgs+0x153()
> fffffffffbc447d8::mutex
            ADDR  TYPE             HELD MINSPL OLDSPL WAITERS
fffffffffbc447d8 adapt               no      -      -      no
> ffffff0072ad83d6::wchaninfo
ADDR             TYPE NWAITERS
ffffff0072ad83d6 cond        1
>

On the client, after a while we get:

[root@6823ef52-00a3-4c12-990e-90b08923f110 ~]# mount -F nfs 192.168.128.5:/exports/data /var/tmp/data/
NFS server 192.168.128.5 not responding still trying
NFS server 192.168.128.5 ok

and the filesystem is sucessfully mounted.

On the server, we get the following in the logs:

[2017-07-19T22:43:20.738Z] ERROR: server.js/RpcServer/64595 on 4464b653-0872-4fd5-82d5-ae8dbb6bfdf9 (/opt/smartdc/nfsserver/node_modules/sdc-nfs/node_modules/oncrpc/lib/server.js:56): connection error occurred (err.code=ECONNRESET)
    Error: read ECONNRESET
        at errnoException (net.js:905:11)
        at TCP.onread (net.js:559:19)
[2017-07-19T22:43:42.343Z]  INFO: server.js/RpcServer/64595 on 4464b653-0872-4fd5-82d5-ae8dbb6bfdf9 (/opt/smartdc/nfsserver/node_modules/sdc-nfs/lib/mount.js:119): mount(/exports/data) from (192.168.128.8): done -> 386d7b0d-63d0-4285-bb85-63ad54d2d932 (procedure=mnt, xid=1500135784)
ping -I 1 192.168.128.5

from the container mounting the volume to the volume itself did not output any error and had 0 packet loss.

Another repro had the following similar but different symptoms:

[root@d3976b0e-0686-61f8-f44f-82ce62503fee ~]# pstack $(pgrep vi)
93728:  vi /var/tmp/data/baz.txt
 00000000 chmod    (8291b00, 1a4)
> 0t93728::pid2proc | ::walk thread | ::findstack
stack pointer for thread ffffff39a6af8080: ffffff00f6d7edb0
[ ffffff00f6d7edb0 _resume_from_idle+0x112() ]
  ffffff00f6d7ede0 swtch+0x141()
  ffffff00f6d7ee80 cv_timedwait_sig_hires+0x39d()
  ffffff00f6d7eed0 cv_timedwait_sig+0x49()
  ffffff00f6d7f070 clnt_cots_kcallit+0x956()
  ffffff00f6d7f1c0 rfscall+0x4ee()
  ffffff00f6d7f2a0 rfs3call+0xae()
  ffffff00f6d7f4c0 nfs3write+0x1b8()
  ffffff00f6d7f540 nfs3_bio+0x216()
  ffffff00f6d7f5d0 nfs3_rdwrlbn+0xd7()
  ffffff00f6d7f660 nfs3_sync_putapage+0x57()
  ffffff00f6d7f720 nfs3_putapage+0x101()
  ffffff00f6d7f7f0 pvn_vplist_dirty+0x413()
  ffffff00f6d7f8a0 nfs_putpages+0xf9()
  ffffff00f6d7f920 nfs3_putpage+0xe5()
  ffffff00f6d7fbc0 nfs3setattr+0x8d()
  ffffff00f6d7fce0 nfs3_setattr+0xed()
  ffffff00f6d7fd60 fop_setattr+0xa8()
  ffffff00f6d7fdd0 fsetattrat+0x137()
  ffffff00f6d7fe90 fchmodat+0x4d()
  ffffff00f6d7feb0 chmod+0x21()
  ffffff00f6d7ff10 _sys_sysenter_post_swapgs+0x153()
> ffffff3d7fc8b690::wchaninfo
ADDR             TYPE NWAITERS
ffffff3d7fc8b690 cond        1
> ffffff3d7fc8b698::mutex
            ADDR  TYPE             HELD MINSPL OLDSPL WAITERS
ffffff3d7fc8b698 adapt               no      -      -      no
>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment