Project

General

Profile

Bug #8876

nfsd: can't register svc name - NFS shares showing unavailable to vSphere hosts

Added by Brian Edgar over 5 years ago. Updated about 3 years ago.

Status:
Closed: User Config Issue
Priority:
Important
Assignee:
Jordan Hubbard
Category:
OS
Target version:
Severity:
New
Reason for Closing:
Reason for Blocked:
Needs QA:
Yes
Needs Doc:
Yes
Needs Merging:
Yes
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

NFS shares become unavailable and console reports:

nfsd: can't register svc name.

During troubleshooting I came across Bug #7775 which appears to be a similar issue. I also do not enable NFSv4. I am running the newest stable build available at posting time - FreeNAS-9.3-STABLE-201503200528.iso

If I shut the NFS service down and remove the shares, create a new share, start the NFS service, I can see the datastore in order to migrate my VM's. The service does not survive rebooting and VM's are unavailable.

Please let me know any other information needed.

Brian

Associated revisions

Revision d9b3dee1 (diff)
Added by Josh Paetzel over 5 years ago

Add rpcinfo -p output to freenas-debug Ticket: #8876

Revision b7ad0002 (diff)
Added by Josh Paetzel over 5 years ago

Add rpcinfo -p output to freenas-debug Ticket: #8876 (cherry picked from commit d9b3dee143562be1e1e492e31d119610f8892558)

Revision b6e23ccd (diff)
Added by Josh Paetzel over 5 years ago

Add rpcinfo -p output to freenas-debug Ticket: #8876 (cherry picked from commit d9b3dee143562be1e1e492e31d119610f8892558)

History

#1 Updated by Jordan Hubbard over 5 years ago

  • Status changed from Unscreened to 15
  • Assignee set to Josh Paetzel

BRB: Please attach save debug output when the system is in this state. Thanks.

#2 Updated by Brian Edgar over 5 years ago

  • File NFS-debug.zip added

Added debug file as requested

freenas-debug -N

#3 Updated by Josh Paetzel over 5 years ago

Looks like I need to fix the debug tool. I also need the output of rpcinfo -p which wasn't included in freenas-debug.

#4 Updated by Brian Edgar over 5 years ago

Here is the rpcinfo -p output

program vers proto   port  service
100000 4 tcp 111 rpcbind
100000 3 tcp 111 rpcbind
100000 2 tcp 111 rpcbind
100000 4 udp 111 rpcbind
100000 3 udp 111 rpcbind
100000 2 udp 111 rpcbind
100000 4 local 111 rpcbind
100000 3 local 111 rpcbind
100000 2 local 111 rpcbind
100005 1 udp 999 mountd
100005 3 udp 999 mountd
100005 1 tcp 999 mountd
100005 3 tcp 999 mountd
100003 2 tcp 2049 nfs
100003 3 tcp 2049 nfs
100024 1 udp 825 status
100024 1 tcp 825 status
100021 0 udp 673 nlockmgr
100021 0 tcp 817 nlockmgr
100021 1 udp 673 nlockmgr
100021 1 tcp 817 nlockmgr
100021 3 udp 673 nlockmgr
100021 3 tcp 817 nlockmgr
100021 4 udp 673 nlockmgr
100021 4 tcp 817 nlockmgr

Thank you.

#5 Updated by Brian Edgar over 5 years ago

I was able to migrate most of my data from the NFS locations with the exception of data in one more shared folder. I am now unable to get the workaround functioning just to finish migrating the remaining data to an iSCSI LUN until the nfs svc issue gets resolved. I hope that this issue gets elevated.

Regards,

Brian

#6 Updated by Josh Paetzel over 5 years ago

  • Category set to 92

Can you attach /var/log/messages after vmware tries an NFS mount and fails?

#7 Updated by Brian Edgar over 5 years ago

  • File putty-session-vmkernel.log added

Here is a section from my /var/log/vmkernel.log file. I am running ESXI 5.1 and /var/log/messages does not exist. As an experiment I enabled all hosts to be able to attach to the NFS share. I previously restricted it to 10.10.93.0/24 now it is set for 0.0.0.0/0. If you look at the portion of the log that I provided you will see it fail for 10.10.93.189 and succeed on 172.30.9.189. The only difference on these interfaces is that the 10.10.93.189 interface on the FreeNAS server is the mtu is set to 8500, the 172.30.9.189 interface is default. I will try other scenarios, but as a note, the 10.10.93.189 interface's mtu was not set to 8500 after initial install on 9.3 and upload of config from 9.2.1.5 build. It was set to 8500 in order to get it working but has remained flaky at best.

This was all functional on build 9.2.1.5.

Please let me know if you need more information.

#8 Updated by Josh Paetzel over 5 years ago

How about /var/log/messages from the FreeNAS box.

#9 Updated by Josh Paetzel over 5 years ago

You mention an MTU of 8500...are you able to ping -s 8500 the vmware box?

This sounds a lot like general networking problems and not a bug per sae.

#10 Updated by Brian Edgar over 5 years ago

From FreeNAS

[root@phi-san02] /mnt/PHI-SAN02-ZFS01/NFS/NFS05# ping -s 8500 10.10.93.16
PING 10.10.93.16 (10.10.93.16): 8500 data bytes
8508 bytes from 10.10.93.16: icmp_seq=1 ttl=64 time=0.416 ms
8508 bytes from 10.10.93.16: icmp_seq=2 ttl=64 time=0.399 ms
8508 bytes from 10.10.93.16: icmp_seq=3 ttl=64 time=0.397 ms
8508 bytes from 10.10.93.16: icmp_seq=4 ttl=64 time=0.394 ms
8508 bytes from 10.10.93.16: icmp_seq=5 ttl=64 time=0.398 ms

From vShere host

/var/log # vmkping -s 8500 10.10.93.189
PING 10.10.93.189 (10.10.93.189): 8500 data bytes
8508 bytes from 10.10.93.189: icmp_seq=0 ttl=64 time=0.469 ms
8508 bytes from 10.10.93.189: icmp_seq=1 ttl=64 time=0.436 ms
8508 bytes from 10.10.93.189: icmp_seq=2 ttl=64 time=0.438 ms

--- 10.10.93.189 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.436/0.448/0.469 ms
/var/log #

Only change was moving to 9.3.

#11 Updated by Brian Edgar over 5 years ago

Additional Info:

From CentOS server that can mount NFS on 172.30.9.x network also cannot mount NFS on FreeNAS 10.10.93.189 interface.

I put the "\" in from of the -s only in the pasting of the data here to remove the strike-thru.

[root@rndlinux ~]# ping 10.10.93.189 \-s 8500
PING 10.10.93.189 (10.10.93.189) 8500(8528) bytes of data.
8508 bytes from 10.10.93.189: icmp_seq=1 ttl=64 time=0.527 ms
8508 bytes from 10.10.93.189: icmp_seq=2 ttl=64 time=0.921 ms
8508 bytes from 10.10.93.189: icmp_seq=3 ttl=64 time=0.574 ms
8508 bytes from 10.10.93.189: icmp_seq=4 ttl=64 time=0.536 ms
8508 bytes from 10.10.93.189: icmp_seq=5 ttl=64 time=0.517 ms
8508 bytes from 10.10.93.189: icmp_seq=6 ttl=64 time=0.565 ms
^C
--- 10.10.93.189 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5550ms
rtt min/avg/max/mdev = 0.517/0.606/0.921/0.144 ms
[root@rndlinux ~]# mount 10.10.93.189:/mnt/PHI-SAN02-ZFS01/NFS/NFS04 /mnt/transfer
^C
[root@rndlinux ~]# mount 10.10.93.189:/mnt/PHI-SAN02-ZFS01/NFS/NFS04 /mnt/transfer
mount.nfs: Connection timed out
[root@rndlinux ~]#

#12 Updated by Josh Paetzel over 5 years ago

  • Status changed from 15 to Unscreened
  • Assignee changed from Josh Paetzel to Jordan Hubbard
  • Target version set to Unspecified

#13 Updated by Brian Edgar over 5 years ago

Is there any more information that I can provide in order to help move this issue along?

Regards,

Brian Edgar

#14 Updated by Josh Paetzel over 5 years ago

The output from /var/log/messages on the FreeNAS side when a ESXi fails to mount could help.

The fact that you are trying to mount from a different subnet makes firewall issues come in to play. You can try telnetting to the various NFS services as seen in rpcinfo -p. If telnet doesn't work to the mountd port then it won't be able to mount.

NFS requires working forward and reverse DNS from the server and the client. If dig -x ip.of.vmware.box and dig mount.name.of.esx.box doesn't work from FreeNAS and likewise from the esxi host dig -x ip.of.freenas.box and dig hostname.of.freenas.box doesn't work NFS won't work.

#15 Updated by Brian Edgar over 5 years ago

Josh,

Thanks for the reply. If you reference a update to this thread above you will see that I am not on different subnets. The FreeNAS server is multi-homed and the ESXi and FreeNAS server are connecting (and used to connect using 9.2.1.9) on the 10.10.93.x/24 subnet. I am also using IP addresses and not using hostnames while troubleshooting this.

Error on ESXi side:

Call "HostDatastoreSystem.CreateNasDatastore" for object "datastoreSystem-145" on vCenter Server "PHI-VCENTER01" failed.
NFS mount 10.10.93.189:/mnt/PHI-SAN02-ZFS01/NFS/NFS01 failed: Unable to connect to NFS server.

Output from messages on FreeNAS:

Apr 20 13:46:30 phi-san02 mountd50318: mount request succeeded from 10.10.93.16 for /mnt/PHI-SAN02-ZFS01/NFS/NFS01
Apr 20 13:46:42 phi-san02 mountd50318: mount request succeeded from 10.10.93.16 for /mnt/PHI-SAN02-ZFS01/NFS/NFS01
Apr 20 13:46:58 phi-san02 mountd50318: mount request succeeded from 10.10.93.16 for /mnt/PHI-SAN02-ZFS01/NFS/NFS01

Console message on FreeNAS server:

nfsd: can't register svc name

#16 Updated by Josh Paetzel over 5 years ago

The can't register name thing is a red herring.

Even using IPs you need working reverse DNS.

#17 Updated by Jordan Hubbard over 5 years ago

  • Status changed from Unscreened to Closed: User Config Issue

#18 Updated by Brian Edgar over 5 years ago

Isn't reverse DNS for NFS only required for NFS v4? I am not selecting that option and it worked in 9.2.1.9. Is there a way to disable that requirement manually in a config file? With a multi-homed system reverse DNS will be a nightmare.

Regards,

#19 Updated by Josh Paetzel over 5 years ago

Reverse DNS is a requirement of NFSv3

#20 Updated by Scott Kamp over 5 years ago

  • Seen in changed from to 9.3-STABLE-201506042008

Sorry, as this morning on an updated to a TN 9.3-Nightlies-201506120012 what was working without reverse DNS is now no longer working with the same error......
with or without nfsv4 enabled

nfsd: can't register svc name

#21 Avatar?id=14398&size=24x24 Updated by Kris Moore about 3 years ago

  • Target version changed from Unspecified to N/A

#22 Updated by Dru Lavigne almost 3 years ago

  • File deleted (NFS-debug.zip)

#23 Updated by Dru Lavigne almost 3 years ago

  • File deleted (putty-session-vmkernel.log)

Also available in: Atom PDF