List of NFS daemons

11.5 NFS Problem Determination

Troubleshooting NFS problems involves a strategy for tracking NFS problems, recognizing NFS-related error messages, and selecting the appropriate solutions. When tracking down an NFS problem, isolate each of the three main points of failure to determine which is not working: The server, the client, or the network itself.

11.5.1 Identifying NFS Problems Checklist

If a client is having NFS trouble, perform the following tasks:

  1. Verify that the network connections are functioning properly.

  2. Verify that the inetd, portmap, and biod daemons are running on the client (see 11.3.1 Get the Current Status of the NFS Daemons ).

  3. Verify that a valid mount point exists on the client system for the file system to be mounted. For more information, see Establishing Predefined NFS Mounts.

  4. Verify that the server is up and running by executing the following command at the shell prompt of the client machine:
     /usr/bin/rpcinfo -p server_name

    where server_name is the name of the server being verified.

    If the server is up, a list of programs, versions, protocols, and port numbers is printed similar to the following:

    program  vers proto   port
    100000    2   tcp    111  portmapper
    100000    2   udp    111  portmapper
    100005    1   udp   1025  mountd
    100001    1   udp   1030  rstatd
    100001    2   udp   1030  rstatd
    100001    3   udp   1030  rstatd
    100002    1   udp   1036  rusersd
    100002    2   udp   1036  rusersd
    100008    1   udp   1040  walld
    100012    1   udp   1043  sprayd
    100005    1   tcp    694  mountd
    100003    2   udp   2049  nfs
    100024    1   udp    713  status
    100024    1   tcp    715  status
    100021    1   tcp    716  nlockmgr
    100021    1   udp    718  nlockmgr
    100021    3   tcp    721  nlockmgr
    100021    3   udp    723  nlockmgr
    100020    1   udp    726  llockmgr
    100020    1   tcp    728  llockmgr
    100021    2   tcp    731  nlockmgr

    If a similar response is not returned, log in to the server at the server console and check the status of the inetd daemon by following the instructions in 11.3.1 Get the Current Status of the NFS Daemons .

  5. Verify that the mountd, portmap, and nfsd daemons are running on the NFS server by entering the following commands at the client shell prompt:

    The program numbers correspond to the commands, respectively, as shown in the preceding example. If a similar response is not returned, log in to the server at the server console and check the status of the daemons by following the instructions in 11.3.1 Get the Current Status of the NFS Daemons .

  6. Verify that the /etc/exports file on the server lists the name of the file system that the client wants to mount and that the file system is exported. Do this by entering the command:
     showmount -e server_name

    This command will list all the file systems currently exported by the server_name.

11.5.2 Checking Network Connections

If the biod daemons are working, check the network connections. The nfsstat command determines whether you are dropping packets. Use the nfsstat -c and nfsstat -s commands to determine if the client or server is retransmitting large blocks. Retransmissions are always a possibility due to lost packets or busy servers. A retransmission rate of 5 percent is considered high.

The probability of retransmissions can be reduced by changing the communication adapter transmit queue parameters, no settings, to name two. SMIT or the no command can be used to change these parameters.

11.5.3 NFS Error Messages

The following sections explain error codes that can be generated while using NFS. Hard-Mounted and Soft-Mounted File Problems

When the network or server has problems, programs that access hard-mounted remote files fail differently from those that access soft-mounted remote files.

If a server fails to respond to a hard-mount request, NFS prints the message:

NFS server hostname not responding, still trying

If a server fails to respond to a soft-mount request, NFS prints the message:

Connection timed out Bad Sendreply Error Message

Insufficient transmit buffers on your network can cause the following error message:

nfs_server: bad sendreply

To increase transmit buffers, use the System Management Interface Tool (SMIT) fast path smitty commodev. Then select your adapter type and increase the number of transmit buffers. Server Not Responding

Use the procedure in 11.5.1 Identifying NFS Problems Checklist to troubleshoot this error. The error usually occurs if the NFS daemons have not been started or have been stopped. If the mountd or the nfsd daemons were not started or were stopped on the server, then when a client tries to mount an exported file system, an 1831-010 error message is displayed.

For example, if the rpc.mountd daemon dies after starting, and this error is received at a client machine, then do the following:

  1. Telnet to the server and log in as root.

  2. cd to the /etc directory

  3. Enter stopsrc -g nfs.

  4. Enter stopsrc -s portmap.

  5. Enter rm -rf state sm sm.bak xtab rmtab.

  6. Enter startsrc -s portmap.

  7. Enter startsrc -g nfs.

  8. Enter exportfs -a.

  9. showmount -e servername.

The rm -rf command clears the mountd files that may be too large for mountd to handle. If this procedure does not work, then refer to section 11.5 NFS Problem Determination. Remote Mounting Errors

The following list provides common mounting errors and their probable causes.

  1. A remote mounting process can fail in several ways. The error messages associated with mounting failures are as follows:

    mount: ... already mounted

    The file system that you are trying to mount is already mounted.

    mount: ... not found in /etc/filesystems

    The specified file system or directory name cannot be matched.

  2. If you issue the mount command with either a directory or file system name but not both. The command looks in the /etc/filesystems file for an entry whose file system or directory field matches the argument. If the mount command finds an entry, such as the following:
    nodename = d61server
    type = nfs
    mount = false

    then it performs the mount as if you had entered the following at the command line:

    /usr/sbin/mount -n danger -o rw,hard /usr/src /dancer.src

    If you receive the following message:

    mount... not in hosts database

    1. On a network without Network Information Service (NIS), this message indicates that the host specified to the mount command is not in the /etc/hosts file. On a network running NIS, the message indicates that NIS could not find the host name in the /etc/hosts database or that the NIS ypbind daemon on your machine has terminated. If the /etc/resolv.conf file exists, so that the name server is being used for host name resolution, there can be a problem in the named database.

      Check the spelling and the syntax in your mount command. If the command is correct, and your network does not run NIS, and you only get this message for this host name, check the entry in the /etc/hosts file.

    2. If your network is running NIS, make sure that the ypbind daemon is running by entering the following at the command line:

      ps -ef | grep ypbind

      You should see an entry for the ypbind daemon. Try using the rlogin command to log in remotely to another machine, or use the rcp command to remote-copy something to another machine. If this also fails, your ypbind daemon is probably stopped or hung.

  3. If you only get this message for this host name, you should check the /etc/hosts entry on the NIS server.
    mount: ... server not responding: port mapper failure - RPC timed out

    Either the server you are trying to mount from is down, or its port mapper is stopped or hung. Try restarting the inetd, portmap, and ypbind daemons.

    If you cannot log in to the server remotely with the rlogin command, but the server is up, you should check the network connection by trying to log in remotely to some other machine. You should also check the server's network connection.

  4. 1831-019 mount: ... server not responding: program not registered

    This means that the mount command got through to the port mapper, but the rpc.mountd NFS mount daemon was not registered.

  5. mount: access denied...

    Your machine name is not in the export list for the file system you are trying to mount from the server.

    You can get a list of the server's exported file systems by running the following command at the command line:

    showmount -e hostname

    If the file system you want is not in the list, or your machine name or netgroup name is not in the user list for the file system, log in to the server and check the /etc/exports file for the correct file system entry. A file system name that appears in the /etc/exports file, but not in the output from the showmount command, indicates a failure in the mountd daemon. Either the daemon could not parse that line in the file, it could not find the directory, or the directory name was not a locally mounted directory. If the /etc/exports file looks correct and your network runs NIS, check the server's ypbind daemon. It may be stopped or hung.

  6. mount: ...: Permission denied

    This message is a generic indication that some part of authentication failed on the server. It may be that, in the previous example, you are not in the export list, the server could not recognize your machine's ypbind daemon, or that the server does not accept the identity you provided.

    Check the server's /etc/exports file and, if applicable, the ypbind daemon. In this case you can just change your host name with the hostname command and retry the mount command.

  7. mount: ...: Not a directory

    Either the remote path or the local path is not a directory. Check the spelling in your command and try to run it on both the remote and local paths.

  8. mount: ...: You are not allowed

    You must have root authority or be a member of the system group to run the mount command on your machine because it affects the file system for all users on that machine. NFS mounts and unmounts are only allowed for root users and members of the system group.

11.6 Quiz