A guide to NFS: Use cases, issues, and troubleshooting in Linux

Network file sharing (NFS) is a protocol used for sharing files and directories between computers on a network. NFS is useful when there is a need to quickly share a large number of files across many devices. Once set up on a file server, any computer on the network that the client services are installed on can connect to the NFS server and access all files and directories. On a client computer, an NFS drive is mounted much like a local drive, making it easy for users to navigate.

However, users access the same files stored on the NFS drive and not copies, which means changes made to a file on an NFS drive will also change the original on the NFS server. Users should mindful of the implications before making changes.

In this article, we explore the benefits of NFS as well as specific use cases. We also cover common NFS-related issues and how to solve them, using specific Linux commands.

When is NFS needed?

NFS is typically used for sharing big files or a large volume of files across multiple computers within a network.

For example, large video files are shared between editors at a production house. A single file could be in the hundreds of gigabytes, with multiple collaborators working on it. Not only is sharing such large files with each editor challenging, but also storing them locally on each computer can result in extremely high storage costs. Having a central storage space, however, can significantly lower expenses.

Per-computer licensing fees for professional applications can also increase costs. Installing these applications on an NFS server allows multiple users to access the application remotely with the same license, thereby reducing costs.

Another common example involves shared study materials and resources in a classroom or a lab. Using NFS, the school can upload resources to one central location. This makes it easy for students and faculty to access them through their personal computers as if the resources were present locally.

Because NFS works with multiple technologies and environments, it doesn't matter what environment the client is running as long as they can access the NFS server. This makes it easy to configure workflows in a heterogeneous fashion.

Common NFS problems and fixes

Despite its wide use and benefits, issues often arise when using NFS. Let’s examine some of the most common issues and how to fix them.

Server not responding

The client uses remote procedure calls (RPC) to communicate with the NFS server. This requires an established connection and functional connection paths from the client to the server and from the server back to the client. A drop in this connection or the paths will result in a "server not responding" error.

Solution

In such cases, common Linux commands such as ping, traceroute, and tracepath can be used to check where the connections are dropping. These commands trace an RPC from the client to the server. If none of the commands provide useful information for debugging, examining the network interface card could help. For this, the ifconfig and ethtool commands can be used.

Even if the physical connection to the server is up and running, the "server not responding" error may appear if the server is unable to respond to client requests. This usually takes place when the server is overloaded and unable to respond within the configured timeout interval. Increasing the timeout should fix the issue but a longer-term solution would involve either scaling the server or finding a way to reduce the load.

No route to host

The "no route to host" error signals that a firewall is blocking the connection. The firewall could either be on the server or on the client’s side. This issue usually arises when a client tries to mount an NFS file system and a firewall is blocking them.

Solution

By default, NFS uses port 2049 for communication between clients and servers, so it’s important to make sure that any configured firewalls are allowing traffic on this port. Switch off the firewall on both the server and the client’s end, and remount the file system for error validation. . You can easily turn off firewalls by stopping iptables using the command below:

service iptables stop 

If everything goes well when the firewall is turned off, you can switch the firewall back on for either the server or client first, test if everything works as expected, and then switch on the other firewall. If the firewall is not configured to allow the client to connect to the NFS server on the designated port, the connection might fail again—so make sure the firewall is configured correctly.

RPC: Port mapper failure/unable to receive

For NFS to work properly, both the server and the client need to have the NFS service and the port mapper up and running. If the port mapper is down or misconfigured, you’ll receive this error.

Solution

It’s fairly simple to verify if these services are running and listening to requests on the expected port. Here’s the command to use:

rpcinfo -p 

This command will provide information on the program version, the protocol used, the port, and the service itself.

Another way to check if the port mapper service is running is to get the status of the service. This is the command to use:

service portmap status 

Here portmap is the name of the service. If the service is not running, starting it should fix the RPC issue. The command for starting the port mapper service is:

chkconfig portmap on 

Stale file handle

Programs that work with files on an NFS-mounted drive or directory access these files the same way they’d access any local file—by using the open(2)command. This command returns a handle to the file, or a file descriptor. However, unlike local files, if an NFS directory file is deleted, the open(2)command will not create a file, because NFS doesn't support this feature.

When a program tries to open a file that's deleted in NFS, the "stale file handle" error is thrown.

Solution

Here’s a simple fix: Either unmount the file system and remount it or kill the process that's trying to access the file.

Access denied

The "access denied" error is displayedwhen a client doesn’t have export permissions to an NFS file system.

Solution

The only way to fix this is through the export permissions configuration, using one of these two commands:

showmount -e  

or

exportfs -a  

A common mistake that leads to this error is an extra space in the export permissions configuration. For example, notice the extra space between the * and the (ro) in the configuration below:

/nfs/data * (ro)  

That extra space is not giving read permission to the intended client but rather access to all users, which is not the intention here. Removing that space will fix the permission issue:

/nfs/data *(ro)  

The /etc/exports file on the NFS server contains the permissions configuration. Checking that file for any such issues and fixing them should also solve the denied permission issue.

RPC timed out

This error usually appears when we configure a new NFS system. The "RPC timed out" error is thrown when the network is not able to resolve the DNS for the NFS server.

A simple fix for this is to check the DNS configuration in the /etc/hosts file and fix the server name there.

Commands to identify issues and other metrics

The nfsstat and the nfsiostat commands can be used to identify issues with NFS and proactively fix them before they create problems. These commands are part of the nfs-utils package.

nfsstat command

This command lets you check the statistical information of the NFS and RPC services. There are options for both the server and the client. Option s is for the server; it provides the number of badcalls or bad RPC calls the server has received over a period of time. If this number is greater than zero, the underlying network should be examined for issues.

Option c, for the client, provides information such as the retrans metric, a measure of retransmission requests received by the client from the server. A significantly high number of retransmission requests means you need to adjust the data transfer buffer size.

nfsiostat command

You can use this command to understand the I/O performance of the NFS drives mounted on client computers. It uses the data in the file /proc/self/mountstats to display the stats. Of all the information shown by this command, two are particularly important: retrans and avg RTT.

The retransmission metric is similar to the one shown by the nfsstat command. The avg RTT metric measures the time elapsed from when the client sends the RPC request to when it receives a reply from the server. A high average (in milliseconds) round-trip time indicates that there is latency in the network between the client and the server, which affects performance. This would warrant a deeper analysis of the network.

Summary

NFS is a highly useful tool for sharing files and directories over a network, allowing users to access shared files similar to how they would access a locally stored one. Moreover, it promotes collaboration among team members.

Despite its advantages, NFS, like other tools, comes with its own set of problems. Understanding each error and knowing how to quickly debug them can prevent significant downtime.

You can also use the above commands to understand the performance of your underlying network and the NFS service itself, thus preventing a number of issues. These commands allow you to proactively maintain NFS services.