Tag: File locking

Isilon,Isilon OneFS

This article describes the best practices and recommendations for client-side settings and mount options when using the NFS protocol to connect to an Isilon cluster and applies to all currently supported versions of OneFS.

Supported Protocol Versions

At this time Isilon OneFS supports NFS versions 3 and 4. NFS version 2 has not been supported since the move to the 7.2.X code family.

NFSv3

NFS version 3 is the most widely used version of the NFS protocol today, and is generally considered to have the widest client and filer adoption. Here are key components of this version:

• Stateless – A client does not technically need to establish a new session if it has the correct information to ask for files, etc. This allows for simple failover between OneFS nodes via dynamic IP pools.
• User and Group info is presented numerically – Client and Server communicate user information by numeric identifiers, allowing the same user to possible appear as different names between client and server.
• File Locking is out of band – Version 3 of NFS uses a helper protocol called NLM to perform locks. This requires the client to respond to RPC messages from the server to confirm locks have been granted, etc.
• Can run over TCP or UDP – This version of the protocol can run over UDP instead of TCP, leaving handling of loss and retransmission to the software instead of the operating system. We always recommend using TCP.

NFSv4

NFS version 4 is the newest major revision of the NFS protocol, and is increasing in adoption. At this time NFSv4 is generally less performant than v3 against the same workflow due to the greater amount of identity mapping and session tracking work required to reply. Here are some of the key differences between v3 and v4

• Stateful – NFSv4 uses sessions in order to handle communication, as such both client and server need to track session state to continue communicating.
  • Prior to OneFS 8.X this meant that NFSv4 clients required static IP pools on the Isilon or could encounter issues.
• User and Group info is presented as strings – Both the client and server need to resolve the names of the numeric information stored. The server needs to lookup names to present, while the client needs to remap those to numbers on its end.
• File Locking is in band – Version 4 no longer users a separate protocol for file locking, instead making it a type of call that is usually compounded with OPENs, CREATES, or WRITES.
• Compound Calls – Version 4 can bundle a series of calls in a single packet, allowing the server to process all of them and reply at the end. This is used to reduce the number of calls involved in common operations.
• Only supports TCP – Version 4 of NFS has left loss and retransmission up to the underlying operating system.

NFSv4.1 and Beyond

At this time OneFS does not support NFS version 4.1. If you need specific features of version 4.1, speak with your account team to see if that is something that we can provide via OneFS’s unique featureset as an NFS filer.

OneFS Version specific concerns

For customers that have been using Isilon OneFS since versions 7.1 or before, changes made in the 7.2.0 version of OneFS, and remaining in place until OneFS 8.1.1, might impact how clients using encoding that differs from the cluster’s are able to view and interact with directory listings. For more details review ETA 483840.

This is not an issue if you began using OneFS on version 7.2 or beyond.

Mount options

While we do not have hard requirements for mount options, we do make some recommendations on how clients connection. We have not provided specific mount strings, as the syntax used to define these options varies depending on the operating system in use. You should refer to your distribution maintainers documentation for specific mount syntax.

Defining Retries and Timeouts

While the Isilon generally replies to client communication very quickly, during instances when a node has lost power or network connectivity, it might take a few seconds for its IP addresses to move to a functional node, as such it is important to have correctly defined timeout and retry values. Isilon generally recommends a timeout of 60 seconds to account for a worst case failover scenario, set to retry two times before reporting a failure.

Soft vs Hard Mounts

Hard mounts cause the client to retry its operations indefinitely on timeout or error. This ensures that the client does not disconnect the mount in circumstances where the Isilon cluster moves IP addresses from one node to another. A soft mount will instead error out and expire the mount requiring a remount to restore access after the IP address moves.

Allowing interrupt

By default, most clients do no allow you to interrupt an input/output or I/O wait, meaning you cannot use ctrl+c, etc, to end the waiting process if the cluster is hanging, including the interrupt mount option allows those signals to pass normally instead.

Local versus Remote Locking

When mounting an NFS export, you can specify whether a like will perform its locks locally, or using the lock co-ordinator on the cluster. Most clients default to remote locking, and this is generally the best option when multiple clients will be accessing the same directory, however there can be performance benefits to performing local locking when a client does not need to share access to the directory it is working with. In addition, some databases and softwares will request you use local locking, as they have their own coordinator.