Been asked several times recently how to get the usage and capacity of individual pools across a heterogeneous cluster. As such, it seemed like a useful topic to explore in an article.
Depending on the configuration and licensed services, OneFS provides three main approaches for reporting utilization and capacity at a per-pool granularity. The following table shows the OneFS feature dependencies for each of the three reporting methods:
Let’s look at each of these options:
1. If a cluster has tiering configured, or has any other SmartPools policy in place, the easiest method to get accurate per-pool usage, percentage, and overall capacity data from OneFS is via the following CLI command:
# isi stat –all-nodepools
Theisi stat –all-nodepoolscommand only works for OneFS 8.0 and later release. For clusters still running OneFS 7.x releases, the following syntax can be used instead:
# isi stat –d
Alternatively, the following CLI command will also yield a similar output:
# isi storagepool list -v –format=table
The same information can be found in the WebUI by navigating to Storage Pools > SmartPools > Tiers & Node Pools and selecting ‘View node pool details’ for the desired pool:
As an alternative, the OneFS RESTful platform API can also be used to query nodepool statistics. Use the following syntax, substituting ‘cluster_ip_address’ for the IPv4 dotted quad address of your cluster:
For each nodepool or tier, the following metrics will be reported:
With the exception of the percentage (pct) data, all the values are in bytes, so simple math will be required in order to convert to more human readable values.
2. For the next option, if everything on the cluster is being written to a single pool which is mapped to a particular directory tree under /ifs, another approach for nodepool capacity and usage reporting is to configure a container advisory quota. Note that this method also requires an active SmartQuotas license on the cluster.
Quota containers help compartmentalize /ifs, so that a directory with a container will appear as its own separate file system slice. As such, setting the container flag directly influences the metrics returned by the CLI command ‘df’. For instance, if you run ‘df’ on a container directory, it will report the free space that the quota container has been allocated.
To configure a directory quota with a 4TB container on /ifs/data/container1, you could use the following CLI command:
# isi quota quotas create /ifs/data/container1 directory –hard-threshold 4T –container true
Under the covers, the ‘container’ argument interacts with the ‘statfs’ system call. This provides the capacity statistics based on the hard limit of the container quota, as opposed to the entire cluster. So, for the example above, 4TB – the size of the container – is reported.
# df -h /ifs/data/container1
Filesystem Size Used Avail Capacity Mounted on
OneFS 4.0T 3.0G 4.0T 0% /ifs
Be aware that OneFS only currently supports containers for directory quotas, not for user or group quotas.
3. Finally, if the FSAnalyze job is running on a cluster, plus you have an instance of InsightIQ version 4.0 or above monitoring the cluster, this can be used to provide per-pool reporting. From the InsightIQ WebUI, select the ‘Cluster Capacity’ report with ‘Breakout Total Usage by Nodepool’. For example:
As an added benefit, using InsightIQ reporting also has the option of historical nodepool usage data. This allows for simple trending analysis and explicit point-in-time statistics.