Many of these can be changed by writing new values. These are denoted by
+Change|Dynamic in the PARAMETER details below.
+If the parameter is not dynamically adjustable, an error can occur and
+the value will not be set. It can be helpful to check the permissions
+for the PARAMETER file in SysFS.
+In some cases, the parameter must be set prior to loading the kernel
+modules or it is desired to have the parameters set automatically at
+boot time. For many distros, this can be accomplished by creating a file
+named /etc/modprobe.d/zfs.conf containing a text line for each
+module parameter using the format:
+Some parameters related to ZFS operations are located in module
+parameters other than in the zfs kernel module. These are documented
+in the individual parameter description. Unless otherwise noted, the
+tunable applies to the zfs kernel module. For example, the icp
+kernel module parameters are visible in the
+/sys/module/icp/parameters directory and can be set by default at
+boot time by changing the /etc/modprobe.d/icp.conf file.
+
+Module Parameters
+
+ignore_hole_birth
+When set, the hole_birth optimization will not be used and all holes
+will always be sent by zfs send In the source code,
+ignore_hole_birth is an alias for and SysFS PARAMETER for
+send_holes_without_birth_time.
+
+
+ignore_hole_birth |
+Notes |
+
|---|
+
+
+Tags |
+send |
+
+When to change |
+Enable if you suspect your datasets are
+affected by a bug in hole_birth during
+zfs send operations |
+
+Data Type |
+boolean |
+
+Range |
+0=disabled, 1=enabled |
+
+Default |
+1 (hole birth optimization is ignored) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+TBD |
+
+
+
+
+
+l2arc_exclude_special
+Controls whether buffers present on special vdevs are eligible for
+caching into L2ARC.
+
+
+l2arc_exclude_special |
+Notes |
+
|---|
+
+
+Tags |
+ARC,
+L2ARC,
+special_vdev, |
+
+When to change |
+If cache and special devices exist and caching
+data on special devices in L2ARC is not desired |
+
+Data Type |
+boolean |
+
+Range |
+0=disabled, 1=enabled |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+TBD |
+
+
+
+
+
+l2arc_feed_again
+Turbo L2ARC cache warm-up. When the L2ARC is cold the fill interval will
+be set to aggressively fill as fast as possible.
+
+
+l2arc_feed_again |
+Notes |
+
|---|
+
+
+Tags |
+ARC, L2ARC |
+
+When to change |
+If cache devices exist and it is desired to
+fill them as fast as possible |
+
+Data Type |
+boolean |
+
+Range |
+0=disabled, 1=enabled |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+TBD |
+
+
+
+
+
+l2arc_feed_min_ms
+Minimum time period for aggressively feeding the L2ARC. The L2ARC feed
+thread wakes up once per second (see
+l2arc_feed_secs) to look for data to feed into
+the L2ARC. l2arc_feed_min_ms only affects the turbo L2ARC cache
+warm-up and allows the aggressiveness to be adjusted.
+
+
+l2arc_feed_min_ms |
+Notes |
+
|---|
+
+
+Tags |
+ARC, L2ARC |
+
+When to change |
+If cache devices exist and
+l2arc_feed_again and
+the feed is too aggressive, then this tunable
+can be adjusted to reduce the impact of the
+fill |
+
+Data Type |
+uint64 |
+
+Units |
+milliseconds |
+
+Range |
+0 to (1000 * l2arc_feed_secs) |
+
+Default |
+200 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+0.6 and later |
+
+
+
+
+
+l2arc_feed_secs
+Seconds between waking the L2ARC feed thread. One feed thread works for
+all cache devices in turn.
+If the pool that owns a cache device is imported readonly, then the feed
+thread is delayed 5 * l2arc_feed_secs before
+moving onto the next cache device. If multiple pools are imported with
+cache devices and one pool with cache is imported readonly, the L2ARC
+feed rate to all caches can be slowed.
+
+
+l2arc_feed_secs |
+Notes |
+
|---|
+
+
+Tags |
+ARC, L2ARC |
+
+When to change |
+Do not change |
+
+Data Type |
+uint64 |
+
+Units |
+seconds |
+
+Range |
+1 to UINT64_MAX |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+0.6 and later |
+
+
+
+
+
+l2arc_headroom
+How far through the ARC lists to search for L2ARC cacheable content,
+expressed as a multiplier of l2arc_write_max
+
+
+l2arc_headroom |
+Notes |
+
|---|
+
+
+Tags |
+ARC, L2ARC |
+
+When to change |
+If the rate of change in the ARC is faster than
+the overall L2ARC feed rate, then increasing
+l2arc_headroom can increase L2ARC efficiency.
+Setting the value too large can cause the L2ARC
+feed thread to consume more CPU time looking
+for data to feed. |
+
+Data Type |
+uint64 |
+
+Units |
+unit |
+
+Range |
+0 to UINT64_MAX |
+
+Default |
+2 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+0.6 and later |
+
+
+
+
+
+l2arc_headroom_boost
+Percentage scale for l2arc_headroom when L2ARC
+contents are being successfully compressed before writing.
+
+
+l2arc_headroom_boost |
+Notes |
+
|---|
+
+
+Tags |
+ARC, L2ARC |
+
+When to change |
+If average compression efficiency is greater
+than 2:1, then increasing
+l2a
+rc_headroom_boost
+can increase the L2ARC feed rate |
+
+Data Type |
+uint64 |
+
+Units |
+percent |
+
+Range |
+100 to UINT64_MAX, when set to 100, the
+L2ARC headroom boost feature is effectively
+disabled |
+
+Default |
+200 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+l2arc_nocompress
+Disable writing compressed data to cache devices. Disabling allows the
+legacy behavior of writing decompressed data to cache devices.
+
+
+l2arc_nocompress |
+Notes |
+
|---|
+
+
+Tags |
+ARC, L2ARC |
+
+When to change |
+When testing compressed L2ARC feature |
+
+Data Type |
+boolean |
+
+Range |
+0=store compressed blocks in cache device,
+1=store uncompressed blocks in cache device |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+deprecated in v0.7.0 by new compressed ARC
+design |
+
+
+
+
+
+
+l2arc_mfuonly
+Controls whether only MFU metadata and data are cached from ARC into L2ARC.
+This may be desirable to avoid wasting space on L2ARC when reading/writing
+large amounts of data that are not expected to be accessed more than once.
+By default both MRU and MFU data and metadata are cached in the L2ARC.
+
+
+l2arc_mfuonly |
+Notes |
+
|---|
+
+
+Tags |
+ARC, L2ARC |
+
+When to change |
+When accessing a large amount of data only
+once. |
+
+Data Type |
+boolean |
+
+Range |
+0=store MRU and MFU blocks in cache device,
+1=store MFU blocks in cache device |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v2.0 and later |
+
+
+
+
+
+l2arc_noprefetch
+Disables writing prefetched, but unused, buffers to cache devices.
+
+
+l2arc_noprefetch |
+Notes |
+
|---|
+
+
+Tags |
+ARC, L2ARC,
+prefetch |
+
+When to change |
+Setting to 0 can increase L2ARC hit rates for
+workloads where the ARC is too small for a read
+workload that benefits from prefetching. Also,
+if the main pool devices are very slow, setting
+to 0 can improve some workloads such as
+backups. |
+
+Data Type |
+boolean |
+
+Range |
+0=write prefetched but unused buffers to cache
+devices, 1=do not write prefetched but unused
+buffers to cache devices |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.0 and later |
+
+
+
+
+
+l2arc_norw
+Disables writing to cache devices while they are being read.
+
+
+l2arc_norw |
+Notes |
+
|---|
+
+
+Tags |
+ARC, L2ARC |
+
+When to change |
+In the early days of SSDs, some devices did not
+perform well when reading and writing
+simultaneously. Modern SSDs do not have these
+issues. |
+
+Data Type |
+boolean |
+
+Range |
+0=read and write simultaneously, 1=avoid writes
+when reading for antique SSDs |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+l2arc_rebuild_blocks_min_l2size
+The minimum required size (in bytes) of an L2ARC device in order to
+write log blocks in it. The log blocks are used upon importing the pool
+to rebuild the persistent L2ARC. For L2ARC devices less than 1GB the
+overhead involved offsets most of benefit so log blocks are not written
+for cache devices smaller than this.
+
+
+l2arc_rebuild_blocks_min_l2size |
+Notes |
+
|---|
+
+
+Tags |
+ARC,
+L2ARC |
+
+When to change |
+The cache device is small and
+the pool is frequently imported. |
+
+Data Type |
+bytes |
+
+Range |
+0 to UINT64_MAX |
+
+Default |
+1,073,741,824 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v2.0 and later |
+
+
+
+
+
+l2arc_rebuild_enabled
+Rebuild the persistent L2ARC when importing a pool.
+
+
+l2arc_rebuild_enabled |
+Notes |
+
|---|
+
+
+Tags |
+ARC, L2ARC |
+
+When to change |
+If there are problems importing a pool or
+attaching an L2ARC device. |
+
+Data Type |
+boolean |
+
+Range |
+0=disable persistent L2ARC rebuild,
+1=enable persistent L2ARC rebuild |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v2.0 and later |
+
+
+
+
+
+l2arc_trim_ahead
+Once the cache device has been filled TRIM ahead of the current write size
+l2arc_write_max on L2ARC devices by this percentage. This can speed
+up future writes depending on the performance characteristics of the
+cache device.
+When set to 100% TRIM twice the space required to accommodate upcoming
+writes. A minimum of 64MB will be trimmed. If set it enables TRIM of
+the whole L2ARC device when it is added to a pool. By default, this
+option is disabled since it can put significant stress on the underlying
+storage devices.
+
+
+l2arc_trim_ahead |
+Notes |
+
|---|
+
+
+Tags |
+ARC, L2ARC |
+
+When to change |
+Consider setting for cache devices which
+effeciently handle TRIM commands. |
+
+Data Type |
+ulong |
+
+Units |
+percent of l2arc_write_max |
+
+Range |
+0 to 100 |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v2.0 and later |
+
+
+
+
+
+l2arc_write_boost
+Until the ARC fills, increases the L2ARC fill rate
+l2arc_write_max by l2arc_write_boost.
+
+
+l2arc_write_boost |
+Notes |
+
|---|
+
+
+Tags |
+ARC, L2ARC |
+
+When to change |
+To fill the cache devices more aggressively
+after pool import. |
+
+Data Type |
+uint64 |
+
+Units |
+bytes |
+
+Range |
+0 to UINT64_MAX |
+
+Default |
+8,388,608 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+l2arc_write_max
+Maximum number of bytes to be written to each cache device for each
+L2ARC feed thread interval (see l2arc_feed_secs).
+The actual limit can be adjusted by
+l2arc_write_boost. By default
+l2arc_feed_secs is 1 second, delivering a maximum
+write workload to cache devices of 8 MiB/sec.
+
+
+l2arc_write_max |
+Notes |
+
|---|
+
+
+Tags |
+ARC, L2ARC |
+
+When to change |
+If the cache devices can sustain the write
+workload, increasing the rate of cache device
+fill when workloads generate new data at a rate
+higher than l2arc_write_max can increase L2ARC
+hit rate |
+
+Data Type |
+uint64 |
+
+Units |
+bytes |
+
+Range |
+1 to UINT64_MAX |
+
+Default |
+8,388,608 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+spa_config_path
+By default, the zpool import command searches for pool information
+in the zpool.cache file. If the pool to be imported has an entry in
+zpool.cache then the devices do not have to be scanned to determine
+if they are pool members. The path to the cache file is spa_config_path.
+For more information on zpool import and the -o cachefile and
+-d options, see the man page for zpool(8)
+See also zfs_autoimport_disable
+
+
+spa_config_path |
+Notes |
+
|---|
+
+
+Tags |
+import |
+
+When to change |
+If creating a non-standard distribution and the
+cachefile property is inconvenient |
+
+Data Type |
+string |
+
+Default |
+/etc/zfs/zpool.cache
|
+
+Change |
+Dynamic, applies only to the next invocation of
+zpool import |
+
+Versions Affected |
+all |
+
+
+
+
+
+spa_asize_inflation
+Multiplication factor used to estimate actual disk consumption from the
+size of data being written. The default value is a worst case estimate,
+but lower values may be valid for a given pool depending on its
+configuration. Pool administrators who understand the factors involved
+may wish to specify a more realistic inflation factor, particularly if
+they operate close to quota or capacity limits.
+The worst case space requirement for allocation is single-sector
+max-parity RAIDZ blocks, in which case the space requirement is exactly
+4 times the size, accounting for a maximum of 3 parity blocks. This is
+added to the maximum number of ZFS copies parameter (copies max=3).
+Additional space is required if the block could impact deduplication
+tables. Altogether, the worst case is 24.
+If the estimation is not correct, then quotas or out-of-space conditions
+can lead to optimistic expectations of the ability to allocate.
+Applications are typically not prepared to deal with such failures and
+can misbehave.
+
+
+spa_asize_inflation |
+Notes |
+
|---|
+
+
+Tags |
+allocation, SPA |
+
+When to change |
+If the allocation requirements for the
+workload are well known and quotas are used |
+
+Data Type |
+uint64 |
+
+Units |
+unit |
+
+Range |
+1 to 24 |
+
+Default |
+24 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.3 and later |
+
+
+
+
+
+spa_load_verify_data
+An extreme rewind import (see zpool import -X) normally performs a
+full traversal of all blocks in the pool for verification. If this
+parameter is set to 0, the traversal skips non-metadata blocks. It can
+be toggled once the import has started to stop or start the traversal of
+non-metadata blocks. See also
+spa_load_verify_metadata.
+
+
+spa_load_verify_data |
+Notes |
+
|---|
+
+
+Tags |
+allocation, SPA |
+
+When to change |
+At the risk of data integrity, to speed
+extreme import of large pool |
+
+Data Type |
+boolean |
+
+Range |
+0=do not verify data upon pool import,
+1=verify pool data upon import |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.4 and later |
+
+
+
+
+
+
+spa_load_verify_maxinflight
+Maximum number of concurrent I/Os during the data verification performed
+during an extreme rewind import (see zpool import -X)
+
+
+spa_load_verify_maxinflight |
+Notes |
+
|---|
+
+
+Tags |
+import |
+
+When to change |
+During an extreme rewind import, to
+match the concurrent I/O capabilities
+of the pool devices |
+
+Data Type |
+int |
+
+Units |
+I/Os |
+
+Range |
+1 to MAX_INT |
+
+Default |
+10,000 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.4 and later |
+
+
+
+
+
+spa_slop_shift
+Normally, the last 3.2% (1/(2^spa_slop_shift)) of pool space is
+reserved to ensure the pool doesn’t run completely out of space, due to
+unaccounted changes (e.g. to the MOS). This also limits the worst-case
+time to allocate space. When less than this amount of free space exists,
+most ZPL operations (e.g. write, create) return error:no space (ENOSPC).
+Changing spa_slop_shift affects the currently loaded ZFS module and all
+imported pools. spa_slop_shift is not stored on disk. Beware when
+importing full pools on systems with larger spa_slop_shift can lead to
+over-full conditions.
+The minimum SPA slop space is limited to 128 MiB.
+The maximum SPA slop space is limited to 128 GiB.
+
+
+spa_slop_shift |
+Notes |
+
|---|
+
+
+Tags |
+allocation, SPA |
+
+When to change |
+For large pools, when 3.2% may be too
+conservative and more usable space is desired,
+consider increasing spa_slop_shift |
+
+Data Type |
+int |
+
+Units |
+shift |
+
+Range |
+1 to MAX_INT, however the practical upper limit
+is 15 for a system with 4TB of RAM |
+
+Default |
+5 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.5 and later (max. slop space since v2.1.0) |
+
+
+
+
+
+zfetch_array_rd_sz
+If prefetching is enabled, do not prefetch blocks larger than
+zfetch_array_rd_sz size.
+
+
+zfetch_array_rd_sz |
+Notes |
+
|---|
+
+
+Tags |
+prefetch |
+
+When to change |
+To allow prefetching when using large block sizes |
+
+Data Type |
+unsigned long |
+
+Units |
+bytes |
+
+Range |
+0 to MAX_ULONG |
+
+Default |
+1,048,576 (1 MiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfetch_max_distance
+Limits the maximum number of bytes to prefetch per stream.
+
+
+zfetch_max_distance |
+Notes |
+
|---|
+
+
+Tags |
+prefetch |
+
+When to change |
+Consider increasing read workloads that use
+large blocks and exhibit high prefetch hit
+ratios |
+
+Data Type |
+uint |
+
+Units |
+bytes |
+
+Range |
+0 to UINT_MAX |
+
+Default |
+8,388,608 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 |
+
+
+
+
+
+zfetch_max_streams
+Maximum number of prefetch streams per file.
+For version v0.7.0 and later, when prefetching small files the number of
+prefetch streams is automatically reduced below to prevent the streams
+from overlapping.
+
+
+zfetch_max_streams |
+Notes |
+
|---|
+
+
+Tags |
+prefetch |
+
+When to change |
+If the workload benefits from prefetching and
+has more than zfetch_max_streams
+concurrent reader threads |
+
+Data Type |
+uint |
+
+Units |
+streams |
+
+Range |
+1 to MAX_UINT |
+
+Default |
+8 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfetch_min_sec_reap
+Prefetch streams that have been accessed in zfetch_min_sec_reap
+seconds are automatically stopped.
+
+
+zfetch_min_sec_reap |
+Notes |
+
|---|
+
+
+Tags |
+prefetch |
+
+When to change |
+To test prefetch efficiency |
+
+Data Type |
+uint |
+
+Units |
+seconds |
+
+Range |
+0 to MAX_UINT |
+
+Default |
+2 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_arc_dnode_limit_percent
+Percentage of ARC metadata space that can be used for dnodes.
+The value calculated for zfs_arc_dnode_limit_percent can be
+overridden by zfs_arc_dnode_limit.
+
+
+zfs_arc_dnode_limit_percent |
+Notes |
+
|---|
+
+
+Tags |
+ARC |
+
+When to change |
+Consider increasing if arc_prune
+is using excessive system time and
+/proc/spl/kstat/zfs/arcstats
+shows arc_dnode_size is near or
+over arc_dnode_limit |
+
+Data Type |
+int |
+
+Units |
+percent of arc_meta_limit |
+
+Range |
+0 to 100 |
+
+Default |
+10 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_arc_dnode_limit
+When the number of bytes consumed by dnodes in the ARC exceeds
+zfs_arc_dnode_limit bytes, demand for new metadata can take from the
+space consumed by dnodes.
+The default value 0, indicates that a percent which is based on
+zfs_arc_dnode_limit_percent of the
+ARC meta buffers that may be used for dnodes.
+zfs_arc_dnode_limit is similar to
+zfs_arc_meta_prune which serves a similar
+purpose for metadata.
+
+
+zfs_arc_dnode_limit |
+Notes |
+
|---|
+
+
+Tags |
+ARC |
+
+When to change |
+Consider increasing if arc_prune is using
+excessive system time and
+/proc/spl/kstat/zfs/arcstats shows
+arc_dnode_size is near or over
+arc_dnode_limit |
+
+Data Type |
+uint64 |
+
+Units |
+bytes |
+
+Range |
+0 to MAX_UINT64 |
+
+Default |
+0 (uses
+zfs_arc_dnode_lim
+it_percent) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_arc_dnode_reduce_percent
+Percentage of ARC dnodes to try to evict in response to demand for
+non-metadata when the number of bytes consumed by dnodes exceeds
+zfs_arc_dnode_limit.
+
+
+zfs_arc_dnode_reduce_percent |
+Notes |
+
|---|
+
+
+Tags |
+ARC |
+
+When to change |
+Testing dnode cache efficiency |
+
+Data Type |
+uint64 |
+
+Units |
+percent of size of dnode space used
+above
+zfs_arc_d
+node_limit |
+
+Range |
+0 to 100 |
+
+Default |
+10 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_arc_average_blocksize
+The ARC’s buffer hash table is sized based on the assumption of an
+average block size of zfs_arc_average_blocksize. The default of 8
+KiB uses approximately 1 MiB of hash table per 1 GiB of physical memory
+with 8-byte pointers.
+
+
+zfs_arc_average_blocksize |
+Notes |
+
|---|
+
+
+Tags |
+ARC, memory |
+
+When to change |
+For workloads where the known average
+blocksize is larger, increasing
+zfs_arc_average_blocksize can
+reduce memory usage |
+
+Data Type |
+int |
+
+Units |
+bytes |
+
+Range |
+512 to 16,777,216 |
+
+Default |
+8,192 |
+
+Change |
+Prior to zfs module load |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_arc_evict_batch_limit
+Number ARC headers to evict per sublist before proceeding to another
+sublist. This batch-style operation prevents entire sublists from being
+evicted at once but comes at a cost of additional unlocking and locking.
+
+
+zfs_arc_evict_batch_limit |
+Notes |
+
|---|
+
+
+Tags |
+ARC |
+
+When to change |
+Testing ARC multilist features |
+
+Data Type |
+int |
+
+Units |
+count of ARC headers |
+
+Range |
+1 to INT_MAX |
+
+Default |
+10 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.5 and later |
+
+
+
+
+
+zfs_arc_grow_retry
+When the ARC is shrunk due to memory demand, do not retry growing the
+ARC for zfs_arc_grow_retry seconds. This operates as a damper to
+prevent oscillating grow/shrink cycles when there is memory pressure.
+If zfs_arc_grow_retry = 0, the internal default of 5 seconds is
+used.
+
+
+zfs_arc_grow_retry |
+Notes |
+
|---|
+
+
+Tags |
+ARC, memory |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+seconds |
+
+Range |
+1 to MAX_INT |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.5 and later |
+
+
+
+
+
+zfs_arc_lotsfree_percent
+Throttle ARC memory consumption, effectively throttling I/O, when free
+system memory drops below this percentage of total system memory.
+Setting zfs_arc_lotsfree_percent to 0 disables the throttle.
+The arcstat_memory_throttle_count counter in
+/proc/spl/kstat/arcstats can indicate throttle activity.
+
+
+zfs_arc_lotsfree_percent |
+Notes |
+
|---|
+
+
+Tags |
+ARC, memory |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+percent |
+
+Range |
+0 to 100 |
+
+Default |
+10 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.5 and later |
+
+
+
+
+
+zfs_arc_max
+Maximum size of ARC in bytes.
+If set to 0 then the maximum size of ARC
+is determined by the amount of system memory installed:
+
+zfs_arc_max can be changed dynamically with some caveats. It cannot
+be set back to 0 while running and reducing it below the current ARC
+size will not cause the ARC to shrink without memory pressure to induce
+shrinking.
+
+
+zfs_arc_max |
+Notes |
+
|---|
+
+
+Tags |
+ARC, memory |
+
+When to change |
+Reduce if ARC competes too much with other
+applications, increase if ZFS is the primary
+application and can use more RAM |
+
+Data Type |
+uint64 |
+
+Units |
+bytes |
+
+Range |
+67,108,864 to RAM size in bytes |
+
+Default |
+0 (see description above, OS-dependent) |
+
+Change |
+Dynamic (see description above) |
+
+Verification |
+c column in arcstats.py or
+/proc/spl/kstat/zfs/arcstats entry
+c_max
|
+
+Versions Affected |
+all |
+
+
+
+
+
+
+
+
+
+
+
+zfs_arc_min
+Minimum ARC size limit. When the ARC is asked to shrink, it will stop
+shrinking at c_min as tuned by zfs_arc_min.
+
+
+zfs_arc_min |
+Notes |
+
|---|
+
+
+Tags |
+ARC |
+
+When to change |
+If the primary focus of the system is ZFS, then
+increasing can ensure the ARC gets a minimum
+amount of RAM |
+
+Data Type |
+uint64 |
+
+Units |
+bytes |
+
+Range |
+33,554,432 to c_max |
+
+Default |
+For kernel: greater of 33,554,432 (32 MiB) and
+memory size / 32. For user-land: greater of
+33,554,432 (32 MiB) and c_max / 2. |
+
+Change |
+Dynamic |
+
+Verification |
+/proc/spl/kstat/zfs/arcstats entry
+c_min
|
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_arc_min_prefetch_ms
+Minimum time prefetched blocks are locked in the ARC.
+A value of 0 represents the default of 1 second. However, once changed,
+dynamically setting to 0 will not return to the default.
+
+
+zfs_arc_min_prefetch_ms |
+Notes |
+
|---|
+
+
+Tags |
+ARC, prefetch |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+milliseconds |
+
+Range |
+1 to INT_MAX |
+
+Default |
+0 (use internal default of 1000 ms) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.8.0 and later |
+
+
+
+
+
+zfs_arc_min_prescient_prefetch_ms
+Minimum time “prescient prefetched” blocks are locked in the ARC. These
+blocks are meant to be prefetched fairly aggressively ahead of the code
+that may use them.
+A value of 0 represents the default of 6 seconds. However, once changed,
+dynamically setting to 0 will not return to the default.
+
+
+z
+fs_arc_min_prescient_prefetch_ms |
+Notes |
+
|---|
+
+
+Tags |
+ARC,
+prefetch |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+milliseconds |
+
+Range |
+1 to INT_MAX |
+
+Default |
+0 (use internal default of 6000
+ms) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.8.0 and later |
+
+
+
+
+
+zfs_multilist_num_sublists
+To allow more fine-grained locking, each ARC state contains a series of
+lists (sublists) for both data and metadata objects. Locking is
+performed at the sublist level. This parameters controls the number of
+sublists per ARC state, and also applies to other uses of the multilist
+data structure.
+
+
+zfs_multilist_num_sublists |
+Notes |
+
|---|
+
+
+Tags |
+ARC |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+lists |
+
+Range |
+1 to INT_MAX |
+
+Default |
+0 (internal value is greater of number
+of online CPUs or 4) |
+
+Change |
+Prior to zfs module load |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_arc_overflow_shift
+The ARC size is considered to be overflowing if it exceeds the current
+ARC target size (/proc/spl/kstat/zfs/arcstats entry c) by a
+threshold determined by zfs_arc_overflow_shift. The threshold is
+calculated as a fraction of c using the formula: (ARC target size)
+c >> zfs_arc_overflow_shift
+The default value of 8 causes the ARC to be considered to be overflowing
+if it exceeds the target size by 1/256th (0.3%) of the target size.
+When the ARC is overflowing, new buffer allocations are stalled until
+the reclaim thread catches up and the overflow condition no longer
+exists.
+
+
+zfs_arc_overflow_shift |
+Notes |
+
|---|
+
+
+Tags |
+ARC |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+shift |
+
+Range |
+1 to INT_MAX |
+
+Default |
+8 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.5 and later |
+
+
+
+
+
+zfs_arc_p_min_shift
+arc_p_min_shift is used to shift of ARC target size
+(/proc/spl/kstat/zfs/arcstats entry c) for calculating both
+minimum and maximum most recently used (MRU) target size
+(/proc/spl/kstat/zfs/arcstats entry p)
+A value of 0 represents the default setting of arc_p_min_shift = 4.
+However, once changed, dynamically setting zfs_arc_p_min_shift to 0
+will not return to the default.
+
+
+zfs_arc_p_min_shift |
+Notes |
+
|---|
+
+
+Tags |
+ARC |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+shift |
+
+Range |
+1 to INT_MAX |
+
+Default |
+0 (internal default = 4) |
+
+Change |
+Dynamic |
+
+Verification |
+Observe changes to
+/proc/spl/kstat/zfs/arcstats entry p |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_arc_p_dampener_disable
+When data is being added to the ghost lists, the MRU target size is
+adjusted. The amount of adjustment is based on the ratio of the MRU/MFU
+sizes. When enabled, the ratio is capped to 10, avoiding large
+adjustments.
+
+
+zfs_arc_p_dampener_disable |
+Notes |
+
|---|
+
+
+Tags |
+ARC |
+
+When to change |
+Testing ARC ghost list behaviour |
+
+Data Type |
+boolean |
+
+Range |
+0=avoid large adjustments, 1=permit
+large adjustments |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.4 and later |
+
+
+
+
+
+zfs_arc_shrink_shift
+arc_shrink_shift is used to adjust the ARC target sizes when large
+reduction is required. The current ARC target size, c, and MRU size
+p can be reduced by by the current size >> arc_shrink_shift. For
+the default value of 7, this reduces the target by approximately 0.8%.
+A value of 0 represents the default setting of arc_shrink_shift = 7.
+However, once changed, dynamically setting arc_shrink_shift to 0 will
+not return to the default.
+
+
+zfs_arc_shrink_shift |
+Notes |
+
|---|
+
+
+Tags |
+ARC, memory |
+
+When to change |
+During memory shortfall, reducing
+zfs_arc_shrink_shift increases the rate
+of ARC shrinkage |
+
+Data Type |
+int |
+
+Units |
+shift |
+
+Range |
+1 to INT_MAX |
+
+Default |
+0 (arc_shrink_shift = 7) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_arc_pc_percent
+zfs_arc_pc_percent allows ZFS arc to play more nicely with the
+kernel’s LRU pagecache. It can guarantee that the arc size won’t
+collapse under scanning pressure on the pagecache, yet still allows arc
+to be reclaimed down to zfs_arc_min if necessary. This value is
+specified as percent of pagecache size (as measured by
+NR_FILE_PAGES) where that percent may exceed 100. This only operates
+during memory pressure/reclaim.
+
+
+zfs_arc_pc_percent |
+Notes |
+
|---|
+
+
+Tags |
+ARC, memory |
+
+When to change |
+When using file systems under memory
+shortfall, if the page scanner causes the ARC
+to shrink too fast, then adjusting
+zfs_arc_pc_percent can reduce the shrink
+rate |
+
+Data Type |
+int |
+
+Units |
+percent |
+
+Range |
+0 to 100 |
+
+Default |
+0 (disabled) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_arc_sys_free
+zfs_arc_sys_free is the target number of bytes the ARC should leave
+as free memory on the system. Defaults to the larger of 1/64 of physical
+memory or 512K. Setting this option to a non-zero value will override
+the default.
+A value of 0 represents the default setting of larger of 1/64 of
+physical memory or 512 KiB. However, once changed, dynamically setting
+zfs_arc_sys_free to 0 will not return to the default.
+
+
+zfs_arc_sys_free |
+Notes |
+
|---|
+
+
+Tags |
+ARC, memory |
+
+When to change |
+Change if more free memory is desired as a
+margin against memory demand by applications |
+
+Data Type |
+ulong |
+
+Units |
+bytes |
+
+Range |
+0 to ULONG_MAX |
+
+Default |
+0 (default to larger of 1/64 of physical memory
+or 512 KiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.5 and later |
+
+
+
+
+
+zfs_autoimport_disable
+Disable reading zpool.cache file (see
+spa_config_path) when loading the zfs module.
+
+
+zfs_autoimport_disable |
+Notes |
+
|---|
+
+
+Tags |
+import |
+
+When to change |
+Leave as default so that zfs behaves as
+other Linux kernel modules |
+
+Data Type |
+boolean |
+
+Range |
+0=read zpool.cache at module load,
+1=do not read zpool.cache at module
+load |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.4 and later |
+
+
+
+
+
+zfs_commit_timeout_pct
+zfs_commit_timeout_pct controls the amount of time that a log (ZIL)
+write block (lwb) remains “open” when it isn’t “full” and it has a
+thread waiting to commit to stable storage. The timeout is scaled based
+on a percentage of the last lwb latency to avoid significantly impacting
+the latency of each individual intent log transaction (itx).
+
+
+zfs_commit_timeout_pct |
+Notes |
+
|---|
+
+
+Tags |
+ZIL |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+percent |
+
+Range |
+1 to 100 |
+
+Default |
+5 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.8.0 |
+
+
+
+
+
+zfs_dbgmsg_enable
+
+
Internally ZFS keeps a small log to facilitate debugging. The contents
+of the log are in the /proc/spl/kstat/zfs/dbgmsg file.
+
Writing 0 to /proc/spl/kstat/zfs/dbgmsg file clears the log.
+
See also zfs_dbgmsg_maxsize
+
+
+zfs_dbgmsg_enable |
+Notes |
+
|---|
+
+
+Tags |
+debug |
+
+When to change |
+To view ZFS internal debug log |
+
+Data Type |
+boolean |
+
+Range |
+0=do not log debug messages, 1=log debug messages |
+
+Default |
+0 (1 for debug builds) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.5 and later |
+
+
+
+
+
+zfs_dbgmsg_maxsize
+The /proc/spl/kstat/zfs/dbgmsg file size limit is set by
+zfs_dbgmsg_maxsize.
+See also zfs_dbgmsg_enable
+
+
+zfs_dbgmsg_maxsize |
+Notes |
+
|---|
+
+
+Tags |
+debug |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+bytes |
+
+Range |
+0 to INT_MAX |
+
+Default |
+4 MiB |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.5 and later |
+
+
+
+
+
+zfs_dbuf_state_index
+The zfs_dbuf_state_index feature is currently unused. It is normally
+used for controlling values in the /proc/spl/kstat/zfs/dbufs file.
+
+
+zfs_dbuf_state_index |
+Notes |
+
|---|
+
+
+Tags |
+debug |
+
+When to change |
+Do not change |
+
+Data Type |
+int |
+
+Units |
+TBD |
+
+Range |
+TBD |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.5 and later |
+
+
+
+
+
+zfs_deadman_enabled
+When a pool sync operation takes longer than zfs_deadman_synctime_ms
+milliseconds, a “slow spa_sync” message is logged to the debug log (see
+zfs_dbgmsg_enable). If zfs_deadman_enabled
+is set to 1, then all pending IO operations are also checked and if any
+haven’t completed within zfs_deadman_synctime_ms milliseconds, a “SLOW
+IO” message is logged to the debug log and a “deadman” system event (see
+zpool events command) with the details of the hung IO is posted.
+
+
+zfs_deadman_enabled |
+Notes |
+
|---|
+
+
+Tags |
+debug |
+
+When to change |
+To disable logging of slow I/O |
+
+Data Type |
+boolean |
+
+Range |
+0=do not log slow I/O, 1=log slow I/O |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.8.0 |
+
+
+
+
+
+zfs_deadman_checktime_ms
+Once a pool sync operation has taken longer than
+zfs_deadman_synctime_ms milliseconds,
+continue to check for slow operations every
+zfs_deadman_checktime_ms milliseconds.
+
+
+zfs_deadman_checktime_ms |
+Notes |
+
|---|
+
+
+Tags |
+debug |
+
+When to change |
+When debugging slow I/O |
+
+Data Type |
+ulong |
+
+Units |
+milliseconds |
+
+Range |
+1 to ULONG_MAX |
+
+Default |
+60,000 (1 minute) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.8.0 |
+
+
+
+
+
+zfs_deadman_ziotime_ms
+When an individual I/O takes longer than zfs_deadman_ziotime_ms
+milliseconds, then the operation is considered to be “hung”. If
+zfs_deadman_enabled is set then the deadman
+behaviour is invoked as described by the
+zfs_deadman_failmode option.
+
+
+zfs_deadman_ziotime_ms |
+Notes |
+
|---|
+
+
+Tags |
+debug |
+
+When to change |
+Testing ABD features |
+
+Data Type |
+ulong |
+
+Units |
+milliseconds |
+
+Range |
+1 to ULONG_MAX |
+
+Default |
+300,000 (5 minutes) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.8.0 |
+
+
+
+
+
+zfs_deadman_synctime_ms
+The I/O deadman timer expiration time has two meanings
+
+determines when the spa_deadman() logic should fire, indicating
+the txg sync has not completed in a timely manner
determines if an I/O is considered “hung”
+In version v0.8.0, any I/O that has not completed in
+zfs_deadman_synctime_ms is considered “hung” resulting in one of
+three behaviors controlled by the
+zfs_deadman_failmode parameter.
+zfs_deadman_synctime_ms takes effect if
+zfs_deadman_enabled = 1.
+
+
+zfs_deadman_synctime_ms |
+Notes |
+
|---|
+
+
+Tags |
+debug |
+
+When to change |
+When debugging slow I/O |
+
+Data Type |
+ulong |
+
+Units |
+milliseconds |
+
+Range |
+1 to ULONG_MAX |
+
+Default |
+600,000 (10 minutes) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.5 and later |
+
+
+
+
+
+zfs_deadman_failmode
+zfs_deadman_failmode controls the behavior of the I/O deadman timer when
+it detects a “hung” I/O. Valid values are:
+
+
+
+zfs_deadman_failmode |
+Notes |
+
|---|
+
+
+Tags |
+debug |
+
+When to change |
+In some cluster cases, panic can be appropriate |
+
+Data Type |
+string |
+
+Range |
+wait, continue, or panic |
+
+Default |
+wait |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.8.0 |
+
+
+
+
+
+zfs_dedup_prefetch
+ZFS can prefetch deduplication table (DDT) entries.
+zfs_dedup_prefetch allows DDT prefetches to be enabled.
+
+
+zfs_dedup_prefetch |
+Notes |
+
|---|
+
+
+Tags |
+prefetch, memory |
+
+When to change |
+For systems with limited RAM using the dedup
+feature, disabling deduplication table
+prefetch can reduce memory pressure |
+
+Data Type |
+boolean |
+
+Range |
+0=do not prefetch, 1=prefetch dedup table
+entries |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.5 and later |
+
+
+
+
+
+zfs_delete_blocks
+zfs_delete_blocks defines a large file for the purposes of delete.
+Files containing more than zfs_delete_blocks will be deleted
+asynchronously while smaller files are deleted synchronously. Decreasing
+this value reduces the time spent in an unlink(2) system call at the
+expense of a longer delay before the freed space is available.
+The zfs_delete_blocks value is specified in blocks, not bytes. The
+size of blocks can vary and is ultimately limited by the filesystem’s
+recordsize property.
+
+
+zfs_delete_blocks |
+Notes |
+
|---|
+
+
+Tags |
+filesystem,
+delete |
+
+When to change |
+If applications delete large files and blocking
+on unlink(2) is not desired |
+
+Data Type |
+ulong |
+
+Units |
+blocks |
+
+Range |
+1 to ULONG_MAX |
+
+Default |
+20,480 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_delay_min_dirty_percent
+The ZFS write throttle begins to delay each transaction when the amount
+of dirty data reaches the threshold zfs_delay_min_dirty_percent of
+zfs_dirty_data_max. This value should be >=
+zfs_vdev_async_write_active_max_dirty_percent.
+
+
+zfs_delay_min_dirty_percent |
+Notes |
+
|---|
+
+
+Tags |
+write_throttle |
+
+When to change |
+See section “ZFS TRANSACTION DELAY” |
+
+Data Type |
+int |
+
+Units |
+percent |
+
+Range |
+0 to 100 |
+
+Default |
+60 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.4 and later |
+
+
+
+
+
+zfs_delay_scale
+zfs_delay_scale controls how quickly the ZFS write throttle
+transaction delay approaches infinity. Larger values cause longer delays
+for a given amount of dirty data.
+For the smoothest delay, this value should be about 1 billion divided by
+the maximum number of write operations per second the pool can sustain.
+The throttle will smoothly handle between 10x and 1/10th
+zfs_delay_scale.
+Note: zfs_delay_scale *
+zfs_dirty_data_max must be < 2^64.
+
+
+zfs_delay_scale |
+Notes |
+
|---|
+
+
+Tags |
+write_throttle |
+
+When to change |
+See section “ZFS TRANSACTION DELAY” |
+
+Data Type |
+ulong |
+
+Units |
+scalar (nanoseconds) |
+
+Range |
+0 to ULONG_MAX |
+
+Default |
+500,000 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.4 and later |
+
+
+
+
+
+zfs_dirty_data_max
+zfs_dirty_data_max is the ZFS write throttle dirty space limit. Once
+this limit is exceeded, new writes are delayed until space is freed by
+writes being committed to the pool.
+zfs_dirty_data_max takes precedence over
+zfs_dirty_data_max_percent.
+
+
+zfs_dirty_data_max |
+Notes |
+
|---|
+
+
+Tags |
+write_throttle |
+
+When to change |
+See section “ZFS TRANSACTION DELAY” |
+
+Data Type |
+ulong |
+
+Units |
+bytes |
+
+Range |
+1 to
+zfs_d
+irty_data_max_max |
+
+Default |
+10% of physical RAM |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.4 and later |
+
+
+
+
+
+zfs_dirty_data_max_percent
+zfs_dirty_data_max_percent is an alternative method of specifying
+zfs_dirty_data_max, the ZFS write throttle
+dirty space limit. Once this limit is exceeded, new writes are delayed
+until space is freed by writes being committed to the pool.
+zfs_dirty_data_max takes precedence over
+zfs_dirty_data_max_percent.
+
+
+zfs_dirty_data_max_percent |
+Notes |
+
|---|
+
+
+Tags |
+write_throttle |
+
+When to change |
+See section “ZFS TRANSACTION DELAY” |
+
+Data Type |
+int |
+
+Units |
+percent |
+
+Range |
+1 to 100 |
+
+Default |
+10% of physical RAM |
+
+Change |
+Prior to zfs module load or a memory
+hot plug event |
+
+Versions Affected |
+v0.6.4 and later |
+
+
+
+
+
+zfs_dirty_data_max_max
+zfs_dirty_data_max_max is the maximum allowable value of
+zfs_dirty_data_max.
+zfs_dirty_data_max_max takes precedence over
+zfs_dirty_data_max_max_percent.
+
+
+zfs_dirty_data_max_max |
+Notes |
+
|---|
+
+
+Tags |
+write_throttle |
+
+When to change |
+See section “ZFS TRANSACTION DELAY” |
+
+Data Type |
+ulong |
+
+Units |
+bytes |
+
+Range |
+1 to physical RAM size |
+
+Default |
+physical_ram/4
+since v0.7: min(physical_ram/4, 4GiB)
+since v2.0 for 32-bit systems: min(physical_ram/4, 1GiB)
+ |
+
+Change |
+Prior to zfs module load |
+
+Versions Affected |
+v0.6.4 and later |
+
+
+
+
+
+zfs_dirty_data_max_max_percent
+zfs_dirty_data_max_max_percent an alternative to
+zfs_dirty_data_max_max for setting the
+maximum allowable value of zfs_dirty_data_max
+zfs_dirty_data_max_max takes precedence
+over zfs_dirty_data_max_max_percent
+
+
+zfs_dirty_data_max_max_percent |
+Notes |
+
|---|
+
+
+Tags |
+write_throttle |
+
+When to change |
+See section “ZFS TRANSACTION DELAY” |
+
+Data Type |
+int |
+
+Units |
+percent |
+
+Range |
+1 to 100 |
+
+Default |
+25% of physical RAM |
+
+Change |
+Prior to zfs module load |
+
+Versions Affected |
+v0.6.4 and later |
+
+
+
+
+
+zfs_dirty_data_sync
+When there is at least zfs_dirty_data_sync dirty data, a transaction
+group sync is started. This allows a transaction group sync to occur
+more frequently than the transaction group timeout interval (see
+zfs_txg_timeout) when there is dirty data to be
+written.
+
+
+zfs_dirty_data_sync |
+Notes |
+
|---|
+
+
+Tags |
+write_throttle,
+ZIO_scheduler |
+
+When to change |
+TBD |
+
+Data Type |
+ulong |
+
+Units |
+bytes |
+
+Range |
+1 to ULONG_MAX |
+
+Default |
+67,108,864 (64 MiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.4 through v0.8.x, deprecation planned
+for v2 |
+
+
+
+
+
+zfs_dirty_data_sync_percent
+When there is at least zfs_dirty_data_sync_percent of
+zfs_dirty_data_max dirty data, a transaction
+group sync is started. This allows a transaction group sync to occur
+more frequently than the transaction group timeout interval (see
+zfs_txg_timeout) when there is dirty data to be
+written.
+
+
+
+zfs_fletcher_4_impl
+Fletcher-4 is the default checksum algorithm for metadata and data. When
+the zfs kernel module is loaded, a set of microbenchmarks are run to
+determine the fastest algorithm for the current hardware. The
+zfs_fletcher_4_impl parameter allows a specific implementation to be
+specified other than the default (fastest). Selectors other than
+fastest and scalar require instruction set extensions to be
+available and will only appear if ZFS detects their presence. The
+scalar implementation works on all processors.
+The results of the microbenchmark are visible in the
+/proc/spl/kstat/zfs/fletcher_4_bench file. Larger numbers indicate
+better performance. Since ZFS is processor endian-independent, the
+microbenchmark is run against both big and little-endian transformation.
+
+
+zfs_fletcher_4_impl |
+Notes |
+
|---|
+
+
+Tags |
+CPU, checksum |
+
+When to change |
+Testing Fletcher-4 algorithms |
+
+Data Type |
+string |
+
+Range |
+fastest, scalar, superscalar,
+superscalar4, sse2, ssse3, avx2,
+avx512f, or aarch64_neon depending on
+hardware support |
+
+Default |
+fastest |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_free_bpobj_enabled
+The processing of the free_bpobj object can be enabled by
+zfs_free_bpobj_enabled
+
+
+zfs_free_bpobj_enabled |
+Notes |
+
|---|
+
+
+Tags |
+delete |
+
+When to change |
+If there’s a problem with processing
+free_bpobj (e.g. i/o error or bug) |
+
+Data Type |
+boolean |
+
+Range |
+0=do not process free_bpobj objects,
+1=process free_bpobj objects |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_free_max_blocks
+zfs_free_max_blocks sets the maximum number of blocks to be freed in
+a single transaction group (txg). For workloads that delete (free) large
+numbers of blocks in a short period of time, the processing of the frees
+can negatively impact other operations, including txg commits.
+zfs_free_max_blocks acts as a limit to reduce the impact.
+
+
+zfs_free_max_blocks |
+Notes |
+
|---|
+
+
+Tags |
+filesystem,
+delete |
+
+When to change |
+For workloads that delete large files,
+zfs_free_max_blocks can be adjusted to
+meet performance requirements while reducing
+the impacts of deletion |
+
+Data Type |
+ulong |
+
+Units |
+blocks |
+
+Range |
+1 to ULONG_MAX |
+
+Default |
+100,000 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_vdev_async_read_max_active
+Maximum asynchronous read I/Os active to each device.
+
+
+
+zfs_vdev_async_read_min_active
+Minimum asynchronous read I/Os active to each device.
+
+
+
+
+
+zfs_vdev_async_write_max_active
+zfs_vdev_async_write_max_active sets the maximum asynchronous write
+I/Os active to each device.
+
+
+zfs_vdev_async_write_max_active |
+Notes |
+
|---|
+
+
+Tags |
+vdev,
+`
+ZIO_scheduler <#zio-scheduler>`__ |
+
+When to change |
+See ZFS I/O
+S
+cheduler |
+
+Data Type |
+uint32 |
+
+Units |
+I/O operations |
+
+Range |
+1 to
+zfs_vdev_max
+_active |
+
+Default |
+10 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.4 and later |
+
+
+
+
+
+zfs_vdev_async_write_min_active
+zfs_vdev_async_write_min_active sets the minimum asynchronous write
+I/Os active to each device.
+Lower values are associated with better latency on rotational media but
+poorer resilver performance. The default value of 2 was chosen as a
+compromise. A value of 3 has been shown to improve resilver performance
+further at a cost of further increasing latency.
+
+
+zfs_vdev_async_write_min_active |
+Notes |
+
|---|
+
+
+Tags |
+vdev,
+`
+ZIO_scheduler <#zio-scheduler>`__ |
+
+When to change |
+See ZFS I/O
+S
+cheduler |
+
+Data Type |
+uint32 |
+
+Units |
+I/O operations |
+
+Range |
+1 to
+zfs
+_vdev_async_write_max_active |
+
+Default |
+1 for v0.6.x, 2 for v0.7.0 and
+later |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.4 and later |
+
+
+
+
+
+zfs_vdev_max_active
+The maximum number of I/Os active to each device. Ideally,
+zfs_vdev_max_active >= the sum of each queue’s max_active.
+Once queued to the device, the ZFS I/O scheduler is no longer able to
+prioritize I/O operations. The underlying device drivers have their own
+scheduler and queue depth limits. Values larger than the device’s
+maximum queue depth can have the affect of increased latency as the I/Os
+are queued in the intervening device driver layers.
+
+
+zfs_vdev_max_active |
+Notes |
+
|---|
+
+
+Tags |
+vdev,
+ZIO_scheduler |
+
+When to change |
+See ZFS I/O
+Scheduler |
+
+Data Type |
+uint32 |
+
+Units |
+I/O operations |
+
+Range |
+sum of each queue’s min_active to UINT32_MAX |
+
+Default |
+1,000 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.4 and later |
+
+
+
+
+
+zfs_vdev_scrub_max_active
+zfs_vdev_scrub_max_active sets the maximum scrub or scan read I/Os
+active to each device.
+
+
+
+zfs_vdev_scrub_min_active
+zfs_vdev_scrub_min_active sets the minimum scrub or scan read I/Os
+active to each device.
+
+
+
+zfs_vdev_sync_read_max_active
+Maximum synchronous read I/Os active to each device.
+
+
+
+zfs_vdev_sync_read_min_active
+zfs_vdev_sync_read_min_active sets the minimum synchronous read I/Os
+active to each device.
+
+
+
+zfs_vdev_sync_write_max_active
+zfs_vdev_sync_write_max_active sets the maximum synchronous write
+I/Os active to each device.
+
+
+
+zfs_vdev_sync_write_min_active
+zfs_vdev_sync_write_min_active sets the minimum synchronous write
+I/Os active to each device.
+
+
+
+zfs_vdev_queue_depth_pct
+Maximum number of queued allocations per top-level vdev expressed as a
+percentage of
+zfs_vdev_async_write_max_active.
+This allows the system to detect devices that are more capable of
+handling allocations and to allocate more blocks to those devices. It
+also allows for dynamic allocation distribution when devices are
+imbalanced as fuller devices will tend to be slower than empty devices.
+Once the queue depth reaches (zfs_vdev_queue_depth_pct *
+zfs_vdev_async_write_max_active /
+100) then allocator will stop allocating blocks on that top-level device
+and switch to the next.
+See also zio_dva_throttle_enabled
+
+
+zfs_vdev_queue_depth_pct |
+Notes |
+
|---|
+
+
+Tags |
+vdev,
+ZIO_scheduler |
+
+When to change |
+See ZFS I/O
+Scheduler |
+
+Data Type |
+uint32 |
+
+Units |
+I/O operations |
+
+Range |
+1 to UINT32_MAX |
+
+Default |
+1,000 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_disable_dup_eviction
+Disable duplicate buffer eviction from ARC.
+
+
+zfs_disable_dup_eviction |
+Notes |
+
|---|
+
+
+Tags |
+ARC, dedup |
+
+When to change |
+TBD |
+
+Data Type |
+boolean |
+
+Range |
+0=duplicate buffers can be evicted, 1=do
+not evict duplicate buffers |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.5, deprecated in v0.7.0 |
+
+
+
+
+
+zfs_expire_snapshot
+Snapshots of filesystems are normally automounted under the filesystem’s
+.zfs/snapshot subdirectory. When not in use, snapshots are unmounted
+after zfs_expire_snapshot seconds.
+
+
+zfs_expire_snapshot |
+Notes |
+
|---|
+
+
+Tags |
+filesystem,
+snapshot |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+seconds |
+
+Range |
+0 disables automatic unmounting, maximum time
+is INT_MAX |
+
+Default |
+300 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.1 and later |
+
+
+
+
+
+zfs_admin_snapshot
+Allow the creation, removal, or renaming of entries in the
+.zfs/snapshot subdirectory to cause the creation, destruction, or
+renaming of snapshots. When enabled this functionality works both
+locally and over NFS exports which have the “no_root_squash” option set.
+
+
+zfs_admin_snapshot |
+Notes |
+
|---|
+
+
+Tags |
+filesystem,
+snapshot |
+
+When to change |
+TBD |
+
+Data Type |
+boolean |
+
+Range |
+0=do not allow snapshot manipulation via the
+filesystem, 1=allow snapshot manipulation via
+the filesystem |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.5 and later |
+
+
+
+
+
+zfs_flags
+Set additional debugging flags (see
+zfs_dbgmsg_enable)
+
+
+flag value |
+symbolic name |
+description |
+
|---|
+
+
+0x1 |
+ZFS_DEBUG_DPRINTF |
+Enable dprintf entries in
+the debug log |
+
+0x2 |
+ZFS_DEBUG_DBUF_VERIFY |
+Enable extra dnode
+verifications |
+
+0x4 |
+ZFS_DEBUG_DNODE_VERIFY |
+Enable extra dnode
+verifications |
+
+0x8 |
+ZFS_DEBUG_SNAPNAMES |
+Enable snapshot name
+verification |
+
+0x10 |
+ZFS_DEBUG_MODIFY |
+Check for illegally
+modified ARC buffers |
+
+0x20 |
+ZFS_DEBUG_SPA |
+Enable spa_dbgmsg entries
+in the debug log |
+
+0x40 |
+ZFS_DEBUG_ZIO_FREE |
+Enable verification of
+block frees |
+
+0x80 |
+Z
+FS_DEBUG_HISTOGRAM_VERIFY |
+Enable extra spacemap
+histogram verifications |
+
+0x100 |
+ZFS_DEBUG_METASLAB_VERIFY |
+Verify space accounting
+on disk matches in-core
+range_trees |
+
+0x200 |
+ZFS_DEBUG_SET_ERROR |
+Enable SET_ERROR and
+dprintf entries in the
+debug log |
+
+
+
+
+
+zfs_flags |
+Notes |
+
|---|
+
+
+Tags |
+debug |
+
+When to change |
+When debugging ZFS |
+
+Data Type |
+int |
+
+Default |
+0 no debug flags set, for debug builds: all
+except ZFS_DEBUG_DPRINTF and ZFS_DEBUG_SPA |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.4 and later |
+
+
+
+
+
+zfs_free_leak_on_eio
+If destroy encounters an I/O error (EIO) while reading metadata (eg
+indirect blocks), space referenced by the missing metadata cannot be
+freed. Normally, this causes the background destroy to become “stalled”,
+as the destroy is unable to make forward progress. While in this stalled
+state, all remaining space to free from the error-encountering
+filesystem is temporarily leaked. Set zfs_free_leak_on_eio = 1 to
+ignore the EIO, permanently leak the space from indirect blocks that can
+not be read, and continue to free everything else that it can.
+The default, stalling behavior is useful if the storage partially fails
+(eg some but not all I/Os fail), and then later recovers. In this case,
+we will be able to continue pool operations while it is partially
+failed, and when it recovers, we can continue to free the space, with no
+leaks. However, note that this case is rare.
+Typically pools either:
+
+fail completely (but perhaps temporarily (eg a top-level vdev going
+offline)
have localized, permanent errors (eg disk returns the wrong data due
+to bit flip or firmware bug)
+In case (1), the zfs_free_leak_on_eio setting does not matter
+because the pool will be suspended and the sync thread will not be able
+to make forward progress. In case (2), because the error is permanent,
+the best effort do is leak the minimum amount of space. Therefore, it is
+reasonable for zfs_free_leak_on_eio be set, but by default the more
+conservative approach is taken, so that there is no possibility of
+leaking space in the “partial temporary” failure case.
+
+
+zfs_free_leak_on_eio |
+Notes |
+
|---|
+
+
+Tags |
+debug |
+
+When to change |
+When debugging I/O errors during destroy |
+
+Data Type |
+boolean |
+
+Range |
+0=normal behavior, 1=ignore error and
+permanently leak space |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.5 and later |
+
+
+
+
+
+zfs_free_min_time_ms
+During a zfs destroy operation using feature@async_destroy a
+minimum of zfs_free_min_time_ms time will be spent working on
+freeing blocks per txg commit.
+
+
+zfs_free_min_time_ms |
+Notes |
+
|---|
+
+
+Tags |
+delete |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+milliseconds |
+
+Range |
+1 to (zfs_txg_timeout * 1000) |
+
+Default |
+1,000 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.0 and later |
+
+
+
+
+
+
+zfs_max_recordsize
+ZFS supports logical record (block) sizes from 512 bytes to 16 MiB. The
+benefits of larger blocks, and thus larger average I/O sizes, can be
+weighed against the cost of copy-on-write of large block to modify one
+byte. Additionally, very large blocks can have a negative impact on both
+I/O latency at the device level and the memory allocator. The
+zfs_max_recordsize parameter limits the upper bound of the dataset
+volblocksize and recordsize properties.
+Larger blocks can be created by enabling zpool large_blocks
+feature and changing this zfs_max_recordsize. Pools with larger
+blocks can always be imported and used, regardless of the value of
+zfs_max_recordsize.
+For 32-bit systems, zfs_max_recordsize also limits the size of
+kernel virtual memory caches used in the ZFS I/O pipeline (zio_buf_*
+and zio_data_buf_*).
+See also the zpool large_blocks feature.
+
+
+zfs_max_recordsize |
+Notes |
+
|---|
+
+
+Tags |
+filesystem,
+memory, volume |
+
+When to change |
+To create datasets with larger volblocksize or
+recordsize |
+
+Data Type |
+int |
+
+Units |
+bytes |
+
+Range |
+512 to 16,777,216 (valid block sizes) |
+
+Default |
+1,048,576 |
+
+Change |
+Dynamic, set prior to creating volumes or
+changing filesystem recordsize |
+
+Versions Affected |
+v0.6.5 and later |
+
+
+
+
+
+zfs_mdcomp_disable
+zfs_mdcomp_disable allows metadata compression to be disabled.
+
+
+zfs_mdcomp_disable |
+Notes |
+
|---|
+
+
+Tags |
+CPU, metadata |
+
+When to change |
+When CPU cycles cost less than I/O |
+
+Data Type |
+boolean |
+
+Range |
+0=compress metadata, 1=do not compress metadata |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+from v0.6.0 to v0.8.0 |
+
+
+
+
+
+
+zfs_mg_fragmentation_threshold
+Metaslab groups (top-level vdevs) are considered eligible for
+allocations if their fragmentation percentage metric is less than or
+equal to zfs_mg_fragmentation_threshold. If a metaslab group exceeds
+this threshold then it will be skipped unless all metaslab groups within
+the metaslab class have also crossed the
+zfs_mg_fragmentation_threshold threshold.
+
+
+zfs_mg_fragmentation_threshold |
+Notes |
+
|---|
+
+
+Tags |
+allocation,
+`
+fragmentation <#fragmentation>`__,
+vdev |
+
+When to change |
+Testing metaslab allocation |
+
+Data Type |
+int |
+
+Units |
+percent |
+
+Range |
+1 to 100 |
+
+Default |
+85 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.4 and later |
+
+
+
+
+
+zfs_mg_noalloc_threshold
+Metaslab groups (top-level vdevs) with free space percentage greater
+than zfs_mg_noalloc_threshold are eligible for new allocations. If a
+metaslab group’s free space is less than or equal to the threshold, the
+allocator avoids allocating to that group unless all groups in the pool
+have reached the threshold. Once all metaslab groups have reached the
+threshold, all metaslab groups are allowed to accept allocations. The
+default value of 0 disables the feature and causes all metaslab groups
+to be eligible for allocations.
+This parameter allows one to deal with pools having heavily imbalanced
+vdevs such as would be the case when a new vdev has been added. Setting
+the threshold to a non-zero percentage will stop allocations from being
+made to vdevs that aren’t filled to the specified percentage and allow
+lesser filled vdevs to acquire more allocations than they otherwise
+would under the older zfs_mg_alloc_failures facility.
+
+
+zfs_mg_noalloc_threshold |
+Notes |
+
|---|
+
+
+Tags |
+allocation,
+fragmentation,
+vdev |
+
+When to change |
+To force rebalancing as top-level vdevs
+are added or expanded |
+
+Data Type |
+int |
+
+Units |
+percent |
+
+Range |
+0 to 100 |
+
+Default |
+0 (disabled) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_multihost_history
+The pool multihost multimodifier protection (MMP) subsystem can
+record historical updates in the
+/proc/spl/kstat/zfs/POOL_NAME/multihost file for debugging purposes.
+The number of lines of history is determined by zfs_multihost_history.
+
+
+zfs_multihost_history |
+Notes |
+
|---|
+
+
+Tags |
+MMP, import |
+
+When to change |
+When testing multihost feature |
+
+Data Type |
+int |
+
+Units |
+lines |
+
+Range |
+0 to INT_MAX |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_multihost_interval
+zfs_multihost_interval controls the frequency of multihost writes
+performed by the pool multihost multimodifier protection (MMP)
+subsystem. The multihost write period is (zfs_multihost_interval /
+number of leaf-vdevs) milliseconds. Thus on average a multihost write
+will be issued for each leaf vdev every zfs_multihost_interval
+milliseconds. In practice, the observed period can vary with the I/O
+load and this observed value is the delay which is stored in the
+uberblock.
+On import the multihost activity check waits a minimum amount of time
+determined by (zfs_multihost_interval *
+zfs_multihost_import_intervals)
+with a lower bound of 1 second. The activity check time may be further
+extended if the value of mmp delay found in the best uberblock indicates
+actual multihost updates happened at longer intervals than
+zfs_multihost_interval
+Note: the multihost protection feature applies to storage devices that
+can be shared between multiple systems.
+
+
+zfs_multihost_interval |
+Notes |
+
|---|
+
+
+Tags |
+MMP, import,
+vdev |
+
+When to change |
+To optimize pool import time against
+possibility of simultaneous import by
+another system |
+
+Data Type |
+ulong |
+
+Units |
+milliseconds |
+
+Range |
+100 to ULONG_MAX |
+
+Default |
+1000 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_multihost_import_intervals
+zfs_multihost_import_intervals controls the duration of the activity
+test on pool import for the multihost multimodifier protection (MMP)
+subsystem. The activity test can be expected to take a minimum time of
+(zfs_multihost_import_intervals *
+zfs_multihost_interval * random(25%))
+milliseconds. The random period of up to 25% improves simultaneous
+import detection. For example, if two hosts are rebooted at the same
+time and automatically attempt to import the pool, then is is highly
+probable that one host will win.
+Smaller values of zfs_multihost_import_intervals reduces the import
+time but increases the risk of failing to detect an active pool. The
+total activity check time is never allowed to drop below one second.
+Note: the multihost protection feature applies to storage devices that
+can be shared between multiple systems.
+
+
+zfs_multihost_import_intervals |
+Notes |
+
|---|
+
+
+Tags |
+MMP, import |
+
+When to change |
+TBD |
+
+Data Type |
+uint |
+
+Units |
+intervals |
+
+Range |
+1 to UINT_MAX |
+
+Default |
+20 since v0.8, previously 10 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_multihost_fail_intervals
+zfs_multihost_fail_intervals controls the behavior of the pool when
+write failures are detected in the multihost multimodifier protection
+(MMP) subsystem.
+If zfs_multihost_fail_intervals = 0 then multihost write failures
+are ignored. The write failures are reported to the ZFS event daemon
+(zed) which can take action such as suspending the pool or offlining
+a device.
+
+
If
zfs_multihost_fail_intervals > 0 then sequential multihost
+write failures will cause the pool to be suspended. This occurs when
+(
zfs_multihost_fail_intervals *
+
zfs_multihost_interval) milliseconds
+have passed since the last successful multihost write.
+
This guarantees the activity test will see multihost writes if the
+pool is attempted to be imported by another system.
+
+
+zfs_multihost_fail_intervals |
+Notes |
+
|---|
+
+
+Tags |
+MMP, import |
+
+When to change |
+TBD |
+
+Data Type |
+uint |
+
+Units |
+intervals |
+
+Range |
+0 to UINT_MAX |
+
+Default |
+10 since v0.8, previously 5 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_delays_per_second
+The ZFS Event Daemon (zed) processes events from ZFS. However, it can be
+overwhelmed by high rates of error reports which can be generated by
+failing, high-performance devices. zfs_delays_per_second limits the
+rate of delay events reported to zed.
+
+
+zfs_delays_per_second |
+Notes |
+
|---|
+
+
+Tags |
+zed, delay |
+
+When to change |
+If processing delay events at a higher rate
+is desired |
+
+Data Type |
+uint |
+
+Units |
+events per second |
+
+Range |
+0 to UINT_MAX |
+
+Default |
+20 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.7 and later |
+
+
+
+
+
+zfs_checksums_per_second
+The ZFS Event Daemon (zed) processes events from ZFS. However, it can be
+overwhelmed by high rates of error reports which can be generated by
+failing, high-performance devices. zfs_checksums_per_second limits
+the rate of checksum events reported to zed.
+Note: do not set this value lower than the SERD limit for checksum
+in zed. By default, checksum_N = 10 and checksum_T = 10 minutes,
+resulting in a practical lower limit of 1.
+
+
+zfs_checksums_per_second |
+Notes |
+
|---|
+
+
+Tags |
+zed, checksum |
+
+When to change |
+If processing checksum error events at a
+higher rate is desired |
+
+Data Type |
+uint |
+
+Units |
+events per second |
+
+Range |
+0 to UINT_MAX |
+
+Default |
+20 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.7 and later |
+
+
+
+
+
+zfs_no_scrub_io
+When zfs_no_scrub_io = 1 scrubs do not actually scrub data and
+simply doing a metadata crawl of the pool instead.
+
+
+zfs_no_scrub_io |
+Notes |
+
|---|
+
+
+Tags |
+scrub |
+
+When to change |
+Testing scrub feature |
+
+Data Type |
+boolean |
+
+Range |
+0=perform scrub I/O, 1=do not perform scrub I/O |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.0 and later |
+
+
+
+
+
+zfs_no_scrub_prefetch
+When zfs_no_scrub_prefetch = 1, prefetch is disabled for scrub I/Os.
+
+
+zfs_no_scrub_prefetch |
+Notes |
+
|---|
+
+
+Tags |
+prefetch, scrub |
+
+When to change |
+Testing scrub feature |
+
+Data Type |
+boolean |
+
+Range |
+0=prefetch scrub I/Os, 1=do not prefetch scrub I/Os |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.4 and later |
+
+
+
+
+
+zfs_nocacheflush
+ZFS uses barriers (volatile cache flush commands) to ensure data is
+committed to permanent media by devices. This ensures consistent
+on-media state for devices where caches are volatile (eg HDDs).
+For devices with nonvolatile caches, the cache flush operation can be a
+no-op. However, in some RAID arrays, cache flushes can cause the entire
+cache to be flushed to the backing devices.
+To ensure on-media consistency, keep cache flush enabled.
+
+
+zfs_nocacheflush |
+Notes |
+
|---|
+
+
+Tags |
+disks |
+
+When to change |
+If the storage device has nonvolatile cache,
+then disabling cache flush can save the cost of
+occasional cache flush commands |
+
+Data Type |
+boolean |
+
+Range |
+0=send cache flush commands, 1=do not send
+cache flush commands |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_nopwrite_enabled
+The NOP-write feature is enabled by default when a
+crytographically-secure checksum algorithm is in use by the dataset.
+zfs_nopwrite_enabled allows the NOP-write feature to be completely
+disabled.
+
+
+zfs_nopwrite_enabled |
+Notes |
+
|---|
+
+
+Tags |
+checksum, debug |
+
+When to change |
+TBD |
+
+Data Type |
+boolean |
+
+Range |
+0=disable NOP-write feature, 1=enable
+NOP-write feature |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.0 and later |
+
+
+
+
+
+zfs_dmu_offset_next_sync
+zfs_dmu_offset_next_sync enables forcing txg sync to find holes.
+This causes ZFS to act like older versions when SEEK_HOLE or
+SEEK_DATA flags are used: when a dirty dnode causes txgs to be
+synced so the previous data can be found.
+
+
+zfs_dmu_offset_next_sync |
+Notes |
+
|---|
+
+
+Tags |
+DMU |
+
+When to change |
+to exchange strict hole reporting for
+performance |
+
+Data Type |
+boolean |
+
+Range |
+0=do not force txg sync to find holes,
+1=force txg sync to find holes |
+
+Default |
+1 since v2.1.5, previously 0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_pd_bytes_max
+zfs_pd_bytes_max limits the number of bytes prefetched during a pool
+traversal (eg zfs send or other data crawling operations). These
+prefetches are referred to as “prescient prefetches” and are always 100%
+hit rate. The traversal operations do not use the default data or
+metadata prefetcher.
+
+
+zfs_pd_bytes_max |
+Notes |
+
|---|
+
+
+Tags |
+prefetch, send |
+
+When to change |
+TBD |
+
+Data Type |
+int32 |
+
+Units |
+bytes |
+
+Range |
+0 to INT32_MAX |
+
+Default |
+52,428,800 (50 MiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+TBD |
+
+
+
+
+
+zfs_per_txg_dirty_frees_percent
+zfs_per_txg_dirty_frees_percent as a percentage of
+zfs_dirty_data_max controls the percentage of
+dirtied blocks from frees in one txg. After the threshold is crossed,
+additional dirty blocks from frees wait until the next txg. Thus, when
+deleting large files, filling consecutive txgs with deletes/frees, does
+not throttle other, perhaps more important, writes.
+A side effect of this throttle can impact zfs receive workloads that
+contain a large number of frees and the
+ignore_hole_birth optimization is disabled. The
+symptom is that the receive workload causes an increase in the frequency
+of txg commits. The frequency of txg commits is observable via the
+otime column of /proc/spl/kstat/zfs/POOLNAME/txgs. Since txg
+commits also flush data from volatile caches in HDDs to media, HDD
+performance can be negatively impacted. Also, since the frees do not
+consume much bandwidth over the pipe, the pipe can appear to stall. Thus
+the overall progress of receives is slower than expected.
+A value of zero will disable this throttle.
+
+
+zfs_per_txg_dirty_frees_percent |
+Notes |
+
|---|
+
+
+Tags |
+delete |
+
+When to change |
+For zfs receive workloads,
+consider increasing or disabling.
+See section ZFS I/O
+S
+cheduler |
+
+Data Type |
+ulong |
+
+Units |
+percent |
+
+Range |
+0 to 100 |
+
+Default |
+30 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_prefetch_disable
+zfs_prefetch_disable controls the predictive prefetcher.
+Note that it leaves “prescient” prefetch (eg prefetch for zfs send)
+intact (see zfs_pd_bytes_max)
+
+
+zfs_prefetch_disable |
+Notes |
+
|---|
+
+
+Tags |
+prefetch |
+
+When to change |
+In some case where the workload is
+completely random reads, overall performance
+can be better if prefetch is disabled |
+
+Data Type |
+boolean |
+
+Range |
+0=prefetch enabled, 1=prefetch disabled |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Verification |
+prefetch efficacy is observed by
+arcstat, arc_summary, and the
+relevant entries in
+/proc/spl/kstat/zfs/arcstats |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_read_chunk_size
+zfs_read_chunk_size is the limit for ZFS filesystem reads. If an
+application issues a read() larger than zfs_read_chunk_size,
+then the read() is divided into multiple operations no larger than
+zfs_read_chunk_size
+
+
+zfs_read_chunk_size |
+Notes |
+
|---|
+
+
+Tags |
+filesystem |
+
+When to change |
+TBD |
+
+Data Type |
+ulong |
+
+Units |
+bytes |
+
+Range |
+512 to ULONG_MAX |
+
+Default |
+1,048,576 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_read_history
+Historical statistics for the last zfs_read_history reads are
+available in /proc/spl/kstat/zfs/POOL_NAME/reads
+
+
+zfs_read_history |
+Notes |
+
|---|
+
+
+Tags |
+debug |
+
+When to change |
+To observe read operation details |
+
+Data Type |
+int |
+
+Units |
+lines |
+
+Range |
+0 to INT_MAX |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_read_history_hits
+When zfs_read_history> 0,
+zfs_read_history_hits controls whether ARC hits are displayed in the
+read history file, /proc/spl/kstat/zfs/POOL_NAME/reads
+
+
+zfs_read_history_hits |
+Notes |
+
|---|
+
+
+Tags |
+debug |
+
+When to change |
+To observe read operation details with ARC
+hits |
+
+Data Type |
+boolean |
+
+Range |
+0=do not include data for ARC hits,
+1=include ARC hit data |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_recover
+zfs_recover can be set to true (1) to attempt to recover from
+otherwise-fatal errors, typically caused by on-disk corruption. When
+set, calls to zfs_panic_recover() will turn into warning messages
+rather than calling panic()
+
+
+zfs_recover |
+Notes |
+
|---|
+
+
+Tags |
+import |
+
+When to change |
+zfs_recover should only be used as a last
+resort, as it typically results in leaked
+space, or worse |
+
+Data Type |
+boolean |
+
+Range |
+0=normal operation, 1=attempt recovery zpool
+import |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Verification |
+check output of dmesg and other logs for
+details |
+
+Versions Affected |
+v0.6.4 or later |
+
+
+
+
+
+zfs_resilver_min_time_ms
+Resilvers are processed by the sync thread in syncing context. While
+resilvering, ZFS spends at least zfs_resilver_min_time_ms time
+working on a resilver between txg commits.
+The zfs_txg_timeout tunable sets a nominal
+timeout value for the txg commits. By default, this timeout is 5 seconds
+and the zfs_resilver_min_time_ms is 3 seconds. However, many
+variables contribute to changing the actual txg times. The measured txg
+interval is observed as the otime column (in nanoseconds) in the
+/proc/spl/kstat/zfs/POOL_NAME/txgs file.
+See also zfs_txg_timeout and
+zfs_scan_min_time_ms
+
+
+zfs_resilver_min_time_ms |
+Notes |
+
|---|
+
+
+Tags |
+resilver |
+
+When to change |
+In some resilvering cases, increasing
+zfs_resilver_min_time_ms can result
+in faster completion |
+
+Data Type |
+int |
+
+Units |
+milliseconds |
+
+Range |
+1 to
+zfs_txg_timeout
+converted to milliseconds |
+
+Default |
+3,000 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_scan_min_time_ms
+Scrubs are processed by the sync thread in syncing context. While
+scrubbing, ZFS spends at least zfs_scan_min_time_ms time working on
+a scrub between txg commits.
+See also zfs_txg_timeout and
+zfs_resilver_min_time_ms
+
+
+zfs_scan_min_time_ms |
+Notes |
+
|---|
+
+
+Tags |
+scrub |
+
+When to change |
+In some scrub cases, increasing
+zfs_scan_min_time_ms can result in
+faster completion |
+
+Data Type |
+int |
+
+Units |
+milliseconds |
+
+Range |
+1 to zfs_txg_timeout
+converted to milliseconds |
+
+Default |
+1,000 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_scan_checkpoint_intval
+To preserve progress across reboots the sequential scan algorithm
+periodically needs to stop metadata scanning and issue all the
+verifications I/Os to disk every zfs_scan_checkpoint_intval seconds.
+
+
+zfs_scan_checkpoint_intval |
+Notes |
+
|---|
+
+
+Tags |
+resilver, scrub |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+seconds |
+
+Range |
+1 to INT_MAX |
+
+Default |
+7,200 (2 hours) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.8.0 and later |
+
+
+
+
+
+zfs_scan_fill_weight
+This tunable affects how scrub and resilver I/O segments are ordered. A
+higher number indicates that we care more about how filled in a segment
+is, while a lower number indicates we care more about the size of the
+extent without considering the gaps within a segment.
+
+
+zfs_scan_fill_weight |
+Notes |
+
|---|
+
+
+Tags |
+resilver, scrub |
+
+When to change |
+Testing sequential scrub and resilver |
+
+Data Type |
+int |
+
+Units |
+scalar |
+
+Range |
+0 to INT_MAX |
+
+Default |
+3 |
+
+Change |
+Prior to zfs module load |
+
+Versions Affected |
+v0.8.0 and later |
+
+
+
+
+
+zfs_scan_issue_strategy
+zfs_scan_issue_strategy controls the order of data verification
+while scrubbing or resilvering.
+
+
+value |
+description |
+
|---|
+
+
+0 |
+fs will use strategy 1 during normal verification and
+strategy 2 while taking a checkpoint |
+
+1 |
+data is verified as sequentially as possible, given the
+amount of memory reserved for scrubbing (see
+zfs_scan_mem_lim_fact). This
+can improve scrub performance if the pool’s data is heavily
+fragmented. |
+
+2 |
+the largest mostly-contiguous chunk of found data is
+verified first. By deferring scrubbing of small segments,
+we may later find adjacent data to coalesce and increase
+the segment size. |
+
+
+
+
+
+zfs_scan_issue_strategy |
+Notes |
+
|---|
+
+
+Tags |
+resilver, scrub |
+
+When to change |
+TBD |
+
+Data Type |
+enum |
+
+Range |
+0 to 2 |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+TBD |
+
+
+
+
+
+zfs_scan_legacy
+Setting zfs_scan_legacy = 1 enables the legacy scan and scrub
+behavior instead of the newer sequential behavior.
+
+
+zfs_scan_legacy |
+Notes |
+
|---|
+
+
+Tags |
+resilver, scrub |
+
+When to change |
+In some cases, the new scan mode can consumer
+more memory as it collects and sorts I/Os;
+using the legacy algorithm can be more memory
+efficient at the expense of HDD read efficiency |
+
+Data Type |
+boolean |
+
+Range |
+0=use new method: scrubs and resilvers will
+gather metadata in memory before issuing
+sequential I/O, 1=use legacy algorithm will be
+used where I/O is initiated as soon as it is
+discovered |
+
+Default |
+0 |
+
+Change |
+Dynamic, however changing to 0 does not affect
+in-progress scrubs or resilvers |
+
+Versions Affected |
+v0.8.0 and later |
+
+
+
+
+
+zfs_scan_max_ext_gap
+zfs_scan_max_ext_gap limits the largest gap in bytes between scrub
+and resilver I/Os that will still be considered sequential for sorting
+purposes.
+
+
+zfs_scan_max_ext_gap |
+Notes |
+
|---|
+
+
+Tags |
+resilver, scrub |
+
+When to change |
+TBD |
+
+Data Type |
+ulong |
+
+Units |
+bytes |
+
+Range |
+512 to ULONG_MAX |
+
+Default |
+2,097,152 (2 MiB) |
+
+Change |
+Dynamic, however changing to 0 does not
+affect in-progress scrubs or resilvers |
+
+Versions Affected |
+v0.8.0 and later |
+
+
+
+
+
+zfs_scan_mem_lim_fact
+zfs_scan_mem_lim_fact limits the maximum fraction of RAM used for
+I/O sorting by sequential scan algorithm. When the limit is reached
+scanning metadata is stopped and data verification I/O is started. Data
+verification I/O continues until the memory used by the sorting
+algorithm drops by
+zfs_scan_mem_lim_soft_fact
+Memory used by the sequential scan algorithm can be observed as the kmem
+sio_cache. This is visible from procfs as
+grep sio_cache /proc/slabinfo and can be monitored using
+slab-monitoring tools such as slabtop
+
+
+zfs_scan_mem_lim_fact |
+Notes |
+
|---|
+
+
+Tags |
+memory,
+resilver,
+scrub |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+divisor of physical RAM |
+
+Range |
+TBD |
+
+Default |
+20 (physical RAM / 20 or 5%) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.8.0 and later |
+
+
+
+
+
+zfs_scan_mem_lim_soft_fact
+zfs_scan_mem_lim_soft_fact sets the fraction of the hard limit,
+zfs_scan_mem_lim_fact, used to determined
+the RAM soft limit for I/O sorting by the sequential scan algorithm.
+After zfs_scan_mem_lim_fact has been
+reached, metadata scanning is stopped until the RAM usage drops by
+zfs_scan_mem_lim_soft_fact
+
+
+zfs_scan_mem_lim_soft_fact |
+Notes |
+
|---|
+
+
+Tags |
+resilver,
+scrub |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+divisor of (physical RAM /
+zfs_scan_mem
+_lim_fact) |
+
+Range |
+1 to INT_MAX |
+
+Default |
+20 (for default
+zfs_scan_mem
+_lim_fact,
+0.25% of physical RAM) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.8.0 and later |
+
+
+
+
+
+zfs_scan_vdev_limit
+zfs_scan_vdev_limit is the maximum amount of data that can be
+concurrently issued at once for scrubs and resilvers per leaf vdev.
+zfs_scan_vdev_limit attempts to strike a balance between keeping the
+leaf vdev queues full of I/Os while not overflowing the queues causing
+high latency resulting in long txg sync times. While
+zfs_scan_vdev_limit represents a bandwidth limit, the existing I/O
+limit of zfs_vdev_scrub_max_active
+remains in effect, too.
+
+
+zfs_scan_vdev_limit |
+Notes |
+
|---|
+
+
+Tags |
+resilver, scrub,
+vdev |
+
+When to change |
+TBD |
+
+Data Type |
+ulong |
+
+Units |
+bytes |
+
+Range |
+512 to ULONG_MAX |
+
+Default |
+4,194,304 (4 MiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.8.0 and later |
+
+
+
+
+
+zfs_send_corrupt_data
+zfs_send_corrupt_data enables zfs send to send of corrupt data
+by ignoring read and checksum errors. The corrupted or unreadable blocks
+are replaced with the value 0x2f5baddb10c (ZFS bad block)
+
+
+zfs_send_corrupt_data |
+Notes |
+
|---|
+
+
+Tags |
+send |
+
+When to change |
+When data corruption exists and an attempt
+to recover at least some data via
+zfs send is needed |
+
+Data Type |
+boolean |
+
+Range |
+0=do not send corrupt data, 1=replace
+corrupt data with cookie |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.0 and later |
+
+
+
+
+
+zfs_sync_pass_deferred_free
+The SPA sync process is performed in multiple passes. Once the pass
+number reaches zfs_sync_pass_deferred_free, frees are no long
+processed and must wait for the next SPA sync.
+The zfs_sync_pass_deferred_free value is expected to be removed as a
+tunable once the optimal value is determined during field testing.
+The zfs_sync_pass_deferred_free pass must be greater than 1 to
+ensure that regular blocks are not deferred.
+
+
+zfs_sync_pass_deferred_free |
+Notes |
+
|---|
+
+
+Tags |
+SPA |
+
+When to change |
+Testing SPA sync process |
+
+Data Type |
+int |
+
+Units |
+SPA sync passes |
+
+Range |
+1 to INT_MAX |
+
+Default |
+2 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_sync_pass_dont_compress
+The SPA sync process is performed in multiple passes. Once the pass
+number reaches zfs_sync_pass_dont_compress, data block compression
+is no longer processed and must wait for the next SPA sync.
+The zfs_sync_pass_dont_compress value is expected to be removed as a
+tunable once the optimal value is determined during field testing.
+
+
+zfs_sync_pass_dont_compress |
+Notes |
+
|---|
+
+
+Tags |
+SPA |
+
+When to change |
+Testing SPA sync process |
+
+Data Type |
+int |
+
+Units |
+SPA sync passes |
+
+Range |
+1 to INT_MAX |
+
+Default |
+5 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_sync_pass_rewrite
+The SPA sync process is performed in multiple passes. Once the pass
+number reaches zfs_sync_pass_rewrite, blocks can be split into gang
+blocks.
+The zfs_sync_pass_rewrite value is expected to be removed as a
+tunable once the optimal value is determined during field testing.
+
+
+zfs_sync_pass_rewrite |
+Notes |
+
|---|
+
+
+Tags |
+SPA |
+
+When to change |
+Testing SPA sync process |
+
+Data Type |
+int |
+
+Units |
+SPA sync passes |
+
+Range |
+1 to INT_MAX |
+
+Default |
+2 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_sync_taskq_batch_pct
+zfs_sync_taskq_batch_pct controls the number of threads used by the
+DSL pool sync taskq, dp_sync_taskq
+
+
+zfs_sync_taskq_batch_pct |
+Notes |
+
|---|
+
+
+Tags |
+SPA |
+
+When to change |
+to adjust the number of
+dp_sync_taskq threads |
+
+Data Type |
+int |
+
+Units |
+percent of number of online CPUs |
+
+Range |
+1 to 100 |
+
+Default |
+75 |
+
+Change |
+Prior to zfs module load |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_txg_history
+Historical statistics for the last zfs_txg_history txg commits are
+available in /proc/spl/kstat/zfs/POOL_NAME/txgs
+The work required to measure the txg commit (SPA statistics) is low.
+However, for debugging purposes, it can be useful to observe the SPA
+statistics.
+
+
+zfs_txg_history |
+Notes |
+
|---|
+
+
+Tags |
+debug |
+
+When to change |
+To observe details of SPA sync behavior. |
+
+Data Type |
+int |
+
+Units |
+lines |
+
+Range |
+0 to INT_MAX |
+
+Default |
+0 for version v0.6.0 to v0.7.6, 100 for version v0.8.0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_txg_timeout
+The open txg is committed to the pool periodically (SPA sync) and
+zfs_txg_timeout represents the default target upper limit.
+txg commits can occur more frequently and a rapid rate of txg commits
+often indicates a busy write workload, quota limits reached, or the free
+space is critically low.
+Many variables contribute to changing the actual txg times. txg commits
+can also take longer than zfs_txg_timeout if the ZFS write throttle
+is not properly tuned or the time to sync is otherwise delayed (eg slow
+device). Shorter txg commit intervals can occur due to
+zfs_dirty_data_sync for write-intensive
+workloads. The measured txg interval is observed as the otime column
+(in nanoseconds) in the /proc/spl/kstat/zfs/POOL_NAME/txgs file.
+See also zfs_dirty_data_sync and
+zfs_txg_history
+
+
+zfs_txg_timeout |
+Notes |
+
|---|
+
+
+Tags |
+SPA,
+ZIO_scheduler |
+
+When to change |
+To optimize the work done by txg commit
+relative to the pool requirements. See also
+section ZFS I/O
+Scheduler |
+
+Data Type |
+int |
+
+Units |
+seconds |
+
+Range |
+1 to INT_MAX |
+
+Default |
+5 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_vdev_aggregation_limit
+To reduce IOPs, small, adjacent I/Os can be aggregated (coalesced) into
+a large I/O. For reads, aggregations occur across small adjacency gaps.
+For writes, aggregation can occur at the ZFS or disk level.
+zfs_vdev_aggregation_limit is the upper bound on the size of the
+larger, aggregated I/O.
+Setting zfs_vdev_aggregation_limit = 0 effectively disables
+aggregation by ZFS. However, the block device scheduler can still merge
+(aggregate) I/Os. Also, many devices, such as modern HDDs, contain
+schedulers that can aggregate I/Os.
+In general, I/O aggregation can improve performance for devices, such as
+HDDs, where ordering I/O operations for contiguous LBAs is a benefit.
+For random access devices, such as SSDs, aggregation might not improve
+performance relative to the CPU cycles needed to aggregate. For devices
+that represent themselves as having no rotation, the
+zfs_vdev_aggregation_limit_non_rotating
+parameter is used instead of zfs_vdev_aggregation_limit
+
+
+zfs_vdev_aggregation_limit |
+Notes |
+
|---|
+
+
+Tags |
+vdev,
+ZIO_scheduler |
+
+When to change |
+If the workload does not benefit from
+aggregation, the
+zfs_vdev_aggregation_limit can be
+reduced to avoid aggregation attempts |
+
+Data Type |
+int |
+
+Units |
+bytes |
+
+Range |
+0 to 1,048,576 (default) or 16,777,216
+(if zpool large_blocks feature
+is enabled) |
+
+Default |
+1,048,576, or 131,072 for <v0.8 |
+
+Change |
+Dynamic |
+
+Verification |
+ZFS aggregation is observed with
+zpool iostat -r and the block
+scheduler merging is observed with
+iostat -x |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_vdev_cache_size
+Note: with the current ZFS code, the vdev cache is not helpful and in
+some cases actually harmful. Thusit is disabled by setting the
+zfs_vdev_cache_size = 0
+zfs_vdev_cache_size is the size of the vdev cache.
+
+
+zfs_vdev_cache_size |
+Notes |
+
|---|
+
+
+Tags |
+vdev,
+vdev_cache |
+
+When to change |
+Do not change |
+
+Data Type |
+int |
+
+Units |
+bytes |
+
+Range |
+0 to MAX_INT |
+
+Default |
+0 (vdev cache is disabled) |
+
+Change |
+Dynamic |
+
+Verification |
+vdev cache statistics are available in the
+/proc/spl/kstat/zfs/vdev_cache_stats file |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_vdev_cache_bshift
+Note: with the current ZFS code, the vdev cache is not helpful and in
+some cases actually harmful. Thus it is disabled by setting the
+zfs_vdev_cache_size to zero. This related
+tunable is, by default, inoperative.
+All read I/Os smaller than zfs_vdev_cache_max
+are turned into (1 << zfs_vdev_cache_bshift) byte reads by the vdev
+cache. At most zfs_vdev_cache_size bytes will
+be kept in each vdev’s cache.
+
+
+zfs_vdev_cache_bshift |
+Notes |
+
|---|
+
+
+Tags |
+vdev, vdev_cache |
+
+When to change |
+Do not change |
+
+Data Type |
+int |
+
+Units |
+shift |
+
+Range |
+1 to INT_MAX |
+
+Default |
+16 (65,536 bytes) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_vdev_cache_max
+Note: with the current ZFS code, the vdev cache is not helpful and in
+some cases actually harmful. Thus it is disabled by setting the
+zfs_vdev_cache_size to zero. This related
+tunable is, by default, inoperative.
+All read I/Os smaller than zfs_vdev_cache_max will be turned into
+(1 <<zfs_vdev_cache_bshift byte reads
+by the vdev cache. At most zfs_vdev_cache_size bytes will be kept in
+each vdev’s cache.
+
+
+zfs_vdev_cache_max |
+Notes |
+
|---|
+
+
+Tags |
+vdev, vdev_cache |
+
+When to change |
+Do not change |
+
+Data Type |
+int |
+
+Units |
+bytes |
+
+Range |
+512 to INT_MAX |
+
+Default |
+16,384 (16 KiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_vdev_mirror_rotating_inc
+The mirror read algorithm uses current load and an incremental weighting
+value to determine the vdev to service a read operation. Lower values
+determine the preferred vdev. The weighting value is
+zfs_vdev_mirror_rotating_inc for rotating media and
+zfs_vdev_mirror_non_rotating_inc
+for nonrotating media.
+Verify the rotational setting described by a block device in sysfs by
+observing /sys/block/DISK_NAME/queue/rotational
+
+
+zfs_vdev_mirror_rotating_inc |
+Notes |
+
|---|
+
+
+Tags |
+vdev,
+mirror, HDD |
+
+When to change |
+Increasing for mirrors with both
+rotating and nonrotating media more
+strongly favors the nonrotating
+media |
+
+Data Type |
+int |
+
+Units |
+scalar |
+
+Range |
+0 to MAX_INT |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_vdev_mirror_non_rotating_inc
+The mirror read algorithm uses current load and an incremental weighting
+value to determine the vdev to service a read operation. Lower values
+determine the preferred vdev. The weighting value is
+zfs_vdev_mirror_rotating_inc for
+rotating media and zfs_vdev_mirror_non_rotating_inc for nonrotating
+media.
+Verify the rotational setting described by a block device in sysfs by
+observing /sys/block/DISK_NAME/queue/rotational
+
+
+zfs_vdev_mirror_non_rotating_inc |
+Notes |
+
|---|
+
+
+Tags |
+vdev,
+mirror,
+SSD |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+scalar |
+
+Range |
+0 to INT_MAX |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_vdev_mirror_rotating_seek_inc
+For rotating media in a mirror, if the next I/O offset is within
+zfs_vdev_mirror_rotating_seek_offset
+then the weighting factor is incremented by
+(zfs_vdev_mirror_rotating_seek_inc / 2). Otherwise the weighting
+factor is increased by zfs_vdev_mirror_rotating_seek_inc. This
+algorithm prefers rotating media with lower seek distance.
+Verify the rotational setting described by a block device in sysfs by
+observing /sys/block/DISK_NAME/queue/rotational
+
+
+z
+fs_vdev_mirror_rotating_seek_inc |
+Notes |
+
|---|
+
+
+Tags |
+vdev,
+mirror,
+HDD |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+scalar |
+
+Range |
+0 to INT_MAX |
+
+Default |
+5 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_vdev_mirror_rotating_seek_offset
+For rotating media in a mirror, if the next I/O offset is within
+zfs_vdev_mirror_rotating_seek_offset then the weighting factor is
+incremented by
+(zfs_vdev_mirror_rotating_seek_inc/ 2).
+Otherwise the weighting factor is increased by
+zfs_vdev_mirror_rotating_seek_inc. This algorithm prefers rotating
+media with lower seek distance.
+Verify the rotational setting described by a block device in sysfs by
+observing /sys/block/DISK_NAME/queue/rotational
+
+
+zfs_vdev_mirror_rotating_seek_off
+set |
+Notes |
+
|---|
+
+
+Tags |
+vdev,
+mirror,
+HDD |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+bytes |
+
+Range |
+0 to INT_MAX |
+
+Default |
+1,048,576 (1 MiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_vdev_mirror_non_rotating_seek_inc
+For nonrotating media in a mirror, a seek penalty is applied as
+sequential I/O’s can be aggregated into fewer operations, avoiding
+unnecessary per-command overhead, often boosting performance.
+Verify the rotational setting described by a block device in SysFS by
+observing /sys/block/DISK_NAME/queue/rotational
+
+
+zfs_v
+dev_mirror_non_rotating_seek_inc |
+Notes |
+
|---|
+
+
+Tags |
+vdev,
+mirror,
+SSD |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+scalar |
+
+Range |
+0 to INT_MAX |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_vdev_read_gap_limit
+To reduce IOPs, small, adjacent I/Os are aggregated (coalesced) into
+into a large I/O. For reads, aggregations occur across small adjacency
+gaps where the gap is less than zfs_vdev_read_gap_limit
+
+
+zfs_vdev_read_gap_limit |
+Notes |
+
|---|
+
+
+Tags |
+vdev,
+ZIO_scheduler |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+bytes |
+
+Range |
+0 to INT_MAX |
+
+Default |
+32,768 (32 KiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_vdev_write_gap_limit
+To reduce IOPs, small, adjacent I/Os are aggregated (coalesced) into
+into a large I/O. For writes, aggregations occur across small adjacency
+gaps where the gap is less than zfs_vdev_write_gap_limit
+
+
+zfs_vdev_write_gap_limit |
+Notes |
+
|---|
+
+
+Tags |
+vdev,
+ZIO_scheduler |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+bytes |
+
+Range |
+0 to INT_MAX |
+
+Default |
+4,096 (4 KiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_vdev_scheduler
+Prior to version 0.8.3, when the pool is imported, for whole disk vdevs,
+the block device I/O scheduler is set to zfs_vdev_scheduler.
+The most common schedulers are: noop, cfq, bfq, and deadline.
+In some cases, the scheduler is not changeable using this method.
+Known schedulers that cannot be changed are: scsi_mq and none.
+In these cases, the scheduler is unchanged and an error message can be
+reported to logs.
+The parameter was disabled in v0.8.3 but left in place to avoid breaking
+loading of the zfs module if the parameter is specified in modprobe
+configuration on existing installations. It is recommended that users
+leave the default scheduler “unless you’re encountering a specific
+problem, or have clearly measured a performance improvement for your
+workload,”
+and if so, to change it via the /sys/block/<device>/queue/scheduler
+interface and/or udev rule.
+
+
+zfs_vdev_scheduler |
+Notes |
+
|---|
+
+
+Tags |
+vdev,
+ZIO_scheduler |
+
+When to change |
+since ZFS has its own I/O scheduler, using a
+simple scheduler can result in more consistent
+performance |
+
+Data Type |
+string |
+
+Range |
+expected: noop, cfq, bfq, and deadline |
+
+Default |
+noop |
+
+Change |
+Dynamic, but takes effect upon pool creation
+or import |
+
+Versions Affected |
+all, but no effect since v0.8.3 |
+
+
+
+
+
+zfs_vdev_raidz_impl
+zfs_vdev_raidz_impl overrides the raidz parity algorithm. By
+default, the algorithm is selected at zfs module load time by the
+results of a microbenchmark of algorithms based on the current hardware.
+Once the module is loaded, the content of
+/sys/module/zfs/parameters/zfs_vdev_raidz_impl shows available
+options with the currently selected enclosed in []. Details of the
+results of the microbenchmark are observable in the
+/proc/spl/kstat/zfs/vdev_raidz_bench file.
+
+
+algorithm |
+architecture |
+description |
+
|---|
+
+
+fastest |
+all |
+fastest implementation
+selected by
+microbenchmark |
+
+original |
+all |
+original raidz
+implementation |
+
+scalar |
+all |
+scalar raidz
+implementation |
+
+sse2 |
+64-bit x86 |
+uses SSE2 instruction
+set |
+
+ssse3 |
+64-bit x86 |
+uses SSSE3 instruction
+set |
+
+avx2 |
+64-bit x86 |
+uses AVX2 instruction
+set |
+
+avx512f |
+64-bit x86 |
+uses AVX512F
+instruction set |
+
+avx512bw |
+64-bit x86 |
+uses AVX512F & AVX512BW
+instruction sets |
+
+aarch64_neon |
+aarch64/64 bit ARMv8 |
+uses NEON |
+
+aarch64_neonx2 |
+aarch64/64 bit ARMv8 |
+uses NEON with more
+unrolling |
+
+
+
+
+
+zfs_vdev_raidz_impl |
+Notes |
+
|---|
+
+
+Tags |
+CPU, raidz, vdev |
+
+When to change |
+testing raidz algorithms |
+
+Data Type |
+string |
+
+Range |
+see table above |
+
+Default |
+fastest |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_zevent_cols
+zfs_zevent_cols is a soft wrap limit in columns (characters) for ZFS
+events logged to the console.
+
+
+zfs_zevent_cols |
+Notes |
+
|---|
+
+
+Tags |
+debug |
+
+When to change |
+if 80 columns isn’t enough |
+
+Data Type |
+int |
+
+Units |
+characters |
+
+Range |
+1 to INT_MAX |
+
+Default |
+80 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_zevent_console
+If zfs_zevent_console is true (1), then ZFS events are logged to the
+console.
+More logging and log filtering capabilities are provided by zed
+
+
+zfs_zevent_console |
+Notes |
+
|---|
+
+
+Tags |
+debug |
+
+When to change |
+to log ZFS events to the console |
+
+Data Type |
+boolean |
+
+Range |
+0=do not log to console, 1=log to console |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_zevent_len_max
+zfs_zevent_len_max is the maximum ZFS event queue length. A value of
+0 results in a calculated value (16 * number of CPUs) with a minimum of
+64. Events in the queue can be viewed with the zpool events command.
+
+
+zfs_zevent_len_max |
+Notes |
+
|---|
+
+
+Tags |
+debug |
+
+When to change |
+increase to see more ZFS events |
+
+Data Type |
+int |
+
+Units |
+events |
+
+Range |
+0 to INT_MAX |
+
+Default |
+0 (calculate as described above) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zfs_zil_clean_taskq_maxalloc
+During a SPA sync, intent log transaction groups (itxg) are cleaned. The
+cleaning work is dispatched to the DSL pool ZIL clean taskq
+(dp_zil_clean_taskq).
+zfs_zil_clean_taskq_minalloc is the
+minimum and zfs_zil_clean_taskq_maxalloc is the maximum number of
+cached taskq entries for dp_zil_clean_taskq. The actual number of
+taskq entries dynamically varies between these values.
+When zfs_zil_clean_taskq_maxalloc is exceeded transaction records
+(itxs) are cleaned synchronously with possible negative impact to the
+performance of SPA sync.
+Ideally taskq entries are pre-allocated prior to being needed by
+zil_clean(), thus avoiding dynamic allocation of new taskq entries.
+
+
+zfs_zil_clean_taskq_maxalloc |
+Notes |
+
|---|
+
+
+Tags |
+ZIL |
+
+When to change |
+If more dp_zil_clean_taskq
+entries are needed to prevent the
+itxs from being synchronously
+cleaned |
+
+Data Type |
+int |
+
+Units |
+dp_zil_clean_taskq taskq entries
|
+
+Range |
+zfs_zil_clean_taskq_minallo
+c
+to INT_MAX |
+
+Default |
+1,048,576 |
+
+Change |
+Dynamic, takes effect per-pool when
+the pool is imported |
+
+Versions Affected |
+v0.8.0 |
+
+
+
+
+
+zfs_zil_clean_taskq_minalloc
+During a SPA sync, intent log transaction groups (itxg) are cleaned. The
+cleaning work is dispatched to the DSL pool ZIL clean taskq
+(dp_zil_clean_taskq). zfs_zil_clean_taskq_minalloc is the
+minimum and
+zfs_zil_clean_taskq_maxalloc is the
+maximum number of cached taskq entries for dp_zil_clean_taskq. The
+actual number of taskq entries dynamically varies between these values.
+zfs_zil_clean_taskq_minalloc is the minimum number of ZIL
+transaction records (itxs).
+Ideally taskq entries are pre-allocated prior to being needed by
+zil_clean(), thus avoiding dynamic allocation of new taskq entries.
+
+
+zfs_zil_clean_taskq_minalloc |
+Notes |
+
|---|
+
+
+Tags |
+ZIL |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+dp_zil_clean_taskq taskq entries |
+
+Range |
+1 to
+zfs_zil_clean_taskq_maxallo
+c |
+
+Default |
+1,024 |
+
+Change |
+Dynamic, takes effect per-pool when
+the pool is imported |
+
+Versions Affected |
+v0.8.0 |
+
+
+
+
+
+zfs_zil_clean_taskq_nthr_pct
+zfs_zil_clean_taskq_nthr_pct controls the number of threads used by
+the DSL pool ZIL clean taskq (dp_zil_clean_taskq). The default value
+of 100% will create a maximum of one thread per cpu.
+
+
+zfs_zil_clean_taskq_nthr_pct |
+Notes |
+
|---|
+
+
+Tags |
+taskq, ZIL |
+
+When to change |
+Testing ZIL clean and SPA sync
+performance |
+
+Data Type |
+int |
+
+Units |
+percent of number of CPUs |
+
+Range |
+1 to 100 |
+
+Default |
+100 |
+
+Change |
+Dynamic, takes effect per-pool when
+the pool is imported |
+
+Versions Affected |
+v0.8.0 |
+
+
+
+
+
+zil_replay_disable
+If zil_replay_disable = 1, then when a volume or filesystem is
+brought online, no attempt to replay the ZIL is made and any existing
+ZIL is destroyed. This can result in loss of data without notice.
+
+
+zil_replay_disable |
+Notes |
+
|---|
+
+
+Tags |
+debug, ZIL |
+
+When to change |
+Do not change |
+
+Data Type |
+boolean |
+
+Range |
+0=replay ZIL, 1=destroy ZIL |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.5 |
+
+
+
+
+
+zil_slog_bulk
+zil_slog_bulk is the log device write size limit per commit executed
+with synchronous priority. Writes below zil_slog_bulk are executed
+with synchronous priority. Writes above zil_slog_bulk are executed
+with lower (asynchronous) priority to reduct potential log device abuse
+by a single active ZIL writer.
+
+
+zil_slog_bulk |
+Notes |
+
|---|
+
+
+Tags |
+ZIL |
+
+When to change |
+See ZFS I/O
+Scheduler |
+
+Data Type |
+ulong |
+
+Units |
+bytes |
+
+Range |
+0 to ULONG_MAX |
+
+Default |
+786,432 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.8.0 |
+
+
+
+
+
+zio_delay_max
+If a ZFS I/O operation takes more than zio_delay_max milliseconds to
+complete, then an event is logged. Note that this is only a logging
+facility, not a timeout on operations. See also zpool events
+
+
+zio_delay_max |
+Notes |
+
|---|
+
+
+Tags |
+debug |
+
+When to change |
+when debugging slow I/O |
+
+Data Type |
+int |
+
+Units |
+milliseconds |
+
+Range |
+1 to INT_MAX |
+
+Default |
+30,000 (30 seconds) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zio_dva_throttle_enabled
+zio_dva_throttle_enabled controls throttling of block allocations in
+the ZFS I/O (ZIO) pipeline. When enabled, the maximum number of pending
+allocations per top-level vdev is limited by
+zfs_vdev_queue_depth_pct
+
+
+zio_dva_throttle_enabled |
+Notes |
+
|---|
+
+
+Tags |
+vdev,
+ZIO_scheduler |
+
+When to change |
+Testing ZIO block allocation algorithms |
+
+Data Type |
+boolean |
+
+Range |
+0=do not throttle ZIO block allocations,
+1=throttle ZIO block allocations |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zio_requeue_io_start_cut_in_line
+zio_requeue_io_start_cut_in_line controls prioritization of a
+re-queued ZFS I/O (ZIO) in the ZIO pipeline by the ZIO taskq.
+
+
+zio_requeue_io_start_cut_in_line |
+Notes |
+
|---|
+
+
+Tags |
+Z
+IO_scheduler |
+
+When to change |
+Do not change |
+
+Data Type |
+boolean |
+
+Range |
+0=don’t prioritize re-queued
+I/Os, 1=prioritize re-queued
+I/Os |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+all |
+
+
+
+
+
+zio_taskq_batch_pct
+zio_taskq_batch_pct sets the number of I/O worker threads as a
+percentage of online CPUs. These workers threads are responsible for IO
+work such as compression and checksum calculations.
+Each block is handled by one worker thread, so maximum overall worker
+thread throughput is function of the number of concurrent blocks being
+processed, the number of worker threads, and the algorithms used. The
+default value of 75% is chosen to avoid using all CPUs which can result
+in latency issues and inconsistent application performance, especially
+when high compression is enabled.
+The taskq batch processes are:
+
+
+taskq |
+process name |
+Notes |
+
|---|
+
+
+Write issue |
+z_wr_iss[_#] |
+Can be CPU intensive, runs at lower
+priority than other taskqs |
+
+
+
+Other taskqs exist, but most have fixed numbers of instances and
+therefore require recompiling the kernel module to adjust.
+
+
+zio_taskq_batch_pct |
+Notes |
+
|---|
+
+
+Tags |
+taskq,
+ZIO_scheduler |
+
+When to change |
+To tune parallelism in multiprocessor systems |
+
+Data Type |
+int |
+
+Units |
+percent of number of CPUs |
+
+Range |
+1 to 100, fractional number of CPUs are
+rounded down |
+
+Default |
+75 |
+
+Change |
+Prior to zfs module load |
+
+Verification |
+The number of taskqs for each batch group can
+be observed using ps and counting the
+threads |
+
+Versions Affected |
+TBD |
+
+
+
+
+
+zvol_inhibit_dev
+zvol_inhibit_dev controls the creation of volume device nodes upon
+pool import.
+
+
+zvol_inhibit_dev |
+Notes |
+
|---|
+
+
+Tags |
+import, volume |
+
+When to change |
+Inhibiting can slightly improve startup time on
+systems with a very large number of volumes |
+
+Data Type |
+boolean |
+
+Range |
+0=create volume device nodes, 1=do not create
+volume device nodes |
+
+Default |
+0 |
+
+Change |
+Dynamic, takes effect per-pool when the pool is
+imported |
+
+Versions Affected |
+v0.6.0 and later |
+
+
+
+
+
+zvol_major
+zvol_major is the default major number for volume devices.
+
+
+zvol_major |
+Notes |
+
|---|
+
+
+Tags |
+volume |
+
+When to change |
+Do not change |
+
+Data Type |
+uint |
+
+Default |
+230 |
+
+Change |
+Dynamic, takes effect per-pool when the pool is
+imported or volumes are created |
+
+Versions Affected |
+all |
+
+
+
+
+
+zvol_max_discard_blocks
+Discard (aka ATA TRIM or SCSI UNMAP) operations done on volumes are done
+in batches zvol_max_discard_blocks blocks. The block size is
+determined by the volblocksize property of a volume.
+Some applications, such as mkfs, discard the whole volume at once
+using the maximum possible discard size. As a result, many gigabytes of
+discard requests are not uncommon. Unfortunately, if a large amount of
+data is already allocated in the volume, ZFS can be quite slow to
+process discard requests. This is especially true if the volblocksize is
+small (eg default=8KB). As a result, very large discard requests can
+take a very long time (perhaps minutes under heavy load) to complete.
+This can cause a number of problems, most notably if the volume is
+accessed remotely (eg via iSCSI), in which case the client has a high
+probability of timing out on the request.
+Limiting the zvol_max_discard_blocks can decrease the amount of
+discard workload request by setting the discard_max_bytes and
+discard_max_hw_bytes for the volume’s block device in SysFS. This
+value is readable by volume device consumers.
+
+
+zvol_max_discard_blocks |
+Notes |
+
|---|
+
+
+Tags |
+discard,
+volume |
+
+When to change |
+if volume discard activity severely
+impacts other workloads |
+
+Data Type |
+ulong |
+
+Units |
+number of blocks of size volblocksize |
+
+Range |
+0 to ULONG_MAX |
+
+Default |
+16,384 |
+
+Change |
+Dynamic, takes effect per-pool when the
+pool is imported or volumes are created |
+
+Verification |
+Observe value of
+/sys/block/
+VOLUME_INSTANCE/queue/discard_max_bytes |
+
+Versions Affected |
+v0.6.0 and later |
+
+
+
+
+
+zvol_prefetch_bytes
+When importing a pool with volumes or adding a volume to a pool,
+zvol_prefetch_bytes are prefetch from the start and end of the
+volume. Prefetching these regions of the volume is desirable because
+they are likely to be accessed immediately by blkid(8) or by the
+kernel scanning for a partition table.
+
+
+zvol_prefetch_bytes |
+Notes |
+
|---|
+
+
+Tags |
+prefetch, volume |
+
+When to change |
+TBD |
+
+Data Type |
+uint |
+
+Units |
+bytes |
+
+Range |
+0 to UINT_MAX |
+
+Default |
+131,072 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.5 and later |
+
+
+
+
+
+zvol_request_sync
+When processing I/O requests for a volume submit them synchronously.
+This effectively limits the queue depth to 1 for each I/O submitter.
+When set to 0 requests are handled asynchronously by the “zvol” thread
+pool.
+See also zvol_threads
+
+
+zvol_request_sync |
+Notes |
+
|---|
+
+
+Tags |
+volume |
+
+When to change |
+Testing concurrent volume requests |
+
+Data Type |
+boolean |
+
+Range |
+0=do concurrent (async) volume requests, 1=do
+sync volume requests |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.2 and later |
+
+
+
+
+
+zvol_threads
+zvol_threads controls the maximum number of threads handling concurrent
+volume I/O requests.
+The default of 32 threads behaves similarly to a disk with a 32-entry
+command queue. The actual number of threads required can vary widely by
+workload and available CPUs. If lock analysis shows high contention in
+the zvol taskq threads, then reducing the number of zvol_threads or
+workload queue depth can improve overall throughput.
+See also zvol_request_sync
+
+
+zvol_threads |
+Notes |
+
|---|
+
+
+Tags |
+volume |
+
+When to change |
+Matching the number of concurrent volume
+requests with workload requirements can improve
+concurrency |
+
+Data Type |
+uint |
+
+Units |
+threads |
+
+Range |
+1 to UINT_MAX |
+
+Default |
+32 |
+
+Change |
+Dynamic, takes effect per-volume when the pool
+is imported or volumes are created |
+
+Verification |
+iostat using avgqu-sz or aqu-sz
+results
|
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zvol_volmode
+zvol_volmode defines volume block devices behaviour when the
+volmode property is set to default
+Note: to maintain compatibility with ZFS on BSD, “geom” is synonymous
+with “full”
+
+
+value |
+volmode |
+Description |
+
|---|
+
+
+1 |
+full |
+legacy fully functional behaviour (default) |
+
+2 |
+dev |
+hide partitions on volume block devices |
+
+3 |
+none |
+not exposing volumes outside ZFS |
+
+
+
+
+
+zvol_volmode |
+Notes |
+
|---|
+
+
+Tags |
+volume |
+
+When to change |
+TBD |
+
+Data Type |
+enum |
+
+Range |
+1, 2, or 3 |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_qat_disable
+zfs_qat_disable controls the Intel QuickAssist Technology (QAT)
+driver providing hardware acceleration for gzip compression. When the
+QAT hardware is present and qat driver available, the default behaviour
+is to enable QAT.
+
+
+zfs_qat_disable |
+Notes |
+
|---|
+
+
+Tags |
+compression, QAT |
+
+When to change |
+Testing QAT functionality |
+
+Data Type |
+boolean |
+
+Range |
+0=use QAT acceleration if available, 1=do not
+use QAT acceleration |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7, renamed to
+zfs_qat_
+compress_disable
+in v0.8 |
+
+
+
+
+
+zfs_qat_checksum_disable
+zfs_qat_checksum_disable controls the Intel QuickAssist Technology
+(QAT) driver providing hardware acceleration for checksums. When the QAT
+hardware is present and qat driver available, the default behaviour is
+to enable QAT.
+
+
+zfs_qat_checksum_disable |
+Notes |
+
|---|
+
+
+Tags |
+checksum, QAT |
+
+When to change |
+Testing QAT functionality |
+
+Data Type |
+boolean |
+
+Range |
+0=use QAT acceleration if available,
+1=do not use QAT acceleration |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.8.0 |
+
+
+
+
+
+zfs_qat_compress_disable
+zfs_qat_compress_disable controls the Intel QuickAssist Technology
+(QAT) driver providing hardware acceleration for gzip compression. When
+the QAT hardware is present and qat driver available, the default
+behaviour is to enable QAT.
+
+
+zfs_qat_compress_disable |
+Notes |
+
|---|
+
+
+Tags |
+compression,
+QAT |
+
+When to change |
+Testing QAT functionality |
+
+Data Type |
+boolean |
+
+Range |
+0=use QAT acceleration if available,
+1=do not use QAT acceleration |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.8.0 |
+
+
+
+
+
+zfs_qat_encrypt_disable
+zfs_qat_encrypt_disable controls the Intel QuickAssist Technology
+(QAT) driver providing hardware acceleration for encryption. When the
+QAT hardware is present and qat driver available, the default behaviour
+is to enable QAT.
+
+
+zfs_qat_encrypt_disable |
+Notes |
+
|---|
+
+
+Tags |
+encryption,
+QAT |
+
+When to change |
+Testing QAT functionality |
+
+Data Type |
+boolean |
+
+Range |
+0=use QAT acceleration if available, 1=do
+not use QAT acceleration |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.8.0 |
+
+
+
+
+
+dbuf_cache_hiwater_pct
+The dbuf_cache_hiwater_pct and
+dbuf_cache_lowater_pct define the
+operating range for dbuf cache evict thread. The hiwater and lowater are
+percentages of the dbuf_cache_max_bytes
+value. When the dbuf cache grows above ((100% +
+dbuf_cache_hiwater_pct) *
+dbuf_cache_max_bytes) then the dbuf cache
+thread begins evicting. When the dbug cache falls below ((100% -
+dbuf_cache_lowater_pct) *
+dbuf_cache_max_bytes) then the dbuf cache
+thread stops evicting.
+
+
+dbuf_cache_hiwater_pct |
+Notes |
+
|---|
+
+
+Tags |
+dbuf_cache |
+
+When to change |
+Testing dbuf cache algorithms |
+
+Data Type |
+uint |
+
+Units |
+percent |
+
+Range |
+0 to UINT_MAX |
+
+Default |
+10 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+dbuf_cache_lowater_pct
+The dbuf_cache_hiwater_pct and dbuf_cache_lowater_pct define the
+operating range for dbuf cache evict thread. The hiwater and lowater are
+percentages of the dbuf_cache_max_bytes
+value. When the dbuf cache grows above ((100% +
+dbuf_cache_hiwater_pct) *
+dbuf_cache_max_bytes) then the dbuf cache
+thread begins evicting. When the dbug cache falls below ((100% -
+dbuf_cache_lowater_pct) *
+dbuf_cache_max_bytes) then the dbuf cache
+thread stops evicting.
+
+
+dbuf_cache_lowater_pct |
+Notes |
+
|---|
+
+
+Tags |
+dbuf_cache |
+
+When to change |
+Testing dbuf cache algorithms |
+
+Data Type |
+uint |
+
+Units |
+percent |
+
+Range |
+0 to UINT_MAX |
+
+Default |
+10 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+dbuf_cache_max_bytes
+The dbuf cache maintains a list of dbufs that are not currently held but
+have been recently released. These dbufs are not eligible for ARC
+eviction until they are aged out of the dbuf cache. Dbufs are added to
+the dbuf cache once the last hold is released. If a dbuf is later
+accessed and still exists in the dbuf cache, then it will be removed
+from the cache and later re-added to the head of the cache. Dbufs that
+are aged out of the cache will be immediately destroyed and become
+eligible for ARC eviction.
+The size of the dbuf cache is set by dbuf_cache_max_bytes. The
+actual size is dynamically adjusted to the minimum of current ARC target
+size (c) >> dbuf_cache_max_shift and the
+default dbuf_cache_max_bytes
+
+
+dbuf_cache_max_bytes |
+Notes |
+
|---|
+
+
+Tags |
+dbuf_cache |
+
+When to change |
+Testing dbuf cache algorithms |
+
+Data Type |
+ulong |
+
+Units |
+bytes |
+
+Range |
+16,777,216 to ULONG_MAX |
+
+Default |
+104,857,600 (100 MiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+dbuf_cache_max_shift
+The dbuf_cache_max_bytes minimum is the
+lesser of dbuf_cache_max_bytes and the
+current ARC target size (c) >> dbuf_cache_max_shift
+
+
+dbuf_cache_max_shift |
+Notes |
+
|---|
+
+
+Tags |
+dbuf_cache |
+
+When to change |
+Testing dbuf cache algorithms |
+
+Data Type |
+int |
+
+Units |
+shift |
+
+Range |
+1 to 63 |
+
+Default |
+5 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+dmu_object_alloc_chunk_shift
+Each of the concurrent object allocators grabs
+2^dmu_object_alloc_chunk_shift dnode slots at a time. The default is
+to grab 128 slots, or 4 blocks worth. This default value was
+experimentally determined to be the lowest value that eliminates the
+measurable effect of lock contention in the DMU object allocation code
+path.
+
+
+dmu_object_alloc_chunk_shift |
+Notes |
+
|---|
+
+
+Tags |
+allocation,
+DMU |
+
+When to change |
+If the workload creates many files
+concurrently on a system with many
+CPUs, then increasing
+dmu_object_alloc_chunk_shift can
+decrease lock contention |
+
+Data Type |
+int |
+
+Units |
+shift |
+
+Range |
+7 to 9 |
+
+Default |
+7 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+
+zfs_abd_scatter_enabled
+zfs_abd_scatter_enabled controls the ARC Buffer Data (ABD)
+scatter/gather feature.
+When disabled, the legacy behaviour is selected using linear buffers.
+For linear buffers, all the data in the ABD is stored in one contiguous
+buffer in memory (from a zio_[data_]buf_* kmem cache).
+When enabled (default), the data in the ABD is split into equal-sized
+chunks (from the abd_chunk_cache kmem_cache), with pointers to the
+chunks recorded in an array at the end of the ABD structure. This allows
+more efficient memory allocation for buffers, especially when large
+recordsizes are used.
+
+
+zfs_abd_scatter_enabled |
+Notes |
+
|---|
+
+
+Tags |
+ABD, memory |
+
+When to change |
+Testing ABD |
+
+Data Type |
+boolean |
+
+Range |
+0=use linear allocation only, 1=allow
+scatter/gather |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Verification |
+ABD statistics are observable in
+/proc/spl/kstat/zfs/abdstats. Slab
+allocations are observable in
+/proc/slabinfo |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_abd_scatter_max_order
+zfs_abd_scatter_max_order sets the maximum order for physical page
+allocation when ABD is enabled (see
+zfs_abd_scatter_enabled)
+See also Buddy Memory Allocation in the Linux kernel documentation.
+
+
+zfs_abd_scatter_max_order |
+Notes |
+
|---|
+
+
+Tags |
+ABD, memory |
+
+When to change |
+Testing ABD features |
+
+Data Type |
+int |
+
+Units |
+orders |
+
+Range |
+1 to 10 (upper limit is
+hardware-dependent) |
+
+Default |
+10 |
+
+Change |
+Dynamic |
+
+Verification |
+ABD statistics are observable in
+/proc/spl/kstat/zfs/abdstats |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_compressed_arc_enabled
+When compression is enabled for a dataset, later reads of the data can
+store the blocks in ARC in their on-disk, compressed state. This can
+increse the effective size of the ARC, as counted in blocks, and thus
+improve the ARC hit ratio.
+
+
+zfs_compressed_arc_enabled |
+Notes |
+
|---|
+
+
+Tags |
+ABD,
+compression |
+
+When to change |
+Testing ARC compression feature |
+
+Data Type |
+boolean |
+
+Range |
+0=compressed ARC disabled (legacy
+behaviour), 1=compress ARC data |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Verification |
+raw ARC statistics are observable in
+/proc/spl/kstat/zfs/arcstats and
+ARC hit ratios can be observed using
+arcstat |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_key_max_salt_uses
+For encrypted datasets, the salt is regenerated every
+zfs_key_max_salt_uses blocks. This automatic regeneration reduces
+the probability of collisions due to the Birthday problem. When set to
+the default (400,000,000) the probability of collision is approximately
+1 in 1 trillion.
+
+
+zfs_key_max_salt_uses |
+Notes |
+
|---|
+
+
+Tags |
+encryption |
+
+When to change |
+Testing encryption features |
+
+Data Type |
+ulong |
+
+Units |
+blocks encrypted |
+
+Range |
+1 to ULONG_MAX |
+
+Default |
+400,000,000 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.8.0 and later |
+
+
+
+
+
+zfs_object_mutex_size
+zfs_object_mutex_size facilitates resizing the the per-dataset znode
+mutex array for testing deadlocks therein.
+
+
+zfs_object_mutex_size |
+Notes |
+
|---|
+
+
+Tags |
+debug |
+
+When to change |
+Testing znode mutex array deadlocks |
+
+Data Type |
+uint |
+
+Units |
+orders |
+
+Range |
+1 to UINT_MAX |
+
+Default |
+64 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 and later |
+
+
+
+
+
+zfs_scan_strict_mem_lim
+When scrubbing or resilvering, by default, ZFS checks to ensure it is
+not over the hard memory limit before each txg commit. If finer-grained
+control of this is needed zfs_scan_strict_mem_lim can be set to 1 to
+enable checking before scanning each block.
+
+
+zfs_scan_strict_mem_lim |
+Notes |
+
|---|
+
+
+Tags |
+memory,
+resilver,
+scrub |
+
+When to change |
+Do not change |
+
+Data Type |
+boolean |
+
+Range |
+0=normal scan behaviour, 1=check hard
+memory limit strictly during scan |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.8.0 |
+
+
+
+
+
+zfs_send_queue_length
+zfs_send_queue_length is the maximum number of bytes allowed in the
+zfs send queue.
+
+
+zfs_send_queue_length |
+Notes |
+
|---|
+
+
+Tags |
+send |
+
+When to change |
+When using the largest recordsize or
+volblocksize (16 MiB), increasing can
+improve send efficiency |
+
+Data Type |
+int |
+
+Units |
+bytes |
+
+Range |
+Must be at least twice the maximum
+recordsize or volblocksize in use |
+
+Default |
+16,777,216 bytes (16 MiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.8.1 |
+
+
+
+
+
+zfs_recv_queue_length
+zfs_recv_queue_length is the maximum number of bytes allowed in the
+zfs receive queue.
+
+
+zfs_recv_queue_length |
+Notes |
+
|---|
+
+
+Tags |
+receive |
+
+When to change |
+When using the largest recordsize or
+volblocksize (16 MiB), increasing can
+improve receive efficiency |
+
+Data Type |
+int |
+
+Units |
+bytes |
+
+Range |
+Must be at least twice the maximum
+recordsize or volblocksize in use |
+
+Default |
+16,777,216 bytes (16 MiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.8.1 |
+
+
+
+
+
+zfs_arc_min_prefetch_lifespan
+arc_min_prefetch_lifespan is the minimum time for a prefetched block
+to remain in ARC before it is eligible for eviction.
+
+
+zfs_arc_min_prefetch_lifespan |
+Notes |
+
|---|
+
+
+Tags |
+ARC |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+clock ticks |
+
+Range |
+0 = use default value |
+
+Default |
+1 second (as expressed in clock ticks) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 |
+
+
+
+
+
+zfs_scan_ignore_errors
+zfs_scan_ignore_errors allows errors discovered during scrub or
+resilver to be ignored. This can be tuned as a workaround to remove the
+dirty time list (DTL) when completing a pool scan. It is intended to be
+used during pool repair or recovery to prevent resilvering when the pool
+is imported.
+
+
+zfs_scan_ignore_errors |
+Notes |
+
|---|
+
+
+Tags |
+resilver |
+
+When to change |
+See description above |
+
+Data Type |
+boolean |
+
+Range |
+0 = do not ignore errors, 1 = ignore
+errors during pool scrub or resilver |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.8.1 |
+
+
+
+
+
+zfs_top_maxinflight
+zfs_top_maxinflight is used to limit the maximum number of I/Os
+queued to top-level vdevs during scrub or resilver operations. The
+actual top-level vdev limit is calculated by multiplying the number of
+child vdevs by zfs_top_maxinflight This limit is an additional cap
+over and above the scan limits
+
+
+zfs_top_maxinflight |
+Notes |
+
|---|
+
+
+Tags |
+resilver, scrub,
+ZIO_scheduler |
+
+When to change |
+for modern ZFS versions, the ZIO scheduler
+limits usually take precedence |
+
+Data Type |
+int |
+
+Units |
+I/O operations |
+
+Range |
+1 to MAX_INT |
+
+Default |
+32 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.0 |
+
+
+
+
+
+zfs_resilver_delay
+zfs_resilver_delay sets a time-based delay for resilver I/Os. This
+delay is in addition to the ZIO scheduler’s treatment of scrub
+workloads. See also zfs_scan_idle
+
+
+zfs_resilver_delay |
+Notes |
+
|---|
+
+
+Tags |
+resilver,
+ZIO_scheduler |
+
+When to change |
+increasing can reduce impact of resilver
+workload on dynamic workloads |
+
+Data Type |
+int |
+
+Units |
+clock ticks |
+
+Range |
+0 to MAX_INT |
+
+Default |
+2 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.0 |
+
+
+
+
+
+zfs_scrub_delay
+zfs_scrub_delay sets a time-based delay for scrub I/Os. This delay
+is in addition to the ZIO scheduler’s treatment of scrub workloads. See
+also zfs_scan_idle
+
+
+zfs_scrub_delay |
+Notes |
+
|---|
+
+
+Tags |
+scrub,
+ZIO_scheduler |
+
+When to change |
+increasing can reduce impact of scrub workload
+on dynamic workloads |
+
+Data Type |
+int |
+
+Units |
+clock ticks |
+
+Range |
+0 to MAX_INT |
+
+Default |
+4 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.0 |
+
+
+
+
+
+zfs_scan_idle
+When a non-scan I/O has occurred in the past zfs_scan_idle clock
+ticks, then zfs_resilver_delay or
+zfs_scrub_delay are enabled.
+
+
+zfs_scan_idle |
+Notes |
+
|---|
+
+
+Tags |
+resilver, scrub,
+ZIO_scheduler |
+
+When to change |
+as part of a resilver/scrub tuning effort |
+
+Data Type |
+int |
+
+Units |
+clock ticks |
+
+Range |
+0 to MAX_INT |
+
+Default |
+50 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.0 |
+
+
+
+
+
+icp_aes_impl
+By default, ZFS will choose the highest performance, hardware-optimized
+implementation of the AES encryption algorithm. The icp_aes_impl
+tunable overrides this automatic choice.
+Note: icp_aes_impl is set in the icp kernel module, not the
+zfs kernel module.
+To observe the available options
+cat /sys/module/icp/parameters/icp_aes_impl The default option is
+shown in brackets ‘[]’
+
+
+icp_aes_impl |
+Notes |
+
|---|
+
+
+Tags |
+encryption |
+
+Kernel module |
+icp |
+
+When to change |
+debugging ZFS encryption on hardware |
+
+Data Type |
+string |
+
+Range |
+varies by hardware |
+
+Default |
+automatic, depends on the hardware |
+
+Change |
+dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+icp_gcm_impl
+By default, ZFS will choose the highest performance, hardware-optimized
+implementation of the GCM encryption algorithm. The icp_gcm_impl
+tunable overrides this automatic choice.
+Note: icp_gcm_impl is set in the icp kernel module, not the
+zfs kernel module.
+To observe the available options
+cat /sys/module/icp/parameters/icp_gcm_impl The default option is
+shown in brackets ‘[]’
+
+
+icp_gcm_impl |
+Notes |
+
|---|
+
+
+Tags |
+encryption |
+
+Kernel module |
+icp |
+
+When to change |
+debugging ZFS encryption on hardware |
+
+Data Type |
+string |
+
+Range |
+varies by hardware |
+
+Default |
+automatic, depends on the hardware |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_abd_scatter_min_size
+zfs_abd_scatter_min_size changes the ARC buffer data (ABD)
+allocator’s threshold for using linear or page-based scatter buffers.
+Allocations smaller than zfs_abd_scatter_min_size use linear ABDs.
+Scatter ABD’s use at least one page each, so sub-page allocations waste
+some space when allocated as scatter allocations. For example, 2KB
+scatter allocation wastes half of each page. Using linear ABD’s for
+small allocations results in slabs containing many allocations. This can
+improve memory efficiency, at the expense of more work for ARC evictions
+attempting to free pages, because all the buffers on one slab need to be
+freed in order to free the slab and its underlying pages.
+Typically, 512B and 1KB kmem caches have 16 buffers per slab, so it’s
+possible for them to actually waste more memory than scatter
+allocations:
+
+Spill blocks are typically 512B and are heavily used on systems running
+selinux with the default dnode size and the xattr=sa property set.
+By default, linear allocations for 512B and 1KB, and scatter allocations
+for larger (>= 1.5KB) allocation requests.
+
+
+zfs_abd_scatter_min_size |
+Notes |
+
|---|
+
+
+Tags |
+ARC |
+
+When to change |
+debugging memory allocation, especially
+for large pages |
+
+Data Type |
+int |
+
+Units |
+bytes |
+
+Range |
+0 to MAX_INT |
+
+Default |
+1536 (512B and 1KB allocations will be
+linear) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_unlink_suspend_progress
+zfs_unlink_suspend_progress changes the policy for removing pending
+unlinks. When enabled, files will not be asynchronously removed from the
+list of pending unlinks and the space they consume will be leaked. Once
+this option has been disabled and the dataset is remounted, the pending
+unlinks will be processed and the freed space returned to the pool.
+
+
+zfs_unlink_suspend_progress |
+Notes |
+
|---|
+
+
+Tags |
+ |
+
+When to change |
+used by the ZFS test suite (ZTS) to
+facilitate testing |
+
+Data Type |
+boolean |
+
+Range |
+0 = use async unlink removal, 1 = do
+not async unlink thus leaking space |
+
+Default |
+0 |
+
+Change |
+prior to dataset mount |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+spa_load_verify_shift
+spa_load_verify_shift sets the fraction of ARC that can be used by
+inflight I/Os when verifying the pool during import. This value is a
+“shift” representing the fraction of ARC target size
+(grep -w c /proc/spl/kstat/zfs/arcstats). The ARC target size is
+shifted to the right. Thus a value of ‘2’ results in the fraction = 1/4,
+while a value of ‘4’ results in the fraction = 1/8.
+For large memory machines, pool import can consume large amounts of ARC:
+much larger than the value of maxinflight. This can result in
+spa_load_verify_maxinflight having a
+value of 0 causing the system to hang. Setting spa_load_verify_shift
+can reduce this limit and allow importing without hanging.
+
+
+spa_load_verify_shift |
+Notes |
+
|---|
+
+
+Tags |
+import, ARC,
+SPA |
+
+When to change |
+troubleshooting pool import on large memory
+machines |
+
+Data Type |
+int |
+
+Units |
+shift |
+
+Range |
+1 to MAX_INT |
+
+Default |
+4 |
+
+Change |
+prior to importing a pool |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+spa_load_print_vdev_tree
+spa_load_print_vdev_tree enables printing of the attempted pool
+import’s vdev tree to kernel message to the ZFS debug message log
+/proc/spl/kstat/zfs/dbgmsg Both the provided vdev tree and MOS vdev
+tree are printed, which can be useful for debugging problems with the
+zpool cachefile
+
+
+spa_load_print_vdev_tree |
+Notes |
+
|---|
+
+
+Tags |
+import, SPA |
+
+When to change |
+troubleshooting pool import failures |
+
+Data Type |
+boolean |
+
+Range |
+0 = do not print pool configuration in
+logs, 1 = print pool configuration in
+logs |
+
+Default |
+0 |
+
+Change |
+prior to pool import |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_max_missing_tvds
+When importing a pool in readonly mode
+(zpool import -o readonly=on ...) then up to
+zfs_max_missing_tvds top-level vdevs can be missing, but the import
+can attempt to progress.
+Note: This is strictly intended for advanced pool recovery cases since
+missing data is almost inevitable. Pools with missing devices can only
+be imported read-only for safety reasons, and the pool’s failmode
+property is automatically set to continue
+The expected use case is to recover pool data immediately after
+accidentally adding a non-protected vdev to a protected pool.
+
+With 1 missing top-level vdev, ZFS should be able to import the pool
+and mount all datasets. User data that was not modified after the
+missing device has been added should be recoverable. Thus snapshots
+created prior to the addition of that device should be completely
+intact.
With 2 missing top-level vdevs, some datasets may fail to mount since
+there are dataset statistics that are stored as regular metadata.
+Some data might be recoverable if those vdevs were added recently.
With 3 or more top-level missing vdevs, the pool is severely damaged
+and MOS entries may be missing entirely. Chances of data recovery are
+very low. Note that there are also risks of performing an inadvertent
+rewind as we might be missing all the vdevs with the latest
+uberblocks.
+
+
+zfs_max_missing_tvds |
+Notes |
+
|---|
+
+
+Tags |
+import |
+
+When to change |
+troubleshooting pools with missing devices |
+
+Data Type |
+int |
+
+Units |
+missing top-level vdevs |
+
+Range |
+0 to MAX_INT |
+
+Default |
+0 |
+
+Change |
+prior to pool import |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+
+
+dbuf_cache_shift
+dbuf_cache_shift sets the size of the dbuf cache as a fraction of
+ARC target size. This is an alternate method for setting dbuf cache size
+than dbuf_cache_max_bytes.
+dbuf_cache_max_bytes overrides
+dbuf_cache_shift
+This value is a “shift” representing the fraction of ARC target size
+(grep -w c /proc/spl/kstat/zfs/arcstats). The ARC target size is
+shifted to the right. Thus a value of ‘2’ results in the fraction = 1/4,
+while a value of ‘5’ results in the fraction = 1/32.
+Performance tuning of dbuf cache can be monitored using:
+
+dbufstat command
node_exporter ZFS
+module for prometheus environments
telegraf ZFS plugin for
+general-purpose metric collection
/proc/spl/kstat/zfs/dbufstats kstat
+
+
+dbuf_cache_shift |
+Notes |
+
|---|
+
+
+Tags |
+ARC, dbuf_cache |
+
+When to change |
+to improve performance of read-intensive
+channel programs |
+
+Data Type |
+int |
+
+Units |
+shift |
+
+Range |
+5 to MAX_INT |
+
+Default |
+5 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+dbuf_cache_max_bytes
+dbuf_cache_max_bytes sets the size of the dbuf cache in bytes. This
+is an alternate method for setting dbuf cache size than
+dbuf_cache_shift
+Performance tuning of dbuf cache can be monitored using:
+
+dbufstat command
node_exporter ZFS
+module for prometheus environments
telegraf ZFS plugin for
+general-purpose metric collection
/proc/spl/kstat/zfs/dbufstats kstat
+
+
+dbuf_cache_max_bytes |
+Notes |
+
|---|
+
+
+Tags |
+ARC, dbuf_cache |
+
+When to change |
+ |
+
+Data Type |
+int |
+
+Units |
+bytes |
+
+Range |
+0 = use
+dbuf_cache_shift to
+ARC c_max |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+
+zfs_vdev_default_ms_count
+When adding a top-level vdev, zfs_vdev_default_ms_count is the
+target number of metaslabs.
+
+
+zfs_vdev_default_ms_count |
+Notes |
+
|---|
+
+
+Tags |
+allocation |
+
+When to change |
+for development testing purposes only |
+
+Data Type |
+int |
+
+Range |
+16 to MAX_INT |
+
+Default |
+200 |
+
+Change |
+prior to creating a pool or adding a
+top-level vdev |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+vdev_removal_max_span
+During top-level vdev removal, chunks of data are copied from the vdev
+which may include free space in order to trade bandwidth for IOPS.
+vdev_removal_max_span sets the maximum span of free space included
+as unnecessary data in a chunk of copied data.
+
+
+vdev_removal_max_span |
+Notes |
+
|---|
+
+
+Tags |
+vdev_removal |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+bytes |
+
+Range |
+0 to MAX_INT |
+
+Default |
+32,768 (32 MiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_removal_ignore_errors
+When removing a device, zfs_removal_ignore_errors controls the
+process for handling hard I/O errors. When set, if a device encounters a
+hard IO error during the removal process the removal will not be
+cancelled. This can result in a normally recoverable block becoming
+permanently damaged and is not recommended. This should only be used as
+a last resort when the pool cannot be returned to a healthy state prior
+to removing the device.
+
+
+zfs_removal_ignore_errors |
+Notes |
+
|---|
+
+
+Tags |
+vdev_removal |
+
+When to change |
+See description for caveat |
+
+Data Type |
+boolean |
+
+Range |
+during device removal: 0 = hard errors
+are not ignored, 1 = hard errors are
+ignored |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_removal_suspend_progress
+zfs_removal_suspend_progress is used during automated testing of the
+ZFS code to incease test coverage.
+
+
+zfs_removal_suspend_progress |
+Notes |
+
|---|
+
+
+Tags |
+vdev_removal |
+
+When to change |
+do not change |
+
+Data Type |
+boolean |
+
+Range |
+0 = do not suspend during vdev removal |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_condense_indirect_commit_entry_delay_ms
+During vdev removal, the vdev indirection layer sleeps for
+zfs_condense_indirect_commit_entry_delay_ms milliseconds during
+mapping generation. This parameter is used during automated testing of
+the ZFS code to improve test coverage.
+
+
+zfs_condens
+e_indirect_commit_entry_delay_ms |
+Notes |
+
|---|
+
+
+Tags |
+vdev_removal |
+
+When to change |
+do not change |
+
+Data Type |
+int |
+
+Units |
+milliseconds |
+
+Range |
+0 to MAX_INT |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_condense_indirect_vdevs_enable
+During vdev removal, condensing process is an attempt to save memory by
+removing obsolete mappings. zfs_condense_indirect_vdevs_enable
+enables condensing indirect vdev mappings. When set, ZFS attempts to
+condense indirect vdev mappings if the mapping uses more than
+zfs_condense_min_mapping_bytes
+bytes of memory and if the obsolete space map object uses more than
+zfs_condense_max_obsolete_bytes
+bytes on disk.
+
+
+zf
+s_condense_indirect_vdevs_enable |
+Notes |
+
|---|
+
+
+Tags |
+vdev_removal |
+
+When to change |
+TBD |
+
+Data Type |
+boolean |
+
+Range |
+0 = do not save memory, 1 = save
+memory by condensing obsolete
+mapping after vdev removal |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_condense_max_obsolete_bytes
+After vdev removal, zfs_condense_max_obsolete_bytes sets the limit
+for beginning the condensing process. Condensing begins if the obsolete
+space map takes up more than zfs_condense_max_obsolete_bytes of
+space on disk (logically). The default of 1 GiB is small enough relative
+to a typical pool that the space consumed by the obsolete space map is
+minimal.
+See also
+zfs_condense_indirect_vdevs_enable
+
+
+zfs_condense_max_obsolete_bytes |
+Notes |
+
|---|
+
+
+Tags |
+vdev_removal |
+
+When to change |
+no not change |
+
+Data Type |
+ulong |
+
+Units |
+bytes |
+
+Range |
+0 to MAX_ULONG |
+
+Default |
+1,073,741,824 (1 GiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_condense_min_mapping_bytes
+After vdev removal, zfs_condense_min_mapping_bytes is the lower
+limit for determining when to condense the in-memory obsolete space map.
+The condensing process will not continue unless a minimum of
+zfs_condense_min_mapping_bytes of memory can be freed.
+See also
+zfs_condense_indirect_vdevs_enable
+
+
+zfs_condense_min_mapping_bytes |
+Notes |
+
|---|
+
+
+Tags |
+vdev_removal |
+
+When to change |
+do not change |
+
+Data Type |
+ulong |
+
+Units |
+bytes |
+
+Range |
+0 to MAX_ULONG |
+
+Default |
+128 KiB |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_vdev_initializing_max_active
+zfs_vdev_initializing_max_active sets the maximum initializing I/Os
+active to each device.
+
+
+
+zfs_vdev_initializing_min_active
+zfs_vdev_initializing_min_active sets the minimum initializing I/Os
+active to each device.
+
+
+
+zfs_vdev_removal_max_active
+zfs_vdev_removal_max_active sets the maximum top-level vdev removal
+I/Os active to each device.
+
+
+
+zfs_vdev_removal_min_active
+zfs_vdev_removal_min_active sets the minimum top-level vdev removal
+I/Os active to each device.
+
+
+
+zfs_vdev_trim_max_active
+zfs_vdev_trim_max_active sets the maximum trim I/Os active to each
+device.
+
+
+
+zfs_vdev_trim_min_active
+zfs_vdev_trim_min_active sets the minimum trim I/Os active to each
+device.
+
+
+
+zfs_initialize_value
+When initializing a vdev, ZFS writes patterns of
+zfs_initialize_value bytes to the device.
+
+
+zfs_initialize_value |
+Notes |
+
|---|
+
+
+Tags |
+vdev_initialize |
+
+When to change |
+when debugging initialization code |
+
+Data Type |
+uint32 or uint64 |
+
+Default |
+0xdeadbeef for 32-bit systems,
+0xdeadbeefdeadbeee for 64-bit systems |
+
+Change |
+prior to running zpool initialize |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_lua_max_instrlimit
+zfs_lua_max_instrlimit limits the maximum time for a ZFS channel
+program to run.
+
+
+zfs_lua_max_instrlimit |
+Notes |
+
|---|
+
+
+Tags |
+channel_programs |
+
+When to change |
+to enforce a CPU usage limit on ZFS
+channel programs |
+
+Data Type |
+ulong |
+
+Units |
+LUA instructions |
+
+Range |
+0 to MAX_ULONG |
+
+Default |
+100,000,000 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_lua_max_memlimit
+‘zfs_lua_max_memlimit’ is the maximum memory limit for a ZFS channel
+program.
+
+
+zfs_lua_max_memlimit |
+Notes |
+
|---|
+
+
+Tags |
+channel_programs |
+
+When to change |
+ |
+
+Data Type |
+ulong |
+
+Units |
+bytes |
+
+Range |
+0 to MAX_ULONG |
+
+Default |
+104,857,600 (100 MiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_max_dataset_nesting
+zfs_max_dataset_nesting limits the depth of nested datasets. Deeply
+nested datasets can overflow the stack. The maximum stack depth depends
+on kernel compilation options, so it is impractical to predict the
+possible limits. For kernels compiled with small stack sizes,
+zfs_max_dataset_nesting may require changes.
+
+
+zfs_max_dataset_nesting |
+Notes |
+
|---|
+
+
+Tags |
+dataset |
+
+When to change |
+can be tuned temporarily to fix existing
+datasets that exceed the predefined limit |
+
+Data Type |
+int |
+
+Units |
+datasets |
+
+Range |
+0 to MAX_INT |
+
+Default |
+50 |
+
+Change |
+Dynamic, though once on-disk the value
+for the pool is set |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_ddt_data_is_special
+zfs_ddt_data_is_special enables the deduplication table (DDT) to
+reside on a special top-level vdev.
+
+
+zfs_ddt_data_is_special |
+Notes |
+
|---|
+
+
+Tags |
+dedup,
+special_vdev |
+
+When to change |
+when using a special top-level vdev and
+no dedup top-level vdev and it is desired
+to store the DDT in the main pool
+top-level vdevs |
+
+Data Type |
+boolean |
+
+Range |
+0=do not use special vdevs to store DDT,
+1=store DDT in special vdevs |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_user_indirect_is_special
+If special vdevs are in use, zfs_user_indirect_is_special enables
+user data indirect blocks (a form of metadata) to be written to the
+special vdevs.
+
+
+zfs_user_indirect_is_special |
+Notes |
+
|---|
+
+
+Tags |
+special_vdev |
+
+When to change |
+to force user data indirect blocks
+to remain in the main pool top-level
+vdevs |
+
+Data Type |
+boolean |
+
+Range |
+0=do not write user indirect blocks
+to a special vdev, 1=write user
+indirect blocks to a special vdev |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_reconstruct_indirect_combinations_max
+After device removal, if an indirect split block contains more than
+zfs_reconstruct_indirect_combinations_max many possible unique
+combinations when being reconstructed, it can be considered too
+computationally expensive to check them all. Instead, at most
+zfs_reconstruct_indirect_combinations_max randomly-selected
+combinations are attempted each time the block is accessed. This allows
+all segment copies to participate fairly in the reconstruction when all
+combinations cannot be checked and prevents repeated use of one bad
+copy.
+
+
+zfs_recon
+struct_indirect_combinations_max |
+Notes |
+
|---|
+
+
+Tags |
+vdev_removal |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+attempts |
+
+Range |
+0=do not limit attempts, 1 to
+MAX_INT = limit for attempts |
+
+Default |
+4096 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_send_unmodified_spill_blocks
+zfs_send_unmodified_spill_blocks enables sending of unmodified spill
+blocks in the send stream. Under certain circumstances, previous
+versions of ZFS could incorrectly remove the spill block from an
+existing object. Including unmodified copies of the spill blocks creates
+a backwards compatible stream which will recreate a spill block if it
+was incorrectly removed.
+
+
+zfs_send_unmodified_spill_blocks |
+Notes |
+
|---|
+
+
+Tags |
+send |
+
+When to change |
+TBD |
+
+Data Type |
+boolean |
+
+Range |
+0=do not send unmodified spill
+blocks, 1=send unmodified spill
+blocks |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_spa_discard_memory_limit
+zfs_spa_discard_memory_limit sets the limit for maximum memory used
+for prefetching a pool’s checkpoint space map on each vdev while
+discarding a pool checkpoint.
+
+
+zfs_spa_discard_memory_limit |
+Notes |
+
|---|
+
+
+Tags |
+checkpoint |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+bytes |
+
+Range |
+0 to MAX_INT |
+
+Default |
+16,777,216 (16 MiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+
+zfs_trim_extent_bytes_max
+zfs_trim_extent_bytes_max sets the maximum size of a trim (aka
+discard, scsi unmap) command. Ranges larger than
+zfs_trim_extent_bytes_max are split in to chunks no larger than
+zfs_trim_extent_bytes_max bytes prior to being issued to the device.
+Use zpool iostat -w to observe the latency of trim commands.
+
+
+zfs_trim_extent_bytes_max |
+Notes |
+
|---|
+
+
+Tags |
+trim |
+
+When to change |
+if the device can efficiently handle
+larger trim requests |
+
+Data Type |
+uint |
+
+Units |
+bytes |
+
+Range |
+zfs_trim_extent_by
+tes_min
+to MAX_UINT |
+
+Default |
+134,217,728 (128 MiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_trim_extent_bytes_min
+zfs_trim_extent_bytes_min sets the minimum size of trim (aka
+discard, scsi unmap) commands. Trim ranges smaller than
+zfs_trim_extent_bytes_min are skipped unless they’re part of a
+larger range which was broken in to chunks. Some devices have
+performance degradation during trim operations, so using a larger
+zfs_trim_extent_bytes_min can reduce the total amount of space
+trimmed. Use zpool iostat -w to observe the latency of trim
+commands.
+
+
+zfs_trim_extent_bytes_min |
+Notes |
+
|---|
+
+
+Tags |
+trim |
+
+When to change |
+when trim is in use and device
+performance suffers from trimming small
+allocations |
+
+Data Type |
+uint |
+
+Units |
+bytes |
+
+Range |
+0=trim all unallocated space, otherwise
+minimum physical block size to MAX_ |
+
+Default |
+32,768 (32 KiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+
+zfs_trim_queue_limit
+zfs_trim_queue_limit sets the maximum queue depth for leaf vdevs.
+See also zfs_vdev_trim_max_active and
+zfs_trim_extent_bytes_max Use
+zpool iostat -q to observe trim queue depth.
+
+
+zfs_trim_queue_limit |
+Notes |
+
|---|
+
+
+Tags |
+trim |
+
+When to change |
+to restrict the number of trim commands in the queue |
+
+Data Type |
+uint |
+
+Units |
+I/O operations |
+
+Range |
+1 to MAX_UINT |
+
+Default |
+10 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_trim_txg_batch
+zfs_trim_txg_batch sets the number of transaction groups worth of
+frees which should be aggregated before trim (aka discard, scsi unmap)
+commands are issued to a device. This setting represents a trade-off
+between issuing larger, more efficient trim commands and the delay
+before the recently trimmed space is available for use by the device.
+Increasing this value will allow frees to be aggregated for a longer
+time. This will result is larger trim operations and potentially
+increased memory usage. Decreasing this value will have the opposite
+effect. The default value of 32 was empirically determined to be a
+reasonable compromise.
+
+
+zfs_trim_txg_batch |
+Notes |
+
|---|
+
+
+Tags |
+trim |
+
+When to change |
+TBD |
+
+Data Type |
+uint |
+
+Units |
+metaslabs to stride |
+
+Range |
+1 to MAX_UINT |
+
+Default |
+32 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_vdev_aggregate_trim
+zfs_vdev_aggregate_trim allows trim I/Os to be aggregated. This is
+normally not helpful because the extents to be trimmed will have been
+already been aggregated by the metaslab.
+
+
+zfs_vdev_aggregate_trim |
+Notes |
+
|---|
+
+
+Tags |
+trim, vdev,
+ZIO_scheduler |
+
+When to change |
+when debugging trim code or trim
+performance issues |
+
+Data Type |
+boolean |
+
+Range |
+0=do not attempt to aggregate trim
+commands, 1=attempt to aggregate trim
+commands |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_vdev_aggregation_limit_non_rotating
+zfs_vdev_aggregation_limit_non_rotating is the equivalent of
+zfs_vdev_aggregation_limit for devices
+which represent themselves as non-rotating to the Linux blkdev
+interfaces. Such devices have a value of 0 in
+/sys/block/DEVICE/queue/rotational and are expected to be SSDs.
+
+
+zfs_vde
+v_aggregation_limit_non_rotating |
+Notes |
+
|---|
+
+
+Tags |
+vdev,
+Z
+IO_scheduler |
+
+When to change |
+see
+zfs_vdev_aggregation_limit |
+
+Data Type |
+int |
+
+Units |
+bytes |
+
+Range |
+0 to MAX_INT |
+
+Default |
+131,072 bytes (128 KiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zil_nocacheflush
+ZFS uses barriers (volatile cache flush commands) to ensure data is
+committed to permanent media by devices. This ensures consistent
+on-media state for devices where caches are volatile (eg HDDs).
+zil_nocacheflush disables the cache flush commands that are normally
+sent to devices by the ZIL after a log write has completed.
+The difference between zil_nocacheflush and
+zfs_nocacheflush is zil_nocacheflush applies
+to ZIL writes while zfs_nocacheflush disables
+barrier writes to the pool devices at the end of transaction group syncs.
+WARNING: setting this can cause ZIL corruption on power loss if the
+device has a volatile write cache.
+
+
+zil_nocacheflush |
+Notes |
+
|---|
+
+
+Tags |
+disks, ZIL |
+
+When to change |
+If the storage device has nonvolatile cache,
+then disabling cache flush can save the cost of
+occasional cache flush commands |
+
+Data Type |
+boolean |
+
+Range |
+0=send cache flush commands, 1=do not send
+cache flush commands |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zio_deadman_log_all
+zio_deadman_log_all enables debugging messages for all ZFS I/Os,
+rather than only for leaf ZFS I/Os for a vdev. This is meant to be used
+by developers to gain diagnostic information for hang conditions which
+don’t involve a mutex or other locking primitive. Typically these are
+conditions where a thread in the zio pipeline is looping indefinitely.
+See also zfs_dbgmsg_enable
+
+
+zio_deadman_log_all |
+Notes |
+
|---|
+
+
+Tags |
+debug |
+
+When to change |
+when debugging ZFS I/O pipeline |
+
+Data Type |
+boolean |
+
+Range |
+0=do not log all deadman events, 1=log all
+deadman events |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zio_decompress_fail_fraction
+If non-zero, zio_decompress_fail_fraction represents the denominator
+of the probability that ZFS should induce a decompression failure. For
+instance, for a 5% decompression failure rate, this value should be set
+to 20.
+
+
+zio_decompress_fail_fraction |
+Notes |
+
|---|
+
+
+Tags |
+debug |
+
+When to change |
+when debugging ZFS internal
+compressed buffer code |
+
+Data Type |
+ulong |
+
+Units |
+probability of induced decompression
+failure is
+1/zio_decompress_fail_fraction |
+
+Range |
+0 = do not induce failures, or 1 to
+MAX_ULONG |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zio_slow_io_ms
+An I/O operation taking more than zio_slow_io_ms milliseconds to
+complete is marked as a slow I/O. Slow I/O counters can be observed with
+zpool status -s. Each slow I/O causes a delay zevent, observable
+using zpool events. See also zfs-events(5).
+
+
+zio_slow_io_ms |
+Notes |
+
|---|
+
+
+Tags |
+vdev, zed |
+
+When to change |
+when debugging slow devices and the default
+value is inappropriate |
+
+Data Type |
+int |
+
+Units |
+milliseconds |
+
+Range |
+0 to MAX_INT |
+
+Default |
+30,000 (30 seconds) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+vdev_validate_skip
+vdev_validate_skip disables label validation steps during pool
+import. Changing is not recommended unless you know what you are doing
+and are recovering a damaged label.
+
+
+vdev_validate_skip |
+Notes |
+
|---|
+
+
+Tags |
+vdev |
+
+When to change |
+do not change |
+
+Data Type |
+boolean |
+
+Range |
+0=validate labels during pool import, 1=do not
+validate vdev labels during pool import |
+
+Default |
+0 |
+
+Change |
+prior to pool import |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_async_block_max_blocks
+zfs_async_block_max_blocks limits the number of blocks freed in a
+single transaction group commit. During deletes of large objects, such
+as snapshots, the number of freed blocks can cause the DMU to extend txg
+sync times well beyond zfs_txg_timeout.
+zfs_async_block_max_blocks is used to limit these effects.
+
+
+zfs_async_block_max_blocks |
+Notes |
+
|---|
+
+
+Tags |
+delete, DMU |
+
+When to change |
+TBD |
+
+Data Type |
+ulong |
+
+Units |
+blocks |
+
+Range |
+1 to MAX_ULONG |
+
+Default |
+MAX_ULONG (do not limit) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_checksum_events_per_second
+zfs_checksum_events_per_second is a rate limit for checksum events.
+Note that this should not be set below the zed thresholds (currently
+10 checksums over 10 sec) or else zed may not trigger any action.
+
+
+zfs_checksum_events_per_second |
+Notes |
+
|---|
+
+
+Tags |
+vdev |
+
+When to change |
+TBD |
+
+Data Type |
+uint |
+
+Units |
+checksum events |
+
+Range |
+zed threshold to MAX_UINT
|
+
+Default |
+20 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_disable_ivset_guid_check
+zfs_disable_ivset_guid_check disables requirement for IVset guids to
+be present and match when doing a raw receive of encrypted datasets.
+Intended for users whose pools were created with ZFS on Linux
+pre-release versions and now have compatibility issues.
+For a ZFS raw receive, from a send stream created by zfs send --raw,
+the crypt_keydata nvlist includes a to_ivset_guid to be set on the new
+snapshot. This value will override the value generated by the snapshot
+code. However, this value may not be present, because older
+implementations of the raw send code did not include this value. When
+zfs_disable_ivset_guid_check is enabled, the receive proceeds and a
+newly-generated value is used.
+
+
+zfs_disable_ivset_guid_check |
+Notes |
+
|---|
+
+
+Tags |
+receive |
+
+When to change |
+debugging pre-release ZFS raw sends |
+
+Data Type |
+boolean |
+
+Range |
+0=check IVset guid, 1=do not check
+IVset guid |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_obsolete_min_time_ms
+zfs_obsolete_min_time_ms is similar to
+zfs_free_min_time_ms and used for cleanup of
+old indirection records for vdevs removed using the zpool remove
+command.
+
+
+zfs_obsolete_min_time_ms |
+Notes |
+
|---|
+
+
+Tags |
+delete, remove |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+milliseconds |
+
+Range |
+0 to MAX_INT |
+
+Default |
+500 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_override_estimate_recordsize
+zfs_override_estimate_recordsize overrides the default logic for
+estimating block sizes when doing a zfs send. The default heuristic is
+that the average block size will be the current recordsize.
+
+
+zfs_override_estimate_recordsize |
+Notes |
+
|---|
+
+
+Tags |
+send |
+
+When to change |
+if most data in your dataset is
+not of the current recordsize
+and you require accurate zfs
+send size estimates |
+
+Data Type |
+ulong |
+
+Units |
+bytes |
+
+Range |
+0=do not override, 1 to
+MAX_ULONG |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_remove_max_segment
+zfs_remove_max_segment sets the largest contiguous segment that ZFS
+attempts to allocate when removing a vdev. This can be no larger than
+16MB. If there is a performance problem with attempting to allocate
+large blocks, consider decreasing this. The value is rounded up to a
+power-of-2.
+
+
+zfs_remove_max_segment |
+Notes |
+
|---|
+
+
+Tags |
+remove |
+
+When to change |
+after removing a top-level vdev, consider
+decreasing if there is a performance
+degradation when attempting to allocate
+large blocks |
+
+Data Type |
+int |
+
+Units |
+bytes |
+
+Range |
+maximum of the physical block size of all
+vdevs in the pool to 16,777,216 bytes (16
+MiB) |
+
+Default |
+16,777,216 bytes (16 MiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_resilver_disable_defer
+zfs_resilver_disable_defer disables the resilver_defer pool
+feature. The resilver_defer feature allows ZFS to postpone new
+resilvers if an existing resilver is in progress.
+
+
+zfs_resilver_disable_defer |
+Notes |
+
|---|
+
+
+Tags |
+resilver |
+
+When to change |
+if resilver postponement is not
+desired due to overall resilver time
+constraints |
+
+Data Type |
+boolean |
+
+Range |
+0=allow resilver_defer to postpone
+new resilver operations, 1=immediately
+restart resilver when needed |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_scan_suspend_progress
+zfs_scan_suspend_progress causes a scrub or resilver scan to freeze
+without actually pausing.
+
+
+zfs_scan_suspend_progress |
+Notes |
+
|---|
+
+
+Tags |
+resilver, scrub |
+
+When to change |
+testing or debugging scan code |
+
+Data Type |
+boolean |
+
+Range |
+0=do not freeze scans, 1=freeze scans |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_scrub_min_time_ms
+Scrubs are processed by the sync thread. While scrubbing at least
+zfs_scrub_min_time_ms time is spent working on a scrub between txg
+syncs.
+
+
+zfs_scrub_min_time_ms |
+Notes |
+
|---|
+
+
+Tags |
+scrub |
+
+When to change |
+ |
+
+Data Type |
+int |
+
+Units |
+milliseconds |
+
+Range |
+1 to (zfs_txg_timeout - 1) |
+
+Default |
+1,000 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_slow_io_events_per_second
+zfs_slow_io_events_per_second is a rate limit for slow I/O events.
+Note that this should not be set below the zed thresholds (currently
+10 checksums over 10 sec) or else zed may not trigger any action.
+
+
+zfs_slow_io_events_per_second |
+Notes |
+
|---|
+
+
+Tags |
+vdev |
+
+When to change |
+TBD |
+
+Data Type |
+uint |
+
+Units |
+slow I/O events |
+
+Range |
+zed threshold to MAX_UINT
|
+
+Default |
+20 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_vdev_min_ms_count
+zfs_vdev_min_ms_count is the minimum number of metaslabs to create
+in a top-level vdev.
+
+
+zfs_vdev_min_ms_count |
+Notes |
+
|---|
+
+
+Tags |
+metaslab, vdev |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+metaslabs |
+
+Range |
+16 to
+zfs_vdev_m
+s_count_limit |
+
+Default |
+16 |
+
+Change |
+prior to creating a pool or adding a
+top-level vdev |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+zfs_vdev_ms_count_limit
+zfs_vdev_ms_count_limit is the practical upper limit for the number
+of metaslabs per top-level vdev.
+
+
+zfs_vdev_ms_count_limit |
+Notes |
+
|---|
+
+
+Tags |
+metaslab,
+vdev |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+metaslabs |
+
+Range |
+zfs_vdev
+_min_ms_count
+to 131,072 |
+
+Default |
+131,072 |
+
+Change |
+prior to creating a pool or adding a
+top-level vdev |
+
+Versions Affected |
+planned for v2 |
+
+
+
+
+
+spl_hostid
+
+
spl_hostid is a unique system id number. It originated in Sun’s
+products where most systems had a unique id assigned at the factory.
+This assignment does not exist in modern hardware.
+
In ZFS, the hostid is stored in the vdev label and can be used to
+determine if another system had imported the pool. When set
+
spl_hostid can be used to uniquely identify a system. By default
+this value is set to zero which indicates the hostid is disabled. It
+can be explicitly enabled by placing a unique non-zero value in the
+file shown in
spl_hostid_path
+
+
+spl_hostid |
+Notes |
+
|---|
+
+
+Tags |
+hostid, MMP |
+
+Kernel module |
+spl |
+
+When to change |
+to uniquely identify a system when vdevs can be
+shared across multiple systems |
+
+Data Type |
+ulong |
+
+Range |
+0=ignore hostid, 1 to 4,294,967,295 (32-bits or
+0xffffffff) |
+
+Default |
+0 |
+
+Change |
+prior to importing pool |
+
+Versions Affected |
+v0.6.1 |
+
+
+
+
+
+spl_hostid_path
+spl_hostid_path is the path name for a file that can contain a
+unique hostid. For testing purposes, spl_hostid_path can be
+overridden by the ZFS_HOSTID environment variable.
+
+
+spl_hostid_path |
+Notes |
+
|---|
+
+
+Tags |
+hostid, MMP |
+
+Kernel module |
+spl |
+
+When to change |
+when creating a new ZFS distribution where the
+default value is inappropriate |
+
+Data Type |
+string |
+
+Default |
+“/etc/hostid” |
+
+Change |
+read-only, can only be changed prior to spl
+module load |
+
+Versions Affected |
+v0.6.1 |
+
+
+
+
+
+spl_kmem_alloc_max
+Large kmem_alloc() allocations fail if they exceed KMALLOC_MAX_SIZE,
+as determined by the kernel source. Allocations which are marginally
+smaller than this limit may succeed but should still be avoided due to
+the expense of locating a contiguous range of free pages. Therefore, a
+maximum kmem size with reasonable safely margin of 4x is set.
+kmem_alloc() allocations larger than this maximum will quickly fail.
+vmem_alloc() allocations less than or equal to this value will use
+kmalloc(), but shift to vmalloc() when exceeding this value.
+
+
+spl_kmem_alloc_max |
+Notes |
+
|---|
+
+
+Tags |
+memory |
+
+Kernel module |
+spl |
+
+When to change |
+TBD |
+
+Data Type |
+uint |
+
+Units |
+bytes |
+
+Range |
+TBD |
+
+Default |
+KMALLOC_MAX_SIZE / 4 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 |
+
+
+
+
+
+spl_kmem_alloc_warn
+As a general rule kmem_alloc() allocations should be small,
+preferably just a few pages since they must by physically contiguous.
+Therefore, a rate limited warning is printed to the console for any
+kmem_alloc() which exceeds the threshold spl_kmem_alloc_warn
+The default warning threshold is set to eight pages but capped at 32K to
+accommodate systems using large pages. This value was selected to be
+small enough to ensure the largest allocations are quickly noticed and
+fixed. But large enough to avoid logging any warnings when a allocation
+size is larger than optimal but not a serious concern. Since this value
+is tunable, developers are encouraged to set it lower when testing so
+any new largish allocations are quickly caught. These warnings may be
+disabled by setting the threshold to zero.
+
+
+spl_kmem_alloc_warn |
+Notes |
+
|---|
+
+
+Tags |
+memory |
+
+Kernel module |
+spl |
+
+When to change |
+developers are encouraged lower when testing
+so any new, large allocations are quickly
+caught |
+
+Data Type |
+uint |
+
+Units |
+bytes |
+
+Range |
+0=disable the warnings, |
+
+Default |
+32,768 (32 KiB) |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 |
+
+
+
+
+
+spl_kmem_cache_expire
+Cache expiration is part of default illumos cache behavior. The idea is
+that objects in magazines which have not been recently accessed should
+be returned to the slabs periodically. This is known as cache aging and
+when enabled objects will be typically returned after 15 seconds.
+On the other hand Linux slabs are designed to never move objects back to
+the slabs unless there is memory pressure. This is possible because
+under Linux the cache will be notified when memory is low and objects
+can be released.
+By default only the Linux method is enabled. It has been shown to
+improve responsiveness on low memory systems and not negatively impact
+the performance of systems with more memory. This policy may be changed
+by setting the spl_kmem_cache_expire bit mask as follows, both
+policies may be enabled concurrently.
+
+
+spl_kmem_cache_expire |
+Notes |
+
|---|
+
+
+Tags |
+memory |
+
+Kernel module |
+spl |
+
+When to change |
+TBD |
+
+Data Type |
+bitmask |
+
+Range |
+0x01 - Aging (illumos), 0x02 - Low memory (Linux) |
+
+Default |
+0x02 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.6.1 to v0.8.x |
+
+
+
+
+
+spl_kmem_cache_kmem_limit
+Depending on the size of a memory cache object it may be backed by
+kmalloc() or vmalloc() memory. This is because the size of the
+required allocation greatly impacts the best way to allocate the memory.
+When objects are small and only a small number of memory pages need to
+be allocated, ideally just one, then kmalloc() is very efficient.
+However, allocating multiple pages with kmalloc() gets increasingly
+expensive because the pages must be physically contiguous.
+For this reason we shift to vmalloc() for slabs of large objects
+which which removes the need for contiguous pages. vmalloc() cannot
+be used in all cases because there is significant locking overhead
+involved. This function takes a single global lock over the entire
+virtual address range which serializes all allocations. Using slightly
+different allocation functions for small and large objects allows us to
+handle a wide range of object sizes.
+The spl_kmem_cache_kmem_limit value is used to determine this cutoff
+size. One quarter of the kernel’s compiled PAGE_SIZE is used as the
+default value because
+spl_kmem_cache_obj_per_slab defaults
+to 16. With these default values, at most four contiguous pages are
+allocated.
+
+
+spl_kmem_cache_kmem_limit |
+Notes |
+
|---|
+
+
+Tags |
+memory |
+
+Kernel module |
+spl |
+
+When to change |
+TBD |
+
+Data Type |
+uint |
+
+Units |
+pages |
+
+Range |
+TBD |
+
+Default |
+PAGE_SIZE / 4 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 to v0.8.x |
+
+
+
+
+
+spl_kmem_cache_max_size
+spl_kmem_cache_max_size is the maximum size of a kmem cache slab in
+MiB. This effectively limits the maximum cache object size to
+spl_kmem_cache_max_size /
+spl_kmem_cache_obj_per_slab Kmem
+caches may not be created with object sized larger than this limit.
+
+
+spl_kmem_cache_max_size |
+Notes |
+
|---|
+
+
+Tags |
+memory |
+
+Kernel module |
+spl |
+
+When to change |
+TBD |
+
+Data Type |
+uint |
+
+Units |
+MiB |
+
+Range |
+TBD |
+
+Default |
+4 for 32-bit kernel, 32 for 64-bit kernel |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 |
+
+
+
+
+
+spl_kmem_cache_obj_per_slab
+spl_kmem_cache_obj_per_slab is the preferred number of objects per
+slab in the kmem cache. In general, a larger value will increase the
+caches memory footprint while decreasing the time required to perform an
+allocation. Conversely, a smaller value will minimize the footprint and
+improve cache reclaim time but individual allocations may take longer.
+
+
+spl_kmem_cache_obj_per_slab |
+Notes |
+
|---|
+
+
+Tags |
+memory |
+
+Kernel module |
+spl |
+
+When to change |
+TBD |
+
+Data Type |
+uint |
+
+Units |
+kmem cache objects |
+
+Range |
+TBD |
+
+Default |
+8 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 to v0.8.x |
+
+
+
+
+
+spl_kmem_cache_obj_per_slab_min
+spl_kmem_cache_obj_per_slab_min is the minimum number of objects
+allowed per slab. Normally slabs will contain
+spl_kmem_cache_obj_per_slab objects
+but for caches that contain very large objects it’s desirable to only
+have a few, or even just one, object per slab.
+
+
+spl_kmem_cache_obj_per_slab_min |
+Notes |
+
|---|
+
+
+Tags |
+memory |
+
+Kernel module |
+spl |
+
+When to change |
+debugging kmem cache operations |
+
+Data Type |
+uint |
+
+Units |
+kmem cache objects |
+
+Range |
+TBD |
+
+Default |
+1 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 |
+
+
+
+
+
+spl_kmem_cache_reclaim
+spl_kmem_cache_reclaim prevents Linux from being able to rapidly
+reclaim all the memory held by the kmem caches. This may be useful in
+circumstances where it’s preferable that Linux reclaim memory from some
+other subsystem first. Setting spl_kmem_cache_reclaim increases the
+likelihood out of memory events on a memory constrained system.
+
+
+spl_kmem_cache_reclaim |
+Notes |
+
|---|
+
+
+Tags |
+memory |
+
+Kernel module |
+spl |
+
+When to change |
+TBD |
+
+Data Type |
+boolean |
+
+Range |
+0=enable rapid memory reclaim from kmem
+caches, 1=disable rapid memory reclaim
+from kmem caches |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 |
+
+
+
+
+
+spl_kmem_cache_slab_limit
+For small objects the Linux slab allocator should be used to make the
+most efficient use of the memory. However, large objects are not
+supported by the Linux slab allocator and therefore the SPL
+implementation is preferred. spl_kmem_cache_slab_limit is used to
+determine the cutoff between a small and large object.
+Objects of spl_kmem_cache_slab_limit or smaller will be allocated
+using the Linux slab allocator, large objects use the SPL allocator. A
+cutoff of 16 KiB was determined to be optimal for architectures using 4
+KiB pages.
+
+
+spl_kmem_cache_slab_limit |
+Notes |
+
|---|
+
+
+Tags |
+memory |
+
+Kernel module |
+spl |
+
+When to change |
+TBD |
+
+Data Type |
+uint |
+
+Units |
+bytes |
+
+Range |
+TBD |
+
+Default |
+16,384 (16 KiB) when kernel PAGE_SIZE =
+4KiB, 0 for other PAGE_SIZE values |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 |
+
+
+
+
+
+spl_max_show_tasks
+spl_max_show_tasks is the limit of tasks per pending list in each
+taskq shown in /proc/spl/taskq and /proc/spl/taskq-all. Reading
+the ProcFS files walks the lists with lock held and it could cause a
+lock up if the list grow too large. If the list is larger than the
+limit, the string `”(truncated)” is printed.
+
+
+spl_max_show_tasks |
+Notes |
+
|---|
+
+
+Tags |
+taskq |
+
+Kernel module |
+spl |
+
+When to change |
+TBD |
+
+Data Type |
+uint |
+
+Units |
+tasks reported |
+
+Range |
+0 disables the limit, 1 to MAX_UINT |
+
+Default |
+512 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 |
+
+
+
+
+
+spl_panic_halt
+spl_panic_halt enables kernel panic upon assertion failures. When
+not enabled, the asserting thread is halted to facilitate further
+debugging.
+
+
+spl_panic_halt |
+Notes |
+
|---|
+
+
+Tags |
+debug, panic |
+
+Kernel module |
+spl |
+
+When to change |
+when debugging assertions and kernel core dumps
+are desired |
+
+Data Type |
+boolean |
+
+Range |
+0=halt thread upon assertion, 1=panic kernel
+upon assertion |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 |
+
+
+
+
+
+spl_taskq_kick
+Upon writing a non-zero value to spl_taskq_kick, all taskqs are
+scanned. If any taskq has a pending task more than 5 seconds old, the
+taskq spawns more threads. This can be useful in rare deadlock
+situations caused by one or more taskqs not spawning a thread when it
+should.
+
+
+spl_taskq_kick |
+Notes |
+
|---|
+
+
+Tags |
+taskq |
+
+Kernel module |
+spl |
+
+When to change |
+See description above |
+
+Data Type |
+uint |
+
+Units |
+N/A |
+
+Default |
+0 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 |
+
+
+
+
+
+spl_taskq_thread_bind
+spl_taskq_thread_bind enables binding taskq threads to specific
+CPUs, distributed evenly over the available CPUs. By default, this
+behavior is disabled to allow the Linux scheduler the maximum
+flexibility to determine where a thread should run.
+
+
+spl_taskq_thread_bind |
+Notes |
+
|---|
+
+
+Tags |
+CPU, taskq |
+
+Kernel module |
+spl |
+
+When to change |
+when debugging CPU scheduling options |
+
+Data Type |
+boolean |
+
+Range |
+0=taskqs are not bound to specific CPUs,
+1=taskqs are bound to CPUs |
+
+Default |
+0 |
+
+Change |
+prior to loading spl kernel module |
+
+Versions Affected |
+v0.7.0 |
+
+
+
+
+
+spl_taskq_thread_dynamic
+spl_taskq_thread_dynamic enables taskqs to set the TASKQ_DYNAMIC
+flag will by default create only a single thread. New threads will be
+created on demand up to a maximum allowed number to facilitate the
+completion of outstanding tasks. Threads which are no longer needed are
+promptly destroyed. By default this behavior is enabled but it can be d.
+See also
+zfs_zil_clean_taskq_nthr_pct,
+zio_taskq_batch_pct
+
+
+spl_taskq_thread_dynamic |
+Notes |
+
|---|
+
+
+Tags |
+taskq |
+
+Kernel module |
+spl |
+
+When to change |
+disable for performance analysis or
+troubleshooting |
+
+Data Type |
+boolean |
+
+Range |
+0=taskq threads are not dynamic, 1=taskq
+threads are dynamically created and
+destroyed |
+
+Default |
+1 |
+
+Change |
+prior to loading spl kernel module |
+
+Versions Affected |
+v0.7.0 |
+
+
+
+
+
+spl_taskq_thread_priority
+
+
spl_taskq_thread_priority allows newly created taskq threads to
+set a non-default scheduler priority. When enabled the priority
+specified when a taskq is created will be applied to all threads
+created by that taskq.
+
When disabled all threads will use the default Linux kernel thread
+priority.
+
+
+spl_taskq_thread_priority |
+Notes |
+
|---|
+
+
+Tags |
+CPU, taskq |
+
+Kernel module |
+spl |
+
+When to change |
+when troubleshooting CPU
+scheduling-related performance issues |
+
+Data Type |
+boolean |
+
+Range |
+0=taskq threads use the default Linux
+kernel thread priority, 1= |
+
+Default |
+1 |
+
+Change |
+prior to loading spl kernel module |
+
+Versions Affected |
+v0.7.0 |
+
+
+
+
+
+spl_taskq_thread_sequential
+spl_taskq_thread_sequential is the number of items a taskq worker
+thread must handle without interruption before requesting a new worker
+thread be spawned. spl_taskq_thread_sequential controls how quickly
+taskqs ramp up the number of threads processing the queue. Because Linux
+thread creation and destruction are relatively inexpensive a small
+default value has been selected. Thus threads are created aggressively,
+which is typically desirable. Increasing this value results in a slower
+thread creation rate which may be preferable for some configurations.
+
+
+spl_taskq_thread_sequential |
+Notes |
+
|---|
+
+
+Tags |
+CPU, taskq |
+
+Kernel module |
+spl |
+
+When to change |
+TBD |
+
+Data Type |
+int |
+
+Units |
+taskq items |
+
+Range |
+1 to MAX_INT |
+
+Default |
+4 |
+
+Change |
+Dynamic |
+
+Versions Affected |
+v0.7.0 |
+
+
+
+
+
+spl_kmem_cache_kmem_threads
+spl_kmem_cache_kmem_threads shows the current number of
+spl_kmem_cache threads. This task queue is responsible for
+allocating new slabs for use by the kmem caches. For the majority of
+systems and workloads only a small number of threads are required.
+
+
+spl_kmem_cache_kmem_threads |
+Notes |
+
|---|
+
+
+Tags |
+CPU, memory |
+
+Kernel module |
+spl |
+
+When to change |
+read-only |
+
+Data Type |
+int |
+
+Range |
+1 to MAX_INT |
+
+Units |
+threads |
+
+Default |
+4 |
+
+Change |
+read-only, can only be changed prior
+to spl module load |
+
+Versions Affected |
+v0.7.0 |
+
+
+
+
+
+spl_kmem_cache_magazine_size
+spl_kmem_cache_magazine_size shows the current . Cache magazines are
+an optimization designed to minimize the cost of allocating memory. They
+do this by keeping a per-cpu cache of recently freed objects, which can
+then be reallocated without taking a lock. This can improve performance
+on highly contended caches. However, because objects in magazines will
+prevent otherwise empty slabs from being immediately released this may
+not be ideal for low memory machines.
+For this reason spl_kmem_cache_magazine_size can be used to set a
+maximum magazine size. When this value is set to 0 the magazine size
+will be automatically determined based on the object size. Otherwise
+magazines will be limited to 2-256 objects per magazine (eg per CPU).
+Magazines cannot be disabled entirely in this implementation.
+
+
+spl_kmem_cache_magazine_size |
+Notes |
+
|---|
+
+
+Tags |
+CPU, memory |
+
+Kernel module |
+spl |
+
+When to change |
+ |
+
+Data Type |
+int |
+
+Units |
+threads |
+
+Range |
+0=automatically scale magazine size,
+otherwise 2 to 256 |
+
+Default |
+0 |
+
+Change |
+read-only, can only be changed prior
+to spl module load |
+
+Versions Affected |
+v0.7.0 |
+
+
+
+
+
+
