Project

General

Profile

Bug #32736

Change default ZFS indirect block size from 128 to 32 for new files and zvols

Added by Dru Lavigne about 1 year ago. Updated 11 months ago.

Status:
Done
Priority:
No priority
Assignee:
Alexander Motin
Category:
OS
Target version:
Seen in:
Severity:
Low
Reason for Closing:
Reason for Blocked:
Needs QA:
No
Needs Doc:
No
Needs Merging:
No
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

I've merged patch (will be in 11.1-U5 and 11.2), removing limit on maximal amount of decompressed ARC data of 100MB, allowing it to reach up to 3% of ARC. It should reduce the decompression overhead for metadata and some small part of very hot data.

We may also consider tuning of default indirect block size, but that is still being investigated.

builder-labstor02.svg (1.1 MB) builder-labstor02.svg Nick Wolff, 03/12/2018 12:45 PM

Subtasks

Bug #34894: Measure performance effects of different indirect block sizesDoneNick Principe

Related issues

Copied from FreeNAS - Bug #23893: Enhance compressed ARC performanceDone

Associated revisions

Revision e0818e8d (diff)
Added by Alexander Motin 12 months ago

Reduce default indirect block size from 128K to 32K.

It dramatically reduces write inflation and metadata (de)compression time
in case of random writes. It may reduce delete/UNMAP performance due
to more head seeks to read the smaller fragmented indirect blocks,
but that operation is prefetchable, so it should not be too bad.

This value originally was set to 16K, but increased to 128K exactly to
improve delete/UNMAP performance. But that was done before compressed
ARC support, which introduced new factors.

Ticket: #32736

Revision 291ecc3d (diff)
Added by Alexander Motin 12 months ago

Reduce default indirect block size from 128K to 32K.

It dramatically reduces write inflation and metadata (de)compression time
in case of random writes. It may reduce delete/UNMAP performance due
to more head seeks to read the smaller fragmented indirect blocks,
but that operation is prefetchable, so it should not be too bad.

This value originally was set to 16K, but increased to 128K exactly to
improve delete/UNMAP performance. But that was done before compressed
ARC support, which introduced new factors.

Ticket: #32736
(cherry picked from commit e0818e8db9ca1cc944580e3acebae1848b3ce4fb)

Revision 9987e2bb (diff)
Added by Alexander Motin 3 months ago

Reduce default indirect block size from 128K to 32K.

It dramatically reduces write inflation and metadata (de)compression time
in case of random writes. It may reduce delete/UNMAP performance due
to more head seeks to read the smaller fragmented indirect blocks,
but that operation is prefetchable, so it should not be too bad.

This value originally was set to 16K, but increased to 128K exactly to
improve delete/UNMAP performance. But that was done before compressed
ARC support, which introduced new factors.

Ticket: #32736

Revision 99bd9b60 (diff)
Added by Alexander Motin 3 months ago

Revert "Reduce default indirect block size from 128K to 32K."

This reverts commit 291ecc3de471e9772f5f59cf42ca87f7e0f622f3.

I've moved this to sysctl.conf instead.

Ticket: #32736

History

#1 Updated by Dru Lavigne about 1 year ago

  • Copied from Bug #23893: Enhance compressed ARC performance added

#2 Updated by Alexander Motin about 1 year ago

  • Status changed from In Progress to Blocked
  • Reason for Blocked set to Dependent on a related task to be completed

#3 Updated by Alexander Motin about 1 year ago

  • Subject changed from Tune default indirect block size to Tune default ZFS indirect block size

New default indirect block size of 128KB (originally was 16KB) can be too big for random access pattern since compressed ARC introduction. It may have sense to reduce it to 64KB or may be even 32KB to reduce CPU overhead to reasonable level with not so big hit to large sequential operations (delete/unmap).

#4 Updated by Dru Lavigne 12 months ago

  • Target version changed from 11.2-RC2 to 11.2-BETA3

#5 Updated by Dru Lavigne 12 months ago

  • Target version changed from 11.2-BETA3 to 11.2-U2

#6 Updated by Alexander Motin 12 months ago

  • Status changed from Blocked to Ready for Testing
  • Target version changed from 11.2-U2 to 11.2-BETA3
  • Reason for Blocked deleted (Dependent on a related task to be completed)
  • Needs QA changed from Yes to No
  • Needs Doc changed from Yes to No
  • Needs Merging changed from Yes to No

freenas/11-stable commit changing the value from 17 (128KB) to 15 (32KB): https://github.com/freenas/os/commit/291ecc3de471e9772f5f59cf42ca87f7e0f622f3

It can be verified or tuned live (affecting only new files or zvols) via sysctl vfs.zfs.default_ibs.

#7 Updated by Dru Lavigne 11 months ago

  • Subject changed from Tune default ZFS indirect block size to Change default ZFS indirect block size from 128 to 32 for new files and zvols
  • Needs QA changed from No to Yes
  • Needs Doc changed from No to Yes

#8 Updated by Timothy Moore II 11 months ago

  • Status changed from Ready for Testing to Passed Testing
  • Needs QA changed from Yes to No

Testing with FreeNAS system updated to FreeNAS-11.2-MASTER-201808140859:

Opened shell and entered `sysctl vfs.zfs.default_ibs`: `15` returned. Entered `sysctl vfs.zfs.default_ibs=17` and updated back to 128 KB.

#9 Updated by Timothy Moore II 11 months ago

  • Needs QA changed from No to Yes

#11 Updated by Timothy Moore II 11 months ago

  • Status changed from Passed Testing to Ready for Testing

#13 Updated by Timothy Moore II 11 months ago

  • Status changed from Ready for Testing to Passed Testing
  • Needs QA changed from Yes to No

#15 Updated by Timothy Moore II 11 months ago

  • Needs Doc changed from Yes to No

#16 Updated by Dru Lavigne 11 months ago

  • Status changed from Passed Testing to Done

Also available in: Atom PDF