Project

General

Profile

Bug #26687

Improve performance for speculative prefetch in ZFS for misaligned reads

Added by Alexander Motin about 1 year ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Critical
Assignee:
Alexander Motin
Category:
OS
Target version:
Seen in:
Severity:
New
Reason for Closing:
Reason for Blocked:
Needs QA:
No
Needs Doc:
Yes
Needs Merging:
Yes
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

Sequential reads not aligned to block size are not detected by ZFS prefetcher as sequential, killing prefetch and severely hurting performance:

# dd if=/dev/zvol/test0/zzz of=/dev/null bs=128k
819200+0 records in
819200+0 records out
107374182400 bytes transferred in 253.025509 secs (424361096 bytes/sec)
# dd if=/dev/gpt/zzz of=/dev/null bs=128k
^C815838+0 records in
815838+0 records out
106933518336 bytes transferred in 683.459370 secs (156459218 bytes/sec)
# dd if=/dev/zvol/test0/zzz of=/dev/null bs=128k
^C486400+0 records in
486400+0 records out
63753420800 bytes transferred in 150.415240 secs (423849477 bytes/sec)
# dd if=/dev/gpt/zzz of=/dev/null bs=128k
^C22129+0 records in
22129+0 records out
2900492288 bytes transferred in 18.267249 secs (158781015 bytes/sec)

It is caused by dmu_zfetch() in case of misaligned sequential accesses being called with overlap of one block:

dmu_zfetch(fffff80198dd0ae0, 27347, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27355, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27363, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27371, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27379, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27387, 9, 1)

zz_zfetch1.patch (970 Bytes) zz_zfetch1.patch Slightly updated version Alexander Motin, 11/20/2017 09:25 AM

Associated revisions

Revision 43e086c4 (diff)
Added by Alexander Motin about 1 year ago

8835 Speculative prefetch in ZFS not working for misaligned reads

In case of misaligned I/O sequential requests are not detected as such
due to overlaps in logical block sequence:

dmu_zfetch(fffff80198dd0ae0, 27347, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27355, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27363, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27371, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27379, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27387, 9, 1)

This patch makes single block overlap to be counted as a stream hit,
improving performance up to several times.

Ticket: #26687

Revision 43e086c4 (diff)
Added by Alexander Motin about 1 year ago

8835 Speculative prefetch in ZFS not working for misaligned reads

In case of misaligned I/O sequential requests are not detected as such
due to overlaps in logical block sequence:

dmu_zfetch(fffff80198dd0ae0, 27347, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27355, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27363, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27371, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27379, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27387, 9, 1)

This patch makes single block overlap to be counted as a stream hit,
improving performance up to several times.

Ticket: #26687

Revision 013cacf8 (diff)
Added by Alexander Motin about 1 year ago

8835 Speculative prefetch in ZFS not working for misaligned reads

In case of misaligned I/O sequential requests are not detected as such
due to overlaps in logical block sequence:

dmu_zfetch(fffff80198dd0ae0, 27347, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27355, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27363, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27371, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27379, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27387, 9, 1)

This patch makes single block overlap to be counted as a stream hit,
improving performance up to several times.

Ticket: #26687
(cherry picked from commit 43e086c4e5de752ff139ec72033d2fb0e906bd77)

Revision 013cacf8 (diff)
Added by Alexander Motin about 1 year ago

8835 Speculative prefetch in ZFS not working for misaligned reads

In case of misaligned I/O sequential requests are not detected as such
due to overlaps in logical block sequence:

dmu_zfetch(fffff80198dd0ae0, 27347, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27355, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27363, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27371, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27379, 9, 1)
dmu_zfetch(fffff80198dd0ae0, 27387, 9, 1)

This patch makes single block overlap to be counted as a stream hit,
improving performance up to several times.

Ticket: #26687
(cherry picked from commit 43e086c4e5de752ff139ec72033d2fb0e906bd77)

History

#2 Updated by Alexander Motin about 1 year ago

I've sent email with review request to Matt Ahrens since he is the author of the present prefetcher code.

#3 Updated by Alexander Motin about 1 year ago

  • Target version changed from 11.1-U1 to 11.1

#4 Updated by Alexander Motin about 1 year ago

  • File zz_zfetch.patch added

#5 Updated by Sean Fagan about 1 year ago

I've only looked at the patch, not in a greater context, and it seems okay.

#6 Updated by Alexander Motin about 1 year ago

#7 Updated by Alexander Motin about 1 year ago

  • File deleted (zz_zfetch.patch)

#8 Updated by Alexander Motin about 1 year ago

  • Status changed from Fix In Progress to 19

I've merged the patch.

#9 Updated by Dru Lavigne about 1 year ago

  • Target version changed from 11.1 to 11.1-RC2

#10 Updated by Dru Lavigne about 1 year ago

  • Subject changed from Speculative prefetch in ZFS not working for misaligned reads to Improve performance for speculative prefetch in ZFS for misaligned reads

#11 Avatar?id=14398&size=24x24 Updated by Kris Moore about 1 year ago

  • Status changed from 19 to Ready For Release

#12 Avatar?id=14398&size=24x24 Updated by Kris Moore about 1 year ago

  • Status changed from Ready For Release to Screened
  • Assignee changed from Alexander Motin to Sean Fagan

Sean,

Alexander had some review comments on the PR:

https://github.com/freenas/os/pull/45

If you are wanting to fix those lets do it, otherwise we can merge as is since it seems relatively minor.

#13 Avatar?id=14398&size=24x24 Updated by Kris Moore about 1 year ago

  • Status changed from Screened to Ready For Release
  • Assignee changed from Sean Fagan to Alexander Motin

Opps, disregard, wrong ticket / PR ;)

#14 Updated by Dru Lavigne about 1 year ago

  • Target version changed from 11.1-RC2 to 11.1-RC3

#15 Updated by Nick Wolff about 1 year ago

  • Needs QA changed from Yes to No
  • QA Status Test Passes FreeNAS, Test Passes TrueNAS added
  • QA Status deleted (Not Tested)

Testing of patch showing 2-3x read speed improvement in original test condition that we found this issue with

#16 Updated by Dru Lavigne about 1 year ago

  • Status changed from Ready For Release to Resolved

Also available in: Atom PDF