Project

General

Profile

Bug #42803

Unblock speculative prefetcher on pool creation

Added by Alexander Motin about 2 years ago. Updated about 2 years ago.

Status:
Done
Priority:
No priority
Assignee:
Alexander Motin
Category:
OS
Target version:
Seen in:
Severity:
Medium
Reason for Closing:
Reason for Blocked:
Needs QA:
No
Needs Doc:
No
Needs Merging:
No
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

Fix at r331950 appeared to be incomplete, fixing only case of pool import, but not pool creation, leaving prefetcher still blocked for newly created pools. It dramatically reduces sequential read speed between pool creation and first reboot or reimport. It can be diagnosed by not growing numbers in `sysctl kstat.zfs.misc.zfetchstats` during sequential read.

Associated revisions

Revision fb319eab (diff)
Added by Alexander Motin about 2 years ago

Unblock speculative prefetcher also on pool creation. Fix at r331950 appeared to be incomplete, fixing only case of pool import, but not pool creation, leaving prefetcher still blocked for newly created pools. Ticket: #42803 (cherry picked from commit 6dafcae1dddb8858d56d3337f4ae49071dff752a)

History

#1 Updated by Alexander Motin about 2 years ago

  • Status changed from Unscreened to In Progress

#2 Updated by Dru Lavigne about 2 years ago

  • Target version changed from 11.2-RC1 to 11.2-BETA3

#3 Updated by Dru Lavigne about 2 years ago

  • Needs Doc changed from Yes to No
  • Needs Merging changed from No to Yes

#4 Updated by Alexander Motin about 2 years ago

  • Status changed from In Progress to Ready for Testing

#5 Updated by Dru Lavigne about 2 years ago

  • Subject changed from Speculative prefetcher blocked on pool creation to Unblock speculative prefetcher on pool creation
  • Needs Merging changed from Yes to No

#7 Updated by Michael Reynolds about 2 years ago

Grabbing a CentOS ISO as it is a 4.16G ISO

kstat.zfs.misc.zfetchstats values before starting the wget
root@freenas[/mnt/pool1]# sysctl kstat.zfs.misc.zfetchstats
kstat.zfs.misc.zfetchstats.max_streams: 18586419
kstat.zfs.misc.zfetchstats.misses: 18599666
kstat.zfs.misc.zfetchstats.hits: 4974

wget http://mirror.fileplanet.com/centos/7/isos/x86_64/CentOS-7-x86_64-DVD-1804.iso

Values after starting the wget
root@freenas[/mnt/pool1]# sysctl kstat.zfs.misc.zfetchstats
kstat.zfs.misc.zfetchstats.max_streams: 18591686
kstat.zfs.misc.zfetchstats.misses: 18604975
kstat.zfs.misc.zfetchstats.hits: 5906
root@freenas[/mnt/pool1]# sysctl kstat.zfs.misc.zfetchstats
kstat.zfs.misc.zfetchstats.max_streams: 18797352
kstat.zfs.misc.zfetchstats.misses: 18810746
kstat.zfs.misc.zfetchstats.hits: 14960

A random time during the wget
root@freenas[/mnt/pool1]# sysctl kstat.zfs.misc.zfetchstats
kstat.zfs.misc.zfetchstats.max_streams: 18664013
kstat.zfs.misc.zfetchstats.misses: 18677391
kstat.zfs.misc.zfetchstats.hits: 12631

After wget finsished the download
root@freenas[/mnt/pool1]# sysctl kstat.zfs.misc.zfetchstats
kstat.zfs.misc.zfetchstats.max_streams: 19210404
kstat.zfs.misc.zfetchstats.misses: 19224005
kstat.zfs.misc.zfetchstats.hits: 39209

Started a cp of the ISO from 1 pool to another after the wget finished

cp CentOS-7-x86_64-DVD-1804.iso ../poolio
root@freenas[/mnt/pool1]# sysctl kstat.zfs.misc.zfetchstats
kstat.zfs.misc.zfetchstats.max_streams: 19612637
kstat.zfs.misc.zfetchstats.misses: 19626509
kstat.zfs.misc.zfetchstats.hits: 41607

When the cp of the ISO finished
root@freenas[/mnt/pool1]# sysctl kstat.zfs.misc.zfetchstats
kstat.zfs.misc.zfetchstats.max_streams: 19643510
kstat.zfs.misc.zfetchstats.misses: 19657394
kstat.zfs.misc.zfetchstats.hits: 43590

Change between start and finish.
kstat.zfs.misc.zfetchstats.max_streams Difference between start and finish: 1057091
kstat.zfs.misc.zfetchstats.misses Difference between start and finish: 1057728
kstat.zfs.misc.zfetchstats.hits Difference between start and finish: 38616

#9 Updated by Michael Reynolds about 2 years ago

Using dd to test this change more specifically.
Since I already fetched the CentOS ISO may as well use it.

before dd
root@freenas[~]# sysctl kstat.zfs.misc.zfetchstats
kstat.zfs.misc.zfetchstats.max_streams: 30477903
kstat.zfs.misc.zfetchstats.misses: 30499898
kstat.zfs.misc.zfetchstats.hits: 57077

dd if=CentOS-7-x86_64-DVD-1804.iso of=/dev/null bs=128k

Right after the dd
root@freenas[~]# sysctl kstat.zfs.misc.zfetchstats
kstat.zfs.misc.zfetchstats.max_streams: 30800414
kstat.zfs.misc.zfetchstats.misses: 30822993
kstat.zfs.misc.zfetchstats.hits: 93811

#11 Updated by Michael Reynolds about 2 years ago

  • Status changed from Ready for Testing to Passed Testing
  • Needs QA changed from Yes to No

Turned off compression
root@freenas[~]# sysctl kstat.zfs.misc.zfetchstats
kstat.zfs.misc.zfetchstats.max_streams: 117022
kstat.zfs.misc.zfetchstats.misses: 117599
kstat.zfs.misc.zfetchstats.hits: 338

ran dd if=/dev/zero of=BUG-42803 bs=1024k count=32768

root@freenas[~]# sysctl kstat.zfs.misc.zfetchstats
kstat.zfs.misc.zfetchstats.max_streams: 124182
kstat.zfs.misc.zfetchstats.misses: 124794
kstat.zfs.misc.zfetchstats.hits: 394

root@freenas[~]# dd if=BUG-42803 of=/dev/null bs=1024k count=32768
5363+0 records in
5363+0 records out
5623513088 bytes transferred in 0.683319 secs (8229698419 bytes/sec)
root@freenas[~]# sysctl kstat.zfs.misc.zfetchstats
kstat.zfs.misc.zfetchstats.max_streams: 151175
kstat.zfs.misc.zfetchstats.misses: 151933
kstat.zfs.misc.zfetchstats.hits: 5819

max_streams went up by 26381
misses went up by 27139
hits went up by 5425

Checked with mav to see if this is acceptable.

He said the number of hits looks right, the number of misses seems high, but that's a question for a different ticket.

#12 Updated by Dru Lavigne about 2 years ago

  • Status changed from Passed Testing to Done

Also available in: Atom PDF