Project

General

Profile

Feature #24371

Change default boot scrub interval to 7 days

Added by Cyber Jock almost 4 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Important
Assignee:
Nikola Gigic
Category:
Middleware
Target version:
Estimated time:
Severity:
New
Reason for Closing:
Reason for Blocked:
Needs QA:
No
Needs Doc:
Yes
Needs Merging:
Yes
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:

Description

Per discussions with Adam Morales (who discussed this with Kris Moore) as well as recurring problems with SATA DOMs, it seems reasonable to make our boot device scrub interval default shorter than it currently is. Based on the fact that scrubs are read-only and the only real consequence is the more busy boot device for a few minutes while it runs, this seems like a great way for us to identify failing SATA DOMs more proactively.

This idea was originally brought back up my me as we had SATA DOM failures on a B node for Gladstone, and after replacing them we realized that the A node had a boot device scrub that identified failing SATA DOMs on the A node.

This feature request may be better applied as a FreeNAS and TrueNAS update. I'll leave it to the development team to decide whether we want to change this for TrueNAS only or both products.

I did a scrub on my own Z20 HA, and despite having a bunch of boot environments, the scrub finished in just 2 minutes.

See below:

[root@stripes] ~# zpool status
pool: freenas-boot
state: ONLINE
scan: scrub repaired 0 in 0h2m with 0 errors on Mon Jun 5 15:24:35 2017
config:

NAME                                            STATE     READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
gptid/0ceb56e8-d22b-11e5-a3ff-fcaa14de34e8 ONLINE 0 0 0
gptid/0cffa35c-d22b-11e5-a3ff-fcaa14de34e8 ONLINE 0 0 0

errors: No known data errors

[root@stripes] ~# beadm list
BE Active Mountpoint Space Created Nickname
default - - 3.7M 2016-02-13 01:23 default
Initial-Install - - 3.0M 2016-02-13 02:26 Initial-Install
TrueNAS-TrueNAS-9.3-STABLE-201602032047 - - 296.5K 2016-02-13 12:14 TrueNAS-TrueNAS-9.3-STABLE-201602032047
TrueNAS-TrueNAS-9.3-STABLE-201604070138 - - 541.2M 2016-04-08 08:07 TrueNAS-TrueNAS-9.3-STABLE-201604070138
TrueNAS-9.3-STABLE-201604112122 - - 135.0M 2016-04-12 11:00 TrueNAS-9.3-STABLE-201604112122
TrueNAS-9.3-STABLE-201604202233 - - 119.1M 2016-04-21 07:01 TrueNAS-9.3-STABLE-201604202233
TrueNAS-9.3-STABLE-201605241903 - - 119.1M 2016-05-25 10:11 TrueNAS-9.3-STABLE-201605241903
TrueNAS-913bde210e8736026e696cffc47591e5 - - 402.1M 2016-07-05 09:48 9.10
9.10.1 - - 161.1M 2016-08-26 16:49 9.10.1
9.10.1-U2 - - 143.1M 2016-10-31 11:24 9.10.1-U2
9.10.1-U2.1 - - 369.5K 2016-11-22 02:18 9.10.1-U2.1
9_10_1_U2_1_Clone - - 487.0K 2016-12-27 00:57 9_10_1_U2_1_Clone
9_10_1_U2_1_fullcrashdumps - - 453.2M 2017-01-13 18:39 9_10_1_U2_1_fullcrashdumps
9.10.2-INTERNAL16-U1 - - 697.2M 2017-02-08 18:30 9.10.2-INTERNAL16-U1
9.10.2-U1 - - 697.3M 2017-02-14 22:11 9.10.2-U1
9.10.2-U2 - - 706.4M 2017-04-10 11:14 9.10.2-U2
9.10.2-U3 - - 704.1M 2017-05-02 16:11 9.10.2-U3
9.10.2-U4 NR / 5.9G 2017-06-01 13:28 9.10.2-U4

Thanks.

Associated revisions

Revision 4658b667 (diff)
Added by Nikola Gigic over 3 years ago

feat(gui): change default scrub interval for boot pool (#276) feat(gui): change default scrub interval for boot pool Ticket: #24371

Revision 188ff7d7 (diff)
Added by Nikola Gigic over 3 years ago

feat(gui): change default scrub interval for boot pool (#276) feat(gui): change default scrub interval for boot pool Ticket: #24371 (cherry picked from commit 4658b667b5aa54c1fda28928ac806e10c055b0d1)

Revision c5f3c7f1 (diff)
Added by Nikola Gigic over 3 years ago

feat(gui): change default scrub interval for boot pool Ticket: #24371

Revision 608863de (diff)
Added by Warren Block over 3 years ago

Update boot scrub interval. Ticket: #24371

Revision 189a9486 (diff)
Added by Warren Block over 3 years ago

Mention the new boot volume scrub interval. Ticket: #24371

Revision 78917167 (diff)
Added by Nikola Gigic over 3 years ago

feat(gui): change default scrub interval for boot pool (#276) feat(gui): change default scrub interval for boot pool Ticket: #24371

History

#1 Updated by William Grzybowski almost 4 years ago

  • Status changed from Unscreened to Screened
  • Target version set to TrueNAS 11.1-U1

#2 Updated by Adam Morales almost 4 years ago

Hi William,

We have been attempting to track sata dom failures with the switch to the new Apacer models and it would be really helpful to get this into a build as soon as possible.

Thanks,
Adam Morales
Director, Storage & Software Support

#3 Updated by William Grzybowski almost 4 years ago

Adam Morales wrote:

Hi William,

We have been attempting to track sata dom failures with the switch to the new Apacer models and it would be really helpful to get this into a build as soon as possible.

Thanks,
Adam Morales
Director, Storage & Software Support

I understand that but I currently have a bunch of much more urgent tickets to work on. I am afraid we have missed our window to get new features into 11.0.
If you really need that please bring that up to Kris so he can assign another person for this task.

Thanks,

#4 Avatar?id=14398&size=24x24 Updated by Kris Moore almost 4 years ago

  • Assignee changed from William Grzybowski to Suraj Ravichandran

Yea, 11.0 is already past the point of new features like this. I'm going to assign to Suraj for now. Im thinking maybe a drop down toggle to change the scrub between monthly / weekly will work.

#5 Updated by Adam Morales almost 4 years ago

@Kris, would it be more expedient to just change the value that we have set for the default boot pool scrub interval? Rather then any changes to the GUI..

Best,
Adam Morales

#6 Avatar?id=14398&size=24x24 Updated by Kris Moore almost 4 years ago

Sure thats an option, but I'll have to see if there's any reason why that could be considered a negative change to some. Give me a day or two and I'll get back if thats a good default or not.

#7 Avatar?id=14398&size=24x24 Updated by Kris Moore almost 4 years ago

  • Target version changed from TrueNAS 11.1-U1 to 11.0-U1

Ok, now that I've had some sleep ;)

This may be a bit more involved than just flipping a "days" integer. William can correct me if I'm wrong, but in src/freenas/usr/local/libexec/nas/scrub it appears to iterate through all the pools and kickoff scrubs. The default value of "35" days for the boot pool exists as a migration script setting this value in the DB, which in turn I believe requires us to write another migration script to set a new default? This late in the release cycle I'm a tad loathe to introduce any new code, especially something that the docs team will scream about it needing them to make last-second UI docs changes ;) I'll move this over to 11.0-U1 though, so we can get the fix in sooner than our next quarterly update.

However the UI in System -> Boot does let us tune down the number of days for boot-pool scrubs, we are free to use that to set a 7 day schedule for the time being.

#8 Updated by Cyber Jock almost 4 years ago

On my home grown FreeNAS machine I've been doing daily scrubs of my boot device and scrubs of my main zpools on the 1st and 15th of every month since 9.3 came out (this was when zfs-based boot devices were first used). So I'm quite perplexed by this...

~# zpool status
pool: freenas-boot
state: ONLINE
scan: scrub repaired 0 in 0h0m with 0 errors on Wed Jun 7 03:45:25 2017
config:
NAME        STATE     READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
ada0p2 ONLINE 0 0 0

errors: No known data errors

pool: tank
state: ONLINE
scan: scrub repaired 0 in 33h59m with 0 errors on Fri Jun 2 13:59:51 2017
config:
NAME                                            STATE     READ WRITE CKS                                                                                                                                                                                                                                             UM
tank ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
gptid/25c7f56c-3a81-11e7-b1b8-0cc47aa96998 ONLINE 0 0 0
gptid/be01fed1-2435-11e4-bde8-001b21633721 ONLINE 0 0 0
gptid/be0da381-2435-11e4-bde8-001b21633721 ONLINE 0 0 0
gptid/be1c4ffe-2435-11e4-bde8-001b21633721 ONLINE 0 0 0
gptid/be29e880-2435-11e4-bde8-001b21633721 ONLINE 0 0 0
gptid/be3758c8-2435-11e4-bde8-001b21633721 ONLINE 0 0 0
gptid/be45dd7f-2435-11e4-bde8-001b21633721 ONLINE 0 0 0
gptid/be539f95-2435-11e4-bde8-001b21633721 ONLINE 0 0 0
gptid/be7130bb-2435-11e4-bde8-001b21633721 ONLINE 0 0 0
gptid/be61ff8d-2435-11e4-bde8-001b21633721 ONLINE 0 0 0

errors: No known data errors

pool: tank2
state: ONLINE
scan: scrub repaired 0 in 5h7m with 0 errors on Tue May 16 20:34:58 2017
config:
NAME                                            STATE     READ WRITE CKS                                                                                                                                                                                                                                             UM
tank2 ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
gptid/3824e311-2be0-11e7-812e-0cc47aa96998 ONLINE 0 0 0
gptid/5cc2f5b8-a540-11e6-a46c-0cc47aa96998 ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
gptid/4cefbdc5-2d25-11e7-a68f-0cc47aa96998 ONLINE 0 0 0
gptid/5feb075c-a540-11e6-a46c-0cc47aa96998 ONLINE 0 0 0
logs
gptid/e9e3a5cf-c4d3-11e6-8865-0cc47aa96998 ONLINE 0 0 0
cache
gptid/96e9ffdb-c4d3-11e6-8865-0cc47aa96998 ONLINE 0 0 0
gptid/9763c32d-c4d3-11e6-8865-0cc47aa96998 ONLINE 0 0 0
gptid/97de41ab-c4d3-11e6-8865-0cc47aa96998 ONLINE 0 0 0

errors: No known data errors

#9 Updated by Suraj Ravichandran almost 4 years ago

@Kris when is 11.0-U1, given my current status I dunno if I shall be back before that.

#10 Avatar?id=14398&size=24x24 Updated by Kris Moore almost 4 years ago

@suraj - no worries. We don't have a specific ETA for -U1 yet, if you aren't back by then we can reassign it.

#11 Updated by Suraj Ravichandran almost 4 years ago

Thanks

#12 Updated by Suraj Ravichandran almost 4 years ago

  • Project changed from TrueNAS to FreeNAS
  • Category changed from 14 to 55
  • Target version changed from 11.0-U1 to 11.0-U1

I am moving this ticket to FreeNAS, if anyone has any objections please inform here.

Also updating category to Reporting (could not think of anything better)

Thanks

#13 Updated by Vaibhav Chauhan almost 4 years ago

  • Target version changed from 11.0-U1 to 11.0-U2

#14 Updated by Vaibhav Chauhan almost 4 years ago

  • Target version changed from 11.0-U2 to 11.0-U3

#15 Avatar?id=14398&size=24x24 Updated by Kris Moore over 3 years ago

  • Status changed from Screened to 46

Suraj - Any update here?

#16 Updated by Suraj Ravichandran over 3 years ago

  • Status changed from 46 to Screened

Will get this in by U3

#17 Updated by William Grzybowski over 3 years ago

  • Status changed from Screened to Unscreened
  • Assignee changed from Suraj Ravichandran to Nikola Gigic

#18 Updated by Nikola Gigic over 3 years ago

  • Status changed from Unscreened to Screened

#19 Updated by Nikola Gigic over 3 years ago

  • Status changed from Screened to Ready For Release

#20 Updated by Nikola Gigic over 3 years ago

  • Status changed from Ready For Release to Needs Developer Review
  • Assignee changed from Nikola Gigic to William Grzybowski

#21 Updated by William Grzybowski over 3 years ago

  • Status changed from Needs Developer Review to Reviewed by Developer
  • Assignee changed from William Grzybowski to Nikola Gigic

#22 Updated by Vaibhav Chauhan over 3 years ago

Please create a PR against freenas/11.0-stable branch so that I can merge changes in for internal 11.0-U3 iso.

#24 Updated by Vaibhav Chauhan over 3 years ago

  • Status changed from Reviewed by Developer to 47

#25 Updated by Dru Lavigne over 3 years ago

  • Subject changed from Change scrub interval from the current default (30 days?) to something like 7 days or shorter to Change default scrub interval to 7 days

#26 Updated by Bonnie Follweiler over 3 years ago

12254

#27 Updated by Bonnie Follweiler over 3 years ago

12255

I was looking in the wrong place. It was the Automatic scrub interval (in days)

#28 Updated by Warren Block over 3 years ago

  • Subject changed from Change default scrub interval to 7 days to Change default boot scrub interval to 7 days

#29 Updated by Dru Lavigne over 3 years ago

  • Status changed from Ready For Release to Resolved

Also available in: Atom PDF