Project

General

Profile

Bug #20230

Clarify scrub schedule fields in Guide

Added by john john almost 2 years ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Important
Assignee:
Warren Block
Category:
Middleware
Target version:
Seen in:
Severity:
New
Reason for Closing:
Reason for Blocked:
Needs QA:
No
Needs Doc:
Yes
Needs Merging:
Yes
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:

HP Proliant Gen8 Microserver
16 GB RAM ECC
2 x 4 TB mirrored
2 x 8 TB mirrored

ChangeLog Required:
No

Description

I have 2 FreeNAS machines that don't follow their scrub schedule. They have been set to do scrubs every 1st and 15th of the month at 1pm with treshold days 10. They are running every 10 days, but not on the set days of month.

An excerpt from crontab:

00 01 01,15 * 1,3,4,5,6,7 root PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/root/bin" /usr/local/libexec/nas/scrub -t 10 DATA

The machine with the crontab excerpt has been updated to 9.10.1-U4 and now to 9.10.2 and the problem persists. I've more details in a forum post https://forums.freenas.org/index.php?threads/scrub-schedule-not-running-properly.48719/#post-339291 .

Associated revisions

Revision a4a8e3d1 (diff)
Added by Warren Block about 1 year ago

Clarify scrub schedule fields

Ticket: #20230

History

#1 Updated by Bonnie Follweiler almost 2 years ago

  • Assignee set to Kris Moore

#3 Updated by Bonnie Follweiler almost 2 years ago

Would you please provide a debug?

In System -> Advanced -> Save Debug

#4 Avatar?id=14398&size=24x24 Updated by Kris Moore almost 2 years ago

  • Assignee changed from William Grzybowski to Suraj Ravichandran

Lets have Suraj take a look next week if possible.

#6 Avatar?id=14398&size=24x24 Updated by Kris Moore almost 2 years ago

  • Subject changed from Scrubs don't follow schedule to [regression] Scrubs don't follow schedule
  • Assignee changed from Kris Moore to William Grzybowski
  • Priority changed from No priority to Important
  • Target version set to 9.10.2-U2

#7 Updated by john john almost 2 years ago

Bonnie Follweiler wrote:

Would you please provide a debug?

In System -> Advanced -> Save Debug

Would you like a debug right now, or during the next scrub?

#8 Avatar?id=14398&size=24x24 Updated by Kris Moore almost 2 years ago

It should have enough historical data to be good, now is fine. Thanks!

#9 Updated by john john almost 2 years ago

Kris Moore wrote:

It should have enough historical data to be good, now is fine. Thanks!

I'm sorry but there's too much personal info in the debug file and I'm uncomfortable with posting it. Can I look for something specific?

Also, I've a got 4 rsync Cron Job (not Rsync tasks) that copy some folders from one dataset to another. They were supposed to start every 5th of the month and I noticed one of them ran at some point between the 10th and today.

#10 Updated by Suraj Ravichandran almost 2 years ago

  • Status changed from Unscreened to Screened

This can be related to 20052, let me see what I can do next week

#11 Updated by john john almost 2 years ago

I rescheduled the scrubs. One for yesterday, one for today at different times, 13:33 and 13:36 with treshold days 1. They ran normally. I initially tried to reschedule just the time but they wouldn't run. I suspect because the 10 day treshold hadn't past yet from the last scrubs.

I did the same with the second unit. It had the same schedule, on the 1st and 15th, but it last ran on the 13th. I rescheduled it for today at 13:45 with treshold days 1 and it worked.

I'm scheduling another scrub on the first unit, with treshold days 3, start time 1am, run every, 1st, 15th and 20th. I'll report back with results.

#12 Updated by john john almost 2 years ago

Short update.

I've tried several combinations but dates are always ignored. I've also noticed that my rsync cron jobs which were set to run once a month on the 5th are running daily at 12am. I've disabled those but it doesn't affect the scrub bug. For now, I'm disabling scrubs and cron jobs. I can send the debug file if it's via forum pm or e-mail.

#13 Updated by Suraj Ravichandran almost 2 years ago

John Hennessy john: Thanks for the update I will try to look into this soon (currently triaging other critical bugs)

#14 Avatar?id=14398&size=24x24 Updated by Kris Moore almost 2 years ago

  • Target version changed from 9.10.2-U2 to 9.10.3

#15 Updated by john john almost 2 years ago

Can you suggest a workaround?

#16 Avatar?id=14398&size=24x24 Updated by Kris Moore over 1 year ago

  • Target version changed from 9.10.3 to 9.10.4

#17 Updated by Suraj Ravichandran over 1 year ago

  • Status changed from Screened to Unscreened
  • Assignee changed from Suraj Ravichandran to William Grzybowski

John Hennessy john I am sorry for not being able to get to this in time, I apologize for my absence.

I am going to assign it to my senior who should hopefully get this resolved for you quickly.

#18 Updated by john john over 1 year ago

Suraj Ravichandran wrote:

John Hennessy john I am sorry for not being able to get to this in time, I apologize for my absence.

I am going to assign it to my senior who should hopefully get this resolved for you quickly.

I've upgraded to FreeNAS 10. If everything's ok, I'll stick to it.

One last piece of information that might help:

I've disabled the cron jobs and didn't run them for over 1 month (I ran them manually). Then, on let's say 10th I've enabled the cron job. WIth the bug, it would start the first day+hour cycle, but it ran on schedule on the 15th instead. After the 15th it started on the 25th (treshold days 10 ) instead of the 1st the next month.

#19 Updated by William Grzybowski over 1 year ago

  • Status changed from Unscreened to Screened

#20 Updated by Shaun Currier over 1 year ago

+1, I'm also experiencing something like this as described here: https://forums.freenas.org/index.php?threads/scheduled-scrub-didnt-happen.52219/#post-361049

#21 Avatar?id=14398&size=24x24 Updated by Kris Moore over 1 year ago

  • Target version changed from 9.10.4 to 11.1

#22 Updated by john john over 1 year ago

  • Hardware Configuration updated (diff)

#23 Updated by Joe Schmuck over 1 year ago

Here is an update about the scrub schedules. The issue isn't the scrub, it's the use of crontab.

The "day of week" variable is an "or" statement, not an "and" statement. This means that if you set it to specific days it will run the cron job. HolyK discovered this yesterday while reading through the cron manual. He came up with a unique way to still achieve his goal of running only on Monday and Tuesday during specific days, which I think the developers should look at and consider incorporating. Here is the link to the thread. https://forums.freenas.org/index.php?threads/how-to-fix-scrub-schedule.54678/

This comes right from the crontab(5) page:

For example, ``30 4 1,15 * 5'' would cause a command to be run at
     4:30 am on    the 1st    and 15th of each month, plus every Friday.

Also, if there are no changes to the operation of this setup then the GUI should be changed to add the "OR" condition for the week of day check boxes and explained clearly in the user manual.

Lastly, this would affect any cron job where someone does not choose all the days of the week.

#24 Updated by William Grzybowski over 1 year ago

Thanks for the insight, I'll take a look as soon as I have a chance but might be a while.

#25 Updated by William Grzybowski over 1 year ago

  • Subject changed from [regression] Scrubs don't follow schedule to Scrubs don't follow schedule

#26 Avatar?id=14398&size=24x24 Updated by Kris Moore over 1 year ago

  • Status changed from Screened to Unscreened
  • Assignee changed from William Grzybowski to Vladimir Vinogradenko

#27 Updated by Vladimir Vinogradenko about 1 year ago

  • Status changed from Unscreened to Screened

#28 Updated by Vladimir Vinogradenko about 1 year ago

  • Status changed from Screened to 15
  • Assignee changed from Vladimir Vinogradenko to William Grzybowski

So this is documentation issue: we only need to document that «day of week» works as OR instead of AND. Making workarounds to make it work as AND probably does not worth it (like with https://bugs.freenas.org/issues/24084).

#29 Updated by Dru Lavigne about 1 year ago

  • Status changed from 15 to Unscreened
  • Assignee changed from William Grzybowski to Warren Block

#30 Updated by Warren Block about 1 year ago

  • Status changed from Unscreened to Screened

#31 Updated by Alex B about 1 year ago

Yes, this is more like a documentation thing. Also it would be nice to have this visible directly in the schedule UI. Just a note about the day of week behavior with reference to documentation for more info.

And yea i agree that making any workaround is just highway to hell. Let's stick with the standard CRON and make users to understand its behavior.

#32 Updated by Dru Lavigne about 1 year ago

  • Status changed from Screened to Resolved
  • Target version changed from 11.1 to 11.1-BETA1

#33 Updated by Dru Lavigne about 1 year ago

  • Subject changed from Scrubs don't follow schedule to Clarify scrub schedule fields in Guide

#34 Updated by Bonnie Follweiler about 1 year ago

  • Needs QA changed from Yes to No
  • QA Status Test Passes FreeNAS added
  • QA Status deleted (Not Tested)

Also available in: Atom PDF