Project

General

Profile

Bug #23552

Re-enable hardware watchdog for newer firmware versions of AsrockRack C2750D4I

Added by Grzegorz Krzystek over 1 year ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Nice to have
Assignee:
Vladimir Vinogradenko
Category:
OS
Target version:
Seen in:
Severity:
New
Reason for Closing:
Reason for Blocked:
Needs QA:
No
Needs Doc:
Yes
Needs Merging:
Yes
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

Bug #16190 we switched watchdogd to report only mode, that not arm hardware watchdog due to bug in AsrockRack C2750D4I IPMI/BMC Implementation BUG Causing lot of writes to BMC Flash, making BMC unstable(FreeNAS Mini/Mini XL and all users using same board).
This BMC bug have been solved with 00.30.00 BMC Firmware version.
In general hardware watchdog should be armed to reboot system in case of system Freeze (kernel panic, or hardfreeze)
As ipmi/BMC no longer affected we can:
totally rollback change (this implies arm Watchdog for all users) or, create tunable like "ARM_HARDWARE_WATCHDOG" or even create UI switch to give user choice: enable watchdog yes/no , arm hardware watchdog if exist on board: yes/no
IMHO watchdog if exist should be armed.

Associated revisions

Revision bd421649 (diff)
Added by Vladimir Vinogradenko about 1 year ago

fix(watchdog): Re-enable hardware watchdog for never firmware versions on AsrockRack C2750D4I

This BMC bug have been solved with 00.30.00 BMC Firmware version.

Ticket: #23552

Revision 49ffe50a (diff)
Added by Vladimir Vinogradenko about 1 year ago

fix(watchdog): Re-enable hardware watchdog for never firmware versions on AsrockRack C2750D4I

This BMC bug have been solved with 00.30.00 BMC Firmware version.

Ticket: #23552

Revision be9580a9 (diff)
Added by Dru Lavigne about 1 year ago

Mention watchdog reenable.
Ticket: #23552

History

#1 Updated by Alexander Motin over 1 year ago

  • Status changed from Unscreened to Screened

#2 Updated by Dru Lavigne over 1 year ago

  • Status changed from Screened to 46

Sasha: is this still an issue or can this bug be closed?

#3 Updated by Alexander Motin over 1 year ago

It would be trivial to revert the change, but I am not very confident about it, since I haven't participated closely in original problem. If somebody have more confidence about it -- be my guest to take it.

#4 Updated by Dru Lavigne over 1 year ago

  • Assignee changed from Alexander Motin to Kris Moore

Kris: what are your thoughts on this one?

#5 Avatar?id=14398&size=24x24 Updated by Kris Moore over 1 year ago

  • Status changed from 46 to Unscreened
  • Assignee changed from Kris Moore to William Grzybowski
  • Target version set to 11.1

I'm going to go with lets expose this to the end-user somewhere, since I think its at least conceivable that down the road somebody may want to enable or disable the hardware watchdog for $reasons.

Here's the recent commits for this behavior:

https://github.com/freenas/freenas/commit/d51deaebe3ac5d44c13be4a4011ed015c9d33461
https://github.com/freenas/freenas/commit/bc89ce14b9507b8e97298a7797affccb9534d016

William, if you agree this is safe to do, feel free to re-assign to Vlad or Nikola.

#6 Avatar?id=14398&size=24x24 Updated by Kris Moore over 1 year ago

  • Priority changed from No priority to Nice to have

#7 Updated by William Grzybowski about 1 year ago

  • Assignee changed from William Grzybowski to Vladimir Vinogradenko

Vladimir, can you investigate this?

#8 Updated by Vladimir Vinogradenko about 1 year ago

  • Status changed from Unscreened to 15
  • Assignee changed from Vladimir Vinogradenko to Kris Moore

We may have hardware watchdog disabled by default right after installation (in case of any further hardware errors) and ask user to enable it during wizard.

#9 Updated by Dru Lavigne about 1 year ago

  • Status changed from 15 to Investigation
  • Assignee changed from Kris Moore to Vladimir Vinogradenko

#10 Updated by Vladimir Vinogradenko about 1 year ago

  • Status changed from Investigation to 15
  • Assignee changed from Vladimir Vinogradenko to William Grzybowski

#11 Updated by William Grzybowski about 1 year ago

  • Assignee changed from William Grzybowski to Vladimir Vinogradenko

I dont like the idea of doing that. Most likely people wont even bother. Very few people actually run the wizard.

Lets just re-enable watchdogd for asrock motherboards that have been updated, something alone these lines:

--- a/src/freenas/etc/rc.conf.local
+++ b/src/freenas/etc/rc.conf.local
@@ -743,7 +743,7 @@ _gen_conf()
                        product=`/usr/local/sbin/dmidecode -s baseboard-product-name | /usr/bin/head -n 1`
                        if [ "${product}" = "X9DR3-F" -o "${product}" = "X9DR3-LN4F+" ] ; then
                                echo "watchdogd_enable=\"NO\"" 
-                       elif [ "${product}" = "C2750D4I" -o "${product}" = "C2550D4I" ] ; then
+                       elif [ "${product}" = "C2750D4I" -o "${product}" = "C2550D4I" ] && [ "$(/usr/local/bin/ipmitool mc info 2> /dev/null | grep ^Firmware | cut -d . -f 2)" -lt 30 ]; then
                                echo "watchdogd_flags=\"-t 30 --softtimeout --softtimeout-action log,printf --pretimeout 15 --pretimeout-action log,printf -e 'sleep 1' -w -T 3\"" 
                                echo "watchdogd_enable=\"YES\"" 
                        else

#12 Updated by Vladimir Vinogradenko about 1 year ago

  • Assignee changed from Vladimir Vinogradenko to William Grzybowski

Ok. GUI option still needed?

#13 Updated by William Grzybowski about 1 year ago

  • Assignee changed from William Grzybowski to Vladimir Vinogradenko

No, lets defer on that for now.

#14 Updated by Grzegorz Krzystek about 1 year ago

Vladimir Pustosmekhov wrote:

Ok. GUI option still needed?

IMHO watchdogd should be treated as normal system service, configurable.
so on/off switch , and optionally editing run-time flags should be added.
ASPEED IPMI bug is not only specific to asrockrack boards... probably some users will need to be able switch it off, or set to softmode.

#15 Updated by William Grzybowski about 1 year ago

Grzegorz Krzystek wrote:

Vladimir Pustosmekhov wrote:

Ok. GUI option still needed?

IMHO watchdogd should be treated as normal system service, configurable.
so on/off switch , and optionally editing run-time flags should be added.
ASPEED IPMI bug is not only specific to asrockrack boards... probably some users will need to be able switch it off, or set to softmode.

We see no need for that yet, complicating UI for little to no use. watchdog can still be configured/disabled via Tunables.

#16 Updated by Vladimir Vinogradenko about 1 year ago

  • Status changed from 15 to Needs Developer Review
  • Assignee changed from Vladimir Vinogradenko to William Grzybowski

#17 Updated by William Grzybowski about 1 year ago

  • Status changed from Needs Developer Review to Reviewed by Developer
  • Assignee changed from William Grzybowski to Vladimir Vinogradenko

#18 Updated by Vladimir Vinogradenko about 1 year ago

  • Status changed from Reviewed by Developer to Ready For Release

#19 Updated by Dru Lavigne about 1 year ago

  • Subject changed from Revert changes from Bug #16190 to reenable hardware watchdog to Re-enable hardware watchdog for newer firmware versions of AsrockRack C2750D4I
  • Target version changed from 11.1 to 11.1-BETA1

#20 Updated by Dru Lavigne about 1 year ago

  • Status changed from Ready For Release to Resolved

#21 Updated by Joe Maloney about 1 year ago

  • Needs QA changed from Yes to No
  • QA Status Test Passes FreeNAS added
  • QA Status deleted (Not Tested)

Also available in: Atom PDF