Project

General

Profile

Bug #25728

FreeNAS shuts down while attempting scrub

Added by Michael Franchi about 3 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
No priority
Assignee:
Alexander Motin
Category:
OS
Target version:
Seen in:
Severity:
Reason for Closing:
Reason for Blocked:
Needs QA:
Yes
Needs Doc:
Yes
Needs Merging:
Yes
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:

CPU: AMD A10-6800K APU
Memory: 16265MB

ChangeLog Required:
No

Description

FreeNAS shuts down while attempting to scrub the main ZFS.

Appears to be repetitive every 30min shutting down the pc. On turning back on everything is ok and scrub continues until finished or shutting down if ~30min is reached.

History

#1 Updated by Michael Franchi about 3 years ago

  • File debug-freenas-20170830195115.txz added

#2 Updated by Dru Lavigne about 3 years ago

  • Assignee changed from Release Council to Alexander Motin
  • Seen in changed from Unspecified to 11.0-U2

#3 Updated by Alexander Motin about 3 years ago

  • Status changed from Unscreened to 15
  • Target version set to 11.1

Could you tell more about "shuts down"? Do you mean reboots? Is there any messages on console before that? In your debug data I see only one kernel dump from this year, and it is from June.

#4 Updated by Michael Franchi about 3 years ago

  • Hardware Configuration updated (diff)

I'm sorry I'm not the most knowledgeable with BSD - But to me it acts like FreeNAS is just shutting down completely, not restarting, off until I power on again. Most notable when scrubs are occurring and under heavier load than normal. Haven't had it occur during normal use accessing data, streaming, etc.

These are the only console entries between the last shutdown/boot, which was 10 minutes apart:


Aug 30 19:49:00 freenas uwsgi: [sentry.errors:599] Sentry responded with an error: <urlopen error timed out> (url: https://sentry.ixsystems.com/api/2/store/)
{'culprit': 'freenasUI.support.views in ticket_progress',
 'event_id': '966a4f5dcdde4eacb0d90ae3c5aacc7f',
 'exception': {'values': [{'module': 'builtins',
                           'stacktrace': {'frames': [{'abs_path': '/usr/local/lib/python3.6/site-packages/django/core/handlers/exception.py',
                                                      'context_line': '            '
                                                                      'response '
                                                                      '= '
                                                                      'get_response(request)',
                                                      'filename': 'django/core/handlers/exception.py',
                                                      'function': 'inner',

Aug 30 19:49:00 freenas uwsgi:                                             'lineno': 39,
                                                      'module': 'django.core.handlers.exception',
                                                      'post_context': ['        '
                                                                       'except '
                                                                       'Exception '
                                                                       'as '
                                                                       'exc:',
                                                                       '            '
                                                                       'response '
                                                                       '= '
                                                                       'response_for_exception(request, '

Aug 30 19:49:00 freenas uwsgi:                                'exc)',
                                                                       '        '
                                                                       'return '
                                                                       'response',
                                                                       '    '
                                                                       'return '
                                                                       'inner',
                                                                       ''],
                                                      'pre_context': ['    can '
                                                                      'rely on '
                                                                      'getting '
                                                                      'a '

Aug 30 19:49:00 freenas uwsgi:                                       'response '
                                                                      'instead '
                                                                      'of an '
                                                                      'exception.',
                                                                      '    """',
                                                                      '    '
                                                                      '@wraps(get_response, '
                                                                      'assigned=available_attrs(get_response))',
                                                                      '    def '
                                                                      'inner(request):',
                                                                      '        '

Aug 30 19:49:00 freenas uwsgi:                               'try:'],
                                                      'vars': {'exc': 'FileNotFoundError(2, '
                                                                      "'No " 
                                                                      'such '
                                                                      'file or '
                                                                      "directory')",
                                                               'get_response': '<bound '
                                                                               'method '
                                                                               'BaseHandler._legacy_get_response '
                                                                               'of '
                                                                               '<django.core.handlers.wsgi.WSGIHandler
Aug 30 19:49:00 freenas uwsgi:                                                                        '    '
                                                                       'return '
                                                                       'HttpResponse(json.dumps(data), '
                                                                       "content_type='application/json')"],
                                                      'pre_context': ['',
                                                                      '    '
                                                                      'return '
                                                                      'HttpResponse(json.dumps(data), '
                                                                      "content_type='application/json')",
                                                                      '',

Aug 30 19:49:00 freenas uwsgi: it__: out = \n'
                        'Aug 30 19:44:30 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = \n'
                        'Aug 30 19:44:30 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = id: 10\n'
                        'Aug 30 19:44:30 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = host: transmission_1\n'
                        'Aug 30 19:44:30 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = iface: \n'
                        'Aug 30 19:44:30 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = ipv4: 192.168.1.242/24\n'
                        'Aug 30 19:44:30 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = alias-ipv4: \n'
                        'Aug 30 19:44:30 freenas uwsgi: [common.cmd:208] '

Aug 30 19:49:00 freenas uwsgi:       'cmd_pipe.__init__: out = bridge-ipv4: \n'
                        'Aug 30 19:44:30 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = alias-bridge-ipv4: \n'
                        'Aug 30 19:44:30 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = defaultrouter-ipv4: \n'
                        'Aug 30 19:44:30 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = ipv6: \n'
                        'Aug 30 19:44:30 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = alias-ipv6: \n'
                        'Aug 30 19:44:30 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = bridge-ipv6: \n'
                        'Aug 30 19:44:30 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = alias-bridge-ipv6: \n'
                        'Aug 30 19:44:
Aug 30 19:49:00 freenas uwsgi:  pluginjail\n'
                        'Aug 30 19:44:30 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = flags: '
                        'allow.raw_sockets=true\n'
                        'Aug 30 19:44:30 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = \n'
                        'Aug 30 19:44:30 freenas uwsgi: [common.cmd:219] '
                        'cmd_pipe.__init__: leave\n'
                        'Aug 30 19:44:30 freenas uwsgi: [common.warden:463] '
                        'warden_base.run: leave\n'
                        'Aug 30 19:44:30 freenas uwsgi: '
                        '[middleware.notifier:197] Popen()ing: route -nv show '
                        "default|grep 'interface:'|awk '{ print $2 }'\n" 
                        'Aug 30 19:44:30 freenas uwsgi: '
                        '[middleware.notifier:197] Popen()ing: route -nv show '

Aug 30 19:49:00 freenas uwsgi:          "-inet6 default|grep 'interface:'|awk '{ print $2 }'\n" 
                        'Aug 30 19:44:31 freenas uwsgi: '
                        '[freeadmin.navtree:114] Unable to import '
                        "'freenasUI.support' 'nav': No module named " 
                        "'freenasUI.support.nav'\n" 
                        'Aug 30 19:44:31 freenas uwsgi: '
                        '[freeadmin.navtree:402] App freenasUI.support has no '
                        'nav.py module, skipping\n'
                        'Aug 30 19:44:31 freenas uwsgi: '
                        '[middleware.notifier:197] Popen()ing: route -nv show '
                        "default|grep 'interface:'|awk '{ print $2 }'\n" 
                        'Aug 30 19:44:31 freenas uwsgi: '
                        '[middleware.notifier:197] Popen()ing: route -nv show '
                        "default|grep 'interface:'|awk '{ print $2 }'\n" 
                        '
Aug 30 19:49:00 freenas uwsgi: Aug 30 19:44:31 freenas uwsgi: '
                        '[middleware.notifier:197] Popen()ing: route -nv show '
                        "-inet6 default|grep 'interface:'|awk '{ print $2 }'\n" 
                        'Aug 30 19:44:31 freenas uwsgi: '
                        '[middleware.notifier:197] Popen()ing: route -nv show '
                        "-inet6 default|grep 'interface:'|awk '{ print $2 }'\n" 
                        'Aug 30 19:44:31 freenas uwsgi: '
                        '[middleware.notifier:197] Popen()ing: route -nv show '
                        "default|grep 'interface:'|awk '{ print $2 }'\n" 
                        'Aug 30 19:44:31 freenas uwsgi: '
                        "[middleware.notifier:197] Popen()ing: ifconfig 'em0'\n" 
                        'Aug 30 19:44:31 freenas uwsgi: '
                        '[middleware.notifier:197] Popen()ing: route -nv show '
                        "default|grep 'interface:'|
Aug 30 19:49:00 freenas uwsgi: ave\n'
                        'Aug 30 19:44:32 freenas uwsgi: [common.warden:445] '
                        'warden_base.run: enter\n'
                        'Aug 30 19:44:32 freenas uwsgi: [common.warden:454] '
                        'warden_base.cmd = /usr/local/bin/warden list  -v\n'
                        'Aug 30 19:44:32 freenas uwsgi: [common.cmd:189] '
                        'cmd_pipe.__init__: cmd = /usr/local/bin/warden list  '
                        "-v, kwargs = {'pipeopen_kwargs': {}}\n" 
                        'Aug 30 19:44:33 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = \n'
                        'Aug 30 19:44:33 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = id: 13\n'
                        'Aug 30 19:44:33 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = host: plexmediaserver_1\n'

Aug 30 19:49:00 freenas uwsgi:  'Aug 30 19:44:33 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = iface: \n'
                        'Aug 30 19:44:33 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = ipv4: 192.168.1.240/24\n'
                        'Aug 30 19:44:33 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = alias-ipv4: \n'
                        'Aug 30 19:44:33 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = bridge-ipv4: \n'
                        'Aug 30 19:44:33 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = alias-bridge-ipv4: \n'
                        'Aug 30 19:44:33 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = defaultrouter-ipv4: \n'
                        'Aug 30 19:44:33 freenas uwsgi: [common.cmd:208] '
                        'cmd_pi
Aug 30 19:49:00 freenas uwsgi: gi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = nat: Disabled\n'
                        'Aug 30 19:44:33 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = mac: 8e:94:9c:fa:31:46\n'
                        'Aug 30 19:44:33 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = status: Running\n'
                        'Aug 30 19:44:33 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = type: pluginjail\n'
                        'Aug 30 19:44:33 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = flags: '
                        'allow.raw_sockets=true\n'
                        'Aug 30 19:44:33 freenas uwsgi: [common.cmd:208] '
                        'cmd_pipe.__init__: out = \n'
                        'Aug 30 19:44:33 freenas uwsgi: [common.cmd:208] '
                        'cmd_pi
Aug 30 19:49:00 freenas uwsgi: [sentry.errors.uncaught:621] ["FileNotFoundError: [Errno 2] No such file or directory: '/tmp/.ticketprogress'", '  File "django/core/handlers/exception.py", line 39, in inner', '  File "django/core/handlers/base.py", line 249, in _legacy_get_response', '  File "django/core/handlers/base.py", line 178, in _get_response', '  File "freenasUI/freeadmin/middleware.py", line 162, in process_view', '  File "django/contrib/auth/decorators.py", line 23, in _wrapped_view', '  File "freenasUI/support/views.py", line 259, in ticket_progress']

#5 Updated by Alexander Motin about 3 years ago

If system shuts down completely, then it must be hardware. FreeNAS should never shut down by itself. Check cooling fans, replace thermal interface on your CPU cooler, check power supply.

#6 Updated by Dru Lavigne about 3 years ago

  • File deleted (debug-freenas-20170830195115.txz)

#7 Updated by Dru Lavigne about 3 years ago

  • Status changed from 15 to Closed: Not Applicable
  • Target version changed from 11.1 to N/A
  • Private changed from Yes to No

#8 Avatar?id=14398&size=24x24 Updated by Kris Moore over 2 years ago

  • Status changed from Closed: Not Applicable to Closed

Also available in: Atom PDF