Project

General

Profile

Bug #69159

Increase SQLite retry timeout to prevent locking on slow USB devices

Added by Bug Clerk 5 months ago. Updated 5 months ago.

Status:
Done
Priority:
No priority
Assignee:
Vladimir Vinogradenko
Category:
Middleware
Target version:
Seen in:
Severity:
New
Reason for Closing:
Reason for Blocked:
Needs QA:
No
Needs Doc:
No
Needs Merging:
No
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

I got the following two Alerts in an a couple of emails. When asked about them in the FreeNAS forums I was told to file a bug report.... so here it is.

*
First Alert
*++
New alerts:
  • Unable to run alert source 'Replication'
    Traceback (most recent call last):
    File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
    File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 412, in execute
    execute = self.locked_retry(Database.Cursor.execute, query, params)
    File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 403, in locked_retry
    raise e
    File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 389, in locked_retry
    rv = method(self, *args, **kwargs)
    sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/alert.py", line 358, in run_source
alerts = (await alert_source.check()) or []
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/../alert/source/replication.py", line 10, in check
for replication in await self.middleware.call("replication.query", [["enabled", "=", True]]):
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1113, in call
return await self._call(name, serviceobj, methodobj, params, app=app, pipes=pipes, io_thread=True)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1049, in _call
return await methodobj(*args)
File "/usr/local/lib/python3.6/site-packages/middlewared/schema.py", line 664, in nf
return await f(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/service.py", line 266, in query
'datastore.query', self._config.datastore, [], datastore_options
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1113, in call
return await self._call(name, serviceobj, methodobj, params, app=app, pipes=pipes, io_thread=True)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1063, in _call
return await run_method(methodobj, *args)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1009, in run_in_thread
return await self.loop.run_in_executor(executor, functools.partial(method, *args, **kwargs))
File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/schema.py", line 668, in nf
return f(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/datastore.py", line 175, in query
select=options.get('select'),
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/datastore.py", line 88, in __queryset_serialize
for i in qs:
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 256, in __iter

self._fetch_all()
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 1087, in _fetch_all
self._result_cache = list(self.iterator())
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 54, in iter
results = compiler.execute_sql()
File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 835, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 94, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 412, in execute
execute = self.locked_retry(Database.Cursor.execute, query, params)
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 403, in locked_retry
raise e
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 389, in locked_retry
rv = method(self, *args, **kwargs)
django.db.utils.OperationalError: database is locked

Alerts:
  • Unable to run alert source 'Replication'
    Traceback (most recent call last):
    File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
    File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 412, in execute
    execute = self.locked_retry(Database.Cursor.execute, query, params)
    File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 403, in locked_retry
    raise e
    File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 389, in locked_retry
    rv = method(self, *args, **kwargs)
    sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/alert.py", line 358, in run_source
alerts = (await alert_source.check()) or []
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/../alert/source/replication.py", line 10, in check
for replication in await self.middleware.call("replication.query", [["enabled", "=", True]]):
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1113, in call
return await self._call(name, serviceobj, methodobj, params, app=app, pipes=pipes, io_thread=True)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1049, in _call
return await methodobj(*args)
File "/usr/local/lib/python3.6/site-packages/middlewared/schema.py", line 664, in nf
return await f(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/service.py", line 266, in query
'datastore.query', self._config.datastore, [], datastore_options
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1113, in call
return await self._call(name, serviceobj, methodobj, params, app=app, pipes=pipes, io_thread=True)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1063, in _call
return await run_method(methodobj, *args)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1009, in run_in_thread
return await self.loop.run_in_executor(executor, functools.partial(method, *args, **kwargs))
File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/schema.py", line 668, in nf
return f(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/datastore.py", line 175, in query
select=options.get('select'),
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/datastore.py", line 88, in __queryset_serialize
for i in qs:
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 256, in __iter

self._fetch_all()
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 1087, in _fetch_all
self._result_cache = list(self.iterator())
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 54, in iter
results = compiler.execute_sql()
File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 835, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 94, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 412, in execute
execute = self.locked_retry(Database.Cursor.execute, query, params)
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 403, in locked_retry
raise e
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 389, in locked_retry
rv = method(self, *args, **kwargs)
django.db.utils.OperationalError: database is locked

Second email:

Gone alerts:
  • Unable to run alert source 'Replication'
    Traceback (most recent call last):
    File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
    File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 412, in execute
    execute = self.locked_retry(Database.Cursor.execute, query, params)
    File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 403, in locked_retry
    raise e
    File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 389, in locked_retry
    rv = method(self, *args, **kwargs)
    sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/alert.py", line 358, in run_source
alerts = (await alert_source.check()) or []
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/../alert/source/replication.py", line 10, in check
for replication in await self.middleware.call("replication.query", [["enabled", "=", True]]):
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1113, in call
return await self._call(name, serviceobj, methodobj, params, app=app, pipes=pipes, io_thread=True)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1049, in _call
return await methodobj(*args)
File "/usr/local/lib/python3.6/site-packages/middlewared/schema.py", line 664, in nf
return await f(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/service.py", line 266, in query
'datastore.query', self._config.datastore, [], datastore_options
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1113, in call
return await self._call(name, serviceobj, methodobj, params, app=app, pipes=pipes, io_thread=True)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1063, in _call
return await run_method(methodobj, *args)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1009, in run_in_thread
return await self.loop.run_in_executor(executor, functools.partial(method, *args, **kwargs))
File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/schema.py", line 668, in nf
return f(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/datastore.py", line 175, in query
select=options.get('select'),
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/datastore.py", line 88, in __queryset_serialize
for i in qs:
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 256, in __iter

self._fetch_all()
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 1087, in _fetch_all
self._result_cache = list(self.iterator())
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 54, in iter
results = compiler.execute_sql()
File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 835, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 94, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 412, in execute
execute = self.locked_retry(Database.Cursor.execute, query, params)
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 403, in locked_retry
raise e
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 389, in locked_retry
rv = method(self, *args, **kwargs)
django.db.utils.OperationalError: database is locked

Alerts:


Related issues

Related to FreeNAS - Bug #41304: Unable to run alert source 'HasUpdate'Closed
Has duplicate FreeNAS - Bug #71649: New Alerts and gone alertsClosed
Has duplicate FreeNAS - Bug #70878: "Not an Integer" error on bhyve VM editClosed
Copied from FreeNAS - Bug #66369: * Unable to run alert source 'Replication'Done

History

#1 Updated by Bug Clerk 5 months ago

  • Copied from Bug #66369: * Unable to run alert source 'Replication' added

#2 Updated by Bug Clerk 5 months ago

  • Target version changed from Master - FreeNAS Nightlies to 11.2-U2

#3 Updated by Bug Clerk 5 months ago

  • Status changed from Unscreened to In Progress

#4 Updated by Bug Clerk 5 months ago

  • Status changed from In Progress to Ready for Testing

#5 Updated by Dru Lavigne 5 months ago

  • Subject changed from * Unable to run alert source 'Replication' to Increase SQLite retry timeout to prevent locking on slow USB devices
  • Needs Doc changed from Yes to No
  • Needs Merging changed from Yes to No

#6 Updated by Vladimir Vinogradenko 5 months ago

  • Needs QA changed from Yes to No

#7 Updated by Dru Lavigne 5 months ago

  • Status changed from Ready for Testing to Done

#8 Updated by Dru Lavigne 5 months ago

  • Related to Bug #41304: Unable to run alert source 'HasUpdate' added

#9 Updated by Dru Lavigne 5 months ago

  • Has duplicate Bug #71649: New Alerts and gone alerts added

#10 Updated by Brandon Schneider 4 months ago

  • Has duplicate Bug #70878: "Not an Integer" error on bhyve VM edit added

Also available in: Atom PDF