Project

General

Profile

Bug #23665

OperationalError in Nightlies on upgrade from 9.10.1-U2

Added by Chris Kucharczyk about 4 years ago. Updated almost 4 years ago.

Status:
Resolved
Priority:
Blocks Until Resolved
Assignee:
Suraj Ravichandran
Category:
OS
Target version:
Severity:
New
Reason for Closing:
Reason for Blocked:
Needs QA:
Yes
Needs Doc:
Yes
Needs Merging:
Yes
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:

Processor: Intel Pentium G4400
Motherboard: MSI H110M Eco Micro-ATX Motherboard
RAM: 2 x G.SKILL Aegis 8GB 288-Pin DDR4 SDRAM DDR4 2133
Storage: 2 x 4TB WD Reds

ChangeLog Required:
No

Description

Request Method: GET
Request URL: http://10.201.1.110/system/
Software Version: FreeNAS-11-MASTER-201704280408 (83eff7b)
Exception Type: OperationalError
Exception Value:
no such column: system_advanced.adv_fqdn_syslog
Exception Location: ./freenasUI/freeadmin/sqlite3_ha/base.py in locked_retry, line 406
Server time: Fri, 28 Apr 2017 16:10:52 -0500
Traceback

Environment:

Software Version: FreeNAS-11-MASTER-201704280408 (83eff7b)
Request Method: GET
Request URL: http://10.201.1.110/system/

Traceback:
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py" in execute
64. return self.cursor.execute(sql, params)
File "./freenasUI/freeadmin/sqlite3_ha/base.py" in execute
429. execute = self.locked_retry(Database.Cursor.execute, query, params)
File "./freenasUI/freeadmin/sqlite3_ha/base.py" in locked_retry
406. rv = method(self, *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner
39. response = get_response(request)
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py" in _legacy_get_response
249. response = self._get_response(request)
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
178. response = middleware_method(request, callback, callback_args, callback_kwargs)
File "./freenasUI/freeadmin/middleware.py" in process_view
162. return login_required(view_func)(request, *view_args, **view_kwargs)
File "/usr/local/lib/python3.6/site-packages/django/contrib/auth/decorators.py" in _wrapped_view
23. return view_func(request, *args, **kwargs)
File "./freenasUI/system/views.py" in home
647. tabs = appPool.hook_app_tabs('system', request)
File "./freenasUI/freeadmin/apppool.py" in hook_app_tabs
64. return self._get_array('hook_app_tabs', name, request)
File "./freenasUI/freeadmin/apppool.py" in _get_array
27. method(arr, func(*args, **kwargs))
File "./freenasUI/system/hook.py" in hook_app_tabs_system
86. obj = model.objects.order_by('-id')[0]
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py" in getitem
295. return list(qs)[0]
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py" in iter
256. self._fetch_all()
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py" in _fetch_all
1087. self._result_cache = list(self.iterator())
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py" in iter
54. results = compiler.execute_sql()
File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py" in execute_sql
835. cursor.execute(sql, params)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py" in execute
64. return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.6/site-packages/django/db/utils.py" in exit
94. six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python3.6/site-packages/django/utils/six.py" in reraise
685. raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py" in execute
64. return self.cursor.execute(sql, params)
File "./freenasUI/freeadmin/sqlite3_ha/base.py" in execute
429. execute = self.locked_retry(Database.Cursor.execute, query, params)
File "./freenasUI/freeadmin/sqlite3_ha/base.py" in locked_retry
406. rv = method(self, *args, **kwargs)

Exception Type: OperationalError at /system/
Exception Value: no such column: system_advanced.adv_fqdn_syslog

Request information

GET

No GET data
POST

No POST data
FILES

No FILES data
COOKIES

Variable Value
sessionid 'm9n6eelbqmykpjyh724g7xotx9pmaubt'
csrftoken 'FwwI7B0B8vTgtwbx7XwSyNTbs1sFHrLIzeorkYj9cmy9KSm8oqIGrHHSorJXUVbD'
META

Also receive alert: Update failed. Check /data/update.failed for further details.

I could not run 9.10.2-U2 on this hardware due to previously seen in Bug #20927.


Related issues

Related to FreeNAS - Bug #23478: Trackback from 9.10.1u4 failed upgrade to FreeNAS-11-MASTER-201704200433 (d94329c)Closed: Duplicate2017-04-20

Associated revisions

Revision 8029d741 (diff)
Added by Suraj Ravichandran about 4 years ago

Disable upgrades from 9.10 versions prior to 9.10.2 Ticket: #23665

Revision 5ae9e072 (diff)
Added by Suraj Ravichandran about 4 years ago

Fix ValidateUpdate script to be less bashistic and actually work. Ticket: #23665

Revision 03da7e1c (diff)
Added by Suraj Ravichandran about 4 years ago

Fix ValidateUpdate script to be less bashistic and actually work. Ticket: #23665 (cherry picked from commit 5ae9e0720b737e2257cbbbe79045100bf0bdf9b0)

Revision dd91d106 (diff)
Added by Suraj Ravichandran about 4 years ago

Add a more descriptive message in the ValidateUpdate script. Ticket: #23665

Revision 0306042d (diff)
Added by Suraj Ravichandran about 4 years ago

Add a more descriptive message in the ValidateUpdate script. Ticket: #23665 (cherry picked from commit dd91d1069f14ea1d189d03ee032528b4343670eb) NOTE: THIS IS A DIRECT CHERRY-PICK. NO FIX BRANCH WAS MADE SPECIFICALLY JUST FOR THIS COMMMIT. SINCE IT IS A STRING ONLY CHANGE. PERMISSION WAS TAKEN FROM RELEASE ENGINEER PRIOR TO DOING THIS.

Revision 85bf7000 (diff)
Added by Warren Block about 4 years ago

Mention that upgrading to 11 from versions prior to 9.10.2 is not supported. Also edit and reorder Caveats section. Ticket: #23665

History

#1 Updated by Sean Fagan about 4 years ago

  • Assignee changed from Sean Fagan to William Grzybowski

I know we've seen that one before.

#2 Updated by William Grzybowski about 4 years ago

  • Status changed from Unscreened to 15
  • Priority changed from No priority to Nice to have

Can you attach the database before the upgrade?

Also, as a workaround, can you try running from SSH as root:

migrate93 -f /data/freenas-v1.db
cd /usr/local/www/freenasUI
python manage.py migrate

Catch the output of these if you can/know how.

#3 Updated by William Grzybowski about 4 years ago

  • Related to Bug #23478: Trackback from 9.10.1u4 failed upgrade to FreeNAS-11-MASTER-201704200433 (d94329c) added

#4 Updated by Chris Kucharczyk about 4 years ago

  • File freenas-v1.db added

Database file attached (please let me know if this is the right thing), trying the workaround now.

#5 Updated by William Grzybowski about 4 years ago

  • Private changed from No to Yes

Yes, thats it, thank you. Marked this ticket as private.

#6 Updated by Chris Kucharczyk about 4 years ago

I don't appear to have migrate93... checked in /usr/local/lib. Am I being stupid?

#7 Updated by William Grzybowski about 4 years ago

Chris Kucharczyk wrote:

I don't appear to have migrate93... checked in /usr/local/lib. Am I being stupid?

/usr/local/sbin/migrate93 -f /data/freenas-v1.db

#8 Updated by Chris Kucharczyk about 4 years ago

It's not there either. I re-booted into 9.10.1-U2, do I need to go back to 11?

#9 Updated by William Grzybowski about 4 years ago

Chris Kucharczyk wrote:

It's not there either. I re-booted into 9.10.1-U2, do I need to go back to 11?

Yes, you need to be in 11 with it in "broken" state to try it.

#10 Updated by Chris Kucharczyk about 4 years ago

  • File migrate93_migratepy_output.txt added

migrate93 works, migrate.py fails. Output attached.
Failure messages:
sqlite3.OperationalError: table "account_bsdusers" already exists;
django.db.utils.OperationalError: table "account_bsdusers" already exists

#11 Updated by William Grzybowski about 4 years ago

Chris Kucharczyk wrote:

migrate93 works, migrate.py fails. Output attached.
Failure messages:
sqlite3.OperationalError: table "account_bsdusers" already exists;
django.db.utils.OperationalError: table "account_bsdusers" already exists

My bad, should have been: python manage.py migrate --fake-initial

#12 Updated by William Grzybowski about 4 years ago

  • Priority changed from Nice to have to Blocks Until Resolved
  • Target version set to 11.0

#13 Updated by Chris Kucharczyk about 4 years ago

  • File success_migratepy.txt added

Ok that worked (output attached). Looks like the traceback in the UI is gone as well.

#14 Updated by Sean Fagan about 4 years ago

  • Assignee changed from William Grzybowski to Suraj Ravichandran

A ValidateUpdate script should do this. Suraj has done it more than I have (even though I wrote the code), so he can probably do it more quickly.

Mostly it's just checking ${OLD_VERSION}, but we should know which versions are good or bad. NAS-9.10.1, maybe?

#15 Updated by Sean Fagan about 4 years ago

Or $(uname -r) or $(uname -K), I guess. That can be used with a numeric compare.

#16 Updated by Suraj Ravichandran about 4 years ago

  • Status changed from 15 to Needs Developer Review
  • Assignee changed from Suraj Ravichandran to Sean Fagan

Sef please review this.

#17 Updated by Sean Fagan about 4 years ago

  • Status changed from Needs Developer Review to Reviewed
  • Assignee changed from Sean Fagan to Suraj Ravichandran

Looks good. Regexpy, just the way I like it.

#18 Updated by Suraj Ravichandran about 4 years ago

thanks sef

#19 Updated by William Grzybowski about 4 years ago

  • Status changed from Reviewed to Unscreened

How was this reviewed:

[freenas-update:215] b'./ValidateUpdate: 9: Syntax error: redirection unexpected (expecting word)'
Update not permitted:
b'./ValidateUpdate: 9: Syntax error: redirection unexpected (expecting word)'

#20 Updated by Sean Fagan about 4 years ago

That was tested, not reviewed. That's the "<<<" redirection, which I assumed was a new sh ism (I have seen it before), but I guess is actually part of zsh? Maybe ksh as well.

#21 Updated by Bonnie Follweiler about 4 years ago

I upgraded from FreeNAS 9.10.1-U4 to FreeNAS 11 Nightlies by switching trains and installing
It downloaded and installed but, after reboot and log in, I get the following Traceback error:

Request Method:
GET
Request URL:
http://10.20.1.154/system/
Software Version:
FreeNAS-11-MASTER-201705010408 (8f4a292)
Exception Type:
OperationalError
Exception Value:
no such column: system_advanced.adv_fqdn_syslog
Exception Location:
./freenasUI/freeadmin/sqlite3_ha/base.py in locked_retry, line 406
Server time:
Tue, 2 May 2017 08:46:11 -0700

Traceback

Environment:

Software Version: FreeNAS-11-MASTER-201705010408 (8f4a292)
Request Method: GET
Request URL: http://10.20.1.154/system/

Traceback:
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py" in execute
64. return self.cursor.execute(sql, params)
File "./freenasUI/freeadmin/sqlite3_ha/base.py" in execute
429. execute = self.locked_retry(Database.Cursor.execute, query, params)
File "./freenasUI/freeadmin/sqlite3_ha/base.py" in locked_retry
406. rv = method(self, *args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner
39. response = get_response(request)
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py" in _legacy_get_response
249. response = self._get_response(request)
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
178. response = middleware_method(request, callback, callback_args, callback_kwargs)
File "./freenasUI/freeadmin/middleware.py" in process_view
162. return login_required(view_func)(request, *view_args, **view_kwargs)
File "/usr/local/lib/python3.6/site-packages/django/contrib/auth/decorators.py" in _wrapped_view
23. return view_func(request, *args, **kwargs)
File "./freenasUI/system/views.py" in home
647. tabs = appPool.hook_app_tabs('system', request)
File "./freenasUI/freeadmin/apppool.py" in hook_app_tabs
64. return self._get_array('hook_app_tabs', name, request)
File "./freenasUI/freeadmin/apppool.py" in _get_array
27. method(arr, func(*args, **kwargs))
File "./freenasUI/system/hook.py" in hook_app_tabs_system
86. obj = model.objects.order_by('-id')[0]
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py" in getitem
295. return list(qs)[0]
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py" in iter
256. self._fetch_all()
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py" in _fetch_all
1087. self._result_cache = list(self.iterator())
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py" in iter
54. results = compiler.execute_sql()
File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py" in execute_sql
835. cursor.execute(sql, params)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py" in execute
64. return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.6/site-packages/django/db/utils.py" in exit
94. six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python3.6/site-packages/django/utils/six.py" in reraise
685. raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py" in execute
64. return self.cursor.execute(sql, params)
File "./freenasUI/freeadmin/sqlite3_ha/base.py" in execute
429. execute = self.locked_retry(Database.Cursor.execute, query, params)
File "./freenasUI/freeadmin/sqlite3_ha/base.py" in locked_retry
406. rv = method(self, *args, **kwargs)

Exception Type: OperationalError at /system/
Exception Value: no such column: system_advanced.adv_fqdn_syslog

Request information
GET
No GET data
POST
No POST data
FILES
No FILES data
COOKIES
Variable
Value
csrftoken
'eM6n0YrwPqrCe8RTw6K2EB7ywY1C3S8FrKPVXpXBYN0yUkmKFeZg99wLcj4gloPq'
fntreeSaveStateCookie
'root'
sessionid
'brodyjsd3hffks52em87jkbz3fs7t3zs'
META
Variable
Value

#22 Updated by Bonnie Follweiler about 4 years ago

FWIW - I upgraded to FreeNAS 11 Nightlies the same way, switching trains and installing, from FreeNAS 9.10.2 with no issues

#23 Updated by Suraj Ravichandran about 4 years ago

  • Status changed from Unscreened to Fix In Progress

This is my bad, I shall fix this now.

For what its worth, I did test it on my own laptpo's shell and it worked.

Lemme see what went wrong.

#24 Updated by Suraj Ravichandran about 4 years ago

  • Status changed from Fix In Progress to Needs Developer Review
  • % Done changed from 0 to 100

Have made the fixes and tested them.

This time I tested it in an actual update (both via the gui update as well as tar file update)

Sean please re-review.

Thanks

#25 Updated by Sean Fagan about 4 years ago

Looks okay.

#26 Updated by Vaibhav Chauhan about 4 years ago

  • Status changed from Needs Developer Review to Ready For Release

#27 Updated by Vaibhav Chauhan about 4 years ago

  • Target version changed from 11.0 to 11.0-RC

#28 Updated by Vaibhav Chauhan about 4 years ago

  • Status changed from Ready For Release to Resolved

#29 Updated by Bonnie Follweiler about 4 years ago

I tried it this morning with a FN 9.10.1-u4 and it gave me the "'Cannot upgrade from FreeNAS-9.10.1-U4 to FreeNAS 11'".

#30 Updated by William Grzybowski about 4 years ago

Can we make the message a bit more clearer? e.g. "You have to upgrade to 9.10.2 first" ?

#31 Updated by Chris Kucharczyk about 4 years ago

So if my hardware cannot support 9.10.2* (Bug #20927), I have to do a clean install of FN11?

#32 Updated by William Grzybowski about 4 years ago

Chris Kucharczyk wrote:

So if my hardware cannot support 9.10.2* (Bug #20927), I have to do a clean install of FN11?

You can try going to 9.10-Nightlies first, or use the commands I pasted earlier in the ticket if you go straight to 11.

#33 Updated by William Grzybowski about 4 years ago

William Grzybowski wrote:

Chris Kucharczyk wrote:

So if my hardware cannot support 9.10.2* (Bug #20927), I have to do a clean install of FN11?

You can try going to 9.10-Nightlies first, or use the commands I pasted earlier in the ticket if you go straight to 11.

Actually scratch that 9.10-nightlies, that won't work as well.

#34 Updated by Dru Lavigne almost 4 years ago

  • File deleted (freenas-v1.db)

#35 Updated by Dru Lavigne almost 4 years ago

  • File deleted (migrate93_migratepy_output.txt)

#36 Updated by Dru Lavigne almost 4 years ago

  • File deleted (success_migratepy.txt)

#37 Updated by Dru Lavigne almost 4 years ago

  • Private changed from Yes to No

Also available in: Atom PDF