Project

General

Profile

Bug #26659

Ensure that disks attached to a pool are not available for use as a spare

Added by Michael Dexter over 1 year ago. Updated about 1 year ago.

Status:
Done
Priority:
Nice to have
Assignee:
Waqar Ahmed
Category:
Middleware
Target version:
Seen in:
Severity:
Medium
Reason for Closing:
Reason for Blocked:
Needs QA:
No
Needs Doc:
No
Needs Merging:
No
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

Software Version: FreeNAS-11.0-U2 (e417d8aa5)
Software Version: FreeNAS-11-MASTER-201711130411 (acd0c21

I am experimenting with hot spare behavior and the "Replace" option for a previous disk ID or the replacement "spare-1" throws a traceback:

Request Method: POST
Request URL: http://10.0.0.185/storage/zpool-test/disk/replace/spare-1/
Software Version: FreeNAS-11.0-U2 (e417d8aa5)
Exception Type: MiddlewareError
Exception Value:
[MiddlewareError: b'Disk replacement failed: "cannot replace spare-1 with gptid/ff625985-c97e-11e7-b4c5-3860775656f5: no such device in pool, "']
Exception Location: ./freenasUI/middleware/notifier.py in zfs_replace_disk, line 1141
Server time: Tue, 14 Nov 2017 13:01:38 -0800

Request Method: POST
Request URL: http://10.0.0.185/storage/zpool-test/disk/replace/gptid/efe86da1-c8d5-11e7-87d2-3860775656f5/
Software Version: FreeNAS-11-MASTER-201711130411 (acd0c21)
Exception Type: MiddlewareError
Exception Value:
[MiddlewareError: Disk replacement failed: "cannot replace gptid/efe86da1-c8d5-11e7-87d2-3860775656f5 with gptid/a4e009bd-c906-11e7-87d2-3860775656f5: can only be replaced by another hot spare, "]
Exception Location: ./freenasUI/middleware/notifier.py in zfs_replace_disk, line 1039
Server time: Mon, 13 Nov 2017 22:40:06 -0800

Environment:

Software Version: FreeNAS-11-MASTER-201711130411 (acd0c21)
Request Method: POST
Request URL: http://10.0.0.185/storage/zpool-test/disk/replace/gptid/efe86da1-c8d5-11e7-87d2-3860775656f5/

Traceback:
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner
42. response = get_response(request)
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py" in _legacy_get_response
249. response = self._get_response(request)
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
178. response = middleware_method(request, callback, callback_args, callback_kwargs)
File "./freenasUI/freeadmin/middleware.py" in process_view
162. return login_required(view_func)(request, *view_args, **view_kwargs)
File "/usr/local/lib/python3.6/site-packages/django/contrib/auth/decorators.py" in _wrapped_view
23. return view_func(request, *args, **kwargs)
File "./freenasUI/storage/views.py" in zpool_disk_replace
761. if form.done():
File "./freenasUI/storage/forms.py" in done
2223. passphrase=passfile
File "./freenasUI/middleware/notifier.py" in zfs_replace_disk
1039. raise MiddlewareError('Disk replacement failed: "%s"' % error)

Exception Type: MiddlewareError at /storage/zpool-test/disk/replace/gptid/efe86da1-c8d5-11e7-87d2-3860775656f5/
Exception Value: [MiddlewareError: Disk replacement failed: "cannot replace gptid/efe86da1-c8d5-11e7-87d2-3860775656f5 with gptid/b526fdda-c907-11e7-87d2-3860775656f5: can only be replaced by another hot spare, "]

The correct behavior does not appear to be documented anywhere but this is probably not it.


Related issues

Has duplicate FreeNAS - Bug #24145: Floating "Detach Volume" on top left LogoClosed: Not To Be Fixed2017-05-24
Has duplicate FreeNAS - Bug #27229: Manual disk replacement with spare throws tracebackClosed2017-12-132018-03-09

Associated revisions

Revision db5944cb (diff)
Added by Waqar Ahmed over 1 year ago

Flake errors fixed
Ticket: #26659

Revision 74ba1a3b
Added by Waqar Ahmed over 1 year ago

Merge branch 'FIX-26659'
Ticket: #26659

Revision 51d6d74f (diff)
Added by Waqar Ahmed over 1 year ago

Flake errors fixed
Ticket: #26659

History

#1 Updated by Dru Lavigne over 1 year ago

  • Assignee changed from Release Council to William Grzybowski
  • Private changed from No to Yes

#2 Updated by William Grzybowski over 1 year ago

  • Status changed from Unscreened to Screened
  • Priority changed from No priority to Nice to have
  • Target version set to 11.3

#3 Updated by Michael Dexter over 1 year ago

Confirmed in 11.0-U4 also.

#5 Updated by Dru Lavigne over 1 year ago

  • Has duplicate Bug #24145: Floating "Detach Volume" on top left Logo added

#8 Updated by Dru Lavigne over 1 year ago

Michael: can you attach a debug from the updated system?

#9 Updated by Michael Dexter over 1 year ago

  • File debug-freenas-20171214113959.tgz added

Debug while traceback is on one browser tab. Contains nothing sensitive.

#10 Updated by Michael Dexter over 1 year ago

Performing an "all zeros" wipe of the drive one attempts to replace with does not make a difference.

#11 Updated by Michael Dexter over 1 year ago

For what it's worth, same behavior in 11.1-U1:

Request Method: POST
Request URL: http://10.0.0.185/storage/zpool-testspare/disk/replace/17692221722372214347/
Software Version: FreeNAS-11.1-U1 (f7e246b8f)
Exception Type: MiddlewareError
Exception Value:
[MiddlewareError: Unable to GPT format the disk "da0": gpart: geom 'da0': File exists
]
Exception Location: ./freenasUI/middleware/notifier.py in __gpt_labeldisk, line 359
Server time: Fri, 19 Jan 2018 11:38:53 -0800

Environment:

Software Version: FreeNAS-11.1-U1 (f7e246b8f)
Request Method: POST
Request URL: http://10.0.0.185/storage/zpool-testspare/disk/replace/17692221722372214347/

Traceback:
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner
42. response = get_response(request)
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py" in legacy_get_response
249. response = self._get_response(request)
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
178. response = middleware_method(request, callback, callback_args, callback_kwargs)
File "./freenasUI/freeadmin/middleware.py" in process_view
162. return login_required(view_func)(request, *view_args, **view_kwargs)
File "/usr/local/lib/python3.6/site-packages/django/contrib/auth/decorators.py" in _wrapped_view
23. return view_func(request, *args, **kwargs)
File "./freenasUI/storage/views.py" in zpool_disk_replace
768. if form.done():
File "./freenasUI/storage/forms.py" in done
2224. passphrase=passfile
File "./freenasUI/middleware/notifier.py" in zfs_replace_disk
987. self.
_gpt_labeldisk(type="freebsd-zfs", devname=to_disk, swapsize=swapsize)
File "./freenasUI/middleware/notifier.py" in __gpt_labeldisk
359. raise MiddlewareError(f'Unable to GPT format the disk "{devname}": {error}')

Exception Type: MiddlewareError at /storage/zpool-testspare/disk/replace/17692221722372214347/
Exception Value: [MiddlewareError: Unable to GPT format the disk "da0": gpart: geom 'da0': File exists
]

Request information

GET

No GET data
POST

Variable Value
all ''
force 'on'
replace_disk 'da0'
__form_id 'form_ZFSDiskReplacementForm'
FILES

No FILES data
COOKIES

Variable Value
csrftoken '09aFmhbbMOIdPOF0TxAQd2adlbXEqRPkIhVP8ruKAoROe3PXakXzrIQIV6IP19cg'
fntreeSaveStateCookie 'root%2Croot%2F68%2F69'
sessionid 'esollnzmvjrn5lbuwi91vqffh6h0og0g'
META

Variable Value

#12 Updated by Michael Dexter over 1 year ago

Please give spares a try.

Scenario:

Two mirrored disks with two spares.
Each replacement option in the GUI gives a traceback.
The one "Offline" option, used on the ID of the missing/replaced disk removed the active spare.

This left the once mirrored dives as a single "striped" drive with no option to attach a new mirror. It is left however with two available but unusable spares.

#13 Updated by Dru Lavigne over 1 year ago

  • Status changed from Screened to 47
  • Assignee changed from William Grzybowski to Bonnie Follweiler

#14 Updated by Dru Lavigne over 1 year ago

Bonnie: can you test this on U1 first?

#15 Updated by Bonnie Follweiler over 1 year ago

Dru, Confirmed in FreeNAS-11.1-U1 (f7e246b8f)

I get the error when I try to replace the drive. After I select the spare the following Traceback error is displyed:

POST
Request URL: http://10.231.3.50/storage/zpool-tank/disk/replace/gptid/f695db4a-cbc9-11e7-ab4e-0007433e6540/
Software Version: FreeNAS-11.1-U1 (f7e246b8f)
Exception Type: MiddlewareError
Exception Value:

[MiddlewareError: Unable to GPT format the disk "ada3": gpart: geom 'ada3': File exists
]

Exception Location: ./freenasUI/middleware/notifier.py in __gpt_labeldisk, line 359
Server time: Mon, 22 Jan 2018 10:56:41 -0800

Traceback
Environment:

Software Version: FreeNAS-11.1-U1 (f7e246b8f)
Request Method: POST
Request URL: http://10.231.3.50/storage/zpool-tank/disk/replace/gptid/f695db4a-cbc9-11e7-ab4e-0007433e6540/

Traceback:
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner
42. response = get_response(request)
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py" in legacy_get_response
249. response = self._get_response(request)
File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
178. response = middleware_method(request, callback, callback_args, callback_kwargs)
File "./freenasUI/freeadmin/middleware.py" in process_view
162. return login_required(view_func)(request, *view_args, **view_kwargs)
File "/usr/local/lib/python3.6/site-packages/django/contrib/auth/decorators.py" in _wrapped_view
23. return view_func(request, *args, **kwargs)
File "./freenasUI/storage/views.py" in zpool_disk_replace
768. if form.done():
File "./freenasUI/storage/forms.py" in done
2224. passphrase=passfile
File "./freenasUI/middleware/notifier.py" in zfs_replace_disk
987. self.
_gpt_labeldisk(type="freebsd-zfs", devname=to_disk, swapsize=swapsize)
File "./freenasUI/middleware/notifier.py" in __gpt_labeldisk
359. raise MiddlewareError(f'Unable to GPT format the disk "{devname}": {error}')

Exception Type: MiddlewareError at /storage/zpool-tank/disk/replace/gptid/f695db4a-cbc9-11e7-ab4e-0007433e6540/
Exception Value: [MiddlewareError: Unable to GPT format the disk "ada3": gpart: geom 'ada3': File exists]

Request information
GET

No GET data
POST
Variable Value
all ''
force 'on'
replace_disk 'ada3'
__form_id 'form_ZFSDiskReplacementForm'
FILES

No FILES data
COOKIES
Variable Value
csrftoken 'fcjCPY0nz8grlyEkqJFLgtPxELAK3wFQZGwBomLOMy1o5HyyHSirZ24GUwAnzwv1'
fntreeSaveStateCookie 'root%2Croot%2F1%2Croot%2F1%2F7%2Croot%2F1%2F6'
sessionid 'ugokcuwchon8jjhfs0w5z728fbvy9est'
META
Variable Value

#16 Updated by Bonnie Follweiler over 1 year ago

  • Status changed from 47 to Broken
  • Assignee changed from Bonnie Follweiler to William Grzybowski
  • Seen in changed from 11.1 to 11.1-U1
  • QA Status Test Passes FreeNAS added
  • QA Status deleted (Not Tested)

#17 Updated by Bonnie Follweiler over 1 year ago

  • QA Status Test Fails FreeNAS added
  • QA Status deleted (Test Passes FreeNAS)

#18 Updated by Dru Lavigne over 1 year ago

  • Target version changed from 11.3 to 11.1-U2

#19 Avatar?id=13649&size=24x24 Updated by Ben Gadd over 1 year ago

  • Due date set to 02/12/2018

Due date updated to reflect the code freeze for 11.1U2.

#20 Avatar?id=13649&size=24x24 Updated by Ben Gadd over 1 year ago

  • Severity set to New

#21 Updated by William Grzybowski over 1 year ago

  • Target version changed from 11.1-U2 to 11.2-RC2

This does not seem to be as important to be required for U2 given the time constraint.

#22 Avatar?id=13649&size=24x24 Updated by Ben Gadd over 1 year ago

  • Due date deleted (02/12/2018)

#23 Updated by William Grzybowski over 1 year ago

  • Status changed from Broken to Not Started

#24 Updated by William Grzybowski over 1 year ago

  • Assignee changed from William Grzybowski to Waqar Ahmed
  • Severity changed from New to Medium

Waqar, perhaps this is something you can work on for 11.2.

Let me know of any questions. Thanks!

#25 Updated by Waqar Ahmed over 1 year ago

  • Status changed from Not Started to In Progress

#26 Updated by Waqar Ahmed over 1 year ago

  • % Done changed from 0 to 60

#27 Updated by Waqar Ahmed over 1 year ago

  • % Done changed from 60 to 90

#28 Updated by Waqar Ahmed over 1 year ago

  • Status changed from In Progress to Done
  • % Done changed from 90 to 100

#29 Updated by Dru Lavigne over 1 year ago

  • File deleted (debug-freenas-20171114131634.tgz)

#30 Updated by Dru Lavigne over 1 year ago

  • File deleted (debug-freenas-20171214113959.tgz)

#31 Updated by Dru Lavigne over 1 year ago

  • Subject changed from Replacing spare-1 throws traceback on 11.0-U2 and 11.1 Nightlies to Ensure that disks attached to a pool are not available for use as a spare
  • Target version changed from 11.2-RC2 to 11.2-BETA1
  • Private changed from Yes to No
  • Needs Doc changed from Yes to No
  • Needs Merging changed from Yes to No

#32 Updated by Dru Lavigne over 1 year ago

  • Has duplicate Bug #27229: Manual disk replacement with spare throws traceback added

#33 Updated by Dru Lavigne about 1 year ago

  • Status changed from Done to Ready for Testing

#34 Updated by Bonnie Follweiler about 1 year ago

  • Status changed from Ready for Testing to Passed Testing
  • Needs QA changed from Yes to No

Test Passed in FreeNAS-11-MASTER-201805020407

#35 Updated by Dru Lavigne about 1 year ago

  • Status changed from Passed Testing to Done

Also available in: Atom PDF