Unfortunate behavior WRT mirrored boot devices
The installer will not let you install to two different size devices in a mirror, however if you install to the smaller device, then attach the larger device through the GUI after the install you can get a mirror. The GUI however does not match the gpt partition size of the existing device when creating the gpt partition on the new device, it uses all the space on the larger device you are attaching.
So say you then detach the smaller device, or it fails. If you reboot your FreeNAS system in that state the boot pool will expand to fill the gpt partition of the single larger device...thus creating a situation where you cannot reattach your original device once it is replaced.
This is unexpected behavior. The system should either refuse to use devices of disparate sizes unilaterally, or it should DTRT and partition the devices to match the smallest device selected.
#4 Updated by Jordan Hubbard over 5 years ago
I didn't file the bug - Josh did. You should have this conversation with him. :)
Also, FWIW, it's pretty clear that regardless of however many conversations we've had, people are running into a lot of trouble with the current installer behavior that just gloms onto the whole stick, right down to the last block. I think what Josh is asking for is an optional partitioning step, since that would solve the problem without losing the ability to grow your boot device.
#5 Updated by Josh Paetzel over 5 years ago
I don't believe behaves correctly is the right status for this bug.
The behavior of the current system has resulted in two reinstalls this week by FreeNAS certified customers (one of whom is watching this bug)
If the installer will let you use two differently sized devices then the easy button solution is two partition both devices to the same (smaller) size.
You can easily grow partitions with gpart resize, it just requires an explicit action, which seems safer than implicitly growing at any opportunity.
#7 Updated by Sean Fagan over 5 years ago
If we add UI to the system to partition new boot devices, and to change the partition sizes, and I'll add code to the installer to use the minimum size of all the disks.
Doing the former by itself is useful; doing the latter without the former isn't so much.
#8 Updated by Josh Paetzel over 5 years ago
Changing the GUI in the running system won't solve the problem because the installer alone is setting up the partitions in such a way that the problem will occur.
1) Install two two different sized devices in a mirror.
2) Remove the smaller device.
3) Reboot the system.
Pool grows to fit the larger device. This is unavoidable automagical ZFS behavior.
At this point you can no longer use the smaller device.
I agree that detecting the device size and using the smallest device to partition all of the devices is more complicated than the current code, but it's not "makes the system hideously complicated or fragile" territory.
#9 Updated by Sean Fagan over 5 years ago
- Status changed from Closed: Behaves correctly to Investigation
I give up. Doing this is dumb, short-sighted, and will cause at least as many problems as it is trying to fix, but that doesn't seem to matter.
- The actual description is wrong, since it does work.
- Picking the minimum won't solve the problem, because when a device dies -- as it will, since we're talking about thumb drives -- the replacement, even if it's the same manufacturer and model and alleged size, could fail due to being smaller.
- Speaking of which, the post-install mirroring and replacement needs to be changed as well. Unless we're blowing off that complaint too.
- Rounding the size down (as per another bug) means that people will complain that their 8gb drives actually only have about 6.5g for ZFS. I'm sure that'll cause no problems whatsoever.
- Saying "they can fix it in the command line" also means that they could fix this in the command line. (I, for example, did that when I created a mirror for my boot device. I admit to being unusual here, but somehow we're saying that using the command line is good, and yet it's also not good.)