Project

General

Profile

Bug #62847

HP Gen8 Microserver fails to boot from internal SD slot

Added by george velim 7 months ago. Updated 4 months ago.

Status:
Closed
Priority:
No priority
Assignee:
Ryan Moeller
Category:
OS
Target version:
Seen in:
Severity:
Low
Reason for Closing:
Duplicate Issue
Reason for Blocked:
Needs QA:
Yes
Needs Doc:
Yes
Needs Merging:
No
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

Please notice that I had a successful upgrade from 11.1-u6 --> 11.2-RC2
however it appears that I cannot go to 11.2-REL no matter the approach

Forum link describing the issue:
https://forums.freenas.org/index.php?threads/freenas-11-2-release-has-arrived.71699/post-496979

https://redmine.ixsystems.com/issues/52441


Related issues

Is duplicate of FreeNAS - Bug #65994: Merge in FreeBSD boot loader fixesReady for Testing
Copied to FreeNAS - Bug #77080: HP Gen8 Microserver page fault in pmapClosed

History

#1 Updated by george velim 7 months ago

  • File debug-freenas-20181208221302.txz added
  • Private changed from No to Yes

#2 Updated by Dru Lavigne 7 months ago

  • Category changed from OS to Middleware
  • Assignee changed from Release Council to William Grzybowski

#3 Updated by William Grzybowski 7 months ago

  • Category changed from Middleware to OS
  • Assignee changed from William Grzybowski to Alexander Motin

#4 Updated by Alexander Motin 7 months ago

  • Assignee changed from Alexander Motin to Ryan Moeller

#5 Updated by george velim 7 months ago

45402

UPDATE:

Recovered server back to 11.1-U6 with everything working as they should
System is booting using grub

Step 1: Upgrade using system menu
Upgraded to 11.2-REL and system rebooted with no problem
The system booted using the GRUB bootloader
The 11.1-U6 option wasn't available in the grub menu
System booted up in 11.2-REL and everything was in place and working as expected

Step 2: Reboot system with 11.2-REL
boot failure ... see attached image

has grub been replaced as part of the first boot in 11.2-REL ?

#6 Updated by Alexander Motin 7 months ago

  • Status changed from Unscreened to Blocked
  • Reason for Blocked set to Waiting for feedback

george velim wrote:

has grub been replaced as part of the first boot in 11.2-REL ?

Yes, if FreeNAS updated via UI. If updated by installer CD, then it is done by installer.

Looking on the screenshot, it seems like loader can't properly identify the boot device it was booted from, confusing between different USB drives. We already saw boot issues when more then one USB drive is attached. Can you try to detach disks related to MEDIA_VOL and see whether it help? If so, then I'd recommend to avoid using USB at all, if possible, since it would be good for performance too.

#7 Updated by george velim 7 months ago

  • Private changed from Yes to No

Looking on the screenshot, it seems like loader can't properly identify the boot device it was booted from, confusing between different USB drives. We already saw boot issues when more then one USB drive is attached

There are no other USB devices attached. Both boot sector & OS are on the internal SD-CARD.

Can you try to detach disks related to MEDIA_VOL and see whether it help? If so, then I'd recommend to avoid using USB at all, if possible, since it would be good for performance too.

The MEDIA_VOL is one of my ZFS volumes and not a USB drive. How do I detach the ZFS volume at boot ?

#8 Updated by Sean Fagan 7 months ago

For some reason, what seems to be happening is that the boot loader (which is coming off of your boot device) is not able to load the ZFS boot loader. Note that it got an error -- but that LBA is non-sensical.

You appear to be booting off of an SD card (specifically, "HP iLO Internal SD-CARD 2.10")? But you have two Toshiba USB devices (just under 1TB, attached to the same USB controller as the keyboard).

#9 Updated by george velim 7 months ago

The MEDIA_VOL zfs volume is made of 2 external USB driver and an internal one. This explains why you see USB drives attached.

Does this sheds any light ?

Apologies for confusing you earlier. I thought you said multiple usb keys.

#10 Updated by Gábor ADORJÁNI 7 months ago

  • Seen in changed from 11.1-U6 to 11.2-RELEASE

Hi,

I had a very similar issue with a twist on an HP Microserver Gen7 N40L. The first reboot after upgrading from 11.1-U6 to 11.2-RELEASE worked properly, then all the others since then have failed. I got the same message from the loader that it could not find /boot/kernel/kernel.

Sort of chronological order of the events and the fixes I tried:

1. Upgrading from 11.1-U6 to 11.2-RELEASE worked, as well as the first boot after that.
2. Any subsequent boot attempts failed.
3. I tried to boot the 11.2-RELEASE installer and upgraded the boot drive from it into a new BE. Booting into the new BE failed the same way.
4. I tried a fresh install(!) of 11.2-RELEASE to another USB drive. Booting failed afterwards.
5. Installed 11.1-U6 freshly to another USB drive, restored freenas-v1.db & pwenc_secret from backup - everything works as before.

#11 Updated by Ryan Moeller 6 months ago

  • Reason for Blocked changed from Waiting for feedback to Need additional information from Author

George: to confirm, you said there are no USB devices connected (other than the internal SD slot) when it fails to boot? Are you able to try installing to a USB boot device instead of the SD card?

#12 Updated by Ryan Moeller 6 months ago

  • Subject changed from 11.2-REL upgrade boot failure to HP Gen8 Microserver fails to boot from internal SD slot
  • Severity changed from New to Low

Gábor, please watch https://redmine.ixsystems.com/issues/65994 for the issue with HP N54L Microservers.

George, I'm still hoping to find out if you can boot a fresh install from a USB stick, or if the SD card can boot when the disks for MEDIA_VOL are temporarily removed. Let me know if it is possible for you to test these configurations. Thanks!

#13 Updated by Gábor ADORJÁNI 6 months ago

Ryan Moeller wrote:

Gábor, please watch https://redmine.ixsystems.com/issues/65994 for the issue with HP N54L Microservers.

Cheers Ryan.

#14 Updated by george velim 6 months ago

Ryan, I can boot the 11.2-RELEASE installation iso. I can't recall but it should using the FreeBSD or grub boot-loader ?

I haven't tried to remove the USB disks as of fear of causing permanent damage to the zfs volume. Is there such a risk ? if yes, in what scenario so I can avoid ?

In the mean time, I'll give a try setting up a USB key to boot 11.2 and let you know...

#15 Updated by Ryan Moeller 6 months ago

The 11.2-RELEASE iso uses the FreeBSD loader, though there are some necessary differences between the iso bootloader of the installer and the zfs bootloader of an installed FreeNAS system.

There should not be any risk to the pool if you shutdown, unplug all the pool disks, and then boot into the 11.2 system. ZFS is fairly resilient and would probably be fine even if you smashed a few of your disks with a sledge hammer while the system was still running, but I can't really say the same for your disks, so I recommend the gentler approach.

Thank you for your collaboration with these diagnostics! I'll be looking forward to the results of your latest tests.

#16 Updated by george velim 5 months ago

Have created a 11.2 bootable USB key and it appears to be working however not fully... in detail
(a) installing 11.2 to a usb key completes successfully
(b) first time boot completes successfully and then rebooted
(c) reboot successfull, however when I eventually reach the freenas console menu screen I am getting a page fault in kernel mode and the server goes in reboot...

the (c) is happening all over and I can never recover.

so the external USB key seems to work using the latest iso download. all above took place without unpluging the attached USB drives.

#18 Updated by george velim 5 months ago

  • File debug-freenas-20190220045300.tgz added

#19 Updated by Ryan Moeller 5 months ago

  • Status changed from Blocked to Screened

#20 Updated by Ryan Moeller 5 months ago

  • Status changed from Screened to Blocked
  • Reason for Blocked changed from Need additional information from Author to Waiting for feedback

#21 Updated by Ryan Moeller 5 months ago

  • Subject changed from HP Gen8 Microserver fails to boot from internal SD slot to HP Gen8 Microserver page fault in pmap

#22 Updated by Ryan Moeller 5 months ago

  • Copied to Bug #77080: HP Gen8 Microserver page fault in pmap added

#23 Updated by Ryan Moeller 5 months ago

  • Subject changed from HP Gen8 Microserver page fault in pmap to HP Gen8 Microserver fails to boot from internal SD slot
  • Reason for Blocked changed from Waiting for feedback to Need verification

George, I've created a separate ticket for the panic, to keep things organized. This ticket will track the issue booting from the SD card slot.

#24 Updated by george velim 5 months ago

Tried to put the videos in the other ticket but I get "no authorisation" message.

Here are the 2 videos
(a) install 11.3 to USB stick
(b) first boot into 11.3 and page fault

https://www.youtube.com/playlist?list=PLn25e8QjtreUpAMvQukcesY1IgMRzo4N0

does this help ?

#25 Updated by Ryan Moeller 5 months ago

george velim write:

Tried to put the videos in the other ticket but I get "no authorisation" message.

Sorry, I should have had you open a new ticket much sooner rather than trying to split this ticket myself. It was set to private because of your debug files (which were copied), but I guess it set me as the author rather than you so you got locked out. I removed the debug files and set the ticket back to public access so it should be possible to follow up on that ticket now.

george velim wrote:

does this help ?

Yes I think that will be illuminating, thank you.

#26 Updated by george velim 5 months ago

Assuming there was no page fault, do we have a conclusion with the SD card booting issue?

Btw, The usb key boots with the 2 external USB drives plugged in.

#27 Updated by Ryan Moeller 5 months ago

george velim wrote:

Assuming there was no page fault, do we have a conclusion with the SD card booting issue?

Btw, The usb key boots with the 2 external USB drives plugged in.

Does booting from the SD card slot work now?
I can close the ticket out if you're happy booting from USB instead.

#28 Updated by Ryan Moeller 4 months ago

  • Related to Bug #65994: Merge in FreeBSD boot loader fixes added

#29 Updated by Ryan Moeller 4 months ago

  • Status changed from Blocked to Closed
  • Target version changed from Backlog to 11.2-U4
  • Reason for Closing set to Duplicate Issue
  • Reason for Blocked deleted (Need verification)
  • Needs Merging changed from Yes to No

I expect the SD card boot issue will be solved by the PR for the related USB boot issue. The page fault after boot is a separate issue.

#30 Updated by Ryan Moeller 4 months ago

  • Related to deleted (Bug #65994: Merge in FreeBSD boot loader fixes)

#31 Updated by Ryan Moeller 4 months ago

  • Is duplicate of Bug #65994: Merge in FreeBSD boot loader fixes added

#32 Updated by Ryan Moeller 4 months ago

  • Target version changed from 11.2-U4 to N/A

#33 Updated by Dru Lavigne 4 months ago

  • File deleted (debug-freenas-20181208221302.txz)

#34 Updated by Dru Lavigne 4 months ago

  • File deleted (debug-freenas-20190220045300.tgz)

Also available in: Atom PDF