Configure the serial console to support both BIOS and UEFI
The serial console does not have all the output present in the video console. This is a regression, as around 11.0 both consoles had identical output. This may be related to the new bootloader.
See attached images for examples of missing output in the serial console (top) present in the video console (bottom).
The serial console does show something not present in the video console, namely the following block:
>> FreeBSD EFI boot block Loader path: /boot/loader.efi Initializing modules: ZFS UFS Load Path: \EFI\BOOT\BOOTX64.EFI Load Device: PciRoot(0x0)/Pci(0x1d,0x0)/USB(0x0,0x0)/USB(0x0,0x0)/HD(1,GPT,2e5fadf1-6578-11e6-8c8a-0cc47a073ee2,0x22,0x32000) BootCurrent: 0002 BootOrder: 0002 0003 0004 0008 0009 0005 Probing 24 block devices.....*..+.......+.+.+.+.+.+ done ZFS found the following pools: freenas-boot Flexo UFS found no partitions Consoles: EFI console Command line arguments: loader.efi Image base: 0xcb202000 EFI version: 2.31 EFI Firmware: American Megatrends (rev 4.655) FreeBSD/amd64 EFI loader, Revision 1.1 (Sat Mar 24 08:12:38 UTC 2018 firstname.lastname@example.org) Loading /boot/defaults/loader.conf console vidconsole is invalid! Available consoles: spinconsole \ ______ _ _ _____ | ____| | \ | | /\ / ____| | |__ _ __ ___ ___| \| | / \ | (___ | __| '__/ _ \/ _ \ . ` | / /\ \ \___ \ | | | | | __/ __/ |\ |/ ____ \ ____) | |_| |_| \___|\___|_| \_/_/ \_\_____/ +============Welcome to FreeNAS============================================+ | | | 1. Boot FreeNAS [Enter] | | | | Options: | | 2. Configure Boot [O]ptions... | | 3. Select Boot [E]nvironment... | | | | | | | | | | | | | +==========================================================================+ /boot/kernel/kernel text=0x15e5078 data=0x3c0390+0x5a0ca8 syms=[0x8+0x1a9850+0x8+0x1f650e] /boot/entropy size=0x1000 /boot/kernel/ispfw.ko size 0x1002b8 at 0x26e7000 /boot/modules/freenas_sysctl.ko size 0x8690 at 0x27e8000 Booting... Start @ 0xffffffff80325000 ... EFI framebuffer information: addr, size 0xf6000000, 0x1d4c00 dimensions 800 x 600 stride 800 masks 0x00ff0000, 0x0000ff00, 0x000000ff, 0xff000000
(No image comparison for this one, unfortunately, it requires good timing by the fleshy operator. The video console starts showing everything from
KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2017 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.1-STABLE #0 0df1a1c85(freenas/11-stable): Sat Mar 24 08:23:35 UTC 2018
The environment is as follows:
#2 Updated by Eric Loewenthal over 2 years ago
The issue is that neither console displays the entirety of the boot process. The video console doesn't display the bootloader's menu and the serial console doesn't display a whole bunch of messages (including after boot - see serialconsole4.png - presumably because they're not being sent to this tty).
The video console is the "graphical" console, in other words, the output is VGA (in this case, it is being viewed remotely via the IPMI iKVM application). FreeBSD identifies it as ttyv0.
The serial console is an ASCII output to the server's serial port (FreeNAS is configured to use uart2 at 0x3e8, uart0 is also available at 0x3f8 but is not in use, the server doesn't have a third serial port that could account for uart1. uart2 is being remotely monitored using the IPMI serial redirection functionality, which is piping it into an SSH session established with the BMC). FreeBSD identifies this console as ttyu2, as expected for uart2.
#4 Updated by Erin Clark over 2 years ago
- Status changed from Unscreened to Not Started
- Assignee changed from Erin Clark to William Grzybowski
We are currently using "filesystem.file_tail_follow:/var/log/messages:500"; to get these messages. William, do you know if there is something we are doing wrong or is this something that needs to be fixed on your end?
#7 Updated by Vaibhav Chauhan over 2 years ago
- File Screen Shot 2018-04-09 at 3.16.23 PM.png Screen Shot 2018-04-09 at 3.16.23 PM.png added
- Assignee changed from Lola Yang to Eric Loewenthal
Eric: so we are trying to determine how you are accessing the console(Serial|Video) ? are you using any of VM `VNC|Serial` display options(see attached screenshot). we are asking you this question, so that we can determine if this is a frontend or backend issue ?
#8 Updated by Eric Loewenthal over 2 years ago
This has nothing to do with VMs. The server is neither virtualized nor is it running any VMs. This is FreeNAS itself, with the new bootloader, on a Supermicro X10SLM+-F.
I see two underlying issues here, which may or may not be connected:
Issue 1: The bootloader doesn't recognize vidconsole, only spinconsole. This keeps it from actually displaying anything on VGA. Curiously, it doesn't recognize a serial console either, even though it is outputting on it (as evidenced by the output I pasted).
Issue 2: When the actual OS takes over from the bootloader, it is not outputting all messages on the serial console (VGA console is fine). The output of ttyv0 is complete after FreeBSD takes over from the bootloader, but the output of the serial console, ttyu2 on uart2, is missing messages (see the first screenshot for an example, ttyu2, on the top, is missing everything that is not bolded in ttyv0, bottom).
#10 Updated by Eric Loewenthal over 2 years ago
- Assignee changed from Vaibhav Chauhan to Eric Loewenthal
This ticket from upstream may be of interest for 1):
#13 Updated by Alexander Motin over 2 years ago
- Assignee changed from Alexander Motin to Benno Rice
Init/rc output going to only one console should not be a new issue. I suppose it was like that forever, since serial and video consoles are different TTY devices, and user-space messages are going to only one of them. We may add a control which one to use for that, but not sure we can make them identical, even though I agree that would be very useful.
What's about the loader messages, that indeed likely to be a new issue, related to switch from GRUB to BSD loader. Benno, do you have some system with UEFI and serial console to test that?
#14 Updated by Eric Loewenthal over 2 years ago
Alexander, I'm pretty sure both ttys had complete outputs before. I went back to check an older serial console log I had lying around, and it does include some stuff that it currently missing (the plugin_load messages and the ConnectionRefusedError traceback*, for instance).
Note that this was around September of last year, so it may have been broken independently since then.
BTW: Supermicro X10 and X11 boards support Serial over LAN by SSHing into the BMC (same credentials as the GUI), cding to the sol1 interface (
cd system1/sol1, typically) and entering
start. The SSH session should now show the local serial console, as configured in the BIOS.
Some other systems may present a busybox shell, which may complicate things.
ASRock boards (such as the C2750D4I in the FreeNAS mini) apparently allow for SoL via ipmitool, of all things: https://forums.freenas.org/index.php?resources/how-to-use-ipmitool-sol-with-asrock-motherboards-alternative-to-jviewer.51/
If you can't get your hands on a system for testing, I might be able to provide access next wednesday.
- The ConnectionRefusedError tracebacks have been hard to reproduce and are an unrelated, seemingly cosmetic issue. They just happened to be a decent ilustration.
#16 Updated by Benno Rice over 2 years ago
- Status changed from Not Started to In Progress
Looks like the issue is that we set the following in all cases:
This works for BIOS/VESA/VGA/whatnot but doesn't work for EFI which wants:
I'm working on a patch for this.
#20 Updated by Eric Loewenthal over 2 years ago
- File tty_comparison.PNG tty_comparison.PNG added
- File working_eficonsole.PNG working_eficonsole.PNG added
I can confirm that the bootloader is now correctly recognizing the EFI console, in addition to the serial console. (See attached screenshot)
I can also confirm that after the Kernel takes over, the two ttys are still not identical, which I'm pretty sure is a regression from earlier versions, as described in comment number 14. I can't confirm this 100% at the moment, because my other server's SoL isn't working at all and I don't know where my null modem cable is. (See screenshot, taken using a 2018-05-02 nightly)
There is one cosmetic oddity that didn't happen before:
JH ______ _ _ _____ H| ____| | \ | | /\ / ____| H| |__ _ __ ___ ___| \| | / \ | (___ H| __| '__/ _ \/ _ \ . ` | / /\ \ \___ \ H| | | | | __/ __/ |\ |/ ____ \ ____) | H|_| |_| \___|\___|_| \_/_/ \_\_____/ H+===========HWelcome to FreeNAS===========================================H+ H| H| H| HH.HBoot FreeNAS [Enter] H| H| H| H| HOptions: H| H| HH.HConfigure Boot [O]ptions... H| H| HH.HSelect Boot [E]nvironment... H| H| H| H| H| H| H| H| H| H| H| H| H| H+=========================================================================H+ H H
Some characters aren't correct. If the fix isn't immediately obvious, I'll investigate further and bring it up here or upstream, as appropriate, since it's merely a cosmetic issue. My serial setup is expecting a VT100, by the way.