Bug #14212

iohyve tool fails to create virtual machine

Added by Dave F over 4 years ago. Updated almost 3 years ago.

No priority
Jordan Hubbard
Seen in:
Reason for Closing:
Reason for Blocked:
Needs QA:
Needs Doc:
Needs Merging:
Needs Automation:
Support Suite Ticket:
Hardware Configuration:
ChangeLog Required:


Trying out bhyve suupport under 9.10-RELEASE have found that iohyve is looking for the ishyve dataset in the wrong location. Initial set-up works, however when creating a new VM, it fails.

[root@tardis] ~# iohyve setup pool=tank kmod=1 net=lagg0
Setting up iohyve pool...
Loading kernel modules...
bridge0 is already enabled on this machine...
[root@tardis] ~# zfs list | grep -i iohyve
tank/iohyve                                                    418K  2.80T   140K  /mnt/iohyve
tank/iohyve/Firmware                                           140K  2.80T   140K  /mnt/iohyve/Firmware
tank/iohyve/ISO                                                140K  2.80T   140K  /mnt/iohyve/ISO
[root@tardis] ~# iohyve set debguest loader=grub=bhyve
Setting debguest prop loader=grub=bhyve...
cannot open '/iohyve/debguest': invalid dataset name
[root@tardis] ~#

Looking at the code, it appears to reference /iohyve as it's base directory in the root filesystem.

Build: FreeNAS-9.10-RELEASE (2def9c8)
Platform: Intel(R) Xeon(R) CPU E3-1220 v3 @ 3.10GHz
Memory: 16292MB


#1 Updated by Dave F over 4 years ago

Spotted a spelling mistake, ishyve should be iohyve.

#2 Updated by William Grzybowski over 4 years ago

  • Status changed from Unscreened to Closed: User Config Issue

You must create the VM first, using "iohyve create debguest"

#3 Updated by Dave F over 4 years ago

Seems the instructions at are unclear. I'll notify the developer for iohyve.

Linux flavors:

Try out Debian or Ubuntu (note LVM installs should work with os=d8lvm):

iohyve set debguest loader=grub-bhyve
iohyve set debguest os=debian
iohyve install debguest debian-8.2.0-amd64-i386-netinst.iso
iohyve console debguest

I've run iohyve create and now hit another roadblock:

[root@tardis] ~# iohyve create debguest 40G
Creating debguest...
[root@tardis] ~# iohyve set debguest loader=grub-bhyve
Setting debguest prop loader=grub-bhyve...
[root@tardis] ~# iohyve set debguest os=debian
Setting debguest prop os=debian...
[root@tardis] ~# iohyve install debguest debian-8.2.0-amd64-netinst.iso
Not a valid guest name
[root@tardis] ~# iohyve list
Guest     VMM?  Running?  rcboot?  Description
debguest  NO    NO        NO       Tue_Mar_22_22:23:00_UTC_2016
[root@tardis] ~# ls -l /mnt/iohyve/ISO
total 243493
drwxr-xr-x  2 root  wheel          3 Mar 22 22:24 ./
drwxr-xr-x  5 root  wheel          5 Mar 22 22:23 ../
-rwxr-xr-x  1 root  wheel  258998272 Mar 22 22:24 debian-8.2.0-amd64-netinst.iso*
[root@tardis] ~#

#4 Updated by William Grzybowski over 4 years ago

iohyve isolist shows anything?

#5 Updated by Dave F over 4 years ago

iohyve isolist doesn't show anything. I manually copied my existing Debian 8.2 ISO to /mnt/iohyve/ISO. Also tried using the iohyve fetch command to download the latest Debian ISO and got an error.

[root@tardis] ~# iohyve fetch
fetch: /iohyve/ISO/debian-8.3.0-amd64-netinst.iso: open(): No such file or directory

#6 Updated by Dave F over 4 years ago

After that error, I see a new dataset related to the ISO image, but it's empty.

[root@tardis] ~# iohyve isolist
Listing ISO's...
[root@tardis] ~# ls /mnt/iohyve/ISO/debian-8.3.0-amd64-netinst.iso/
./  ../
[root@tardis] ~#

#7 Updated by William Grzybowski over 4 years ago

  • Status changed from Closed: User Config Issue to Investigation

Apparently thats not how iohyve works, you have to use fetch as it will create a dataset for the iso.

Looks like iohyve doesn't not play well with pools imported with -R.

Try to workaround it with ln -s /mnt/iohyve /iohyve

#8 Updated by Dave F over 4 years ago

Linking /mnt/iohyve to /iohyve allows the ISO to be downloaded and I'm able to install the VM. However, my main concern is will that symbolic link be preserved following a reboot?

Is there any reason to use iohyve vs something like vm-bhyve? I've used vm-bhyve ( with much success on FreeBSD 10.2, also allowing for virtual switches and VLAN support. Or is iohyve becoming part of the middle ware in FN10?

#9 Updated by William Grzybowski over 4 years ago

Yes, it will be preserved following a reboot.

As far as iohyve vs vm-bhyve, I have no clue, I suppose it was an arbitrary choice from Jordan. Its not becoming part of the middleware.

#10 Updated by Jordan Hubbard over 4 years ago

  • Category changed from 201 to 283
  • Assignee changed from William Grzybowski to Jordan Hubbard

I admit that my choice of iohyve was totally arbitrary and quite possibly wrong, since I have had a lot of trouble with its ISO handling sub-commands myself. Before I brought it into 9.10, I basically fetched an ISO manually, registered it manually, then managed to install a FreeBSD 11 VM and attach a console to it. I didn't test Linux.

I will check out vm-bhyve, and if it's a better choice, we can switch sooner rather than later. As I said in the release notes, all of this is just a stop-gap solution in any case.

#11 Updated by Jordan Hubbard over 4 years ago

  • Status changed from Investigation to Resolved

Seems to work with the version of iohyve that we shipped in the 9.10 SU.

#12 Updated by Dru Lavigne almost 3 years ago

  • Target version set to Master - FreeNAS Nightlies

Also available in: Atom PDF