Project

General

Profile

Bug #23357

Add ability to create Docker container in VM using RancherOS

Added by John Hixson over 1 year ago. Updated 10 months ago.

Status:
Resolved
Priority:
Important
Assignee:
Marcelo Araujo
Category:
Middleware
Target version:
Seen in:
Sprint:
Severity:
New
Backlog Priority:
Reason for Closing:
Reason for Blocked:
Needs QA:
Yes
Needs Doc:
Yes
Needs Merging:
Yes
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

We need a checkbox in the VM UI that has something like "Enable RancherOS". It will download and install RancherOS (or grab pre-configured VM, whatever necessary). We will provide a HOWTO for setting it up with NFS/S3 so that Docker containers can use them.

howto.pdf (391 KB) howto.pdf Marcelo Araujo, 11/27/2017 09:44 PM

Related issues

Related to FreeNAS - Feature #23393: Add ability to configure NFS passthrough into a Docker VMClosed2017-04-14

Associated revisions

Revision 63dd42cc (diff)
Added by Warren Block 11 months ago

Document Rancher VM setup

Ticket: #23357

History

#1 Updated by John Hixson over 1 year ago

  • Status changed from Unscreened to Screened

Coming soon. Finishing up another ticket and then on to this.

#2 Updated by John Hixson over 1 year ago

  • Assignee changed from John Hixson to Marcelo Araujo

#3 Updated by Adam Gibson over 1 year ago

Maybe a more generic option to "Enable Docker/Container VM" with ability to select RancherOS or alternative like Portainer (or instances of both).

#4 Updated by Marcelo Araujo over 1 year ago

  • Target version changed from 9.10.3 to 9.10.4

#5 Updated by Marcelo Araujo over 1 year ago

  • Target version changed from 9.10.4 to 9.10.3

Let get it into 9.10.3!

#6 Avatar?id=14398&size=24x24 Updated by Kris Moore over 1 year ago

  • Target version changed from 9.10.3 to 11.0

#7 Avatar?id=14398&size=24x24 Updated by Kris Moore over 1 year ago

  • Target version changed from 11.0 to 11.1

#8 Avatar?id=14398&size=24x24 Updated by Kris Moore 12 months ago

  • Target version changed from 11.1 to 11.1-U1

#9 Updated by Marcelo Araujo 12 months ago

  • Status changed from Screened to Ready For Release

Hi All,

After months of tests and talks, we have decided to use RancherOS as the main system for our docker container.

Rancher brings a lot of advantages for containers, it has a nice UI called RancherUI and you can make an orchestration with lots of other containers just installing a simple agent.

Also soon Rancher 2.0 will bring great user experience with Kubernetes whether managing it using Rancher or existing Kubernetes clusters.

The feature was committed at: https://github.com/freenas/freenas/commit/829484373de48c310e544a72a5a67d4f52963ea0

#10 Updated by Sam Mousa 11 months ago

Marcelo Araujo wrote:

Hi All,

After months of tests and talks, we have decided to use RancherOS as the main system for our docker container.

Rancher brings a lot of advantages for containers, it has a nice UI called RancherUI and you can make an orchestration with lots of other containers just installing a simple agent.

Also soon Rancher 2.0 will bring great user experience with Kubernetes whether managing it using Rancher or existing Kubernetes clusters.

The feature was committed at: https://github.com/freenas/freenas/commit/829484373de48c310e544a72a5a67d4f52963ea0

Hi Marcelo,

I'm rediscovering FreeNAS 11 (last version I think I used was around 6 or so).
I see that you want to use RancherOS combined with RancherUI.
While I'm a very happy user of Rancher, it is quite heavy, also starting from 2.0 everything will be Kubernetes only (their cattle orchestrator will run on top of kubernetes).

For a typical NAS it might make more sense to do something simpler and less resource intensive. I just found https://portainer.io which looks really cool.

Here's a comparison of several options: https://www.servethehome.com/docker-swarm-management-a-quick-overview-of-rancher-portainer-and-shipyard/

In general I don't think you really need to commit to one option since these management systems usually run as a (simple / lightweight) container exposing a web interface.
So if you choose an OS, like RancherOS for virtualization, you could then theoretically choose any container image to use as a management interface.

A reverse proxy could then expose and secure the web interface.
I'm a developer but unfortunately I do not speak Python.

Launching portainer is as easy as:

docker run --rm -p 9000:9000 -v /var/run/docker.sock:/var/run/docker.sock -v ~/portainer:/data portainer/portainer --no-auth

The connection would then go:
1. FreeNAS authenticating proxy.
2. Port forwarded from localhost:xxx to VM:9000 (VM behind NAT)
3. VM:9000 forwarded to Portainer container.

Since RancherOS supports a cloud-config.yml configuration file, that could be used to run the portainer container on boot.
http://rancher.com/docs/os/v1.1/en/system-services/custom-system-services/#launching-services-through-cloud-config

Anyway, I'll keep tracking this and see if I can pitch in somehow!

#11 Updated by Marcelo Araujo 11 months ago

Sam Mousa wrote:

Hi Marcelo,

I'm rediscovering FreeNAS 11 (last version I think I used was around 6 or so).
I see that you want to use RancherOS combined with RancherUI.
While I'm a very happy user of Rancher, it is quite heavy, also starting from 2.0 everything will be Kubernetes only (their cattle orchestrator will run on top of kubernetes).

For a typical NAS it might make more sense to do something simpler and less resource intensive. I just found https://portainer.io which looks really cool.

Here's a comparison of several options: https://www.servethehome.com/docker-swarm-management-a-quick-overview-of-rancher-portainer-and-shipyard/

In general I don't think you really need to commit to one option since these management systems usually run as a (simple / lightweight) container exposing a web interface.
So if you choose an OS, like RancherOS for virtualization, you could then theoretically choose any container image to use as a management interface.

A reverse proxy could then expose and secure the web interface.
I'm a developer but unfortunately I do not speak Python.

Launching portainer is as easy as:

docker run --rm -p 9000:9000 -v /var/run/docker.sock:/var/run/docker.sock -v ~/portainer:/data portainer/portainer --no-auth

The connection would then go:
1. FreeNAS authenticating proxy.
2. Port forwarded from localhost:xxx to VM:9000 (VM behind NAT)
3. VM:9000 forwarded to Portainer container.

Since RancherOS supports a cloud-config.yml configuration file, that could be used to run the portainer container on boot.
http://rancher.com/docs/os/v1.1/en/system-services/custom-system-services/#launching-services-through-cloud-config

Anyway, I'll keep tracking this and see if I can pitch in somehow!

Hi,

I don't install by default RancherUI that will requires a container with at least 2G of RAM, we only use RancherOS that needs at least 1G of RAM! However you are correct, you can install any other options inside RancherOS, you don't need stick on RancherUI and its Agents. Also, you can install any docker image on it and even replace the userland part if you wish. RancherOS is doing a good job with its OS and their community is very responsive.

About portainer, I'm aware of it and it is a nice project too, but on FreeNAS case, you can use anything from RancherUI to Portainer, we only use RancherOS as the OS.

Best,

#12 Updated by Tobias Müllauer 11 months ago

I really love this Energy with this implantation.

I wonder if there are any chans to start testing this new feature today? today i am at freenas-11.0-U2.
if so what do i need to do?

i am really hungry to test this docker technology. and i am really starving =P

#13 Updated by Marcelo Araujo 11 months ago

Tobias Müllauer wrote:

I really love this Energy with this implantation.

I wonder if there are any chans to start testing this new feature today? today i am at freenas-11.0-U2.
if so what do i need to do?

i am really hungry to test this docker technology. and i am really starving =P

Hey, thanks for the hands up!

Right now you can test it on FreeNAS-11-Nightlies, however for any upgrade you must take some precautions like backup your data and configurations prior to upgrade.

I spent a good amount of time testing this new feature, but I would be more than happy if more people are testing it too, report bugs, giving feedbacks and some howto about their own setup.

It all helps at this stage, as this feature is new.

Thanks again!

#14 Updated by Dru Lavigne 11 months ago

  • Status changed from Ready For Release to 46
  • Target version changed from 11.1-U1 to 11.1

Marcelo: can you double-check that this was cherry-picked to stable? I see the merge to master but may have missed the stable commit. If it makes more sense to have this in 11.2, I'll change the target and no cherry-pick is needed.

#15 Updated by Marcelo Araujo 11 months ago

Dru Lavigne wrote:

Marcelo: can you double-check that this was cherry-picked to stable? I see the merge to master but may have missed the stable commit. If it makes more sense to have this in 11.2, I'll change the target and no cherry-pick is needed.

It was done here: https://github.com/freenas/freenas/commit/3629d34b5703001657ab65d1fcb13367ada1e240

This PR is based in the same PR to master, so all the bits must be there. I didn't include anything else related with this feature in master after the PR merge to master.

Br,

#16 Updated by Marcelo Araujo 11 months ago

  • Status changed from 46 to Ready For Release
  • Assignee changed from Marcelo Araujo to Dru Lavigne

Hi,

Not sure if I should send it back to you or not.

Best,

#17 Updated by Dru Lavigne 11 months ago

  • Assignee changed from Dru Lavigne to Marcelo Araujo

#18 Updated by Dru Lavigne 11 months ago

  • Target version changed from 11.1 to 11.1-RC2

#19 Updated by Dru Lavigne 11 months ago

  • Subject changed from Create downloadable RancherOS bhyve image and simple UI to get it for Docker containers to Add ability to crete Docker container in VM using RancherOS

#20 Updated by Dru Lavigne 11 months ago

  • Subject changed from Add ability to crete Docker container in VM using RancherOS to Add ability to create Docker container in VM using RancherOS

#21 Updated by Tim Lee 11 months ago

Marcelo Araujo wrote:

Tobias Müllauer wrote:

I really love this Energy with this implantation.

I wonder if there are any chans to start testing this new feature today? today i am at freenas-11.0-U2.
if so what do i need to do?

i am really hungry to test this docker technology. and i am really starving =P

Hey, thanks for the hands up!

Right now you can test it on FreeNAS-11-Nightlies, however for any upgrade you must take some precautions like backup your data and configurations prior to upgrade.

I spent a good amount of time testing this new feature, but I would be more than happy if more people are testing it too, report bugs, giving feedbacks and some howto about their own setup.

It all helps at this stage, as this feature is new.

Thanks again!

Tobias,
I had tried to test this out but it seems that the raw image wasn't created properly so the VM wouldn't boot. I then opened the device settings for the VM and on the RAW image file there was no path. After looking at the source code I think the issue may have been that when I initially tried to create the VM, I didn't yet have any storage pool besides the freenas-boot pool. Figuring this was the problem, I deleted the VM and tried again after creating a storage pool called "storage". Unfortunately, I didn't see any download progress and the VM was added instantly but still doesn't boot and again didn't have the RAW image path. I have tried each of the following things and after recreating the VM had the same symptoms.
1. Deleting "/tmp/.vm_container_download".
2. Destroying the "storage/.bhyve_containers" volume.
Is there a way to reset this without wiping the system drive and starting over?

Thanks for your hard work!

#22 Updated by Marcelo Araujo 11 months ago

Tim Lee wrote:

Marcelo Araujo wrote:

Tobias Müllauer wrote:

I really love this Energy with this implantation.

I wonder if there are any chans to start testing this new feature today? today i am at freenas-11.0-U2.
if so what do i need to do?

i am really hungry to test this docker technology. and i am really starving =P

Hey, thanks for the hands up!

Right now you can test it on FreeNAS-11-Nightlies, however for any upgrade you must take some precautions like backup your data and configurations prior to upgrade.

I spent a good amount of time testing this new feature, but I would be more than happy if more people are testing it too, report bugs, giving feedbacks and some howto about their own setup.

It all helps at this stage, as this feature is new.

Thanks again!

Tobias,
I had tried to test this out but it seems that the raw image wasn't created properly so the VM wouldn't boot. I then opened the device settings for the VM and on the RAW image file there was no path. After looking at the source code I think the issue may have been that when I initially tried to create the VM, I didn't yet have any storage pool besides the freenas-boot pool. Figuring this was the problem, I deleted the VM and tried again after creating a storage pool called "storage". Unfortunately, I didn't see any download progress and the VM was added instantly but still doesn't boot and again didn't have the RAW image path. I have tried each of the following things and after recreating the VM had the same symptoms.
1. Deleting "/tmp/.vm_container_download".
2. Destroying the "storage/.bhyve_containers" volume.
Is there a way to reset this without wiping the system drive and starting over?

Thanks for your hard work!

Yes there is a way!
You must do 'zfs list' and check where is '.bhyve_containers' as an example on my system:
root@freenas:/mnt/ssd2 # zfs list | grep bhyve
ssd/.bhyve_containers 44.9M 184G 44.9M /mnt/ssd/.bhyve_containers

Then you need to destroy it: zfs destroy ssd/.bhyve_containers

After that you will be able to run it again.

#23 Updated by Tim Lee 11 months ago

Marcelo Araujo wrote:

Tim Lee wrote:

Marcelo Araujo wrote:

Tobias Müllauer wrote:

I really love this Energy with this implantation.

I wonder if there are any chans to start testing this new feature today? today i am at freenas-11.0-U2.
if so what do i need to do?

i am really hungry to test this docker technology. and i am really starving =P

Hey, thanks for the hands up!

Right now you can test it on FreeNAS-11-Nightlies, however for any upgrade you must take some precautions like backup your data and configurations prior to upgrade.

I spent a good amount of time testing this new feature, but I would be more than happy if more people are testing it too, report bugs, giving feedbacks and some howto about their own setup.

It all helps at this stage, as this feature is new.

Thanks again!

Tobias,
I had tried to test this out but it seems that the raw image wasn't created properly so the VM wouldn't boot. I then opened the device settings for the VM and on the RAW image file there was no path. After looking at the source code I think the issue may have been that when I initially tried to create the VM, I didn't yet have any storage pool besides the freenas-boot pool. Figuring this was the problem, I deleted the VM and tried again after creating a storage pool called "storage". Unfortunately, I didn't see any download progress and the VM was added instantly but still doesn't boot and again didn't have the RAW image path. I have tried each of the following things and after recreating the VM had the same symptoms.
1. Deleting "/tmp/.vm_container_download".
2. Destroying the "storage/.bhyve_containers" volume.
Is there a way to reset this without wiping the system drive and starting over?

Thanks for your hard work!

Yes there is a way!
You must do 'zfs list' and check where is '.bhyve_containers' as an example on my system:
root@freenas:/mnt/ssd2 # zfs list | grep bhyve
ssd/.bhyve_containers 44.9M 184G 44.9M /mnt/ssd/.bhyve_containers

Then you need to destroy it: zfs destroy ssd/.bhyve_containers

After that you will be able to run it again.

I had already tried that but just tried it again. I still end up with an instantly created VM with a RAW file with no path. Any other ideas?

#24 Updated by Marcelo Araujo 11 months ago

Tim Lee wrote:

Marcelo Araujo wrote:

Tim Lee wrote:

Marcelo Araujo wrote:

Tobias Müllauer wrote:

I really love this Energy with this implantation.

I wonder if there are any chans to start testing this new feature today? today i am at freenas-11.0-U2.
if so what do i need to do?

i am really hungry to test this docker technology. and i am really starving =P

Hey, thanks for the hands up!

Right now you can test it on FreeNAS-11-Nightlies, however for any upgrade you must take some precautions like backup your data and configurations prior to upgrade.

I spent a good amount of time testing this new feature, but I would be more than happy if more people are testing it too, report bugs, giving feedbacks and some howto about their own setup.

It all helps at this stage, as this feature is new.

Thanks again!

Tobias,
I had tried to test this out but it seems that the raw image wasn't created properly so the VM wouldn't boot. I then opened the device settings for the VM and on the RAW image file there was no path. After looking at the source code I think the issue may have been that when I initially tried to create the VM, I didn't yet have any storage pool besides the freenas-boot pool. Figuring this was the problem, I deleted the VM and tried again after creating a storage pool called "storage". Unfortunately, I didn't see any download progress and the VM was added instantly but still doesn't boot and again didn't have the RAW image path. I have tried each of the following things and after recreating the VM had the same symptoms.
1. Deleting "/tmp/.vm_container_download".
2. Destroying the "storage/.bhyve_containers" volume.
Is there a way to reset this without wiping the system drive and starting over?

Thanks for your hard work!

Yes there is a way!
You must do 'zfs list' and check where is '.bhyve_containers' as an example on my system:
root@freenas:/mnt/ssd2 # zfs list | grep bhyve
ssd/.bhyve_containers 44.9M 184G 44.9M /mnt/ssd/.bhyve_containers

Then you need to destroy it: zfs destroy ssd/.bhyve_containers

After that you will be able to run it again.

I had already tried that but just tried it again. I still end up with an instantly created VM with a RAW file with no path. Any other ideas?

Yes, you need to add a path to that RAW device.
As an example from my system:
My pool is at: /mnt/ssd2
I need to be sure there is no file there with the same name that I will use at the RAW device.

At the RAW device it will looks like:
/mnt/ssd2/MyRancherOS.img <--- You can give any other name, just be sure MyRancherOS.img doesn't exist yet in that path.

Let me know if it works for you.

#25 Updated by Tim Lee 11 months ago

Marcelo Araujo wrote:

Tim Lee wrote:

Marcelo Araujo wrote:

Tim Lee wrote:

Marcelo Araujo wrote:

Tobias Müllauer wrote:

I really love this Energy with this implantation.

I wonder if there are any chans to start testing this new feature today? today i am at freenas-11.0-U2.
if so what do i need to do?

i am really hungry to test this docker technology. and i am really starving =P

Hey, thanks for the hands up!

Right now you can test it on FreeNAS-11-Nightlies, however for any upgrade you must take some precautions like backup your data and configurations prior to upgrade.

I spent a good amount of time testing this new feature, but I would be more than happy if more people are testing it too, report bugs, giving feedbacks and some howto about their own setup.

It all helps at this stage, as this feature is new.

Thanks again!

Tobias,
I had tried to test this out but it seems that the raw image wasn't created properly so the VM wouldn't boot. I then opened the device settings for the VM and on the RAW image file there was no path. After looking at the source code I think the issue may have been that when I initially tried to create the VM, I didn't yet have any storage pool besides the freenas-boot pool. Figuring this was the problem, I deleted the VM and tried again after creating a storage pool called "storage". Unfortunately, I didn't see any download progress and the VM was added instantly but still doesn't boot and again didn't have the RAW image path. I have tried each of the following things and after recreating the VM had the same symptoms.
1. Deleting "/tmp/.vm_container_download".
2. Destroying the "storage/.bhyve_containers" volume.
Is there a way to reset this without wiping the system drive and starting over?

Thanks for your hard work!

Yes there is a way!
You must do 'zfs list' and check where is '.bhyve_containers' as an example on my system:
root@freenas:/mnt/ssd2 # zfs list | grep bhyve
ssd/.bhyve_containers 44.9M 184G 44.9M /mnt/ssd/.bhyve_containers

Then you need to destroy it: zfs destroy ssd/.bhyve_containers

After that you will be able to run it again.

I had already tried that but just tried it again. I still end up with an instantly created VM with a RAW file with no path. Any other ideas?

Yes, you need to add a path to that RAW device.
As an example from my system:
My pool is at: /mnt/ssd2
I need to be sure there is no file there with the same name that I will use at the RAW device.

At the RAW device it will looks like:
/mnt/ssd2/MyRancherOS.img <--- You can give any other name, just be sure MyRancherOS.img doesn't exist yet in that path.

Let me know if it works for you.

That does allow the image to be created but when starting the VM it stops right away. Here are the terminal messages:
"kernel: tap3: promiscuous mode enabled
Nov 27 21:50:13 tlss2 dhclient: New IP Address (ix1): 192.168.10.254
Nov 27 21:50:13 tlss2 dhclient: New Subnet Mask (ix1): 255.255.255.0
Nov 27 21:50:13 tlss2 dhclient: New Broadcast Address (ix1): 192.168.10.255
Nov 27 21:50:13 tlss2 dhclient: New Routers (ix1): 192.168.10.1
Nov 27 21:50:13 tlss2 kernel: ix1: link state changed to UP
Nov 27 21:50:13 tlss2 kernel: ix1: link state changed to UP
Nov 27 21:50:13 tlss2 kernel: bridge0: link state changed to UP
Nov 27 21:50:13 tlss2 kernel: bridge0: link state changed to UP"

Are there more logs for the virtual machine interface? I couldn't find where they are located.

Thanks,

#26 Updated by Marcelo Araujo 11 months ago

Tim Lee wrote:

Marcelo Araujo wrote:

Tim Lee wrote:

Marcelo Araujo wrote:

Tim Lee wrote:

Marcelo Araujo wrote:

Tobias Müllauer wrote:

I really love this Energy with this implantation.

I wonder if there are any chans to start testing this new feature today? today i am at freenas-11.0-U2.
if so what do i need to do?

i am really hungry to test this docker technology. and i am really starving =P

Hey, thanks for the hands up!

Right now you can test it on FreeNAS-11-Nightlies, however for any upgrade you must take some precautions like backup your data and configurations prior to upgrade.

I spent a good amount of time testing this new feature, but I would be more than happy if more people are testing it too, report bugs, giving feedbacks and some howto about their own setup.

It all helps at this stage, as this feature is new.

Thanks again!

Tobias,
I had tried to test this out but it seems that the raw image wasn't created properly so the VM wouldn't boot. I then opened the device settings for the VM and on the RAW image file there was no path. After looking at the source code I think the issue may have been that when I initially tried to create the VM, I didn't yet have any storage pool besides the freenas-boot pool. Figuring this was the problem, I deleted the VM and tried again after creating a storage pool called "storage". Unfortunately, I didn't see any download progress and the VM was added instantly but still doesn't boot and again didn't have the RAW image path. I have tried each of the following things and after recreating the VM had the same symptoms.
1. Deleting "/tmp/.vm_container_download".
2. Destroying the "storage/.bhyve_containers" volume.
Is there a way to reset this without wiping the system drive and starting over?

Thanks for your hard work!

Yes there is a way!
You must do 'zfs list' and check where is '.bhyve_containers' as an example on my system:
root@freenas:/mnt/ssd2 # zfs list | grep bhyve
ssd/.bhyve_containers 44.9M 184G 44.9M /mnt/ssd/.bhyve_containers

Then you need to destroy it: zfs destroy ssd/.bhyve_containers

After that you will be able to run it again.

I had already tried that but just tried it again. I still end up with an instantly created VM with a RAW file with no path. Any other ideas?

Yes, you need to add a path to that RAW device.
As an example from my system:
My pool is at: /mnt/ssd2
I need to be sure there is no file there with the same name that I will use at the RAW device.

At the RAW device it will looks like:
/mnt/ssd2/MyRancherOS.img <--- You can give any other name, just be sure MyRancherOS.img doesn't exist yet in that path.

Let me know if it works for you.

That does allow the image to be created but when starting the VM it stops right away. Here are the terminal messages:
"kernel: tap3: promiscuous mode enabled
Nov 27 21:50:13 tlss2 dhclient: New IP Address (ix1): 192.168.10.254
Nov 27 21:50:13 tlss2 dhclient: New Subnet Mask (ix1): 255.255.255.0
Nov 27 21:50:13 tlss2 dhclient: New Broadcast Address (ix1): 192.168.10.255
Nov 27 21:50:13 tlss2 dhclient: New Routers (ix1): 192.168.10.1
Nov 27 21:50:13 tlss2 kernel: ix1: link state changed to UP
Nov 27 21:50:13 tlss2 kernel: ix1: link state changed to UP
Nov 27 21:50:13 tlss2 kernel: bridge0: link state changed to UP
Nov 27 21:50:13 tlss2 kernel: bridge0: link state changed to UP"

Are there more logs for the virtual machine interface? I couldn't find where they are located.

Thanks,

more logs at /var/log/middlewared.log

It looks like the VM is up and running.

You can share your logs with me if you want and I can take a look.
Go to: System->Advanced->Save Debug and attach it here.

#27 Updated by Tim Lee 11 months ago

  • File debug-tlss2-20171127231723.tgz added

Marcelo Araujo wrote:

Tim Lee wrote:

Marcelo Araujo wrote:

Tim Lee wrote:

Marcelo Araujo wrote:

Tim Lee wrote:

Marcelo Araujo wrote:

Tobias Müllauer wrote:

I really love this Energy with this implantation.

I wonder if there are any chans to start testing this new feature today? today i am at freenas-11.0-U2.
if so what do i need to do?

i am really hungry to test this docker technology. and i am really starving =P

Hey, thanks for the hands up!

Right now you can test it on FreeNAS-11-Nightlies, however for any upgrade you must take some precautions like backup your data and configurations prior to upgrade.

I spent a good amount of time testing this new feature, but I would be more than happy if more people are testing it too, report bugs, giving feedbacks and some howto about their own setup.

It all helps at this stage, as this feature is new.

Thanks again!

Tobias,
I had tried to test this out but it seems that the raw image wasn't created properly so the VM wouldn't boot. I then opened the device settings for the VM and on the RAW image file there was no path. After looking at the source code I think the issue may have been that when I initially tried to create the VM, I didn't yet have any storage pool besides the freenas-boot pool. Figuring this was the problem, I deleted the VM and tried again after creating a storage pool called "storage". Unfortunately, I didn't see any download progress and the VM was added instantly but still doesn't boot and again didn't have the RAW image path. I have tried each of the following things and after recreating the VM had the same symptoms.
1. Deleting "/tmp/.vm_container_download".
2. Destroying the "storage/.bhyve_containers" volume.
Is there a way to reset this without wiping the system drive and starting over?

Thanks for your hard work!

Yes there is a way!
You must do 'zfs list' and check where is '.bhyve_containers' as an example on my system:
root@freenas:/mnt/ssd2 # zfs list | grep bhyve
ssd/.bhyve_containers 44.9M 184G 44.9M /mnt/ssd/.bhyve_containers

Then you need to destroy it: zfs destroy ssd/.bhyve_containers

After that you will be able to run it again.

I had already tried that but just tried it again. I still end up with an instantly created VM with a RAW file with no path. Any other ideas?

Yes, you need to add a path to that RAW device.
As an example from my system:
My pool is at: /mnt/ssd2
I need to be sure there is no file there with the same name that I will use at the RAW device.

At the RAW device it will looks like:
/mnt/ssd2/MyRancherOS.img <--- You can give any other name, just be sure MyRancherOS.img doesn't exist yet in that path.

Let me know if it works for you.

That does allow the image to be created but when starting the VM it stops right away. Here are the terminal messages:
"kernel: tap3: promiscuous mode enabled
Nov 27 21:50:13 tlss2 dhclient: New IP Address (ix1): 192.168.10.254
Nov 27 21:50:13 tlss2 dhclient: New Subnet Mask (ix1): 255.255.255.0
Nov 27 21:50:13 tlss2 dhclient: New Broadcast Address (ix1): 192.168.10.255
Nov 27 21:50:13 tlss2 dhclient: New Routers (ix1): 192.168.10.1
Nov 27 21:50:13 tlss2 kernel: ix1: link state changed to UP
Nov 27 21:50:13 tlss2 kernel: ix1: link state changed to UP
Nov 27 21:50:13 tlss2 kernel: bridge0: link state changed to UP
Nov 27 21:50:13 tlss2 kernel: bridge0: link state changed to UP"

Are there more logs for the virtual machine interface? I couldn't find where they are located.

Thanks,

more logs at /var/log/middlewared.log

It looks like the VM is up and running.

You can share your logs with me if you want and I can take a look.
Go to: System->Advanced->Save Debug and attach it here.

#28 Updated by Tim Lee 11 months ago

Tim Lee wrote:

Marcelo Araujo wrote:

Tim Lee wrote:

Marcelo Araujo wrote:

Tim Lee wrote:

Marcelo Araujo wrote:

Tim Lee wrote:

Marcelo Araujo wrote:

Tobias Müllauer wrote:

I really love this Energy with this implantation.

I wonder if there are any chans to start testing this new feature today? today i am at freenas-11.0-U2.
if so what do i need to do?

i am really hungry to test this docker technology. and i am really starving =P

Hey, thanks for the hands up!

Right now you can test it on FreeNAS-11-Nightlies, however for any upgrade you must take some precautions like backup your data and configurations prior to upgrade.

I spent a good amount of time testing this new feature, but I would be more than happy if more people are testing it too, report bugs, giving feedbacks and some howto about their own setup.

It all helps at this stage, as this feature is new.

Thanks again!

Tobias,
I had tried to test this out but it seems that the raw image wasn't created properly so the VM wouldn't boot. I then opened the device settings for the VM and on the RAW image file there was no path. After looking at the source code I think the issue may have been that when I initially tried to create the VM, I didn't yet have any storage pool besides the freenas-boot pool. Figuring this was the problem, I deleted the VM and tried again after creating a storage pool called "storage". Unfortunately, I didn't see any download progress and the VM was added instantly but still doesn't boot and again didn't have the RAW image path. I have tried each of the following things and after recreating the VM had the same symptoms.
1. Deleting "/tmp/.vm_container_download".
2. Destroying the "storage/.bhyve_containers" volume.
Is there a way to reset this without wiping the system drive and starting over?

Thanks for your hard work!

Yes there is a way!
You must do 'zfs list' and check where is '.bhyve_containers' as an example on my system:
root@freenas:/mnt/ssd2 # zfs list | grep bhyve
ssd/.bhyve_containers 44.9M 184G 44.9M /mnt/ssd/.bhyve_containers

Then you need to destroy it: zfs destroy ssd/.bhyve_containers

After that you will be able to run it again.

I had already tried that but just tried it again. I still end up with an instantly created VM with a RAW file with no path. Any other ideas?

Yes, you need to add a path to that RAW device.
As an example from my system:
My pool is at: /mnt/ssd2
I need to be sure there is no file there with the same name that I will use at the RAW device.

At the RAW device it will looks like:
/mnt/ssd2/MyRancherOS.img <--- You can give any other name, just be sure MyRancherOS.img doesn't exist yet in that path.

Let me know if it works for you.

That does allow the image to be created but when starting the VM it stops right away. Here are the terminal messages:
"kernel: tap3: promiscuous mode enabled
Nov 27 21:50:13 tlss2 dhclient: New IP Address (ix1): 192.168.10.254
Nov 27 21:50:13 tlss2 dhclient: New Subnet Mask (ix1): 255.255.255.0
Nov 27 21:50:13 tlss2 dhclient: New Broadcast Address (ix1): 192.168.10.255
Nov 27 21:50:13 tlss2 dhclient: New Routers (ix1): 192.168.10.1
Nov 27 21:50:13 tlss2 kernel: ix1: link state changed to UP
Nov 27 21:50:13 tlss2 kernel: ix1: link state changed to UP
Nov 27 21:50:13 tlss2 kernel: bridge0: link state changed to UP
Nov 27 21:50:13 tlss2 kernel: bridge0: link state changed to UP"

Are there more logs for the virtual machine interface? I couldn't find where they are located.

Thanks,

more logs at /var/log/middlewared.log

It looks like the VM is up and running.

You can share your logs with me if you want and I can take a look.
Go to: System->Advanced->Save Debug and attach it here.

Here you go. I wasn't able to find anything related to the VM starting in the middlewared.log file.
Thanks for your help.

#29 Updated by Marcelo Araujo 11 months ago

  • File RancherOS-FreeNAS.pdf added

I didn't see anything useful in your logs.
So, please.. can you follow this howto attached?

#30 Updated by Marcelo Araujo 11 months ago

  • File deleted (RancherOS-FreeNAS.pdf)

#31 Updated by Marcelo Araujo 11 months ago

Now this is the correct file.

#32 Updated by Dru Lavigne 11 months ago

  • Target version changed from 11.1-RC2 to 11.1-RC3

#33 Updated by Tim Lee 11 months ago

  • Target version changed from 11.1-RC3 to 11.1-RC2

Marcelo Araujo wrote:

Now this is the correct file.

I'm having issues trying to follow the instructions:
1. The RancherUI VM always states that it is stopped at every stage even after clicking start (I have deleted the .bhyve_containers and started from scratch to follow the instructions)
2. When trying to create the Node0 VM, it doesn't create any devices (such as a NIC or RAW device) and doesn't allow me to create any.
3. This also happens when trying to create Node1.

After updating to the latest nightly (11/28/17) and deleting the the .bhyve_containers dataset and starting over there are no devices in the RancherUI VM to begin with. Any idea what might be happening here?

Thanks for your help.

#34 Updated by Nick Wolff 11 months ago

  • Target version changed from 11.1-RC2 to 11.1-RC3

#35 Updated by Tim Lee 11 months ago

  • File deleted (debug-tlss2-20171127231723.tgz)

#36 Updated by Marcelo Araujo 11 months ago

Tim Lee wrote:

Marcelo Araujo wrote:

Now this is the correct file.

I'm having issues trying to follow the instructions:
1. The RancherUI VM always states that it is stopped at every stage even after clicking start (I have deleted the .bhyve_containers and started from scratch to follow the instructions)
2. When trying to create the Node0 VM, it doesn't create any devices (such as a NIC or RAW device) and doesn't allow me to create any.
3. This also happens when trying to create Node1.

After updating to the latest nightly (11/28/17) and deleting the the .bhyve_containers dataset and starting over there are no devices in the RancherUI VM to begin with. Any idea what might be happening here?

Thanks for your help.

Hello Tim,

As you are using nightly, please update it again! Unfortunately I introduced a regression at Container feature when I fixed a bug related with VMs with unique name. However, this regression was spotted after couple hours and I fixed it already.

Sorry by the regression, update again and let me know how it is.

#37 Updated by Tim Lee 11 months ago

Marcelo Araujo wrote:

Tim Lee wrote:

Marcelo Araujo wrote:

Now this is the correct file.

I'm having issues trying to follow the instructions:
1. The RancherUI VM always states that it is stopped at every stage even after clicking start (I have deleted the .bhyve_containers and started from scratch to follow the instructions)
2. When trying to create the Node0 VM, it doesn't create any devices (such as a NIC or RAW device) and doesn't allow me to create any.
3. This also happens when trying to create Node1.

After updating to the latest nightly (11/28/17) and deleting the the .bhyve_containers dataset and starting over there are no devices in the RancherUI VM to begin with. Any idea what might be happening here?

Thanks for your help.

Hello Tim,

As you are using nightly, please update it again! Unfortunately I introduced a regression at Container feature when I fixed a bug related with VMs with unique name. However, this regression was spotted after couple hours and I fixed it already.

Sorry by the regression, update again and let me know how it is.

So I updated to the 11/29 nightly and now I can get the VM's to start after following the process from the beginning. Unfortunately though when attempting to connect to the VM's consoles, I just get the connected message with no terminal capability. Any ideas? I will look at the logs to see if anything pops up.

#38 Updated by Marcelo Araujo 11 months ago

Tim Lee wrote:

Marcelo Araujo wrote:

Tim Lee wrote:

Marcelo Araujo wrote:

Now this is the correct file.

I'm having issues trying to follow the instructions:
1. The RancherUI VM always states that it is stopped at every stage even after clicking start (I have deleted the .bhyve_containers and started from scratch to follow the instructions)
2. When trying to create the Node0 VM, it doesn't create any devices (such as a NIC or RAW device) and doesn't allow me to create any.
3. This also happens when trying to create Node1.

After updating to the latest nightly (11/28/17) and deleting the the .bhyve_containers dataset and starting over there are no devices in the RancherUI VM to begin with. Any idea what might be happening here?

Thanks for your help.

Hello Tim,

As you are using nightly, please update it again! Unfortunately I introduced a regression at Container feature when I fixed a bug related with VMs with unique name. However, this regression was spotted after couple hours and I fixed it already.

Sorry by the regression, update again and let me know how it is.

So I updated to the 11/29 nightly and now I can get the VM's to start after following the process from the beginning. Unfortunately though when attempting to connect to the VM's consoles, I just get the connected message with no terminal capability. Any ideas? I will look at the logs to see if anything pops up.

Usually for me it shows a message "Connected" and as soon as I hit <enter> I can see all the console. Isn't it your case?

#39 Updated by Tim Lee 11 months ago

Marcelo Araujo wrote:

Tim Lee wrote:

Marcelo Araujo wrote:

Tim Lee wrote:

Marcelo Araujo wrote:

Now this is the correct file.

I'm having issues trying to follow the instructions:
1. The RancherUI VM always states that it is stopped at every stage even after clicking start (I have deleted the .bhyve_containers and started from scratch to follow the instructions)
2. When trying to create the Node0 VM, it doesn't create any devices (such as a NIC or RAW device) and doesn't allow me to create any.
3. This also happens when trying to create Node1.

After updating to the latest nightly (11/28/17) and deleting the the .bhyve_containers dataset and starting over there are no devices in the RancherUI VM to begin with. Any idea what might be happening here?

Thanks for your help.

Hello Tim,

As you are using nightly, please update it again! Unfortunately I introduced a regression at Container feature when I fixed a bug related with VMs with unique name. However, this regression was spotted after couple hours and I fixed it already.

Sorry by the regression, update again and let me know how it is.

So I updated to the 11/29 nightly and now I can get the VM's to start after following the process from the beginning. Unfortunately though when attempting to connect to the VM's consoles, I just get the connected message with no terminal capability. Any ideas? I will look at the logs to see if anything pops up.

Usually for me it shows a message "Connected" and as soon as I hit <enter> I can see all the console. Isn't it your case?

In my case it says connected with a green cursor below and hitting enter doesn't show me the console.

#40 Updated by Marcelo Araujo 11 months ago

OK!

Can we schedule a time for a teamviewer?

I will contact you via email.

#41 Updated by Dru Lavigne 11 months ago

  • Status changed from Ready For Release to Resolved

#42 Updated by Marcelo Araujo 11 months ago

Marcelo Araujo wrote:

OK!

Can we schedule a time for a teamviewer?

I will contact you via email.

Just for record!
I made a teamviewer with Tim and we found a small issue with grub-bhyve password.

I will open another ticket related with it.

Thanks again Tim.

#43 Updated by Luca Weiss 10 months ago

I have recently upgraded to 11.1-RELEASE and I see the same behaviour as reported in the last couple of comments. The "RAW" device doesn't have a "Raw File" and the middlewared.log says [2017/12/28 10:59:26] (ERROR) VMService.run():217 - ===> There is no boot disk for vm: docker

I have not tried the suggestions given in the comments but there's still obviously a problem with the creation of a Docker VM.

#44 Updated by Luca Weiss 10 months ago

sorry, I found the documentation about this (https://doc.freenas.org/11/vms.html#docker-rancher-vm) but maybe the documentation should be linked in the creation box for the docker vm as it's otherwise unclear why you can't start the vm (it just says "starting..." and then is still stopped).

#45 Updated by Dru Lavigne 9 months ago

  • Has duplicate Feature #23393: Add ability to configure NFS passthrough into a Docker VM added

#46 Updated by Dru Lavigne 9 months ago

  • Has duplicate deleted (Feature #23393: Add ability to configure NFS passthrough into a Docker VM)

#47 Updated by Dru Lavigne 9 months ago

  • Related to Feature #23393: Add ability to configure NFS passthrough into a Docker VM added

Also available in: Atom PDF