Project

General

Profile

Bug #29257

Bug #26773: [NEW UI/VM] Add Docker VM in the new UI.

please fix middleware call for downloading rancheros-bhyve-v1.1.3.img.gz

Added by Vaibhav Chauhan over 2 years ago. Updated over 2 years ago.

Status:
Done
Priority:
Critical
Assignee:
Marcelo Araujo
Category:
OS
Severity:
High
Reason for Closing:
Reason for Blocked:
Needs QA:
No
Needs Doc:
No
Needs Merging:
No
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

so I took following steps to point out that `midclt call vm.fetch_image 'RancherOS'` is broken in following way.

root@freenas:/mnt/data/.bhyve_containers/iso_files # rm rancheros-bhyve-v1.1.3.img.gz

root@freenas:/mnt/data/.bhyve_containers/iso_files # midclt call vm.fetch_image 'RancherOS'
20

root@freenas:/mnt/data/.bhyve_containers/iso_files # midclt call vm.get_download_status '20'
{"id": 20, "method": "vm.fetch_image", "arguments": ["RancherOS"], "progress": {"percent": 18, "description": "Downloading", "extra": {"downloaded": 8855552, "total": 47958299}}, "result": null, "error": "dictionary changed size during iteration", "exception": "Traceback (most recent call last):\n  File \"/usr/local/lib/python3.6/site-packages/middlewared/job.py\", line 293, in run\n    await self.future\n  File \"/usr/local/lib/python3.6/asyncio/coroutines.py\", line 128, in throw\n    return self.gen.throw(type, value, traceback)\n  File \"/usr/local/lib/python3.6/site-packages/middlewared/job.py\", line 346, in __run_body\n    rv = await self.method(*([self] + args))\n  File \"/usr/local/lib/python3.6/asyncio/coroutines.py\", line 128, in throw\n    return self.gen.throw(type, value, traceback)\n  File \"/usr/local/lib/python3.6/site-packages/middlewared/schema.py\", line 502, in nf\n    return await f(*args, **kwargs)\n  File \"/usr/local/lib/python3.6/asyncio/coroutines.py\", line 128, in throw\n    return self.gen.throw(type, value, traceback)\n  File \"/usr/local/lib/python3.6/site-packages/middlewared/plugins/vm.py\", line 1023, in fetch_image\n    await self.middleware.run_in_thread(lambda: urlretrieve(\n  File \"/usr/local/lib/python3.6/asyncio/coroutines.py\", line 128, in throw\n    return self.gen.throw(type, value, traceback)\n  File \"/usr/local/lib/python3.6/site-packages/middlewared/main.py\", line 854, in run_in_thread\n    return await self.run_in_thread_pool(self.__threadpool, method, *args, **kwargs)\n  File \"/usr/local/lib/python3.6/asyncio/coroutines.py\", line 128, in throw\n    return self.gen.throw(type, value, traceback)\n  File \"/usr/local/lib/python3.6/site-packages/middlewared/main.py\", line 850, in run_in_thread_pool\n    await task\n  File \"/usr/local/lib/python3.6/concurrent/futures/thread.py\", line 56, in run\n    result = self.fn(*self.args, **self.kwargs)\n  File \"/usr/local/lib/python3.6/site-packages/middlewared/plugins/vm.py\", line 1026, in <lambda>\n    lambda nb, bs, fs, job=job: self.fetch_hookreport(nb, bs, fs, job, file_path)\n  File \"/usr/local/lib/python3.6/urllib/request.py\", line 284, in urlretrieve\n    reporthook(blocknum, bs, size)\n  File \"/usr/local/lib/python3.6/site-packages/middlewared/plugins/vm.py\", line 1026, in <lambda>\n    lambda nb, bs, fs, job=job: self.fetch_hookreport(nb, bs, fs, job, file_path)\n  File \"/usr/local/lib/python3.6/site-packages/middlewared/plugins/vm.py\", line 1005, in fetch_hookreport\n    job.set_progress(int(percent), 'Downloading', {'downloaded': readchunk, 'total': totalsize})\n  File \"/usr/local/lib/python3.6/site-packages/middlewared/job.py\", line 256, in set_progress\n    self.middleware.send_event('core.get_jobs', 'CHANGED', id=self.id, fields=self.__encode__())\n  File \"/usr/local/lib/python3.6/site-packages/middlewared/main.py\", line 952, in send_event\n    for sessionid, wsclient in self.__wsclients.items():\nRuntimeError: dictionary changed size during iteration\n", "state": "FAILED", "time_started": {"$date": 1520471654710}, "time_finished": {"$date": 1520471660370}}

root@freenas:/mnt/data/.bhyve_containers/iso_files # midclt call vm.query
[{"id": 135, "name": "dokcer1", "description": "", "vcpus": 1, "memory": 1024, "bootloader": "GRUB", "vm_type": "Container Provider", "autostart": false, "devices": [{"dtype": "NIC", "attributes": {"type": "E1000", "mac": "00:a0:98:FF:FF:FF", "nic_attach": "igb0"}}, {"dtype": "RAW", "attributes": {"path": "/mnt/data/test-file", "sectorsize": 0, "mode": "AHCI"}}, {"dtype": "VNC", "attributes": {"wait": true, "vnc_port": "5717", "vnc_resolution": "1024x768", "vnc_bind": "10.250.1.33", "vnc_password": "", "vnc_web": true}}]}]

root@freenas:/mnt/data/.bhyve_containers/iso_files # sha256 rancheros-bhyve-v1.1.3.img.gz
SHA256 (rancheros-bhyve-v1.1.3.img.gz) = 7acc8ba0ad693b656587aa8ad64848b9795de0f3014ebe95aa43be90afd07dbd

also please explain how docker vm creation and start work, including what devices are created by default, and the vm process from start to the end. there is relevant documentation to do this.

RancherOS-FreeNAS(1).pdf (193 KB) RancherOS-FreeNAS(1).pdf Marcelo Araujo, 03/13/2018 11:18 PM
Screen Shot 2018-03-14 at 1.34.03 PM.png (23.6 KB) Screen Shot 2018-03-14 at 1.34.03 PM.png Vaibhav Chauhan, 03/14/2018 01:34 PM
15175

Related issues

Blocks FreeNAS - Bug #29018: I don't see a way to create the Rancher VM in the new UI Done2018-03-01

History

#1 Updated by Vaibhav Chauhan over 2 years ago

  • Blocks Bug #29018: I don't see a way to create the Rancher VM in the new UI added

#2 Avatar?id=13649&size=24x24 Updated by Ben Gadd over 2 years ago

  • Category changed from Middleware to OS
  • Target version changed from 11.2-U2 to 11.2-BETA1

#3 Avatar?id=13649&size=24x24 Updated by Ben Gadd over 2 years ago

  • Due date set to 03/13/2018

#4 Updated by Marcelo Araujo over 2 years ago

Hi,

Attached there is a PDF where I'm trying to explain how the Docker feature works. Please, if it is not clear, feel free to ping me at any time, I'm back to work :).

Devices that you need by default when you create a Docker VM:
1) Nic.
2) Raw disk.

Regards the errors that you saw and reported via RocketChat, I made some tests with FreeNAS version: FreeNAS-11-MASTER-201803120410 and it works as expected, see below:

midclt call vm.fetch_image 'RancherOS'
2
midclt call vm.get_download_status 2 {"id": 2, "method": "vm.fetch_image", "arguments": ["RancherOS"], "progress": {"percent": 3, "description": "Downloading", "extra": {"downloaded": 1474560, "total": 47958299}}, "result": null, "error": null, "exception": null, "state": "RUNNING", "time_started": {"$date": 1521007687466}, "time_finished": null}

and after it finish:

midclt call vm.get_download_status 2 {"id": 2, "method": "vm.fetch_image", "arguments": ["RancherOS"], "progress": {"percent": 100, "description": "Downloading", "extra": {"downloaded": 47964160, "total": 47958299}}, "result": null, "error": null, "exception": null, "state": "SUCCESS", "time_started": {"$date": 1521007687466}, "time_finished": {"$date": 1521007807142}}

#5 Updated by Marcelo Araujo over 2 years ago

  • Status changed from Not Started to In Progress

I will ping vb@ to verify if I have provided enough information for him to be able to finish the development in the new UI.

#6 Updated by Vaibhav Chauhan over 2 years ago

15175

so using OLD UI i see that image downloading failed.

[2018/03/14 20:31:59] (DEBUG) VMService.fetch_image():1014 - ==> IMAGE: {'URL': 'http://download.freenas.org/bhyve-templates/rancheros-bhyve-v1.1.3/rancheros-bhyve-v1.1.3.img.gz', 'GZIPFILE': 'rancheros-bhyve-v1.1.3.img.gz', 'SHA256': 'e9288df573e01f5468c1f7e4609fbeab481caa3ffc5855af9d003b49557dde84'}
[2018/03/14 20:32:00] (DEBUG) vm.fetch_image():1022 - ===> Downloading: http://download.freenas.org/bhyve-templates/rancheros-bhyve-v1.1.3/rancheros-bhyve-v1.1.3.img.gz
[2018/03/14 20:32:01] (WARNING) middlewared._loop_monitor_thread():1013 - Task seems blocked:  File "/usr/local/lib/python3.6/site-packages/aiohttp/web.py", line 303, in _handle
    resp = yield from handler(request)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 987, in ws_handler
    await connection.on_message(x)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 298, in on_message
    asyncio.ensure_future(self.call_method(message))
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 84, in debug_wrapper
    return CoroWrapper(gen, None)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 94, in __init__
    self._source_traceback = traceback.extract_stack(sys._getframe(1))
  File "/usr/local/lib/python3.6/traceback.py", line 207, in extract_stack
    stack = StackSummary.extract(walk_stack(f), limit=limit)
  File "/usr/local/lib/python3.6/traceback.py", line 354, in extract
    linecache.checkcache(filename)
  File "/usr/local/lib/python3.6/linecache.py", line 74, in checkcache
    stat = os.stat(fullname)
........

root@freenas:/mnt/demo/.bhyve_containers/iso_files # sha256 rancheros-bhyve-v1.1.3.img.gz
SHA256 (rancheros-bhyve-v1.1.3.img.gz) = 7b4f6d1ec776b0ca7ced292431b12eaa11fef7d751c1208252493e9372320e26

#7 Updated by Marcelo Araujo over 2 years ago

  • Status changed from In Progress to Done

Followed with vb@ all steps to bring this functionality to the new UI.

#8 Updated by Dru Lavigne over 2 years ago

  • Target version changed from 11.2-BETA1 to Master - FreeNAS Nightlies
  • Needs QA changed from Yes to No
  • Needs Doc changed from Yes to No
  • Needs Merging changed from Yes to No

Also available in: Atom PDF