VMware VAAI/Block Support
I would love to see support for VMware VAAI added in the near future. I believe if FreeNAS wants to have a place in the enterprise then it really needs to support this.
#1 Updated by Cyber Jock over 5 years ago
Because of how ZFS works, it is strongly(and I mean STRONGLY) recommended that FreeNAS be used as a VM with VT-d PCI passthrough of the associated SATA/SAS controller. If you are following this recommendation the VAAI feature provides no benefit for FreeNAS users except perhaps UFS users. Consult the forum stickies for more information on virtualizing FreeNAS as well as explanations for why VT-d is the only recommended method for ZFS with a VM.
#2 Updated by Anonymous over 5 years ago
Bite Off, I think you're confusing VAAI with some other feature. VAAI (vStorage APIs for Automation Integration) is storage configuration automation, allowing VMware to perform storage-side copies, snapshots, LUN allocations, and other advanced features on SANs that support it. Its a standard feature on advanced SAN products and some competitors to FreeNAS have VAAI support as well.
More information can be found in this KB article: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1021976
It is a very large project, though; the SMI-S provider itself is probably a 3 man-month project, requiring quite a bit of FreeNAS backend integration, and requires writing a connector loads into the hypervisor and requires certification to use in a VMware-supported environment. On the upside, the SMI-S provider would provide the same features to other compliant management environments, the most notable being XenServer.
#4 Updated by Cyber Jock over 5 years ago
I stand corrected. My initial searches of VAAI turned up a different explanation than what you provided. This would be a useful feature. Considering the amount of developer resources for FreeNAS I would expect that it might be added someday, but is far from high priority . FreeNAS only has a handful of paid developers. The ticket should definitely be left open. It might be something that is easy to implement in the future even though the workload is unrealistically high at the present time.
#6 Updated by Thomas Spalinger over 5 years ago
Why not take a look at Quadstor (opensource http://www.quadstor.com/), it works on FreeBSD and maybe could be integrated, also https://bugs.freenas.org/issues/60 for HA clustering could be eliminated with that.
Don't know if this could be done or if there would be any licensing problems.
Would love to see FreeNAS with iSCSI clustering and VAAI support, the last step for me to eliminate all other NAS appliances I currently have to use because I need some HA and VAAI supported storage for ESXi.
#9 Updated by Nick DeMarco over 4 years ago
Jordan Hubbard wrote:
I'll take this one - this is becoming more of a hot item for us.
I'd help where I can with this one, which may be limited to pulling documents together and contributing some donation hardware.
I'd love to see offloaded eager zeroing for on-SAN thick disk creation.
#10 Updated by Josh Paetzel over 4 years ago
- Block Zeroing (WRITE SAME) -- istgt supports it; for CTL I've added support recently.
- Cloning file blocks (EXTENDED COPY) -- neither istgt, nor CTL support it now.
- Hardware Assisted Locking (COMPARE AND WRITE) -- istgt supports it, unless configured for traditional mode; CTL has no
support, but should not be too difficult to add.
- Thin provisioning (UNMAP) -- istgt doesn't support it; for CTL I've added support recently for device/ZVOL extents.
#12 Updated by Alexander Motin over 4 years ago
In 184.108.40.206-RC2 CTL (new iSCSI target) got supports for COMPARE AND WRITE, so:
- Block Zeroing (WRITE SAME) -- both istgt and CTL support it;
- Cloning file blocks (EXTENDED COPY) -- neither istgt, nor CTL support it now;
- Hardware Assisted Locking (COMPARE AND WRITE) -- both istgt and CTL support it;
- Thin provisioning (UNMAP) -- istgt doesn't support it, CTL does support.
#14 Updated by Jordan Hubbard over 4 years ago
- Subject changed from VMware VAAI Support to VMware VAAI/Block Support
- Assignee changed from Jordan Hubbard to Alexander Motin
- Target version changed from 9.3-BETA to 9.3-RELEASE
Hey Sasha, where are we at these days with respect to the VAAI primitives? I'm going to retitle this bug to just track the VAAI/Block primitives because I think VAAI/NAS should be broken out into its own ticket - it's just too big a chunk of work to be tracked by a single ticket otherwise, and VMWare also differentiates between VAAI/Block and VAAI/NAS in their certification.
#16 Updated by Alexander Motin over 4 years ago
IMO at this point we have reasonable support for both VMWare VAAI and Microsoft ODX acceleration primitives in CTL of both 9.3 and 220.127.116.11 branches. SCSI specifications in that area are endless, but as far as I know I implemented all functionality supported by present VMWare and Windows versions. It would be great if we could try some pre-certification tests to find out whether we meet all their requirements, or we should do something more before 9.3 release.
I am still not completely satisfied with present XCOPY performance in 10GigE environment -- speed is lower and CPU overhead is higher then I would like to see. That is a result of present CTL backends design, optimized for data are transferred outside the system. For internal copy from ZVOL to ZVOL in present implementation it makes data to be copied in memory four times. It is invisible at 1GigE rates, but it creates bottleneck on 10+GigE speed. I am still thinking about better way to optimize that.
#17 Updated by Chris Nickl about 4 years ago
Is this implementation going to require any configuration from the VMware side of things. Some storage providers require certain configurations, and some provide plugins for the VAAI features. Is this something you will be looking at?
I am currently testing vSphere 5.5 U2 and will be updating to 6 shortly. I am also now using 10gb and iSCSI with FreeNAS 9.3 BETA, so i'd be willing to help test anything as well.
#18 Updated by Alexander Motin about 4 years ago
VAAI support should work out of the box without any additional configuration or plugins on VMware side. I think some tuning on that side may potentially affect VAAI performance, but that area is still waiting for experiments.
The only requirement from the FreeNAS side is that you should use ZVOL-backed (not file-backed) iSCSI extents to get thin-provisioning (UNMAP) working. In that case FreeNAS 9.3 should support all VAAI primitives supported by vSphere 5.5.