Feature #3118

VMware VAAI/Block Support

Added by dcprom0 - over 5 years ago. Updated about 4 years ago.

Nice to have
Alexander Motin
Target version:
Estimated time:
Reason for Closing:
Reason for Blocked:
Needs QA:
Needs Doc:
Needs Merging:
Needs Automation:
Support Suite Ticket:
Hardware Configuration:


I would love to see support for VMware VAAI added in the near future. I believe if FreeNAS wants to have a place in the enterprise then it really needs to support this.


#1 Updated by Cyber Jock over 5 years ago


Because of how ZFS works, it is strongly(and I mean STRONGLY) recommended that FreeNAS be used as a VM with VT-d PCI passthrough of the associated SATA/SAS controller. If you are following this recommendation the VAAI feature provides no benefit for FreeNAS users except perhaps UFS users. Consult the forum stickies for more information on virtualizing FreeNAS as well as explanations for why VT-d is the only recommended method for ZFS with a VM.

#2 Updated by Anonymous over 5 years ago

Bite Off, I think you're confusing VAAI with some other feature. VAAI (vStorage APIs for Automation Integration) is storage configuration automation, allowing VMware to perform storage-side copies, snapshots, LUN allocations, and other advanced features on SANs that support it. Its a standard feature on advanced SAN products and some competitors to FreeNAS have VAAI support as well.

More information can be found in this KB article:

It is a very large project, though; the SMI-S provider itself is probably a 3 man-month project, requiring quite a bit of FreeNAS backend integration, and requires writing a connector loads into the hypervisor and requires certification to use in a VMware-supported environment. On the upside, the SMI-S provider would provide the same features to other compliant management environments, the most notable being XenServer.

#3 Updated by Anonymous over 5 years ago

Expansion correction: vStorage APIs for Array Integration (VAAI)

#4 Updated by Cyber Jock over 5 years ago

I stand corrected. My initial searches of VAAI turned up a different explanation than what you provided. This would be a useful feature. Considering the amount of developer resources for FreeNAS I would expect that it might be added someday, but is far from high priority . FreeNAS only has a handful of paid developers. The ticket should definitely be left open. It might be something that is easy to implement in the future even though the workload is unrealistically high at the present time.

#5 Updated by Anonymous over 5 years ago

  • Assignee set to Anonymous

Take this as per Craig Rodrigues request.

#6 Updated by Thomas Spalinger over 5 years ago

Why not take a look at Quadstor (opensource, it works on FreeBSD and maybe could be integrated, also for HA clustering could be eliminated with that.
Don't know if this could be done or if there would be any licensing problems.
Would love to see FreeNAS with iSCSI clustering and VAAI support, the last step for me to eliminate all other NAS appliances I currently have to use because I need some HA and VAAI supported storage for ESXi.

#7 Updated by Craig Rodrigues about 5 years ago

  • Assignee deleted (Anonymous)

#8 Updated by Jordan Hubbard about 5 years ago

  • Category set to OS
  • Status changed from Unscreened to Screened
  • Assignee set to Jordan Hubbard
  • Target version set to 48

I'll take this one - this is becoming more of a hot item for us.

#9 Updated by Nick DeMarco over 4 years ago

Jordan Hubbard wrote:

I'll take this one - this is becoming more of a hot item for us.

I'd help where I can with this one, which may be limited to pulling documents together and contributing some donation hardware.

I'd love to see offloaded eager zeroing for on-SAN thick disk creation.

#10 Updated by Josh Paetzel over 4 years ago

- Block Zeroing (WRITE SAME) -- istgt supports it; for CTL I've added support recently.

- Cloning file blocks (EXTENDED COPY) -- neither istgt, nor CTL support it now.

- Hardware Assisted Locking (COMPARE AND WRITE) -- istgt supports it, unless configured for traditional mode; CTL has no
support, but should not be too difficult to add.

- Thin provisioning (UNMAP) -- istgt doesn't support it; for CTL I've added support recently for device/ZVOL extents.

#11 Updated by Jordan Hubbard over 4 years ago

  • Target version changed from 48 to 9.3-BETA

#12 Updated by Alexander Motin over 4 years ago

In CTL (new iSCSI target) got supports for COMPARE AND WRITE, so:
- Block Zeroing (WRITE SAME) -- both istgt and CTL support it;
- Cloning file blocks (EXTENDED COPY) -- neither istgt, nor CTL support it now;
- Hardware Assisted Locking (COMPARE AND WRITE) -- both istgt and CTL support it;
- Thin provisioning (UNMAP) -- istgt doesn't support it, CTL does support.

#13 Updated by Steve Bennett over 4 years ago

Great progress on the Block Front especially when the xcopy works.

Any thoughts on the NAS/NFS VAAI primitives?

#14 Updated by Jordan Hubbard over 4 years ago

  • Subject changed from VMware VAAI Support to VMware VAAI/Block Support
  • Assignee changed from Jordan Hubbard to Alexander Motin
  • Target version changed from 9.3-BETA to 9.3-RELEASE

Hey Sasha, where are we at these days with respect to the VAAI primitives? I'm going to retitle this bug to just track the VAAI/Block primitives because I think VAAI/NAS should be broken out into its own ticket - it's just too big a chunk of work to be tracked by a single ticket otherwise, and VMWare also differentiates between VAAI/Block and VAAI/NAS in their certification.

#15 Updated by Josh Paetzel over 4 years ago

  • Status changed from Screened to Resolved

9.3 has full support for all VAAI block primitives. ( does as well if you turn on the experimental target)

#16 Updated by Alexander Motin over 4 years ago

IMO at this point we have reasonable support for both VMWare VAAI and Microsoft ODX acceleration primitives in CTL of both 9.3 and branches. SCSI specifications in that area are endless, but as far as I know I implemented all functionality supported by present VMWare and Windows versions. It would be great if we could try some pre-certification tests to find out whether we meet all their requirements, or we should do something more before 9.3 release.

I am still not completely satisfied with present XCOPY performance in 10GigE environment -- speed is lower and CPU overhead is higher then I would like to see. That is a result of present CTL backends design, optimized for data are transferred outside the system. For internal copy from ZVOL to ZVOL in present implementation it makes data to be copied in memory four times. It is invisible at 1GigE rates, but it creates bottleneck on 10+GigE speed. I am still thinking about better way to optimize that.

#17 Updated by Chris Nickl about 4 years ago

Is this implementation going to require any configuration from the VMware side of things. Some storage providers require certain configurations, and some provide plugins for the VAAI features. Is this something you will be looking at?

I am currently testing vSphere 5.5 U2 and will be updating to 6 shortly. I am also now using 10gb and iSCSI with FreeNAS 9.3 BETA, so i'd be willing to help test anything as well.

#18 Updated by Alexander Motin about 4 years ago

VAAI support should work out of the box without any additional configuration or plugins on VMware side. I think some tuning on that side may potentially affect VAAI performance, but that area is still waiting for experiments.

The only requirement from the FreeNAS side is that you should use ZVOL-backed (not file-backed) iSCSI extents to get thin-provisioning (UNMAP) working. In that case FreeNAS 9.3 should support all VAAI primitives supported by vSphere 5.5.

#19 Updated by Chris Nickl about 4 years ago

So i have verified that ZVOL Backed iSCSI devices, have ATS, Clone, Zero & Delete fully supported and showing as so on my ESXi 5.5 U2 servers.

#20 Updated by Reqlez Guy about 4 years ago

It would be nice to add NFS VAAI support as well, for people who like to run over NFS and want to be able to create thick provisioned VMDKs on NFS datastores with ESXi

#21 Updated by Jordan Hubbard about 4 years ago

Please don't add comments to closed bugs asking for new/different features, since there's nothing tracking that and the comments will just be lost. Open a new bug, since this one is just for VAAI/Block. Thanks.

Also available in: Atom PDF