SSDs constantly trimming
I have 2 SSD on a mirror pool. They serve as a iSCSI target for a VMware Cluster.
The pool is new, and I'm observing constant trimming operations. It trimmed 1TB of data on 2 days. It's not the trimming operation at device creation time. Wear level on the drives grew...
It only happens when VMs are running on the pool. It only happens with iSCSI. Tested with and without deduplication. Dedup makes matters even worse.
On the reporting tab there are constant "Delete" operations on both disks when the ESXi are connected to the Pool and VMs are running.
I migrated the pool to NFS, and it stopped. Trim counter is now at 0 and it does not grow in time if I don't delete files, as it should.
#3 Updated by Alexander Motin over 3 years ago
- Status changed from Unscreened to Screened
- Seen in changed from Unspecified to 9.10.2-U5
ZFS sends TRIM to disks only when some space on pool is freed (either something is overwritten or something is deleted (UNMAP received from iSCSI initiator)). Are you sure there are no respective amount of writes to the pool, or VMware cluster does not request UNMAP for previously written data? Unfortunately debug data you attached does not include performance statistics. Please provide additional input about it.
In any case, if you wish to disable any TRIM to your SSDs, you can always set vfs.zfs.trim.enabled loader tunable. It may actually improve performance, though theoretically should not improve SSD life time, if there are indeed writes.
#5 Updated by Andres Biront over 3 years ago
Found out it is an ESXi "issue", or common behavior.
I disabled trimming and the constant writes to disk became evident. The drives don't trim anymore but there are 5GB written per hour to the SSDs. The VMs inside the datastore are writting less than 150KB/s (they are idling).
Found out that ESXi writes and deletes something constantly when a VM has a disk mounted on a datastore. Even if the virtual disk is not initialized on the Guest OS.
Space reclamation, HA heartbeat, I/O control, everything that could be writing to disk was disabled. But the ESXi hosts continue writing/deleting.
It also happens to my HDD pool, but it has 6 disks, which write 1GB per hour each.
You can close the ticket, it's not a FreeNAS bug. Ended up disabling the SSD pool. TLC drives and VMware is defenitely not a good idea...
Thanks for your help!!