Project

General

Profile

Bug #24464

SSDs constantly trimming

Added by Andres Biront over 3 years ago. Updated about 3 years ago.

Status:
Closed: Behaves correctly
Priority:
Important
Assignee:
Alexander Motin
Category:
Middleware
Target version:
Seen in:
Severity:
New
Reason for Closing:
Reason for Blocked:
Needs QA:
Yes
Needs Doc:
Yes
Needs Merging:
Yes
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

I have 2 SSD on a mirror pool. They serve as a iSCSI target for a VMware Cluster.

The pool is new, and I'm observing constant trimming operations. It trimmed 1TB of data on 2 days. It's not the trimming operation at device creation time. Wear level on the drives grew...

It only happens when VMs are running on the pool. It only happens with iSCSI. Tested with and without deduplication. Dedup makes matters even worse.

On the reporting tab there are constant "Delete" operations on both disks when the ESXi are connected to the Pool and VMs are running.

I migrated the pool to NFS, and it stopped. Trim counter is now at 0 and it does not grow in time if I don't delete files, as it should.

History

#1 Updated by Andres Biront over 3 years ago

  • File debug-freenas-20170610211039.txz added

#2 Avatar?id=14398&size=24x24 Updated by Kris Moore over 3 years ago

  • Assignee changed from William Grzybowski to Alexander Motin
  • Priority changed from No priority to Important

#3 Updated by Alexander Motin over 3 years ago

  • Status changed from Unscreened to Screened
  • Seen in changed from Unspecified to 9.10.2-U5

ZFS sends TRIM to disks only when some space on pool is freed (either something is overwritten or something is deleted (UNMAP received from iSCSI initiator)). Are you sure there are no respective amount of writes to the pool, or VMware cluster does not request UNMAP for previously written data? Unfortunately debug data you attached does not include performance statistics. Please provide additional input about it.

In any case, if you wish to disable any TRIM to your SSDs, you can always set vfs.zfs.trim.enabled loader tunable. It may actually improve performance, though theoretically should not improve SSD life time, if there are indeed writes.

#4 Updated by Alexander Motin over 3 years ago

  • Status changed from Screened to 15

#5 Updated by Andres Biront over 3 years ago

Found out it is an ESXi "issue", or common behavior.

I disabled trimming and the constant writes to disk became evident. The drives don't trim anymore but there are 5GB written per hour to the SSDs. The VMs inside the datastore are writting less than 150KB/s (they are idling).

Found out that ESXi writes and deletes something constantly when a VM has a disk mounted on a datastore. Even if the virtual disk is not initialized on the Guest OS.

Space reclamation, HA heartbeat, I/O control, everything that could be writing to disk was disabled. But the ESXi hosts continue writing/deleting.

It also happens to my HDD pool, but it has 6 disks, which write 1GB per hour each.

You can close the ticket, it's not a FreeNAS bug. Ended up disabling the SSD pool. TLC drives and VMware is defenitely not a good idea...

Thanks for your help!!

#6 Updated by Alexander Motin over 3 years ago

  • Status changed from 15 to Closed: Behaves correctly

#7 Updated by Dru Lavigne about 3 years ago

  • File deleted (debug-freenas-20170610211039.txz)

#8 Updated by Dru Lavigne about 3 years ago

  • Target version set to N/A
  • Private changed from Yes to No

Also available in: Atom PDF