Project

General

Profile

Bug #13630

Prevent large file delete from starving out write operations

Added by Josh Paetzel almost 3 years ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Important
Assignee:
Alexander Motin
Category:
OS
Target version:
11.1-BETA1
Seen in:
9.3.1-STABLE-201602031011
Severity:
New
Reason for Closing:
Reason for Blocked:
Needs QA:
No
Needs Doc:
Yes
Needs Merging:
Yes
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

We are definitely going to want this once it's in OpenZFS.

Related issues

Related to FreeNAS - Bug #26313: Merge in additional upstream scrub commitsResolved2017-10-20

Associated revisions

Revision 5d16dfdf (diff)
Added by Dru Lavigne about 1 year ago

Mention ZFS improvements.
Ticket: #13630
Ticket: #18428

History

#1 Updated by Alexander Motin almost 3 years ago

  • Status changed from Unscreened to Screened

Sure. I am merging everything that appears in Illumos.

#2 Updated by Jordan Hubbard over 2 years ago

  • Status changed from Screened to 15

Is this resolved? I thought you had already done all of the pull-ups from Illumos. Thanks.

#3 Updated by Alexander Motin over 2 years ago

We are indeed just a few commits after illumos this moment, but this commit is not one of them. It is still on stage of open pull request asking for review. The review process became terribly slow last time. My own prefetcher patch hangs in the queue from about autumn. :(

#4 Avatar?id=14398&size=24x24 Updated by Kris Moore over 2 years ago

  • Target version changed from Unspecified to 9.10.1-U1

Is this merged in?

#5 Updated by Alexander Motin over 2 years ago

  • Status changed from 15 to Closed: Third party to resolve

Kris Moore wrote:

Is this merged in?

No. There is nothing merge. It was recently closed on GitHub: "alek-p commented 17 days ago: Looks like the async delete part needs more work. I'll try to upstream the rest in pieces, hopefully this week."

#6 Updated by Josh Paetzel over 2 years ago

  • Status changed from Closed: Third party to resolve to Investigation
  • Assignee changed from Alexander Motin to Josh Paetzel
  • Target version changed from 9.10.1-U1 to 9.10.2

We really need this. If it won't be coming from openzfs we need to investigate a solution.

Doing an rm -rf on a large directory of an NFS server will take it to it's knees, as the txgs get immediately filled with deletes and write I/O gets starved.

#7 Updated by Josh Paetzel about 2 years ago

  • Target version changed from 9.10.2 to 9.10.2-U1

#8 Avatar?id=14398&size=24x24 Updated by Kris Moore almost 2 years ago

  • Target version changed from 9.10.2-U1 to 9.10.2-U2

#9 Updated by Josh Paetzel almost 2 years ago

  • Status changed from Investigation to Unscreened
  • Assignee changed from Josh Paetzel to Kris Moore

#10 Avatar?id=14398&size=24x24 Updated by Kris Moore almost 2 years ago

  • Assignee changed from Kris Moore to Alexander Motin
  • Target version changed from 9.10.2-U2 to 9.10.2-U3

#12 Updated by Alexander Motin almost 2 years ago

  • Status changed from Unscreened to Fix In Progress

Josh merged the patch to FreeBSD head.

#13 Avatar?id=14398&size=24x24 Updated by Kris Moore almost 2 years ago

  • Target version changed from 9.10.2-U3 to 9.10.4

#14 Updated by Alexander Motin over 1 year ago

This code is in FreeBSD head now.

#15 Avatar?id=14398&size=24x24 Updated by Kris Moore over 1 year ago

  • Target version changed from 9.10.4 to 11.1

#16 Updated by Dru Lavigne over 1 year ago

  • Status changed from Fix In Progress to 46

Sasha: is there anything to merge here yet?

#17 Updated by Alexander Motin over 1 year ago

  • Status changed from 46 to Ready For Release

It was recently merged into nightly branch and respectively 11.1.

#18 Updated by Dru Lavigne over 1 year ago

  • Subject changed from https://github.com/openzfs/openzfs/pull/61/commits to Prevent large file delete from starving out write operations

#19 Updated by Dru Lavigne about 1 year ago

  • Target version changed from 11.1 to 11.1-BETA1

#20 Updated by Dru Lavigne about 1 year ago

  • Status changed from Ready For Release to Resolved

#22 Updated by Dru Lavigne about 1 year ago

  • Related to Bug #26313: Merge in additional upstream scrub commits added

#23 Updated by Nick Wolff about 1 year ago

Started working on testing this and having issues creating a work load that will show this behavior. A few hundred thousand small files of random data(100m) being deleted don't seem to make a noticeable difference to an async rand write fio test. This is true even when vfs.zfs.per_txg_dirty_frees_percent=0 which from my understanding should display the older behavior.

#24 Updated by Nick Wolff about 1 year ago

  • Needs QA changed from Yes to No
  • QA Status Test Passes FreeNAS added
  • QA Status deleted (Not Tested)

Test passes.

Large deletes (terabytes of data) and fio testing

Also available in: Atom PDF