Project

General

Profile

Bug #52507

Avatar?id=14540&size=50x50

Add Transfers field to Cloud Sync in legacy UI

Added by James Majerus about 2 years ago. Updated over 1 year ago.

Status:
Ready for Testing
Priority:
No priority
Assignee:
Vladimir Vinogradenko
Category:
Middleware
Target version:
Seen in:
Severity:
Low Medium
Reason for Closing:
Reason for Blocked:
Needs QA:
Yes
Needs Doc:
No
Needs Merging:
No
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

As outlined https://rclone.org/b2/, otherwise large cloudsync tasks are slow.

Transfers
Backblaze recommends that you do lots of transfers simultaneously for maximum speed. In tests from my SSD equipped laptop the optimum setting is about --transfers 32 though higher numbers may be used for a slight speed improvement. The optimum number for you may vary depending on your hardware, how big the files are, how much you want to load your computer, etc. The default of --transfers 4 is definitely too low for Backblaze B2 though.

Note that uploading big files (bigger than 200 MB by default) will use a 96 MB RAM buffer by default. There can be at most --transfers of these in use at any moment, so this sets the upper limit on the memory used.


Related issues

Related to FreeNAS - Feature #26131: Add "Auxiliary arguments" field to middlewareDone
Related to FreeNAS - Bug #63258: Add Transfers and snapshot fields to Cloud Sync in new UIReady for Testing

History

#1 Updated by Dru Lavigne about 2 years ago

  • Category changed from GUI (new) to Middleware
  • Assignee changed from Release Council to William Grzybowski

#2 Updated by William Grzybowski about 2 years ago

  • Assignee changed from William Grzybowski to Vladimir Vinogradenko
  • Target version changed from Backlog to 11.3
  • Severity changed from New to Low Medium

#3 Updated by Vladimir Vinogradenko about 2 years ago

  • Related to Feature #26131: Add "Auxiliary arguments" field to middleware added

#4 Updated by Vladimir Vinogradenko about 2 years ago

  • Status changed from Unscreened to Screened

I'll expose --transfers option as a cloud sync option. Some people might want slow background sync that does not affect their NAS and network performance and some may afford pushing 10 GBits to S3 with CPU being used at 199.9%

Risk
Low risk, it's just another option with default value set to what's it already is

Acceptance Criteria
Create a cloud sync task, set "The number of file transfers to run in parallel" option to some value e.g. 5. Set it to sync relatively large amount of data so you can catch it in ps ax | grep rclone while it's running. Run it. Command line should contain --transfers=5 among all other options and cloud sync task should succeed.

#5 Updated by Bug Clerk almost 2 years ago

  • Status changed from Screened to In Progress

#6 Updated by Vladimir Vinogradenko almost 2 years ago

  • Related to Bug #63258: Add Transfers and snapshot fields to Cloud Sync in new UI added

#7 Updated by Bug Clerk almost 2 years ago

  • Status changed from In Progress to Ready for Testing

#8 Updated by Dru Lavigne almost 2 years ago

  • Subject changed from Add ability to increase transfers to Cloud Sync Tasks for B2 (maybe others?) to Expose --transfers Cloud Sync option in middleware
  • Needs Doc changed from Yes to No
  • Needs Merging changed from Yes to No

#9 Updated by Dru Lavigne almost 2 years ago

  • Target version changed from 11.3 to 11.3-BETA1

#10 Updated by Dru Lavigne over 1 year ago

  • Subject changed from Expose --transfers Cloud Sync option in middleware to Expose --transfers Cloud Sync option in legacy UI

#12 Updated by Dru Lavigne over 1 year ago

  • Subject changed from Expose --transfers Cloud Sync option in legacy UI to Add Transfers field to Cloud Sync in legacy UI

Also available in: Atom PDF