Project

General

Profile

Bug #69607

Add note to Guide explaining common rclone sync error

Added by Michael Leonard 5 months ago. Updated 3 months ago.

Status:
Done
Priority:
No priority
Assignee:
Aaron St. John
Category:
Documentation
Target version:
Seen in:
Severity:
New
Reason for Closing:
Reason for Blocked:
Needs QA:
Yes
Needs Doc:
No
Needs Merging:
No
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

Files are removed from Dropbox (cloud - my source) and are not removed from my Freenas (local - destination).

The cloud sync task is setup in the PULL direction, the transfer mode is SYNC.

Previously posted at:
https://forums.freenas.org/index.php?threads/dropbox-cloud-sync-doesnt-remove-files-folders.72745/#post-504545

History

#1 Updated by Dru Lavigne 5 months ago

  • Private changed from No to Yes
  • Reason for Blocked set to Need additional information from Author

Michael: please reproduce then attach a debug (System -> Advanced -> Save debug) to this ticket.

#2 Updated by Michael Leonard 5 months ago

  • File debug-freenas-20190114150516.tar added

Attached is the requested debug file. -Michael

#3 Updated by Dru Lavigne 5 months ago

  • Assignee changed from Release Council to William Grzybowski
  • Reason for Blocked deleted (Need additional information from Author)

#4 Updated by William Grzybowski 5 months ago

  • Assignee changed from William Grzybowski to Vladimir Vinogradenko
  • Target version changed from Backlog to 11.2-U3

#6 Updated by Vladimir Vinogradenko 5 months ago

  • Status changed from Unscreened to Closed
  • Reason for Closing set to Third Party to Resolve

Michael, your cloud sync task is failing, you can see why in legacy UI by clickng at error message.

What's available to us through the submitted debug is

2019/01/14 14:28:54 Failed to sync: failed to open source object: path/restricted_content/..

According to rclone bug tracker https://github.com/ncw/rclone/issues/2301 this happens if you are trying to sync a copyrighted file

What exact file is copyrighted you can see by looking at complete cloud sync task logs e.g. /tmp/middlewared/jobs/1736.log (the logs excerpt window has this path)

Right now you can only delete this file from your Dropbox folder. In 11.3 we'll have an option to exclude certain files/directories from sync.

Note to Dru: we'll probably need to document this under some "Common errors" section in cloud sync doc.

#7 Updated by Dru Lavigne 5 months ago

  • Category changed from Services to Documentation
  • Status changed from Closed to Unscreened
  • Assignee changed from Vladimir Vinogradenko to Warren Block
  • Reason for Closing deleted (Third Party to Resolve)

#8 Updated by Dru Lavigne 5 months ago

  • File deleted (debug-freenas-20190114150516.tar)

#9 Updated by Dru Lavigne 5 months ago

  • Private changed from Yes to No

#10 Updated by Warren Block 5 months ago

  • Status changed from Unscreened to Screened

#11 Updated by Michael Leonard 5 months ago

Thanks for pointing to the classic UI, as the error does not show up in the new UI.

I looked at the 1736.log file. Yes, some media files were flagged for copyright and I understand the reasoning. However, the files/directories that are not being removed are original PDFs, Pages, Photoshop, and other work project files. I could not find these files in that .log file.

Here is the last lines of 1736.log file. Could the 'IO errors' be leading to this issue?

2019/01/14 14:28:54 INFO : Local file system at /mnt/Tank/Dropbox: Waiting for checks to finish
2019/01/14 14:28:54 INFO : Local file system at /mnt/Tank/Dropbox: Waiting for transfers to finish
2019/01/14 14:28:54 ERROR : Local file system at /mnt/Tank/Dropbox: not deleting files as there were IO errors
2019/01/14 14:28:54 ERROR : Local file system at /mnt/Tank/Dropbox: not deleting directories as there were IO errors
2019/01/14 14:28:54 ERROR : Can't retry this error - not attempting retries
2019/01/14 14:28:54 Failed to sync: failed to open source object: path/restricted_content/..

#12 Updated by Vladimir Vinogradenko 5 months ago

However, the files/directories that are not being removed are original PDFs, Pages, Photoshop, and other work project files. I could not find these files in that .log file.

This is intended behavior stated in rclone doc https://rclone.org/commands/rclone_sync/

"Note that files in the destination won’t be deleted if there were any errors at any point."

In other words, rclone will only delete files if there were no errors with any other files.

Could the 'IO errors' be leading to this issue?

What they mean by these IO errors is errors accessing copyrighted files

#13 Updated by Warren Block 5 months ago

  • Status changed from Screened to Unscreened
  • Assignee changed from Warren Block to Aaron St. John

#14 Updated by Aaron St. John 4 months ago

  • Status changed from Unscreened to In Progress

#15 Updated by Dru Lavigne 4 months ago

  • Subject changed from Dropbox cloud sync doesn't remove files & folders to Add note to Guide explaining common rclone sync error
  • Status changed from In Progress to Ready for Testing
  • Needs Doc changed from Yes to No
  • Needs Merging changed from Yes to No

#16 Updated by Dru Lavigne 3 months ago

  • Status changed from Ready for Testing to Failed Testing

#18 Updated by Aaron St. John 3 months ago

  • Status changed from Failed Testing to Ready for Testing

(userguide text fix) angulargui PR: https://github.com/freenas/freenas-docs/pull/777

#19 Updated by Dru Lavigne 3 months ago

  • Status changed from Ready for Testing to Done

Also available in: Atom PDF