Project

General

Profile

Bug #77412

Fix prepending root directory for non-bucket cloud sync remote paths

Added by Paulo Pera dos Santos almost 3 years ago. Updated over 2 years ago.

Status:
Done
Priority:
No priority
Assignee:
Vladimir Vinogradenko
Category:
Middleware
Target version:
Seen in:
Severity:
New
Reason for Closing:
Reason for Blocked:
Needs QA:
No
Needs Doc:
No
Needs Merging:
No
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

I don't know If I'm reporting it in the right place but after I updated to FreeNAS-11.2-U2 I can't use cloud sync task to sync with Dropbox anymore. The last day it was synced was 02/19/2019 and I updated the server on 02/20/2019. Before reporting the problem I tried to find anyone reporting the same problem but I couldn't find anything.
Here's the log it shows when I run the task:

2019/02/26 14:07:26 INFO : remote: Converting token to new format
2019/02/26 14:07:27 ERROR : : error reading destination directory:
2019/02/26 14:07:27 INFO : Dropbox root 'db-backup': Waiting for checks to finish
2019/02/26 14:07:27 INFO : Dropbox root 'db-backup': Waiting for transfers to finish
2019/02/26 14:07:27 ERROR : Dropbox root 'db-backup': not deleting files as there were IO errors
2019/02/26 14:07:27 ERROR : Dropbox root 'db-backup': not deleting directories as there were IO errors
2019/02/26 14:07:27 ERROR : Attempt 1/3 failed with 1 errors and: not deleting files as there were IO errors
2019/02/26 14:07:28 ERROR : : error reading destination directory:
2019/02/26 14:07:28 INFO : Dropbox root 'db-backup': Waiting for checks to finish
2019/02/26 14:07:28 INFO : Dropbox root 'db-backup': Waiting for transfers to finish
2019/02/26 14:07:28 ERROR : Dropbox root 'db-backup': not deleting files as there were IO errors
2019/02/26 14:07:28 ERROR : Dropbox root 'db-backup': not deleting directories as there were IO errors
2019/02/26 14:07:28 ERROR : Attempt 2/3 failed with 1 errors and: not deleting files as there were IO errors
2019/02/26 14:07:28 INFO :
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors: 0
Checks: 0 / 0, -
Transferred: 0 / 0, -
Elapsed time: 2.2s

2019/02/26 14:07:28 ERROR : : error reading destination directory:
2019/02/26 14:07:28 INFO : Dropbox root 'db-backup': Waiting for checks to finish
2019/02/26 14:07:28 INFO : Dropbox root 'db-backup': Waiting for transfers to finish
2019/02/26 14:07:28 ERROR : Dropbox root 'db-backup': not deleting files as there were IO errors
2019/02/26 14:07:28 ERROR : Dropbox root 'db-backup': not deleting directories as there were IO errors
2019/02/26 14:07:28 ERROR : Attempt 3/3 failed with 1 errors and: not deleting files as there were IO errors
2019/02/26 14:07:28 Failed to sync: not deleting files as there were IO errors


Related issues

Copied to FreeNAS - Bug #79077: Cloud Sync Dropbox reporting an errorDone

History

#1 Updated by Dru Lavigne almost 3 years ago

  • Private changed from No to Yes
  • Reason for Blocked set to Need additional information from Author

Paulo: please also attach a debug (System -> Advanced -> Save debug) to this ticket.

#2 Updated by Paulo Pera dos Santos almost 3 years ago

  • File debug-freenas-20190226160804.tgz added

#3 Updated by Paulo Pera dos Santos almost 3 years ago

I'm sorry, at the time of the post I did generated it but didn't attached.
here's the file!

#4 Updated by Paulo Pera dos Santos almost 3 years ago

  • File debug-freenas-20190226162215.tgz added
  • File 18.log added

I installed it in a VM and tried to use cloud sync tasks and I got the same error. I attached the files too.
ps: I know I'm reporting from a production machine but I thought it would help to send this one too.

#5 Updated by Dru Lavigne almost 3 years ago

  • Assignee changed from Release Council to William Grzybowski
  • Reason for Blocked deleted (Need additional information from Author)

#6 Updated by William Grzybowski almost 3 years ago

  • Assignee changed from William Grzybowski to Vladimir Vinogradenko
  • Seen in changed from 11.2-U2 to 11.2-U4

Any ideas here, Vladimir?

#7 Updated by Vladimir Vinogradenko almost 3 years ago

  • Status changed from Unscreened to Blocked
  • Reason for Blocked set to Waiting for feedback

Paulo, is it a pull or push task? Have some files/directories with strange name appeared recently? Our error is:

2019/02/26 14:07:27 ERROR : : error reading destination directory:

Looks like it's trying to read a directory with empty name.

#8 Updated by Paulo Pera dos Santos almost 3 years ago

Vladimir Vinogradenko wrote:

Paulo, is it a pull or push task? Have some files/directories with strange name appeared recently? Our error is:

2019/02/26 14:07:27 ERROR : : error reading destination directory:

Looks like it's trying to read a directory with empty name.

It's a PUSH task and there's no files/directories with strange name. All the files I save in this folder to sync to Dropbox has the same name everyday, so I can use the extended version history (I pay for this) and have one year history of the files.
I didn't change anything prior the update of Freenas. As I said before the last sync was on 02/19/2019 and I updated the server on 02/20/2019.
When I click run in the task I see the Freenas is accessing some Dropbox IP's and has no alerts of block/reject by the firewall.
I'm using this sync for the last 6-9 months I think.
Is there any log I can see more information when I click RUN in the task?
Just to be more specific the token I'm generating on Dropbox Console is for the permission type: App folder
If I create another App on Dropbox Console but with permission type Full Dropbox I can sync the files. I discovered it a minute agora as I was trying to generate another app and saw the Type options (App Folder, Full Dropbox).
I prefere App Folder because the token has access only to the App Folder itself and Full Dropbox has access to all my files.

#10 Updated by William Grzybowski almost 3 years ago

  • Target version changed from Backlog to 11.2-U4
  • Seen in changed from 11.2-U4 to 11.2-U2

#11 Updated by Vladimir Vinogradenko over 2 years ago

Paulo, what do you mean by creating apps on Dropbox console? FreeNAS uses rclone app and it's not currently possible to change this. How does creating an app affect your FreeNAS configuration? How do you obtain access token that you use to configure Dropbox sync in FreeNAS?

#12 Updated by Paulo Pera dos Santos over 2 years ago

57864
57873

Hi Vladimir, in order to create a task in FreeNAS to sync you need to fill the token for the service you want in cloud credentials. So to create the token for Dropbox I have to go to access the Dropbox App Console https://www.dropbox.com/developers/apps to create an App, before generating the token you have two options of access:

1 - Generate a App Folder - Access to a single folder specific to your app (so your FreeNAS doesn't have access to any other folder you have in Dropbox)
2 - Full Dropbox - Access to all files and folders in a user Dropbox (it can access and list any file)

I was using the App Folder previously because it looks the more safe and better solution for me. It would only have access to the files in the App Folder I specified.
When it was not working anymore in the process of debugging to report more infos to you I tried to generate a new token with the App Folder access and didn't succeed. But when I generated a Full Dropbox token access it worked. I could sync the files without errors. But as the Dropbox console shows, I could list any files in my account and not the specific one for the app.
Here you can see the two Apps I created to test it:

#13 Updated by Vladimir Vinogradenko over 2 years ago

Paulo, thank you for your help, problem fixed!

#14 Updated by Bug Clerk over 2 years ago

  • Status changed from Blocked to In Progress

#15 Updated by Paulo Pera dos Santos over 2 years ago

Vladimir Vinogradenko wrote:

Paulo, thank you for your help, problem fixed!

Thanks Valdimir and you all!

#16 Updated by Bug Clerk over 2 years ago

  • Status changed from In Progress to Ready for Testing

#17 Updated by Bug Clerk over 2 years ago

  • Copied to Bug #79077: Cloud Sync Dropbox reporting an error added

#18 Updated by Dru Lavigne over 2 years ago

  • Target version changed from 11.2-U4 to 11.2-U3

#20 Updated by Dru Lavigne over 2 years ago

  • File deleted (211.log)

#21 Updated by Dru Lavigne over 2 years ago

  • File deleted (debug-freenas-20190226160804.tgz)

#22 Updated by Dru Lavigne over 2 years ago

  • File deleted (18.log)

#23 Updated by Dru Lavigne over 2 years ago

  • File deleted (debug-freenas-20190226162215.tgz)

#24 Updated by Dru Lavigne over 2 years ago

  • Subject changed from Cloud Sync Dropbox reporting an error to Fix prepending root directory for non-bucket cloud sync remote paths
  • Private changed from Yes to No
  • Reason for Blocked deleted (Waiting for feedback)
  • Needs Doc changed from Yes to No
  • Needs Merging changed from Yes to No

#25 Updated by Vladimir Vinogradenko over 2 years ago

Acceptance Criteria

Dropbox cloud sync, set up as Paolo described in comment #12 https://redmine.ixsystems.com/issues/77412#note-12, should work

#27 Updated by Jeff Ervin over 2 years ago

61158
61167
61176
61185
61194
61203
61212

Test Passed FreeNAS-11.2-U2-INTERNAL100

Initially getting an error (see screenshot) that the task had failed. Checked Dropbox, all files within the folder had sync'd successfully. Per developer, had to remove the DS_Store file from source folder. After removal of this file, had successful sync with Dropbox.

#28 Updated by Dru Lavigne over 2 years ago

  • Status changed from Passed Testing to Done

Also available in: Atom PDF