Project

General

Profile

Bug #23872

Allow rsync task with / as remote host

Added by 0 4242 over 1 year ago. Updated 10 months ago.

Status:
Resolved
Priority:
Nice to have
Assignee:
Vladimir Vinogradenko
Category:
GUI (new)
Target version:
Seen in:
Sprint:
Severity:
New
Backlog Priority:
Reason for Closing:
Reason for Blocked:
Needs QA:
No
Needs Doc:
Yes
Needs Merging:
Yes
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

As posted on the forum: https://forums.freenas.org/index.php?threads/cant-create-an-rsync-task-with-as-remote-host.54379

Hello,

I'm trying to setup an rsync task. The task should pull from a remote Linux server. The Linux server use the rrsync script to restrict rsync access. It usualy work very well!

Because I use rrsync, I use "/" as remote path. rrsync translate that into my real directory on my Linux server.

First thing, when I create the rsync task with that remote path, it complains the remote path cannot be found. I unchecked the checkbox to disable the validation in order to be able to create the rule.

Then, I try to run the task from the web GUI. It fails silently.

So, I open the /etc/crontab file on my FreeNAS box to see what it is doing and try to replicate manually on the command line.

[rsync@freenas] ~ % /usr/bin/lockf -s -t 0 -k "/mnt/pool0/BACKUP_COPY/Postgresql" /usr/local/bin/rsync -r -t -z -e "ssh -p 22 -o BatchMode=yes -o StrictHostKeyChecking=yes" remote-user@remote.host:\""/"\" "/mnt/pool0/BACKUP_COPY/Postgresql" 
rsync: change_dir "/to/my/real/remote/path//"" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1664) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

Seem like there is a double escaping on the remote path and this is being translated as a double slash on the remote host.

If I replace

user@remote.host:\""/"\"
by
user@remote.host:/
it's working!

But I cannot just change the crontab manually, because the change would be eventually lost. How can I fix my problem using the web GUI?

Thank you!

Associated revisions

Revision 49792d76 (diff)
Added by Vladimir Vinogradenko about 1 year ago

fix(gui): Don't do double-escaping for safe rsync remote paths

Ticket: #23872

Revision 766075e5 (diff)
Added by Vladimir Vinogradenko about 1 year ago

fix(gui): Don't do double-escaping for safe rsync remote paths

Ticket: #23872

History

#1 Updated by William Grzybowski over 1 year ago

  • Status changed from Unscreened to Screened
  • Priority changed from No priority to Nice to have
  • Target version set to 11.1

There isn't anything you can do at your side.

Are you really putting "/" with quotes in the field space?

#2 Updated by 0 4242 over 1 year ago

No, quotes are added by the GUI. I just put / as remote path.

#3 Updated by William Grzybowski over 1 year ago

f 4242 wrote:

No, quotes are added by the GUI. I just put / as remote path.

Can't you workaround that in rrsync by using "/" as path, with quotes?

#4 Updated by 0 4242 over 1 year ago

It doesn't work better.

Remote path in the cron file become:

\"""/""\"
(one more escaping) and the result on the remote server is the same.

#5 Updated by William Grzybowski over 1 year ago

f 4242 wrote:

It doesn't work better.

Remote path in the cron file become: [...] (one more escaping) and the result on the remote server is the same.

I meant on the remote server, not in freenas.

#6 Updated by 0 4242 over 1 year ago

Sorry, I misunderstood you.

Yeah as a workaround, I was able to hack the rrsync script to deal with the input sent by FreeNAS. I still have to disable the remote path validation when I create/edit my task, but the task run fine.

#7 Updated by 0 4242 over 1 year ago

After creating a new task, I found the problem is not only when creating a task with / as remote. It also fails when I try to pull from a subdirectory (example: /server1) because of double quoting. When rsync receive directly the request I assume this work fine, but it fails with the use of rrsync between.

I don't know if the current issue is a feature or a bug. The remote path is double quoted but you don't double quote the local path. If the local path doesn't need to be double quoted, I wonder why the remote path should be.

Meanwhile I updated my workaround in order it work in all situation.

Thanks!

#8 Updated by Vladimir Vinogradenko about 1 year ago

  • Status changed from Screened to Fix In Progress
  • Assignee changed from William Grzybowski to Vladimir Vinogradenko

#9 Updated by Vladimir Vinogradenko about 1 year ago

  • Status changed from Fix In Progress to 15

I think this double-escaping is an attempt to mimic -s, --protect-args option behavior:

       -s, --protect-args
              This option sends all filenames and most options to the remote
              rsync without allowing the remote shell to interpret them.  This
              means that spaces are not split in names, and any non-wildcard
              special characters are not translated (such as ~, $, ;, &,
              etc.).  Wildcards are expanded on the remote host by rsync
              (instead of the shell doing it).

The problem with just adding this option and removing double-escaping is that this must be supported by remote side:

              Since this option was first introduced in 3.0.0, you’ll need to make
              sure it’s disabled if you ever need to interact with a remote
              rsync that is older than that.

Rsync 3.0.0 was released at 2008-Mar-01.

I see two options here:

1) Don't do double-escaping for safe paths (python test: pipes.quote(s) == s)

2) Add -s and don't do double quoting, add option «Remote side is rsync<3.0» with fallback to double quoting

#10 Updated by William Grzybowski about 1 year ago

  • Status changed from 15 to Unscreened

I am pretty sure escaping was added just for the case of rsync of folders with spaces in it.

I believe #1 would be enough here and with less disruption.

#11 Updated by Vladimir Vinogradenko about 1 year ago

  • Status changed from Unscreened to Needs Developer Review

#12 Updated by William Grzybowski about 1 year ago

  • Status changed from Needs Developer Review to Reviewed by Developer

#13 Updated by Vladimir Vinogradenko about 1 year ago

  • Status changed from Reviewed by Developer to Ready For Release

#14 Updated by Dru Lavigne about 1 year ago

  • Subject changed from Can't create an rsync task with / as remote host to Allow rsync task with / as remote host

#15 Updated by Dru Lavigne about 1 year ago

  • Target version changed from 11.1 to 11.1-BETA1

#16 Updated by Dru Lavigne 12 months ago

  • Status changed from Ready For Release to Resolved

#17 Updated by Joe Maloney 10 months ago

13336

See the attached screenshot. I believe that as a side effect of this fix we can now longer browse for local path for an rsync task. 10.231.3.1 root abcd1234 can be used to see the issue.

#18 Updated by Joe Maloney 10 months ago

  • QA Status Test Fails FreeNAS added
  • QA Status deleted (Not Tested)

#19 Updated by Joe Maloney 10 months ago

Sorry for the false alarm. My storage was missing from a previous test. I checked crontab and the remote destination is formatted correctly for rsync over ssh.

"root"@10.20.20.233:/

#20 Updated by Joe Maloney 10 months ago

  • Priority changed from Blocks Until Resolved to Nice to have
  • Needs QA changed from Yes to No
  • QA Status Test Passes FreeNAS added
  • QA Status deleted (Test Fails FreeNAS)

Also available in: Atom PDF