Project

General

Profile

Bug #24732

The replication failed for the local ZFS volume1/**** while attempting to apply incremental send of snapshot

Added by Andrew Sterling over 3 years ago. Updated about 3 years ago.

Status:
Closed: Not To Be Fixed
Priority:
Nice to have
Assignee:
William Grzybowski
Category:
Middleware
Target version:
Seen in:
Severity:
New
Reason for Closing:
Reason for Blocked:
Needs QA:
Yes
Needs Doc:
Yes
Needs Merging:
Yes
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

I set up snapshot replication to a second FreeNAS 11 server. Ever since, a few times a day I get an email saying that it failed to send the most recent snapshot. Sometimes it's every two hours, sometimes 3. It's always different.

Looking at the volume sizes on both servers, they appear to be correct.

This is the email I receive.

Hello,
The replication failed for the local ZFS volume1/users/rheanna while attempting to
apply incremental send of snapshot auto-20170619.2100-1m -> auto-20170619.2200-1m to 192.168.1.135

When I set it up, I used the auto config option. I don't know if it's something I'm doing wrong, or what, but any help would be appreciated.

Thanks in advance.


Related issues

Related to FreeNAS - Bug #25905: ZFS Send Failing To Copy After First SnapshotClosed: Cannot reproduce2017-09-18

History

#1 Updated by Andrew Sterling over 3 years ago

  • File debug-goliath-20170619233339.txz added

#2 Updated by Alexander Motin over 3 years ago

  • Category changed from 200 to 59
  • Assignee changed from Alexander Motin to William Grzybowski

William should know more about how to debug it.

#3 Updated by William Grzybowski over 3 years ago

  • Status changed from Unscreened to 15

Did you ever run Corral on it?

#4 Updated by William Grzybowski over 3 years ago

Jun 19 13:00:10 goliath /autorepl.py: [tools.autorepl:131] Sending zfs snapshot: /sbin/zfs send -V -p -i volume1/users/rheanna@auto-20170619.1200-1m volume1/users/rheanna@auto-20170619.1300-1m | /usr/local/bin/lz4c | /bin/dd obs=1m 2> /dev/null | /bin/dd obs=1m 2> /dev/null | /usr/local/bin/pipewatcher $$ | /usr/local/bin/ssh -i /data/ssh/replication -o BatchMode=yes -o StrictHostKeyChecking=yes -o ConnectTimeout=7 -p 22 192.168.1.135 "/usr/bin/env lz4c -d | /sbin/zfs receive -F -d 'volume1' && echo Succeeded" 
Jun 19 13:00:11 goliath /autorepl.py: [tools.autorepl:150] Replication result: internal error: Invalid argument
Abort (core dumped)

Please get "zfs get all volume1/users/rheanna"

#5 Updated by Andrew Sterling over 3 years ago

William Grzybowski wrote:

Did you ever run Corral on it?

Yes I did. To my recollection I didn't update zfs on it until FN11.

And I'll run that command when I get home from work tonight.

Thank you.

#6 Updated by Andrew Sterling over 3 years ago

William Grzybowski wrote:

[...]

Please get "zfs get all volume1/users/rheanna"

NAME PROPERTY VALUE SOURCE
volume1/users/rheanna type filesystem -
volume1/users/rheanna creation Mon Mar 7 23:02 2016 -
volume1/users/rheanna used 156G -
volume1/users/rheanna available 4.18T -
volume1/users/rheanna referenced 143G -
volume1/users/rheanna compressratio 1.01x -
volume1/users/rheanna mounted yes -
volume1/users/rheanna quota none default
volume1/users/rheanna reservation none default
volume1/users/rheanna recordsize 128K default
volume1/users/rheanna mountpoint /mnt/volume1/users/rheanna default
volume1/users/rheanna sharenfs off default
volume1/users/rheanna checksum on default
volume1/users/rheanna compression gzip-9 inherited from volume1/users
volume1/users/rheanna atime on default
volume1/users/rheanna devices on default
volume1/users/rheanna exec on default
volume1/users/rheanna setuid on default
volume1/users/rheanna readonly off default
volume1/users/rheanna jailed off default
volume1/users/rheanna snapdir hidden default
volume1/users/rheanna aclinherit passthrough inherited from volume1
volume1/users/rheanna canmount on default
volume1/users/rheanna xattr off temporary
volume1/users/rheanna copies 1 default
volume1/users/rheanna version 5 -
volume1/users/rheanna utf8only off -
volume1/users/rheanna normalization none -
volume1/users/rheanna casesensitivity sensitive -
volume1/users/rheanna vscan off default
volume1/users/rheanna nbmand off default
volume1/users/rheanna sharesmb off default
volume1/users/rheanna refquota none default
volume1/users/rheanna refreservation none default
volume1/users/rheanna primarycache all default
volume1/users/rheanna secondarycache all default
volume1/users/rheanna usedbysnapshots 12.9G -
volume1/users/rheanna usedbydataset 143G -
volume1/users/rheanna usedbychildren 0 -
volume1/users/rheanna usedbyrefreservation 0 -
volume1/users/rheanna logbias latency default
volume1/users/rheanna dedup off default
volume1/users/rheanna mlslabel -
volume1/users/rheanna sync standard default
volume1/users/rheanna refcompressratio 1.01x -
volume1/users/rheanna written 0 -
volume1/users/rheanna logicalused 157G -
volume1/users/rheanna logicalreferenced 144G -
volume1/users/rheanna volmode default default
volume1/users/rheanna filesystem_limit none default
volume1/users/rheanna snapshot_limit none default
volume1/users/rheanna filesystem_count none default
volume1/users/rheanna snapshot_count none default
volume1/users/rheanna redundant_metadata all default
volume1/users/rheanna org.freenas:permissions_type ACL local

#7 Updated by William Grzybowski over 3 years ago

Try:

zfs set aclmode=passthrough volume1/users/rheanna

And wait for next replication.

#8 Updated by Andrew Sterling over 3 years ago

William Grzybowski wrote:

Try:

zfs set aclmode=passthrough volume1/users/rheanna

And wait for next replication.

Well, ran that this morning and have received no emails so far today. What exactly did that do? lol

#9 Updated by William Grzybowski over 3 years ago

  • Status changed from 15 to Screened
  • Priority changed from No priority to Nice to have
  • Target version set to 11.0-U1

Andrew Sterling wrote:

William Grzybowski wrote:

Try:

zfs set aclmode=passthrough volume1/users/rheanna

And wait for next replication.

Well, ran that this morning and have received no emails so far today. What exactly did that do? lol

Removed an ZFS option that was exclusive of Corral.

#10 Updated by William Grzybowski over 3 years ago

  • Status changed from Screened to Closed: Not To Be Fixed
  • Target version changed from 11.0-U1 to N/A

We have pondered a migration script in such cases but for now decided its not worth the time. The docs will have this mentioned.

#11 Avatar?id=14398&size=24x24 Updated by Kris Moore over 3 years ago

  • Seen in changed from Unspecified to N/A

#12 Updated by Dru Lavigne about 3 years ago

  • File deleted (debug-goliath-20170619233339.txz)

#13 Updated by Dru Lavigne about 3 years ago

  • Private changed from Yes to No

#14 Updated by Alexander Motin about 3 years ago

  • Related to Bug #25905: ZFS Send Failing To Copy After First Snapshot added

Also available in: Atom PDF