Project

General

Profile

Bug #66430

Clarify replication process in Guide

Added by Mike Sullivan 10 months ago. Updated 7 months ago.

Status:
Done
Priority:
No priority
Assignee:
Timothy Moore II
Category:
Documentation
Target version:
Severity:
New
Reason for Closing:
Reason for Blocked:
Needs QA:
No
Needs Doc:
No
Needs Merging:
No
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

In my case, push and pull systems are both using encrypted pools, and even if a given replication task completes prior to a reboot on the pull system, said replication task starts from scratch on reboot. I believe this is an over looked side effect of combining encrypted pools with replication tasks, or possibly a bug. Confirm would be appreciated.

Further, the user guide provides sufficient particularity in regards to certain nuances related to replication (which I applaud and examples follow); however, once this bug is confirmed, I would suggest note of such be made in the user guide.

//

Note: Replicated data is not visible on the receiving system until the replication task completes.

Note: The target dataset on the receiving system is automatically created in read-only mode to protect the data. To mount or browse the data on the receiving system, create a clone of the snapshot and use the clone. Clones are created in read/write mode, making it possible to browse or mount them. See Snapshots for more information on creating clones.

History

#1 Updated by Dru Lavigne 9 months ago

  • Status changed from Unscreened to Blocked
  • Reason for Blocked set to Need additional information from Author

Mike: restarts should only occur if snapshot names fall outside of configured time or replication task did not complete. If this is not the case, please reproduce then attach a debug to this ticket (System -> Advanced -> Save debug).

#2 Updated by Mike Sullivan 9 months ago

Dru Lavigne wrote:

Mike: restarts should only occur if snapshot names fall outside of configured time or replication task did not complete. If this is not the case, please reproduce then attach a debug to this ticket (System -> Advanced -> Save debug).

Dru, thank you very much for your reply.

Can you clarify more precisely what you mean by snapshot names falling outside of the configured time, please? Is that to suggest that a hypothetical snapshot "" (which runs at 6 PM) would fail if the associated replication task is set to start at 8 PM? And that would cause a restart?

If so, I don't think that would apply to me, but the later remark would, i.e. incomplete replication task. I have yet to complete a full initial replication (I hoped / expected replication would run faster than I'm seeing @ 250 MB/s).

So perhaps the issue is me:
  • Given that I'm basically the only user accessing these systems,
  • and I prefer to have as "near real time" backup as possible (understood FreeNAS doesn't support real time),
  • I have replication configured as follows:

(1) Replication Tasks = Begins 00:00:00 and ends 23:59:00.
(2) Periodic Snapshot Tasks = Lifetime of 1 hour, Interval of 1 hour.

Is this a flawed approach?

#3 Updated by Dru Lavigne 9 months ago

Mike: were you able to get a replication to complete? If not, try the initial replication from the command line (see https://www.ixsystems.com/documentation/freenas/11.2/tasks.html#manual-testing) and wait for it to complete. Once it does, leave a comment on whether or not that solved the configured tasks issue.

#4 Updated by Mike Sullivan 9 months ago

Dru,

Thanks much for the follow up. Yes, I was able to successfully replicate 43 TiB from PUSH to PULL and the ticket can be closed. That was achieved by: (1) Sending the first snapshot via CLI (mbuffer + zfs send / recv specifically - much faster) and (2) then enabling snapshot replication tasks after manual send / recv for a given dataset had completed.

The suggestion that this was "possibly a bug" can be dismissed with further education on my end, specifically understanding that: (1) In order for a replication task to complete successfully (and start any sort of automated replication) both PUSH and PULL must "share" one snapshot in common and (2) Without a "shared" snapshot having been replicated from PUSH to PULL, if replication is interrupted for some reason, then replication will start anew.

I was hard for me to "learn" this with such large datasets and the time needed to send. Thank you for your feedback regarding snapshot life, as understanding of that was necessary to achieve a state of continuous replication.

Thanks, Mike

#5 Updated by Dru Lavigne 9 months ago

  • Category changed from Middleware to Documentation
  • Status changed from Blocked to Unscreened
  • Assignee changed from Release Council to Warren Block
  • Target version changed from Backlog to 11.2-U3
  • Reason for Blocked deleted (Need additional information from Author)

#7 Updated by Warren Block 9 months ago

  • Assignee changed from Warren Block to Timothy Moore II

#8 Updated by Timothy Moore II 8 months ago

  • Status changed from Unscreened to Screened

#9 Updated by Timothy Moore II 8 months ago

  • Status changed from Screened to In Progress

#10 Updated by Timothy Moore II 8 months ago

Docs PRs: https://github.com/freenas/freenas-docs/pull/717 [master], https://github.com/freenas/freenas-docs/pull/716 [angulargui]

Testing: Build the guide and find the "Replication" section ("Tasks" chapter in new UI guide, "Storage" chapter in legacy UI guide). Verify text explaining first-time replications vs later replications is present and explanation that interrupting a replication task forces it to restart from the beginning is present.

#11 Updated by Dru Lavigne 8 months ago

  • Subject changed from Replication Tasks | Potential Bug + User Guide Clarification Suggestion to Clarify replication process in Guide
  • Needs Doc changed from Yes to No
  • Needs Merging changed from Yes to No

#12 Updated by Dru Lavigne 8 months ago

  • Status changed from In Progress to Ready for Testing

#13 Updated by Dru Lavigne 7 months ago

  • Status changed from Ready for Testing to Done
  • Needs QA changed from Yes to No

Also available in: Atom PDF