Project

General

Profile

Bug #66880

Fix dhclient network device flapping

Added by Andrew Kraut 10 months ago. Updated 10 months ago.

Status:
Done
Priority:
No priority
Assignee:
Ryan Moeller
Category:
OS
Target version:
Seen in:
Severity:
Low
Reason for Closing:
Reason for Blocked:
Needs QA:
No
Needs Doc:
No
Needs Merging:
No
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

After updating to 11.2, ethernet device shows flapping link.

Hardware summary:
  • FreeNAS Mini XL, directly connected to an Amplifi HD Router
Troubleshooting attempts:

Log snippet:

Jan  1 12:56:20 walter kernel: igb1: link state changed to DOWN
Jan  1 12:56:20 walter kernel: igb1: link state changed to DOWN
Jan  1 12:56:23 walter kernel: igb1: link state changed to UP
Jan  1 12:56:23 walter kernel: igb1: link state changed to UP
Jan  1 12:56:29 walter dhclient: New IP Address (igb1): 192.168.159.56
Jan  1 12:56:29 walter dhclient: New Subnet Mask (igb1): 255.255.255.0
Jan  1 12:56:29 walter dhclient: New Broadcast Address (igb1): 192.168.159.255
Jan  1 12:56:29 walter dhclient: New Routers (igb1): 192.168.159.1
Jan  1 12:56:30 walter kernel: igb1: link state changed to DOWN
Jan  1 12:56:30 walter kernel: igb1: link state changed to DOWN
Jan  1 12:56:33 walter kernel: igb1: link state changed to UP
Jan  1 12:56:33 walter kernel: igb1: link state changed to UP
Jan  1 12:56:39 walter dhclient: New IP Address (igb1): 192.168.159.56
Jan  1 12:56:39 walter dhclient: New Subnet Mask (igb1): 255.255.255.0
Jan  1 12:56:39 walter dhclient: New Broadcast Address (igb1): 192.168.159.255
Jan  1 12:56:39 walter dhclient: New Routers (igb1): 192.168.159.1
Jan  1 12:56:39 walter kernel: igb1: link state changed to DOWN
Jan  1 12:56:39 walter kernel: igb1: link state changed to DOWN
Jan  1 12:56:43 walter kernel: igb1: link state changed to UP
Jan  1 12:56:43 walter kernel: igb1: link state changed to UP

History

#1 Updated by Dru Lavigne 10 months ago

  • Private changed from No to Yes
  • Reason for Blocked set to Need additional information from Author

Andrew: please attach a debug (System -> Advanced -> Save debug) to this ticket.

#2 Updated by Andrew Kraut 10 months ago

  • File debug-walter-20190101165619.tgz added

System Debug attached

#3 Updated by Dru Lavigne 10 months ago

  • Assignee changed from Release Council to Alexander Motin
  • Reason for Blocked deleted (Need additional information from Author)

#4 Updated by Alexander Motin 10 months ago

  • Status changed from Unscreened to Blocked
  • Assignee changed from Alexander Motin to Ryan Moeller
  • Reason for Blocked set to Need additional information from Author

The possible explanation in referenced FreeBSD bug report sounds plausible. Make sure you applied the proposed workaround correctly (for example, specifying proper interface name). The proper fix mentioned there is already in FreeNAS 11.3-nightly, and we could merge it, if confirmed that it help.

If problem is not in MTU, then we would need to know what exactly your DHCP server provides. Full tcpdump packet capture of several cycles could be interesting. May be I'd try to run dhclient under truss to see what IOCTLs it calls for the interface, that may cause its flapping.

Ryan, could you track this, and may be backport the fix.

#5 Updated by Andrew Kraut 10 months ago

I applied the mtu supersede as outlined in the FreeBSD bug inside /etc/dhclient.conf, but it didn't seem to remedy the issue. Is that the correct place to put that?

I will capture DHCP traffic and attach it later this evening. Is there a reasonably safe way to switch over to 11.3-nightly for testing the fix? This is my daily-driver unit, so I'm keen to avoid data loss if possible. Alternatively, can I manually install just the affected packages that are suspected to fix the issue?

#6 Updated by Ryan Moeller 10 months ago

Note that /etc/dhclient.conf is generated from a template at each boot, so if you rebooted after editing the file, the changes are probably gone. Did you reboot, or manually kill dhclient and rerun it in the shell?

I suggest downloading a backup of your config and perhaps snapshot the boot environment if you are looking for some peace of mind. It is probably preferable to stay on 11.2, since either the workaround will solve your problem, or it won't and upgrading to the nightly train wouldn't help.

#7 Updated by Ryan Moeller 10 months ago

I spoke too soon, the change for handling supersede interface-mtu 0 also is not in 11.2, so that would not have the desired effect. I'll make a PR for both of these, if you can confirm the issue somehow.

Are you able to install a nightly image to a spare USB stick for testing?

#8 Updated by Ryan Moeller 10 months ago

  • Target version changed from Backlog to 11.2-U2
  • Severity changed from New to Low
  • Reason for Blocked changed from Need additional information from Author to Need verification

freenas/11.2-stable PR: https://github.com/freenas/os/pull/170

I made the PR anyway because these updates are good to have, but I would still like to find out if this fixes the issue for you.

Andrew Kraut wrote:

I will capture DHCP traffic and attach it later this evening

That will be helpful, if you can.

Andrew Kraut wrote:

Is there a reasonably safe way to switch over to 11.3-nightly for testing the fix?

You can switch back to a previous boot environment and continue on 11.2-stable after jumping on the 11.3-nightly train. See `System -> Boot Environments` and https://www.ixsystems.com/documentation/freenas/11.2/system.html#boot-environments for details.
Be sure to save a backup of your config when prompted, just in case.

#9 Updated by Ryan Moeller 10 months ago

  • Status changed from Blocked to Ready for Testing
  • Reason for Blocked deleted (Need verification)
  • Needs Merging changed from Yes to No

#10 Updated by Dru Lavigne 10 months ago

  • File deleted (debug-walter-20190101165619.tgz)

#11 Updated by Dru Lavigne 10 months ago

  • Subject changed from Network devices flap after 11.2 update to Fix dhclient network device flapping
  • Private changed from Yes to No
  • Needs Doc changed from Yes to No

#13 Updated by Ryan Moeller 10 months ago

  • Needs QA changed from Yes to No

#15 Updated by Dru Lavigne 10 months ago

  • Status changed from Ready for Testing to Done

#16 Updated by Andrew Kraut 10 months ago

Ok, I can confirm that there's no flapping in the nightly.

Also available in: Atom PDF