Project

General

Profile

Bug #25161

Avatar?id=14398&size=22x22

Consul causing high Load

Added by Haris Khan about 3 years ago. Updated about 3 years ago.

Status:
Closed: Cannot reproduce
Priority:
No priority
Assignee:
Kris Moore
Category:
Middleware
Target version:
Seen in:
Severity:
New
Reason for Closing:
Reason for Blocked:
Needs QA:
Yes
Needs Doc:
Yes
Needs Merging:
Yes
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

[khanh@blackninja ~]$ uptime
4:17PM up 6:54, 1 users, load averages: 4.70, 4.80, 4.21

PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND
2942 root 21 49 0 11111M 11091M uwait 5 559:35 357.23% consul

within minutes of killing consul load comes down to .25

Associated revisions

Revision 9b8a1442 (diff)
Added by Kris Moore about 3 years ago

Update consul rc.d service to use syslog logging, so we can debug any failures in the field Ticket: #25161

History

#1 Updated by Haris Khan about 3 years ago

  • File debug-blackninja-20170716162235.txz added

#2 Updated by Dru Lavigne about 3 years ago

  • Assignee changed from Release Council to William Grzybowski
  • Seen in changed from Unspecified to 11.0-U1

#3 Updated by William Grzybowski about 3 years ago

  • Assignee changed from William Grzybowski to Kris Moore

Any ideas, Kris?

#4 Avatar?id=14398&size=24x24 Updated by Kris Moore about 3 years ago

  • Status changed from Unscreened to 15

Ouch, that is way out of the ordinary.

Try this:

  1. service consul stop
  2. rm -rf /var/db/system/consul
  3. service consul start

Does it jump back up to crazy CPU usage again?

#5 Updated by Haris Khan about 3 years ago

this seems to have resolved the issue, i no longer have high load. hovering on .22 on idle.

#6 Avatar?id=14398&size=24x24 Updated by Kris Moore about 3 years ago

  • Status changed from 15 to Closed: Cannot reproduce

Haris,

Thanks for the info! I've added some extra logging to the next version of FreeNAS so we can try and capture what went wrong next time. (It sounds like something in the local consul DB was corrupted). Additionally we will be bringing in a newer version soon(ish) as well. If it's reproducible on your end please re-open.

#7 Updated by Dru Lavigne about 3 years ago

  • File deleted (debug-blackninja-20170716162235.txz)

#8 Updated by Dru Lavigne about 3 years ago

  • Target version set to N/A
  • Private changed from Yes to No

Also available in: Atom PDF