Bug #15274
SNMP service writes programming code to log file
100%
Description
I'm running a PRTG server (https://www.paessler.com/prtg) for monitoring devices on my network. I noticed since enabling the SNMP service on FreeNAS and started probing it with PRTG, it occasionally spits out what looks like some programming code (see below).
I also see the message:
netsnmp_assert 1 == new_val->high failed int64.c:370 netsnmp_c64_check32_and_update()
However the 'netsnmp_c64_check32' bug has been fixed in upstream (https://lists.freebsd.org/pipermail/freebsd-ports-bugs/2016-February/330579.html) so could the SNMP agent also be upgraded to the latest version?
Here's the output I see in the FreeNAS logs after performing a discovery via PRTG.
May 10 13:14:34 tardis snmpd[6427]: netsnmp_assert 1 == new_val->high failed int64.c:370 netsnmp_c64_check32_and_update()
May 10 13:17:35 tardis snmpd[6427]: netsnmp_assert 1 == new_val->high failed int64.c:370 netsnmp_c64_check32_and_update()
May 10 13:18:35 tardis snmpd[6427]: netsnmp_assert 1 == new_val->high failed int64.c:370 netsnmp_c64_check32_and_update()
May 10 13:41:15 tardis snmpd[6427]: params 1017/1024 -n
#pragma D option quiet
inline int OPT_time = 0;
inline int OPT_txg = 0;
inline int OPT_pool = 0;
inline int OPT_mega = 0;
inline int INTERVAL = 5;
inline int LINES = -1;
inline int COUNTER = 1;
inline int FILTER = 0;
inline string POOL = "";
dtrace:::BEGIN
{
/* starting values */
MEGA = 1000000;
counts = COUNTER;
secs = INTERVAL;
interval = INTERVAL;
interval == 0 ? interval++ : 1;
line = 0;
last_event[""] = 0;
nused=0;
nused_max_per_sec=0;
nused_per_sec=0;
size=0;
size_max_per_sec=0;
size_per_sec=0;
syncops=0;
size_4k=0;
size_4k_32k=0;
size_32k=0;
OPT_txg ? printf("waiting for txg commit...\n") : 1;
}
/*
* collect info when zil_lwb_write_start fires
*/
fbt::zil_lwb_write_start:entry
/OPT_pool == 0 || POOL == args[0]->zl_dmu_pool->dp_spa->spa_name/
{
nused += args[1]->lwb_nused;
nused_per_sec += args[1]->lwb_nused;
size += args[1]->lwb_sz;
size_per_sec += args[1]->lwb_sz;
syncops+
May 10 14:16:58 tardis snmpd[6427]: netsnmp_assert 1 == new_val->high failed int64.c:370 netsnmp_c64_check32_and_update()
May 10 14:35:59 tardis snmpd[6427]: netsnmp_assert 1 == new_val->high failed int64.c:370 netsnmp_c64_check32_and_update()
May 10 14:36:59 tardis snmpd[6427]: netsnmp_assert 1 == new_val->high failed int64.c:370 netsnmp_c64_check32_and_update()
May 10 14:40:59 tardis snmpd[6427]: netsnmp_assert 1 == new_val->high failed int64.c:370 netsnmp_c64_check32_and_update()
May 10 15:01:15 tardis snmpd[6427]: params 1017/1024 -n
#pragma D option quiet
inline int OPT_time = 0;
inline int OPT_txg = 0;
inline int OPT_pool = 0;
inline int OPT_mega = 0;
inline int INTERVAL = 10;
inline int LINES = -1;
inline int COUNTER = 1;
inline int FILTER = 0;
inline string POOL = "";
dtrace:::BEGIN
{
/* starting values */
MEGA = 1000000;
counts = COUNTER;
secs = INTERVAL;
interval = INTERVAL;
interval == 0 ? interval++ : 1;
line = 0;
last_event[""] = 0;
nused=0;
nused_max_per_sec=0;
nused_per_sec=0;
size=0;
size_max_per_sec=0;
size_per_sec=0;
syncops=0;
size_4k=0;
size_4k_32k=0;
size_32k=0;
OPT_txg ? printf("waiting for txg commit...\n") : 1;
}
/*
* collect info when zil_lwb_write_start fires
*/
fbt::zil_lwb_write_start:entry
/OPT_pool == 0 || POOL == args[0]->zl_dmu_pool->dp_spa->spa_name/
{
nused += args[1]->lwb_nused;
nused_per_sec += args[1]->lwb_nused;
size += args[1]->lwb_sz;
size_per_sec += args[1]->lwb_sz;
syncops
May 10 15:51:59 tardis snmpd[6427]: netsnmp_assert 1 == new_val->high failed int64.c:370 netsnmp_c64_check32_and_update()
May 10 15:52:59 tardis snmpd[6427]: netsnmp_assert 1 == new_val->high failed int64.c:370 netsnmp_c64_check32_and_update()
May 10 15:53:59 tardis snmpd[6427]: netsnmp_assert 1 == new_val->high failed int64.c:370 netsnmp_c64_check32_and_update()
May 10 16:30:08 tardis kernel: pid 6412 (python2.7), uid 0: exited on signal 11
History
#1
Updated by Jordan Hubbard over 1 year ago
- Target version set to 261
- Assignee set to Anonymous
BRB; That is fascinating. That's a dtrace script in the "D" language. Why it gets spat to stdout / stderr is totally unknown. We should both upgrade this and try to track this down if it still occurs after the upgrade.
#2 Updated by Anonymous over 1 year ago
To reproduce the scenario, will I have to install the PRTG server?
Also Can you please share the steps to reproduce the bug?
#3
Updated by Suraj Ravichandran over 1 year ago
Pasting slack reply to the same query here for records:
deepikadhiman: no need for reproducing that one before upgrading the port (edited)
[11:23]
i mean you could if need be (just to make it more satisfying) but it was a reported bug in net-snmp (as the guy has a link pointing out to)
[11:24]
deepikadhiman: instead you should track that link and see which version of net-snmp has that fix and if freenas has the same version or not
[11:24]
deepikadhiman: if freenas does not have the same version of the port then you will have to update the port
[11:25]
for 9.10 we use the following ports tree branch (edited)
[11:26]
``` "url": "https://github.com/freebsd/freebsd-ports.git",
"branch": "branches/2016Q2"```
[11:26]
also jfyi this information is available over at the freenas-build repo: https://github.com/freenas/freenas-build/blob/master/build/profiles/freenas9/repos.pyd
GitHub
freenas/freenas-build
freenas-build - FreeNAS 9.10/10 Build System
#4
Updated by Suraj Ravichandran over 1 year ago
Ok I looked into the freebsd ports tree for this and found out that the patch never made it through (was approved but not added for some reason)
So you will have to do the following:
1. Copy the net-mgmt/net-snmp port over to freenas repo under nas_ports (maintain net-mgmt/net-snmp folder hierarchy)
2. add this patch: https://bugs.freebsd.org/bugzilla/attachment.cgi?id=167153&action=edit to nas_ports/net-mgmt/net-snmp/files/ (make sure to name it:patch-snmplib_int64.c)
3. Bump the portrevision int he makefile.
4. Do a build using github.com/freenas/freenas-build with the above changes in to see if everything works
Please consult with Kaustubh for the port patching and building part as even he is doing the same for another port.
#5 Updated by Anonymous over 1 year ago
- Status changed from Unscreened to Screened
#6
Updated by Vaibhav Chauhan over 1 year ago
- Target version changed from 261 to 9.10-U1
#7 Updated by Anonymous over 1 year ago
- Target version changed from 9.10-U1 to 261
@Suraj: I tried the steps you mentioned. But the build is failing while applying "net-mgmt/net-snmp/patch.
Logs :
[00:53:18] ====>> [02][00:00:00] Starting build of net-mgmt/net-snmp
[00:54:39] ====>> [02][00:01:21] Saved net-mgmt/net-snmp wrkdir to: /root/freenas-build/_BE/objs/ports/wrkdirs/ja-p/p/net-snmp-5.7.3_12.tbz
[00:54:40] ====>> [02][00:01:22] Finished build of net-mgmt/net-snmp: Failed: patch
[01:22:53] ====>> Failed ports: devel/gettext-runtime:fetch security/cyrus-sasl2:fetch net-mgmt/net-snmp:patch
[3:43:59] > Installing ports
[3:43:59] > Log file: /root/freenas-build/_BE/objs/logs/pkg-install
[3:44:02] ==> ERROR: Packages installation failed, see log file /root/freenas-build/_BE/objs/logs/pkg-install
Logs at /root/freenas-build/_BE/objs/logs/pkg-install:
Installing pkg-1.7.2...
Extracting pkg-1.7.2: .......... done
Updating local repository catalogue...
Fetching meta.txz: . done
Fetching packagesite.txz: ...... done
Processing entries: .......... done
local repository update completed. 203 packages processed.
Updating database digests format: . done
pkg: No packages available to install matching 'freenas/freenas-ui' have been found in the repositories
I observed that patch was changing the value of portrevision in the makefile, so I tried to apply patch by skipping the step 3 also but it failed at the same port.
I applied the following steps:
make checkout PROFILE=freenas9
make release PROFILE=freenas9
Am I missing something?
#8
Updated by Vaibhav Chauhan over 1 year ago
- Target version changed from 261 to 9.10-U1
Deepika. the reason I am changing target version to 9.10-RELEASE-p1 as I this is the version we are hopefully aiming for the fix.
#9 Updated by Anonymous over 1 year ago
I am adding a new port 'net-snmp' from freebsd to freenas with a patch added. I have new setup of freebsd, clonned the freenas-build repository, done make checkout and copied my new port directory to freenas/nas-ports/net-mgmt. Now I have done make release, but its not picking up my new port and generating an error at that point. Am I missing something before doing make release? Sould I give the path of the new port added to some other conf file or something?
#10 Updated by Anonymous over 1 year ago
Hi Suraj, I have applied the patch successfully after making some changes in it.
I removed the lines from patch which were:
changing the value of portrevision in Makefile from 11 to 12.
As I changed it manually.
After this the is build done successfully and ISO is created.
Where should I share the ISO?
#11 Updated by Anonymous over 1 year ago
I applied the patch and rebuild the freenas.
I tested the scenario using 'snmpwalk' command and now it is not showing any code in the logs.
So, patch worked successfully.
Should I create a pull request with the changes?
#12 Updated by Anonymous over 1 year ago
- Status changed from Screened to Investigation
#13
Updated by Vaibhav Chauhan over 1 year ago
- Target version changed from 9.10-U1 to 9.10.1-U1
#14 Updated by Anonymous over 1 year ago
- Status changed from Investigation to Fix In Progress
#15 Updated by Anonymous over 1 year ago
I have created a pull request for this ticket and waiting to get it reviewed.
https://github.com/freenas/freenas/pull/179
#16
Updated by Kris Moore over 1 year ago
- Assignee changed from Anonymous to Suraj Ravichandran
Suraj, can you review the pull request? The new port looks correct to me, just want you to confirm that the ticket was satisfied.
#17
Updated by Suraj Ravichandran over 1 year ago
- Status changed from Fix In Progress to Build Testing
- % Done changed from 0 to 90
The port and patch look good, have merged them to master.
I will check SNMP on a nightly before making the fix branches and be copying the ticket over and such.
#18 Updated by Anonymous over 1 year ago
- Status changed from Build Testing to Fix In Progress
- % Done changed from 90 to 0
- Assignee changed from Suraj Ravichandran to Anonymous
#19
Updated by Dru Lavigne over 1 year ago
Suraj, has this been merged? If not, will it be in time for U1?
#20 Updated by Anonymous over 1 year ago
@Suraj, I had created a pull request https://github.com/freenas/freenas/pull/179/files.
But as you have pointed out there is some issue in Makefile.
So, should I just change ".if ${SSL_DEFAULT} != base" with ".if defined(WITH_OPENSSL_PORT) || defined(OPENSSL_PORT)". in Makefile in my commit as we want to use the older WITH_OPENSSL_PORT stuff?
#21
Updated by Kris Moore over 1 year ago
Deepika,
Yes, it should look like this:
.if defined(WITH_OPENSSL_PORT) || defined(OPENSSL_PORT)
LCRYPTO= -lcrypto
.else
LCRYPTO=
.endif
#22
Updated by Kris Moore over 1 year ago
- Priority changed from No priority to Nice to have
#23
Updated by Vaibhav Chauhan over 1 year ago
BRB: we need this fix to go in before due date.
#24 Updated by Anonymous over 1 year ago
I have created a pull request for the changes made : https://github.com/freenas/freenas/pull/192
#25
Updated by Kris Moore over 1 year ago
- Target version changed from 9.10.1-U1 to 9.10.1-U2
#26 Updated by Anonymous over 1 year ago
@Suraj: Please have a look at pull request https://github.com/freenas/freenas/pull/192
#27
Updated by Vaibhav Chauhan over 1 year ago
- Target version changed from 9.10.1-U2 to 9.10.1-U3
#28
Updated by Vaibhav Chauhan over 1 year ago
- Target version changed from 9.10.1-U3 to 9.10.1-U2
#29
Updated by Kris Moore over 1 year ago
- Target version changed from 9.10.1-U2 to 9.10.1-U3
#30
Updated by Dru Lavigne about 1 year ago
- Assignee changed from Anonymous to Suraj Ravichandran
#31
Updated by Dru Lavigne about 1 year ago
Will this make it into U3?
#32
Updated by Kris Moore about 1 year ago
- Target version changed from 9.10.1-U3 to 9.10.2-U1
#34
Updated by Suraj Ravichandran about 1 year ago
- Status changed from Fix In Progress to Reviewed
#35
Updated by Suraj Ravichandran about 1 year ago
- % Done changed from 0 to 100
#36
Updated by The m0nkey_ about 1 year ago
What's the delay with this bug? I see it's been pushed back a few times now.
#37
Updated by Suraj Ravichandran about 1 year ago
@The m0nkey_ it should be available in 9.10.1-U3, I have already created a fix branch and marked it as reviewed and our release engineer will merge it in
#38
Updated by Dru Lavigne about 1 year ago
Suraj, should the target version be changed back to U3?
#39
Updated by Suraj Ravichandran about 1 year ago
- Target version changed from 9.10.2-U1 to 9.10.1-U3
#40
Updated by Suraj Ravichandran about 1 year ago
@Dru thanks for reminding me!
#41
Updated by Vaibhav Chauhan about 1 year ago
- Status changed from Reviewed to Ready For Release
#42
Updated by Vaibhav Chauhan about 1 year ago
- Status changed from Ready For Release to Resolved