-
Incident report
-
Resolution: Fixed
-
Major
-
1.8.8
-
FreeBSD 8.2 RELEASE amd64, zabbix-server-1.8.8, net-snmp-5.7
Looks like zabbix_server doesn't listen to Timeout config option for snmp checks on FreeBSD (Don't have Linux, can't check).
Now I have Timeout=30, but I see zabbix log messages like:
1749:20111125:172042.270 SNMP Host [spb9124-F1S2]: first network error, wait for 15 seconds
1749:20111125:172043.271 SNMP Host [spb9124-F2S2]: first network error, wait for 15 seconds
1754:20111125:172058.267 SNMP Host [spb9124-F1S2]: another network error, wait for 15 seconds
Ok, let's do debug = 4.
...
47693:20111123:111039.182 In get_values()
47693:20111123:111039.182 In DCconfig_get_poller_items() poller_type:0
47693:20111123:111039.182 End of DCconfig_get_poller_items():1
47693:20111123:111039.182 In substitute_simple_macros() data:'snmp.cpu'
47693:20111123:111039.182 In substitute_simple_macros() data:'iowa'
47693:20111123:111039.182 In substitute_simple_macros() data:'CISCO-SYSTEM-EXT-MIB::cseSysCPUUtilization.0'
47693:20111123:111039.182 In get_value() key:'snmp.cpu'
47693:20111123:111039.182 In get_value_snmp() key:'snmp.cpu' oid:'CISCO-SYSTEM-EXT-MIB::cseSysCPUUtilization.0'
47693:20111123:111039.182 In snmp_open_session()
47693:20111123:111039.182 SNMP [[email protected]:161]
47693:20111123:111039.183 End of snmp_open_session()
47693:20111123:111039.183 Standard processing
47693:20111123:111039.183 In snmp_normalize(oid:CISCO-SYSTEM-EXT-MIB::cseSysCPUUtilization.0)
47693:20111123:111039.183 End of snmp_normalize():CISCO-SYSTEM-EXT-MIB::cseSysCPUUtilization.0
47693:20111123:111039.183 In get_snmp(oid:CISCO-SYSTEM-EXT-MIB::cseSysCPUUtilization.0)
47695:20111123:111039.199 In get_values()
47695:20111123:111039.199 In DCconfig_get_poller_items() poller_type:0
47695:20111123:111039.199 End of DCconfig_get_poller_items():0
47695:20111123:111039.199 End of get_values()
47695:20111123:111039.199 poller #3 spent 0.000112 seconds while updating 0 values
47695:20111123:111039.199 In DCconfig_get_poller_nextcheck() poller_type:0
47695:20111123:111039.199 End of DCconfig_get_poller_nextcheck():1322032240
47695:20111123:111039.199 sleeping for 1 seconds
....
47693:20111123:111040.185 End of get_snmp():NETWORK_ERROR
47693:20111123:111040.185 In snmp_close_session()
47693:20111123:111040.185 End of snmp_close_session()
47693:20111123:111040.185 End of get_value_snmp():NETWORK_ERROR
47693:20111123:111040.185 Item [spb9124-F1S1:snmp.cpu] error: Timeout while connecting to [192.168.9.42:161]
47693:20111123:111040.185 In zabbix_log()
47693:20111123:111040.185 In DCconfig_get_items() hostid:0 key:'zabbix[log]'
47693:20111123:111040.185 End of DCconfig_get_items():0
47693:20111123:111040.185 End of zabbix_log()
47693:20111123:111040.186 End of get_value():NETWORK_ERROR
47693:20111123:111040.186 query [txnlev:1] [begin;]
47693:20111123:111040.186 query [txnlev:1] [update hosts set snmp_errors_from=1322032240,snmp_disable_until=1322032255 where hostid=10182]
47693:20111123:111040.186 query [txnlev:1] [commit;]
47693:20111123:111040.186 SNMP Host [spb9124-F1S1]: first network error, wait for 15 seconds
.....
About 1 second for timeout. Why?
[root@octans ~]# tcpdump -N -i bce0 host 192.168.9.42
....
17:25:39.675758 IP monitoring.59216 > spb9124-f1s1.snmp: C=iowa GetRequest(33) E:cisco.9.305.1.1.1.0
17:25:39.850199 IP monitoring > spb9124-f1s1: ICMP echo request, id 15879, seq 1280, length 76
17:25:39.850451 IP spb9124-f1s1 > monitoring: ICMP echo reply, id 15879, seq 1280, length 76
17:25:40.050191 IP monitoring > spb9124-f1s1: ICMP echo request, id 15879, seq 1280, length 76
17:25:40.050429 IP spb9124-f1s1 > monitoring: ICMP echo reply, id 15879, seq 1280, length 76
17:25:40.251189 IP monitoring > spb9124-f1s1: ICMP echo request, id 15879, seq 1280, length 76
17:25:40.251428 IP spb9124-f1s1 > monitoring: ICMP echo reply, id 15879, seq 1280, length 76
17:25:40.452795 IP monitoring > spb9124-f1s1: ICMP echo request, id 15879, seq 1280, length 76
17:25:40.453040 IP spb9124-f1s1 > monitoring: ICMP echo reply, id 15879, seq 1280, length 76
17:25:40.651187 IP monitoring > spb9124-f1s1: ICMP echo request, id 15879, seq 1280, length 76
17:25:40.651425 IP spb9124-f1s1 > monitoring: ICMP echo reply, id 15879, seq 1280, length 76
17:25:40.692946 IP spb9124-f1s1.snmp > monitoring.59216: C=iowa GetResponse(34) E:cisco.9.305.1.1.1.0=0
17:25:40.692963 IP monitoring > spb9124-f1s1: ICMP monitoring udp port 59216 unreachable, length 36
....
About 1 second for waiting responce again. 1 second is default timeout parameter for net-snmp.
So let's try net-snmp's snmpget - same tcpdump output.
And this
[root@octans ~]# time snmpget -c iowa -v1 192.168.9.42 1.3.6.1.4.1.9.9.305.1.1.1.0
Timeout: No Response from 192.168.9.42.
1.02s real 0.01s user 0.00s sys
But if I set -t 30 all works good every time.
[root@octans ~]# time snmpget -t 30 -c iowa -v1 192.168.9.42 1.3.6.1.4.1.9.9.305.1.1.1.0
SNMPv2-SMI::enterprises.9.9.305.1.1.1.0 = Gauge32: 0
1.60s real 0.02s user 0.00s sys
P.S.
[root@octans /usr/ports/net-mgmt/zabbix-server]# make showconfig
===> The following configuration options are available for zabbix-server-1.8.8,2:
MYSQL=on "Use MySQL backend"
PGSQL=off "Use PostgreSQL backend"
SQLITE=off "Use SQLite backend"
IPV6=on "Support for IPv6"
FPING=on "Use fping for pinging hosts"
JABBER=on "Support for jabber media type"
CURL=on "Support web monitoring with cURL"
LDAP=off "Support for checking LDAP servers"
IPMI=on "Support for IPMI"
SSH=on "Support for SSH-based checks"
IODBC=off "Support for iODBC"
UNIXODBC=off "Support for unixODBC"
===> Use 'make config' to modify these settings
[root@octans /usr/ports/net-mgmt/net-snmp]# make showconfig
===> The following configuration options are available for net-snmp-5.7_5:
IPV6=on "Build with IPv6 support"
MFD_REWRITES=off "Build with 64-bit Interface Counters"
PERL=on "Install additional perl modules"
PERL_EMBEDDED=on "Build embedded perl"
DUMMY=on "Enable dummy values as placeholders"
TKMIB=off "Install graphical MIB browser"
DMALLOC=off "Enable dmalloc debug memory allocator"
UNPRIVILEGED=off "Allow unprivileged users to execute net-snmp"
===> Use 'make config' to modify these settings