CentOS

Backtrace from the log file for easier searching:

 10750:20141021:230935.069 11: /opt/zabbix/sbin/zabbix_server: poller #25 [got 1 values in 0.110493 sec, getting values](print_fatal_info+0x3c3) [0x80b4fb3]
 10750:20141021:230935.069 10: /opt/zabbix/sbin/zabbix_server: poller #25 [got 1 values in 0.110493 sec, getting values] [0x80b58b5]
 10750:20141021:230935.069 9: [0xb11440]
 10750:20141021:230935.069 8: /opt/zabbix/sbin/zabbix_server: poller #25 [got 1 values in 0.110493 sec, getting values] [0x8064242]
 10750:20141021:230935.069 7: /opt/zabbix/sbin/zabbix_server: poller #25 [got 1 values in 0.110493 sec, getting values](get_values_snmp+0x13a1) [0x8065af1]
 10750:20141021:230935.069 6: /opt/zabbix/sbin/zabbix_server: poller #25 [got 1 values in 0.110493 sec, getting values](poller_thread+0x139a) [0x8067a3a]
 10750:20141021:230935.069 5: /opt/zabbix/sbin/zabbix_server: poller #25 [got 1 values in 0.110493 sec, getting values](zbx_thread_start+0x3c) [0x80b5c6c]
 10750:20141021:230935.069 4: /opt/zabbix/sbin/zabbix_server: poller #25 [got 1 values in 0.110493 sec, getting values](MAIN_ZABBIX_ENTRY+0x402) [0x805c912]
 10750:20141021:230935.069 3: /opt/zabbix/sbin/zabbix_server: poller #25 [got 1 values in 0.110493 sec, getting values](daemon_start+0x1da) [0x80b3daa]
 10750:20141021:230935.069 2: /opt/zabbix/sbin/zabbix_server: poller #25 [got 1 values in 0.110493 sec, getting values](main+0x3b3) [0x805d133]
 10750:20141021:230935.069 1: /lib/libc.so.6(__libc_start_main+0xdc) [0xb27e9c]
 10750:20141021:230935.069 0: /opt/zabbix/sbin/zabbix_server: poller #25 [got 1 values in 0.110493 sec, getting values] [0x80565c1]

Could you please attach the disassembly of Zabbix server? It can be produced with "objdump -DSswx zabbix_server", as mentioned in the log file.

I have included the disassembly of Zabbix server.

Based on the log file and the disassembly, my guess is that you are monitoring some device using SNMPv1 and it occasionally returns a response with a bad error index.

Would it be possible for you to run Zabbix server with DebugLevel=4 enabled? Note that since Zabbix 2.4.0 you can change log level at runtime (increase it for all pollers using "zabbix_server -R log_level_increase=poller", see "Log level change at runtime" at What's new in 2.4.0) and there is no need to restart the server.

A similar issue has been fixed recently under ZBX-8621, but it has not been released yet. Anyway, DebugLevel=4 should help with confirming whether the guess above is really the problem.

Hi Aleksandrs,

Yes we are monitoring some devices using SNMPv1.

I have set DebugLevel on 4. Do you want the log files?

Yes, I would like some DebugLevel=4 log that precedes the crash.

Hi,
I've uploaded the zabbix log file.

Thank you, Kevin, but the attached log does not contain any crash. I only need a part of the log that gets written immediately prior to the crash. So you should wait until the crash happens and attach that log instead. Meanwhile, you might wish to remove the log you have just attached, because it contains some private information.

Hi Aleksandrs,
When the error occurs I will upload the error log. It didnt happen yet for several hours.

Hi,

Zabbix crashed again. I've uploaded the log files for you to see.

Attached log contains the following beginning of the line:

 26750:20141023:224333.639 zbx_snmp_get_values() snmp_synch_response() errindex:-128 oid:'

It can be seen that your MiktoTik device at x.x.x.71 returns "errstat" of SNMP_ERR_NOSUCHNAME and "errindex" of -128, when queried with 128 variables.

It might be useful to do some tcpdump on this device, to confirm that the problem is with the device, rather than Net-SNMP library.

In any case, ZBX-8621 should help with the crash - Zabbix will not crash, but will log that the device is not behaving properly and this suggests to turn SNMP bulk off for this device.

Based on the above, there seems to be nothing to fix on Zabbix side beyond ZBX-8621 (will be available in Zabbix 2.4.2).

If you could capture some traffic for this device, wait until Zabbix server crashes, and attach the last part of the traffic here, that would be useful. We would know where is the problem - the device, Net-SNMP library, or maybe it will show that the problem is actually with Zabbix.

Until then, closing as a duplicate.