SNMP bulk request fallback logic flaw causes poller exhaustion and prevents host from becoming Unreachable

XMLWordPrintable

    • Type: Problem report
    • Resolution: Unresolved
    • Priority: Trivial
    • None
    • Affects Version/s: 7.0.28
    • Component/s: Server (S)
    • None
    • Environment:
      Rocky Linux 8.10

      Steps to reproduce:

      1. Use a standard Zabbix Server configuration with default poller settings (e.g., StartPollers=5, StartPollersUnreachable=1) to easily observe the exhaustion.

      2. Create a host with an SNMP interface.

      3. In the SNMP interface settings, ensure the "Use combined requests" checkbox is enabled.

      4. Create multiple (at least 5) SNMP agent items using the traditional direct OID specification (Individual OID polling, not the newer get/walk[OID] method).
      If you use 3 or fewer, the items might be polled individually (num:1 level:0) due to timing, which bypasses the bug.

      5. Set the exact same "Update interval" (e.g., 1m) for all 5 items so they are scheduled together.

      6. Set DebugLevel=4 on the Zabbix Server to monitor the internal SNMP polling process.

      7. Ensure the snmpd service on the target host is running, and let Zabbix poll the items successfully at least once. Verify in the zabbix_server.log that the items were successfully polled together in a single bulk request. You should see a log entry confirming status:0 and mapping_num:5:

      ... zbx_snmp_get_values() snmp_sess_synch_response() status:0 s_snmp_errno:0 errstat:0 mapping_num:5

      8. Stop the snmpd service on the target host (or use iptables to DROP UDP port 161) to simulate a sudden, complete unresponsiveness/timeout scenario.

      9. Wait for the next polling cycle

      Result:

      1. The host never transitions to the Unreachable state (which should be handled by StartPollersUnreachable). The SNMP availability icon on the frontend remains green.

      2. The Zabbix server's regular Poller process (StartPollers) is held hostage for a long duration doing serial timeouts. (For 5 items, the poller is locked for about 24 seconds. At scale, e.g., with 128 items, this causes severe Poller exhaustion, locking processes for several minutes and delaying all other checks).

      3. In the next polling cycle, the poller starts the bulk request again from the maximum size (level: 0), creating a permanent "silent" exhaustion loop across the entire Zabbix Server.

      4. See log file: The zabbix_server.log shows the fallback process (num:5 level:0 -> num:2 level:1 -> num:1 level:2). When the final single-item query (level:2) times out (status:2), the function wrongly returns SUCCEED instead of an error:

      Expected:

      1. When the final single-item fallback (level: 2) times out, it should report a network error (NETWORK_ERROR), and the host must be placed into the Unreachable state to prevent regular Poller exhaustion.

      2. This matches the official Zabbix documentation for Dynamic SNMP bulk requests, which states: "If it still fails at this point [querying one by one], then the device is definitely not responding and request size is not an issue."

      3. Root Cause (Source Code): In src/libs/zbxpoller/checks_snmp.c, inside zbx_snmp_get_values(), when level == 2 and a timeout occurs, the code hits if (1 <= level) goto halve;. However, under the halve: label, there is no condition for level >= 2. The execution simply falls through without calling zbx_get_snmp_response_error and returns the default SUCCEED. A patch is needed to ensure network errors are properly generated when level >= 2 times out.

        1. bug_report.log
          197 kB
          Kotaro Miyashita

            Assignee:
            Andrejs Poddubnaks
            Reporter:
            Kotaro Miyashita
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: