Ubuntu 18.04.1 LTC Bionic
We are currently monitoring 30 hosts using a custom built SNMPv3 template with 4 items. On about 12 of those hosts we get the error message "Timeout while connecting to "host:port".".
The hosts that are working and that aren't working are using completely the same templates with no modifications done to specific hosts.
We've recreated hosts that aren't working by cloning them from hosts that are.
We've tried changing UDP ports that the hosts are listening on (and then making the corresponding Port change for an SNMP interface for a host in Zabbix, we did not modify the template Port for this test, just the host port).
We've also captured network traffic between the Zabbix server and the hosts that are and aren't working properly.
While capturing traffic on the hosts that didn't work we could see traffic and information sent and received by both the Zabbix server and the host that was being monitored. However Zabbix was still reporting the above error.
We were able to notice a difference between the packets generated by the Zabbix server that were sent to the hosts that were and weren't working.
On the hosts that are working correctly the Zabbix server seemed to respect the "Use bulk requests" setting and it was sending the request all in one packet.
While the hosts that didn't work properly the request was broken up into individual packets even though "Use bulk requests" was checked. Turning off bulk requests didn't fix the issue either.
The individual requests were still being received by the monitored host though, as shown in screenshots bellow, and the responses were being sent back.