[ZBX-7933] zabbix generate TCP queue overflow Created: 2014 Mar 12  Updated: 2025 Apr 10  Resolved: 2021 Aug 25

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Proxy (P), Server (S)
Affects Version/s: 2.2.1, 2.2.2
Fix Version/s: 5.0.15rc1, 5.4.4rc1, 6.0.0alpha1, 6.0 (plan)

Type: Problem report Priority: Major
Reporter: Stefan Sabolowitsch Assignee: Dmitrijs Goloscapovs
Resolution: Fixed Votes: 8
Labels: javagateway, pending
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

FreeBSD 10 i386 / x64

Attachments: File patch-src_libs_zbxcomms_comms.c    
Issue Links:
Causes
causes ZBX-20453 Agent2 compilation broken on macOS Closed
Duplicate
is duplicated by ZBX-17621 zabbix server listen backlog SOMAXCON... Closed
is duplicated by ZBX-17762 Frequent TCP listen queue overflows Closed
Team: Team A
Sprint: Sprint 67 (Aug 2020), Sprint 68 (Sep 2020), Sprint 69 (Oct 2020), Sprint 70 (Nov 2020), Sprint 71 (Dec 2020), Sprint 72 (Jan 2021), Sprint 73 (Feb 2021), Sprint 74 (Mar 2021), Sprint 75 (Apr 2021), Sprint 77 (Jun 2021), Sprint 78 (Jul 2021), Sprint 79 (Aug 2021)
Story Points: 1

 Description   

From time to time generates zabbix a TCP queue overflow.
Then is no traffic from / to ZABBIX more possible, only zabbix restart help here.

 
Mar 12 17:20:17 zabbix-p01 kernel: sonewconn: pcb 0xc809dad4: Listen queue overflow: 193 already in queue awaiting acceptance
Mar 12 17:20:48 zabbix-p01 last message repeated 18 times
Mar 12 17:21:34 zabbix-p01 last message repeated 28 times

 
[root@zabbix-p01 ~]# netstat -Lan
Current listen queue sizes (qlen/incqlen/maxqlen)
Proto Listen         Local Address         
tcp4  0/0/10         127.0.0.1.25           
tcp4  0/0/128        *.22                   
tcp6  0/0/128        *.22                   
tcp4  0/0/128        *.199                  
tcp4  0/0/128        *.11000                
tcp4  193/0/128      *.10051                
tcp4  0/0/128        *.10050                
tcp6  0/0/128        *.10050                
tcp4  0/0/50         *.3306                 
unix  0/0/50         /tmp/mysql.sock
unix  0/0/4          /var/run/devd.pipe


 Comments   
Comment by Oleksii Zagorskyi [ 2014 Mar 13 ]

could it be the same as ZBX-5271 ?

Comment by Stefan Sabolowitsch [ 2014 Mar 13 ]

I think no, have no javagateway active.

Comment by Juris Miščenko (Inactive) [ 2014 Mar 20 ]

Stefan, could you please check whether the traffic to the server process really dies completely and it comes to a stand-still? Ignore the queue overflow messages for now.

Comment by Juris Miščenko (Inactive) [ 2014 Apr 15 ]

I am closing this issue as the reporter hasn't replied with the necessary information for some time now.

When I began working on the issue, I contacted the FreeBSD developers and asked them why this might be happening and the answer is very simple: in FreeBSD10 they simply added an informative message which is now printed by the kernel. Other than that, the behavior of the kernel has not changed. TCP queue overflows were happening before, they simply weren't reported. This is normal behavior. Also, this is not a Zabbix issue.

If you believe that there is still an issue and your Zabbix components are overflowing the queue in such a way that it makes the service unusable, log this behavior and reopen the issue.

Comment by Alan Somers [ 2020 May 18 ]

This is still an issue.  Not a correctness issue; just a performance issue.  All of those dropped TCP connections hurt performance.  On my system, I find that when the maximum listen queue is very large, the backlog quickly spikes from 0 to 215 connections before plateauing.  This strongly suggests that raising the listen queue size is the correct solution.

 

The attached patch raises the listen queue from the old-fashioned SOMAXCONN to INT_MAX instead.  The operating system will limit the actual queue depth, of course.  On FreeBSD, the actual maximum queue depth is set by the kern.ipc.soacceptqueue sysctl.  SOMAXCONN is a fossil that doesn't have any real meaning anymore.

Comment by dimir [ 2020 May 19 ]

Let's re-open this initial one and close ZBX-17762 as duplicate.

Comment by Alan Somers [ 2020 May 19 ]

Find by me.  The only reason that I opened a new issue is because I lack the permissions to reopen an old one, or to change its labels.  You should add the "patch" label to this issue.

Comment by Glebs Ivanovskis [ 2020 May 21 ]

One more recent issue that is related — ZBX-17621.

Comment by Dmitrijs Goloscapovs [ 2021 Jul 27 ]

Available in versions:

Documentation updated:

Generated at Fri May 09 08:35:16 EEST 2025 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.