[ZBX-10781] Communication not working between server and proxy Created: 2016 May 11 Updated: 2018 Feb 02 Resolved: 2016 Jun 13 |
|
| Status: | Closed |
| Project: | ZABBIX BUGS AND ISSUES |
| Component/s: | Proxy (P), Server (S) |
| Affects Version/s: | 3.0.2 |
| Fix Version/s: | None |
| Type: | Incident report | Priority: | Major |
| Reporter: | Stephan Niklas | Assignee: | Unassigned |
| Resolution: | Won't fix | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Proxy: CentOS Linux release 7.2 |
||
| Issue Links: |
|
||||||||
| Description |
|
Hello, I recently updated my Zabbix installation from version 2.2.12 to 3.0.2. The overall migration went pretty flawless except one proxy which isn't working anymore. I always receive the following error message Zabbix server 31238:20160511:164221.214 sending configuration data to proxy "TJP-ZAPR01" at "213.208.148.34", datalen 54385 31238:20160511:164236.214 cannot send configuration data to proxy "TJP-ZAPR01" at "213.208.148.34": ZBX_TCP_WRITE() timed out Zabbix proxy 27000:20160511:161518.057 cannot obtain configuration data from server at "85.126.241.58": ZBX_TCP_READ() timed out The communication itself works without any problems because the, on the proxy installed, agent is communicating without issues with the server. From proxy [root@tjp-zapr01 ~]# telnet 85.126.241.58 10050 Trying 85.126.241.58... Connected to 85.126.241.58. Escape character is '^]'. Connection closed by foreign host. [root@tjp-zapr01 ~]# telnet 85.126.241.58 10051 Trying 85.126.241.58... Connected to 85.126.241.58. Escape character is '^]'. Connection closed by foreign host. From server root@ebcontmonitoring:~# telnet 213.208.148.34 10050 Trying 213.208.148.34... Connected to 213.208.148.34. Escape character is '^]'. ZBXD)ZBX_NOTSUPPORTEDInvalid item key format.Connection closed by foreign host. root@ebcontmonitoring:~# telnet 213.208.148.34 10051 Trying 213.208.148.34... Connected to 213.208.148.34. Escape character is '^]'. Connection closed by foreign host. Please support. Thanks in advance. cheers, |
| Comments |
| Comment by Aleksandrs Saveljevs [ 2016 May 11 ] |
|
Looks like a duplicate of |
| Comment by Stephan Niklas [ 2016 May 11 ] |
|
I forgot to add... Due to this problem all agents behind the proxy are not available with "Zabbix agent on ... is unreachable for 8 minutes" |
| Comment by richlv [ 2016 May 11 ] |
|
hmm, given |
| Comment by Stephan Niklas [ 2016 May 11 ] |
|
I already increased the timeout on both server and proxy side to 15 seconds. Without any success |
| Comment by Stephan Niklas [ 2016 May 11 ] |
8108:20160511:171626.439 sending configuration data to proxy "TJP-ZAPR01" at "213.208.148.34", datalen 54385 8108:20160511:171641.440 cannot send configuration data to proxy "TJP-ZAPR01" at "213.208.148.34": ZBX_TCP_WRITE() timed out |
| Comment by Stephan Niklas [ 2016 May 11 ] |
|
Furthermore I have this problem with just one of five proxies |
| Comment by Stephan Niklas [ 2016 May 11 ] |
|
I'm not sure if it's a duplicate. I have additional proxies in my environment which work without any issues after the upgrade |
| Comment by Aleksandrs Saveljevs [ 2016 May 12 ] |
|
This is a duplicate of |
| Comment by Stephan Niklas [ 2016 May 12 ] |
|
Ok. Thanks for confirmation. Any forecast when there will be an installable package available for the fixed version? |
| Comment by Stephan Niklas [ 2016 May 24 ] |
|
We still have the issue with the timeout after the upgrade to 3.0.3 in the server and the affected proxy Zabbix server 18602:20160524:100529.185 sending configuration data to proxy "TJP-ZAPR01" at "213.208.148.34", datalen 54385 18602:20160524:101029.185 cannot send configuration data to proxy "TJP-ZAPR01" at "213.208.148.34": ZBX_TCP_WRITE() timed out Zabbix proxy 10491:20160524:101529.163 cannot obtain configuration data from server at "85.126.241.58": ZBX_TCP_READ() timed out Please support |
| Comment by Glebs Ivanovskis (Inactive) [ 2016 May 24 ] |
|
Look at the timestamps in log files - server and proxy time out virtually at the same moment. I server wants to send data (but can't) and proxy wants to receive data (but can't) at the same time there must be some sort of communication breakdown on OS kernel TCP implementation level or somewhere between server and proxy. I think your issue is not related to Zabbix. |
| Comment by Stephan Niklas [ 2016 May 24 ] |
|
But why does the communication between the server and the installed zabbix-client work without issues then? |
| Comment by Glebs Ivanovskis (Inactive) [ 2016 May 24 ] |
|
It simply goes through a different port. As I understand, in your environment agent installed on proxy host communicates with server directly, not through proxy. Try to have a look at connection queue and statuses on port where server and proxy communicate using netstat -na or ss -l. |
| Comment by Glebs Ivanovskis (Inactive) [ 2016 Jun 08 ] |
|
sniklas, please check ports one more time. Command in the description (root@ebcontmonitoring:~# telnet 213.208.148.34 10050) and the response (ZBXD)ZBX_NOTSUPPORTEDInvalid item key format.) suggests that you connected to agent instead of proxy. |
| Comment by Stephan Niklas [ 2016 Jun 08 ] |
|
The network connection was fine. The problem was the SELinux configuration on the host (enabled by default on CentOS 7.2). The communication between the server and proxy is now working. The ticket can be closed |
| Comment by Alexei Vladishev [ 2016 Jun 13 ] |
|
Closed, thanks for the follow up. |
| Comment by Dellroc [ 2018 Feb 02 ] |
|
У меня была подобная проблема, только сервер вообще не видел попыток переподключения. В результате проблема оказалась в сетевом оборудовании, которое нет возможности настроить. Решением оказалось перенастроить непопулярный порт 10051 сервера на весьма активно используемый 3389. |