[ZBXNEXT-4839] Unable to start zabbix agent in RHEL7 Created: 2018 Oct 30  Updated: 2020 Aug 12  Resolved: 2020 Aug 12

Status: Closed
Project: ZABBIX FEATURE REQUESTS
Component/s: Installation (I)
Affects Version/s: None
Fix Version/s: None

Type: Change Request Priority: Trivial
Reporter: janakiram pulavarthi Assignee: Unassigned
Resolution: Won't fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: JPEG File zabbix.JPG    

 Description   

Unable to start zabbix agent in RHEL 7.

Snippet below from agent log file : 

150551.023 Starting Zabbix Agent [POD2-Node1]. Zabbix 4.0.0 (revision 85308).
150551.023 **** Enabled features ****
150551.023 IPv6 support: YES
150551.023 TLS support: YES
150551.023 **************************
150551.023 using configuration file: /etc/zabbix/zabbix_agentd.conf
150551.023 listener failed: bind() for [[192.168.220.100]:10050] failed: [99] Cannot assign requested address

snippet from systemctl status 

[root@compute-0 ~]# systemctl status zabbix-agent

  • zabbix-agent.service - Zabbix Agent
    Loaded: loaded (/usr/lib/systemd/system/zabbix-agent.service; disabled; vendor preset: disabled)
    Active: activating (auto-restart) (Result: exit-code) since Tue 2018-10-30 15:17:30 UTC; 2s ago
    Process: 42857 ExecStop=/bin/kill -SIGTERM $MAINPID (code=exited, status=1/FAILURE)
    Process: 42853 ExecStart=/usr/sbin/zabbix_agentd -c $CONFFILE (code=exited, status=0/SUCCESS)
    Main PID: 42855 (code=exited, status=1/FAILURE)

Oct 30 15:17:30 compute-0 kill[42857]: -s, --signal <sig> send specified signal
Oct 30 15:17:30 compute-0 kill[42857]: -q, --queue <sig> use sigqueue(2) rather than kill(2)
Oct 30 15:17:30 compute-0 kill[42857]: -p, --pid print pids without signaling them
Oct 30 15:17:30 compute-0 kill[42857]: -l, --list [=<signal>] list signal names, or convert one to a name
Oct 30 15:17:30 compute-0 kill[42857]: -L, --table list signal names and numbers
Oct 30 15:17:30 compute-0 kill[42857]: -h, --help display this help and exit
Oct 30 15:17:30 compute-0 kill[42857]: -V, --version output version information and exit
Oct 30 15:17:30 compute-0 kill[42857]: For more details see kill(1).
Oct 30 15:17:30 compute-0 systemd[1]: Unit zabbix-agent.service entered failed state.
Oct 30 15:17:30 compute-0 systemd[1]: zabbix-agent.service failed.

 

zabbix server started sucessfully. Port number in the server machine is not used by any other services. 

----------------------------------------------------------------------------------------------------------------

 Comments   
Comment by Vladislavs Sokurenko [ 2018 Oct 30 ]

Could you please be so kind and show output of 

ps -ax | grep zabbix

and

netstat -lpn | grep 10050
Comment by janakiram pulavarthi [ 2018 Oct 31 ]

Now I am facing different issue after changing agent config file 

below are the required snippets 

 

[root@compute-0 ~]# vi /var/log/zabbix/zabbix_agentd.log
[root@compute-0 ~]# ps -ax | grep zabbix
150071 pts/0 S+ 0:00 vi /etc/zabbix/zabbix_agentd.conf
153333 ? S 0:00 /usr/sbin/zabbix_agentd -c /etc/zabbix/zabbix_agentd.conf
153334 ? S 0:00 /usr/sbin/zabbix_agentd: collector [idle 1 sec]
153335 ? S 0:00 /usr/sbin/zabbix_agentd: listener #1 [waiting for connection]
153336 ? S 0:00 /usr/sbin/zabbix_agentd: listener #2 [waiting for connection]
153337 ? S 0:00 /usr/sbin/zabbix_agentd: listener #3 [waiting for connection]
153338 ? S 0:00 /usr/sbin/zabbix_agentd: active checks #1 [idle 1 sec]
153869 pts/1 S+ 0:00 grep --color=auto zabbix

 

[root@compute-0 ~]# netstat -lpn | grep 10050
tcp 0 0 192.168.0.161:10050 0.0.0.0:* LISTEN 153333/zabbix_agent
[root@compute-0 ~]#

Agent log file

153338:20181031:073514.189 agent #5 started active checks #1
153338:20181031:073517.191 active check configuration update from [192.168.220.100:10051] started to fail (cannot connect to [[192.168.220.100]:10051]: [4] Interrupted system call)

Server log file 

5669:20181031:024034.183 server #40 started preprocessing worker #2
5652:20181031:024034.296 Zabbix agent item "system.users.num" on host "Zabbix server" failed: first network error, wait for 15 seconds
5658:20181031:024134.302 temporarily disabling Zabbix agent checks on host "Zabbix server": host unavailable
5658:20181031:024934.440 enabling Zabbix agent checks on host "Zabbix server": host became available
5632:20181031:031033.489 executing housekeeper
5632:20181031:031033.564 housekeeper [deleted 0 hist/trends, 6081 items/triggers, 4 events, 0 problems, 0 sessions, 0 alarms, 0 audit items in 0.073714 sec, idle for 1 hour(s)]
5652:20181031:031252.380 Zabbix agent item "agent.ping" on host "pod2-node10" failed: first network error, wait for 15 seconds
5658:20181031:031352.681 temporarily disabling Zabbix agent checks on host "pod2-node10": host unavailable

 

server output 

 

[root@director zabbix]# netstat -lpn | grep 10050
tcp 0 0 127.0.0.1:10050 0.0.0.0:* LISTEN 6891/zabbix_agentd
[root@director zabbix]# netstat -lpn | grep 10051
tcp 0 0 0.0.0.0:10051 0.0.0.0:* LISTEN 5625/zabbix_server
tcp6 0 0 :::10051 :::* LISTEN 5625/zabbix_server
[root@director zabbix]#

 

[root@director zabbix]# ps -eaf | grep zabbix
zabbix 5625 1 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server -c /etc/zabbix/zabbix_server.conf
zabbix 5627 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: configuration syncer [synced configuration in 0.021127 sec, idle 60 sec]
zabbix 5628 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: ipmi manager #1 [scheduled 0, polled 0 values, idle 5.005153 sec during 5.005166 sec]
zabbix 5629 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: alerter #1 started
zabbix 5630 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: alerter #2 started
zabbix 5631 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: alerter #3 started
zabbix 5632 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: housekeeper [deleted 0 hist/trends, 6081 items/triggers, 4 events, 0 sessions, 0 alarms, 0 audit items in 0.073714 sec, idle for 1 hour(s)]
zabbix 5633 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: timer #1 [updated 0 hosts, suppressed 0 events in 0.000431 sec, idle 59 sec]
zabbix 5634 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: http poller #1 [got 0 values in 0.000862 sec, idle 5 sec]
zabbix 5635 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: http poller #2 [got 0 values in 0.000753 sec, idle 5 sec]
zabbix 5636 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: discoverer #1 [processed 0 rules in 0.000730 sec, idle 60 sec]
zabbix 5637 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: history syncer #1 [processed 0 values, 0 triggers in 0.000013 sec, idle 1 sec]
zabbix 5638 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: history syncer #2 [processed 0 values, 0 triggers in 0.000013 sec, idle 1 sec]
zabbix 5639 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: history syncer #3 [processed 1 values, 1 triggers in 0.001813 sec, idle 1 sec]
zabbix 5640 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: history syncer #4 [processed 0 values, 1 triggers in 0.000425 sec, idle 1 sec]
zabbix 5641 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: escalator #1 [processed 0 escalations in 0.000336 sec, idle 3 sec]
zabbix 5642 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: ipmi poller #1 [polled 0 values, idle 3600.999258 sec during 3600.999305 sec]
zabbix 5643 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: ipmi poller #2 [polled 0 values, idle 3600.998046 sec during 3600.998097 sec]
zabbix 5644 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: ipmi poller #3 [polled 0 values, idle 3600.997131 sec during 3600.997179 sec]
zabbix 5645 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: ipmi poller #4 [polled 0 values, idle 3600.996472 sec during 3600.996528 sec]
zabbix 5646 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: ipmi poller #5 [polled 0 values, idle 3600.996022 sec during 3600.996068 sec]
zabbix 5647 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: snmp trapper [processed data in 0.000016 sec, idle 1 sec]
zabbix 5648 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: proxy poller #1 [exchanged data with 0 proxies in 0.000005 sec, idle 5 sec]
zabbix 5649 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: self-monitoring [processed data in 0.000013 sec, idle 1 sec]
zabbix 5651 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: task manager [processed 0 task(s) in 0.000161 sec, idle 5 sec]
zabbix 5652 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: poller #1 [got 1 values in 0.000123 sec, idle 2 sec]
zabbix 5654 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: poller #2 [got 0 values in 0.000007 sec, idle 2 sec]
zabbix 5655 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: poller #3 [got 0 values in 0.000006 sec, idle 2 sec]
zabbix 5656 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: poller #4 [got 0 values in 0.000005 sec, idle 2 sec]
zabbix 5657 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: poller #5 [got 0 values in 0.000007 sec, idle 2 sec]
zabbix 5658 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: unreachable poller #1 [got 1 values in 0.003398 sec, idle 4 sec]
zabbix 5660 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: trapper #1 [processed data in 0.000392 sec, waiting for connection]
zabbix 5661 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: trapper #2 [processed data in 0.000291 sec, waiting for connection]
zabbix 5662 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: trapper #3 [processed data in 0.000307 sec, waiting for connection]
zabbix 5663 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: trapper #4 [processed data in 0.000270 sec, waiting for connection]
zabbix 5664 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: trapper #5 [processed data in 0.000270 sec, waiting for connection]
zabbix 5665 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: icmp pinger #1 [got 0 values in 0.000008 sec, idle 5 sec]
zabbix 5666 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: alert manager #1 [sent 0, failed 0 alerts, idle 5.005715 sec during 5.005727 sec]
zabbix 5667 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: preprocessing manager #1 [queued 0, processed 10 values, idle 5.001570 sec during 5.001915 sec]
zabbix 5668 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: preprocessing worker #1 started
zabbix 5669 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: preprocessing worker #2 started
zabbix 5671 5625 0 02:40 ? 00:00:00 /usr/sbin/zabbix_server: preprocessing worker #3 started
zabbix 6891 1 0 02:50 ? 00:00:00 /usr/sbin/zabbix_agentd -c /etc/zabbix/zabbix_agentd.conf
zabbix 6892 6891 0 02:50 ? 00:00:00 /usr/sbin/zabbix_agentd: collector [idle 1 sec]
zabbix 6893 6891 0 02:50 ? 00:00:01 /usr/sbin/zabbix_agentd: listener #1 [waiting for connection]
zabbix 6894 6891 0 02:50 ? 00:00:01 /usr/sbin/zabbix_agentd: listener #2 [waiting for connection]
zabbix 6895 6891 0 02:50 ? 00:00:01 /usr/sbin/zabbix_agentd: listener #3 [waiting for connection]
root 15483 28992 0 03:45 pts/2 00:00:00 grep --color=auto zabbix
[root@director zabbix]#

 

 

Comment by Vladislavs Sokurenko [ 2018 Oct 31 ]

Thank you for your report, I am closing this issue as a duplicate of ZBX-15027, please do pkill zabbix_agentd -9 for now and let me know if it helps.

Comment by Vladislavs Sokurenko [ 2018 Oct 31 ]

This could also mean that zabbix agent was not stopped before starting new one, is that the case ?

Comment by janakiram pulavarthi [ 2018 Oct 31 ]

No, zabbix agent was started properly without having any issues. 

above command does't help in this case.

Comment by Vladislavs Sokurenko [ 2018 Oct 31 ]

I am sorry, issue description mentions that zabbix agent cannot be started, is this issue gone now ?

Comment by janakiram pulavarthi [ 2018 Oct 31 ]

yes agent is started now,  but facing different issue while connecting with zabbix server 

snippet below : 

158478:20181031:083844.277 agent #4 started listener #3
158479:20181031:083847.276 active check configuration update from [192.168.220.100:10051] started to fail (cannot connect to [[192.168.220.100]:10051]: [4] Interrupted system call)

 

Comment by Vladislavs Sokurenko [ 2018 Oct 31 ]

How was original issue resolved ?

Comment by janakiram pulavarthi [ 2018 Oct 31 ]

By changing Server, Active server, Listen IP address in agent config file.

Comment by janakiram pulavarthi [ 2018 Nov 07 ]

Please update.

Comment by Vladislavs Sokurenko [ 2018 Nov 07 ]

I am sorry but there is no indication of a bug.

Comment by janakiram pulavarthi [ 2018 Nov 07 ]

Is there any changes i need to perform in zabbix config file or in linux machine to catch the data from agent to server 

Below is the issue I'm facing now

Comment by richlv [ 2018 Nov 07 ]

This issue tracker is for bugreports & feature requests; for support options, you might want to check https://zabbix.org/wiki/Getting_help .

Generated at Fri Apr 26 03:26:39 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.