[ZBX-18576] Can't start zabbix-agent for AIX Created: 2020 Oct 28  Updated: 2024 Apr 10

Status: Need info
Project: ZABBIX BUGS AND ISSUES
Component/s: Agent (G)
Affects Version/s: 5.0.4, 5.0.16
Fix Version/s: None

Type: Incident report Priority: Critical
Reporter: Mikhail Grigorev Assignee: Andrei Gushchin (Inactive)
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

AIX 7.1 - Technology Level 2 - Service Pack 2

Attachments: Text File lslpp.txt     Text File lspp.txt    
Issue Links:
Causes
Duplicate
is duplicated by ZBX-20751 AIX7.1 zabbix_agentd [7798912]: the f... Closed
Team: Team C

 Description   

Steps to reproduce:

  1. Download official agent https://www.zabbix.com/ru/download_agents?version=5.0+LTS&release=5.0.3&os=AIX&os_version=7.1&hardware=powerpc&encryption=OpenSSL&packaging=Archive
  2. Install and run agent
  3. But agent not started, see log...

Result:
See log file

5952888:20201028:150113.838 Starting Zabbix Agent [AIX01]. Zabbix 5.0.3 (revision 146855bff3).
25952888:20201028:150113.838 **** Enabled features ****
25952888:20201028:150113.838 IPv6 support:          YES
25952888:20201028:150113.838 TLS support:           YES
25952888:20201028:150113.838 **************************
25952888:20201028:150113.838 using configuration file: /etc/zabbix/zabbix_agentd.conf
25952888:20201028:150113.838 In zbx_load_modules()
25952888:20201028:150113.838 End of zbx_load_modules():SUCCEED
25952888:20201028:150113.839 In init_collector_data()
25952888:20201028:150113.839 End of init_collector_data()
25952888:20201028:150113.839 agent #0 started [main process]
11403470:20201028:150113.842 agent #1 started [collector]
11403470:20201028:150113.842 perfstat_partition_total: [22] Invalid argument
11403470:20201028:150113.842 In init_cpu_collector()
24772782:20201028:150113.843 agent #2 started [listener #1]
24772782:20201028:150113.843 In zbx_tls_init_child()
8389066:20201028:150113.844 agent #3 started [listener #2]
11403470:20201028:150113.844 End of init_cpu_collector():SUCCEED
8389066:20201028:150113.844 In zbx_tls_init_child()
11403470:20201028:150113.845 zbx_setproctitle() title:'collector [processing data]'
11403470:20201028:150113.845 In update_cpustats()
20447962:20201028:150113.845 agent #4 started [listener #3]
20447962:20201028:150113.846 In zbx_tls_init_child()
11403470:20201028:150113.847 End of update_cpustats()
11403470:20201028:150113.847 In update_cpustats_physical()
zabbix_agentd [11403470]: the first call of perfstat_cpu() failed: [22] Invalid argument
20841332:20201028:150113.847 agent #5 started [active checks #1]
20841332:20201028:150113.847 In zbx_tls_init_child()
25952888:20201028:150113.850 One child process died (PID:11403470,exitcode/signal:1). Exiting ...
25952888:20201028:150113.851 zbx_on_exit() called
24772782:20201028:150113.852 OpenSSL library (version OpenSSL 1.1.1g  21 Apr 2020) initialized
8389066:20201028:150113.855 OpenSSL library (version OpenSSL 1.1.1g  21 Apr 2020) initialized
20447962:20201028:150113.855 OpenSSL library (version OpenSSL 1.1.1g  21 Apr 2020) initialized
20841332:20201028:150113.856 OpenSSL library (version OpenSSL 1.1.1g  21 Apr 2020) initialized
25952888:20201028:150113.858 In zbx_unload_modules()
25952888:20201028:150113.858 End of zbx_unload_modules()
25952888:20201028:150113.858 Zabbix Agent stopped. Zabbix 5.0.3 (revision 146855bff3).


 Comments   
Comment by Andrei Gushchin (Inactive) [ 2020 Oct 29 ]

Thank you for reporting this.
Can you please try to get the output of this command and attach it here:

ldd /usr/bin/zabbix_agentd
oslevel -s
lslpp -l
Comment by Mikhail Grigorev [ 2020 Oct 29 ] 
bash-3.2# ldd /sbin/zabbix_agentd
/sbin/zabbix_agentd needs:
         /usr/lib/libc.a(shr_64.o)
         /usr/lib/libpthreads.a(shr_xpg5_64.o)
         /usr/lib/libperfstat.a(shr_64.o)
         /usr/lib/libiconv.a(shr4_64.o)
         /unix
         /usr/lib/libcrypt.a(shr_64.o)
         /usr/lib/libcfg.a(shr_64.o)
         /usr/lib/libodm.a(shr_64.o)
         /usr/lib/liblvm.a(shr_64.o)
         /usr/lib/libcorcfg.a(shr_64.o)
         /usr/lib/libsrc.a(shr_64.o)


bash-3.2# oslevel -s
7100-02-02-1316

lslpp -l 
see attached file

lslpp.txt

Comment by Mikhail Grigorev [ 2020 Nov 05 ]

Do you need any more information from me?

I tried to start the old version of the agent (zabbix_agent-3.4.0-aix-7.1-powerpc.tar.gz) and it started, but messages about problems with collecting some CPU metrics began to appear in the log:

22675506:20201105:143326.109 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
22675506:20201105:143327.109 __zbx_zbx_setproctitle() title:'collector [processing data]'
22675506:20201105:143327.109 In update_cpustats()
22675506:20201105:143327.110 End of update_cpustats()
22675506:20201105:143327.110 perfstat_partition_total: [22] Invalid argument
22675506:20201105:143327.110 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
21627472:20201105:143327.250 active check "system.stat[memory,avm]" is not supported: No data available in collector.
21627472:20201105:143327.250 In process_value() key:'AIX01:system.stat[memory,avm]' value:'No data available in collector.'
21627472:20201105:143327.250 In send_buffer() host:'192.168.XX.XX' port:11311 entries:1/100
21627472:20201105:143327.250 send_buffer() now:1604576007 lastsent:1604576005 now-lastsent:2 BufferSend:5; will not send now
21627472:20201105:143327.250 End of send_buffer():SUCCEED
21627472:20201105:143327.251 buffer: new element 1
21627472:20201105:143327.251 End of process_value():SUCCEED
22675506:20201105:143328.110 __zbx_zbx_setproctitle() title:'collector [processing data]'
22675506:20201105:143328.110 In update_cpustats()
22675506:20201105:143328.111 End of update_cpustats()
22675506:20201105:143328.111 perfstat_partition_total: [22] Invalid argument
22675506:20201105:143328.111 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
22675506:20201105:143329.111 __zbx_zbx_setproctitle() title:'collector [processing data]'
22675506:20201105:143329.111 In update_cpustats()
22675506:20201105:143329.111 End of update_cpustats()
22675506:20201105:143329.111 perfstat_partition_total: [22] Invalid argument
22675506:20201105:143329.111 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
21627472:20201105:143329.251 active check "system.stat[memory,fre]" is not supported: No data available in collector.
21627472:20201105:143329.251 In process_value() key:'AIX01:system.stat[memory,fre]' value:'No data available in collector.'
21627472:20201105:143329.251 In send_buffer() host:'192.168.XX.XX' port:11311 entries:2/100
21627472:20201105:143329.251 send_buffer() now:1604576009 lastsent:1604576005 now-lastsent:4 BufferSend:5; will not send now
21627472:20201105:143329.251 End of send_buffer():SUCCEED
21627472:20201105:143329.251 buffer: new element 2
21627472:20201105:143329.251 End of process_value():SUCCEED
22675506:20201105:143330.112 __zbx_zbx_setproctitle() title:'collector [processing data]'
22675506:20201105:143330.112 In update_cpustats()
22675506:20201105:143330.112 End of update_cpustats()
22675506:20201105:143330.112 perfstat_partition_total: [22] Invalid argument
22675506:20201105:143330.112 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
22675506:20201105:143331.113 __zbx_zbx_setproctitle() title:'collector [processing data]'
22675506:20201105:143331.113 In update_cpustats()
22675506:20201105:143331.113 End of update_cpustats()
22675506:20201105:143331.113 perfstat_partition_total: [22] Invalid argument
22675506:20201105:143331.113 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
21627472:20201105:143331.251 active check "system.stat[page,fi]" is not supported: No data available in collector.
21627472:20201105:143331.251 In process_value() key:'AIX01:system.stat[page,fi]' value:'No data available in collector.'
21627472:20201105:143331.252 In send_buffer() host:'192.168.XX.XX' port:11311 entries:3/100
Comment by Mikhail Grigorev [ 2020 Nov 05 ]

I tested running zabbix_agent-4.0.7-aix-7.2-powerpc.tar.gz - it also starts, but there are errors when collecting some data on the CPU.

Comment by Mikhail Grigorev [ 2020 Nov 05 ]

I compiled from source 2 versions of zabbix-agent - 5.0.2 and 5.0.3

The problem is in version 5.0.3

Here is the log output:

5177428:20201105:162231.947 Starting Zabbix Agent [AIX01]. Zabbix 5.0.2 (revision 352ca05870).
5177428:20201105:162231.948 **** Enabled features ****
5177428:20201105:162231.948 IPv6 support:           NO
5177428:20201105:162231.948 TLS support:            NO
5177428:20201105:162231.948 **************************
5177428:20201105:162231.948 using configuration file: /etc/zabbix/zabbix_agentd.conf
5177428:20201105:162231.948 agent #0 started [main process]
18482022:20201105:162231.950 agent #1 started [collector]
6488688:20201105:162231.951 agent #2 started [listener #1]
9961692:20201105:162231.952 agent #3 started [listener #2]
13173578:20201105:162231.953 agent #4 started [listener #3]
20513626:20201105:162231.954 agent #5 started [active checks #1]
20513626:20201105:162234.069 active check "system.stat[cpu,ec]" is not supported: No data available in collector.
20513626:20201105:162236.069 active check "system.stat[cpu,id]" is not supported: No data available in collector.
20513626:20201105:162238.071 active check "system.stat[cpu,lbusy]" is not supported: No data available in collector.
20513626:20201105:162240.071 active check "system.stat[cpu,pc]" is not supported: No data available in collector.
20513626:20201105:162242.071 active check "system.stat[cpu,sy]" is not supported: No data available in collector.
20513626:20201105:162244.072 active check "system.stat[cpu,us]" is not supported: No data available in collector.
20513626:20201105:162246.072 active check "system.stat[cpu,wa]" is not supported: No data available in collector.
20513626:20201105:162248.072 active check "system.stat[disk,bps]" is not supported: No data available in collector.
20513626:20201105:162250.074 active check "system.stat[disk,tps]" is not supported: No data available in collector.
20513626:20201105:162252.074 active check "system.stat[ent]" is not supported: No data available in collector.
20513626:20201105:162254.074 active check "system.stat[faults,cs]" is not supported: No data available in collector.
20513626:20201105:162256.077 active check "system.stat[faults,in]" is not supported: No data available in collector.
20513626:20201105:162258.078 active check "system.stat[faults,sy]" is not supported: No data available in collector.
20513626:20201105:162300.078 active check "system.stat[kthr,b]" is not supported: No data available in collector.
20513626:20201105:162302.078 active check "system.stat[kthr,r]" is not supported: No data available in collector.
20513626:20201105:162304.078 active check "system.stat[memory,avm]" is not supported: No data available in collector.
20513626:20201105:162306.079 active check "system.stat[memory,fre]" is not supported: No data available in collector.
20513626:20201105:162308.081 active check "system.stat[page,fi]" is not supported: No data available in collector.
20513626:20201105:162310.081 active check "system.stat[page,fo]" is not supported: No data available in collector.
20513626:20201105:162312.081 active check "system.stat[page,fr]" is not supported: No data available in collector.
20513626:20201105:162314.083 active check "system.stat[page,pi]" is not supported: No data available in collector.
20513626:20201105:162316.083 active check "system.stat[page,po]" is not supported: No data available in collector.
20513626:20201105:162318.083 active check "system.stat[page,sr]" is not supported: No data available in collector.


5177428:20201105:162803.227 Zabbix Agent stopped. Zabbix 5.0.2 (revision 352ca05870).
14352736:20201105:162817.260 Starting Zabbix Agent [AIX01]. Zabbix 5.0.3 (revision 6e02cfb1cf).
14352736:20201105:162817.260 **** Enabled features ****
14352736:20201105:162817.260 IPv6 support:           NO
14352736:20201105:162817.260 TLS support:            NO
14352736:20201105:162817.260 **************************
14352736:20201105:162817.260 using configuration file: /etc/zabbix/zabbix_agentd.conf
14352736:20201105:162817.260 agent #0 started [main process]
20644192:20201105:162817.262 agent #1 started [collector]
zabbix_agentd_503 [20644192]: the first call of perfstat_cpu() failed: [22] Invalid argument
20906340:20201105:162817.266 agent #2 started [listener #1]
14352736:20201105:162817.266 One child process died (PID:20644192,exitcode/signal:1). Exiting ...
14352736:20201105:162817.267 Zabbix Agent stopped. Zabbix 5.0.3 (revision 6e02cfb1cf).

It seems to me that the regression happened at

ZBX-18214 added collecting of physical CPU utilization data on IBM AIX and modified system.cpu.util[] (Andris)

https://support.zabbix.com/browse/ZBX-18214

Comment by Andris Mednis [ 2021 Nov 26 ]

On https://www.ibm.com/docs/en/aix/7.1?topic=interfaces-perfstat-cpu-interface there is an example program of using perfstat_cpu() just as Zabbix agent does:

cputotal =  perfstat_cpu(NULL, NULL, sizeof(perfstat_cpu_t), 0);

Can you compile and try to run this example program on your AIX?

Comment by Mikhail Grigorev [ 2022 Feb 13 ]

Sorry, i changed jobs and am no longer working with AIX and can't test this code.

Comment by André Frederico Lucas da Silva [ 2023 Apr 27 ]

Hello all,
Just to inform you that on version 6.0.16 the issue still occurs.

 

aixview1:root# ldd /usr/sbin/zabbix_agentd

/usr/sbin/zabbix_agentd needs:
         /usr/lib/libc.a(shr_64.o)
         /usr/lib/libpthreads.a(shr_xpg5_64.o)
         /usr/lib/libperfstat.a(shr_64.o)
         /unix
         /usr/lib/libcrypt.a(shr_64.o)
         /usr/lib/libcfg.a(shr_64.o)
         /usr/lib/libodm.a(shr_64.o)
         /usr/lib/liblvm.a(shr_64.o)
         /usr/lib/libcorcfg.a(shr_64.o)
         /usr/lib/libsrc.a(shr_64.o) 
aixview1:root# oslevel -s 7100-02-03-1334

lspp -l
attached file

lspp.txt

Comment by Andris Mednis [ 2023 Apr 27 ]

Can you compile example https://www.ibm.com/docs/en/aix/7.1?topic=interfaces-perfstat-cpu-interface and run it?

Generated at Tue Apr 29 08:32:20 EEST 2025 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.