[ZBX-22868] zabbix-agent2 upgrade breaks postgresql plugin monitoring Created: 2023 May 30 Updated: 2024 Apr 10 Resolved: 2023 Jun 20 |
|
Status: | Closed |
Project: | ZABBIX BUGS AND ISSUES |
Component/s: | Agent2 plugin (G) |
Affects Version/s: | 6.0.18rc1 |
Fix Version/s: | 5.0.36rc1, 6.0.19rc1, 6.4.4rc1, 7.0.0alpha2, 7.0 (plan) |
Type: | Problem report | Priority: | Blocker |
Reporter: | Sandro Bordacchini | Assignee: | Eriks Sneiders |
Resolution: | Fixed | Votes: | 30 |
Labels: | agent, macros | ||
Remaining Estimate: | Not Specified | ||
Time Spent: | Not Specified | ||
Original Estimate: | Not Specified |
Attachments: |
![]() |
||||||||||||||||||||||||||||||||
Issue Links: |
|
||||||||||||||||||||||||||||||||
Team: | |||||||||||||||||||||||||||||||||
Sprint: | Sprint 101 (Jun 2023) | ||||||||||||||||||||||||||||||||
Story Points: | 1 |
Description |
Starting scenario:
Steps to reproduce:
Result: It seems monitoring user is not taken from macros anymore... it fails back to default Console output: Setting up zabbix-agent2-plugin-postgresql (6.0.18-1+debian11) ... Installing new version of config file /etc/zabbix/zabbix_agent2.d/plugins.d/postgresql.conf ... Setting up zabbix-agent2-plugin-mongodb (6.0.18-1+debian11) ... Installing new version of config file /etc/zabbix/zabbix_agent2.d/plugins.d/mongodb.conf ... Setting up zabbix-agent2 (1:6.0.18-1+debian11) ... Installing new version of config file /etc/zabbix/zabbix_agent2.d/plugins.d/ceph.conf ... Installing new version of config file /etc/zabbix/zabbix_agent2.d/plugins.d/memcached.conf ... Installing new version of config file /etc/zabbix/zabbix_agent2.d/plugins.d/mysql.conf ... Installing new version of config file /etc/zabbix/zabbix_agent2.d/plugins.d/oracle.conf ... Installing new version of config file /etc/zabbix/zabbix_agent2.d/plugins.d/redis.conf ...
Postgresql log
2023-05-30 10:20:33.722 UTC [17672] zabbix@user=zbx_monitor FATAL: password authentication failed for user "zabbix" 2023-05-30 10:20:33.722 UTC [17672] zabbix@user=zbx_monitor DETAIL: Role "zabbix" does not exist. Connection matched pg_hba.conf line 92: "host all all 127.0.0.1/32 md5" [... keeps repeating ...] zabbix-agent2 log 2023/05/30 12:20:31.382440 Zabbix Agent 2 stopped. (6.0.17) 2023/05/30 12:20:31.473446 Starting Zabbix Agent 2 (6.0.18) 2023/05/30 12:20:31.474151 OpenSSL library (OpenSSL 1.1.1n 15 Mar 2022) initialized 2023/05/30 12:20:31.474233 using configuration file: /etc/zabbix/zabbix_agent2.custom.conf 2023/05/30 12:20:31.474349 using plugin 'Agent' (built-in) providing following interfaces: exporter [...] 2023/05/30 12:20:31.476317 using plugin 'Modbus' (built-in) providing following interfaces: exporter, configurator 2023/05/30 12:20:31.476338 using plugin 'MongoDB' (/usr/sbin/zabbix-agent2-plugin/zabbix-agent2-plugin-mongodb) providing following interfaces: exporter, runner, configurator 2023/05/30 12:20:31.476352 using plugin 'Mysql' (built-in) providing following interfaces: exporter, runner, configurator 2023/05/30 12:20:31.476363 using plugin 'NetIf' (built-in) providing following interfaces: exporter 2023/05/30 12:20:31.476372 using plugin 'Oracle' (built-in) providing following interfaces: exporter, runner, configurator 2023/05/30 12:20:31.476389 using plugin 'PostgreSQL' (/usr/sbin/zabbix-agent2-plugin/zabbix-agent2-plugin-postgresql) providing following interfaces: exporter, runner, configurator [...] 2023/05/30 12:20:31.476601 using plugin 'ZabbixStats' (built-in) providing following interfaces: exporter, configurator 2023/05/30 12:20:31.476608 lowering the plugin ZabbixSync capacity to 1 as the configured capacity 100 exceeds limits 2023/05/30 12:20:31.476615 using plugin 'ZabbixSync' (built-in) providing following interfaces: exporter 2023/05/30 12:20:31.478990 Plugin communication protocol version is 6.0.13 2023/05/30 12:20:32.020985 Zabbix Agent2 hostname: [node004.xxxxxxxxx.net] 2023/05/30 12:20:33.722582 [PostgreSQL] Connection failed: failed to connect to `host=localhost user=zabbix database=user=zbx_monitor`: server error (FATAL: password authentication failed for user "zabbix" (SQLSTATE 28P01)). [... last line keeps repeating ...]
Expected: |
Comments |
Comment by Andrew Martin [ 2023 May 30 ] |
I see this on 5.0.35-1 on Ubuntu too |
Comment by Alessandro Duvia [ 2023 May 30 ] |
Same problem here. Our logs outputs are identical to yours. It seems like the PG.USER macro is in the database field.... Zabbix-agent2 1:6.0.18-1+debian10 amd64 |
Comment by Sandro Bordacchini [ 2023 May 30 ] |
Ok, I tried to fix the authentication part (dropping the role zbx_monitor and creating a zabbix one) but, as it could be expected (reading accurately the logs above) now the plugin query a evidently non-existent db "user=zabbix". Please see the attached postgres and zabbix-agent2 (level 3) debug log Maybe the query's parameters are not passed correctly? PS: <zabbix_server_ip_address>, <pgsql_password> replace the actual values for privacyZBX-22868.log |
Comment by Sandro Bordacchini [ 2023 May 30 ] |
I think that the only relevant work done between 6.0.17 and 6.0.18 is https://support.zabbix.com/browse/ZBXNEXT-8330 But I don't know how to relate that to this issue... |
Comment by Lasse Osterild [ 2023 May 31 ] |
Same issue on 6.4.3 - server and agent was installed with 6.4.2 where it worked fine, then after todays upgrade to 6.4.3 it breaks. Seemingly something to do with Connection failed: failed to connect to `host=localhost user=zabbix database=user=zbx_monitor |
Comment by Robbert Vercammen [ 2023 Jun 01 ] |
Full zabbix-agent2 log line: 2023/06/01 12:00:53.816001 [PostgreSQL] Connection failed: failed to connect to `host=localhost user=zabbix database=user=nagios`: failed SASL auth (FATAL: password authentication failed for user "zabbix" (SQLSTATE 28P01)). We use a nagios user (as set in the template macro) to connect to the PostgreSQL database. |
Comment by Jonas Nilsson [ 2023 Jun 01 ] |
Updating the postgres plugin but holding the zabbix agent on 6.4.2 produces the same results. |
Comment by Tomasz Berner [ 2023 Jun 02 ] |
Temporary solution is to modify plugin config: Plugins.PostgreSQL.Default.Uri=tcp://localhost:5432 and set empty macros: {$PG.URI}
|
Comment by Bruno Hostalery [ 2023 Jun 03 ] |
Server on debian 11, zabbix server 6.4.3 Client on debian 11,ztabbix-agent2 and zabbix-agent2-plugin-postgresql. Works with macro in server due to need of monitoring multiple instance on same machine. Previous release: 6.4.2-1+debian11 Update to: 6.4.3-1+debian11 Break the monitoring with log:
Looks like plugin find no user and use user=zabbix, and use for db user=USERNAME Back to version 6.4.2-1+debian11. Everything works fine again. |
Comment by Evgeny Yurchenko [ 2023 Jun 04 ] |
Experiencing the same. Looking at PostgreSQL logs: 2023-06-03 22:48:44.669 UTC [52466] zabbix@user=zbx_monitor FATAL: password authentication failed for user "zabbix" 2023-06-03 22:48:44.669 UTC [52466] zabbix@user=zbx_monitor DETAIL: Connection matched pg_hba.conf line 101: "host all all 127.0.0.1/32 scram-sha-256" we can say that plugin suddenly uses user "zabbix" to connect to "user=zbx_monitor" database. |
Comment by John Wayne [ 2023 Jun 06 ] |
I really wonder how this one made it to the final 6.0.18 release as the problem was already reported with 6.0.18rc1? At least the testing workflow may be improved for future releases? Correct me if I'm wrong but the only "real" solution for now is to downgrade to 6.0.17 and wait for a fixed (and hopefully better tested) 6.0.19. |
Comment by Sandro Bordacchini [ 2023 Jun 06 ] |
Hello @JohnWayne , please note that i reported the bug on 6.0.18rc1 but i have discovered it when upgrading to 6.0.18 via deb packaging. So the bug report has been made after the release of 6.0.18. I cannot recall now why I wrote 6.0.18rc1, probably that was the closest option available in the combo box
@Tomasz Berner proposed a temporary workaround that i did not try yet, though. |
Comment by Andris Zeila [ 2023 Jun 09 ] |
As a temporary workaround adding Plugins.PostgreSQL.Default.Database=postgres to agent configuration file should be enough. |
Comment by Andres Tarallo [ 2023 Jun 09 ] |
We were working fine on 6.0.17, the upgrade broke posgres monitoring. We just installed the latest patch and still get errors.
failed: failed to connect to host=localhost user=zabbix database=user=zbx_monitor: failed SASL auth (FATAL: password authentication failed for user "zabbix" (SQLSTATE 28P01)).' to 'X.X.X.X' |
Comment by Maksim Astapenko [ 2023 Jun 11 ] |
On RHEL latest work zabbix-agent2 and PostgreSQL plugin 6.4.0 version. |
Comment by Eriks Sneiders [ 2023 Jun 15 ] |
Fixed in:
|