-
Incident report
-
Resolution: Duplicate
-
Trivial
-
None
-
6.4.3
-
DB: PostgreSQL 13.11 timescaledb 2.7.2 16CPUs 64GB RAM
Ubuntu 22.04.2 LTS (GNU/Linux 5.15.0-73-generic x86_64)
Server: Zabbix - 6.4.3 6 CPU, 24 GB RAM
Ubuntu 22.04.2 LTS (GNU/Linux 5.15.0-73-generic x86_64)
Number of hosts 4882
Number of items 594372
Number of triggers 323943
Required server performance, new values per second 3562.33
Everything is monitored by 9 active zabbix proxies and 1 Passive.
DB: PostgreSQL 13.11 timescaledb 2.7.2 16CPUs 64GB RAM Ubuntu 22.04.2 LTS (GNU/Linux 5.15.0-73-generic x86_64) Server: Zabbix - 6.4.3 6 CPU, 24 GB RAM Ubuntu 22.04.2 LTS (GNU/Linux 5.15.0-73-generic x86_64) Number of hosts 4882 Number of items 594372 Number of triggers 323943 Required server performance, new values per second 3562.33 Everything is monitored by 9 active zabbix proxies and 1 Passive.
Steps to reproduce:
- Changes in configuration...
Zabbix Server Conf : HousekeepingFrequency=1
MaxHousekeeperDelete=100000
zabbixv5=# SELECT count FROM problem ;
count
---------
4119150
(1 row)
zabbixv5=# SELECT count FROM events ;
count
---------
8293480
(1 row)
1988:20230605:183149.912 query [txnlev:0] [delete from problem where (eventid in (1316,1534,1535,1537,1526,1559,1561,1581,1580,1573,1569,1583,1540,1566,....
1988:20230605:183152.973 slow query: 3.063448 sec, "delete from problem where (eventid in (1316,1534,1535,1537,1526,1559,1561,1581,1580,1573,1569,1583,....
1988:20230605:183152.973 query [txnlev:0] [select p1.eventid from problem p1 where p1.r_clock<>0 and p1.r_clock<1685896309 and not exists (select NULL from problem p2 where p1.eventid=p2.cause_eventid) limit 100000]
1988:20230605:183153.384 End of housekeeping_problems():76325
1988:20230605:183153.384 zbx_setproctitle() title:'housekeeper [removing old events]'
1988:20230605:183153.384 In housekeeping_process_rule() table:'events' field_name:'eventid' filter:'events.source=0 and events.object=0 and not exists(select null from problem where events.eventid=problem.eventid) and not exists(select null from problem where events.eventid=problem.r_eventid) and not exists(select null from event_symptom where events.eventid=event_symptom.cause_eventid)' min_clock:1685894247 now:1685982709
1988:20230605:183153.384 End of housekeeping_process_rule():0
1988:20230605:183153.384 In housekeeping_process_rule() table:'events' field_name:'eventid' filter:'events.source=3 and events.object=0 and not exists(select null from problem where events.eventid=problem.eventid) and not exists(select null from problem where events.eventid=problem.r_eventid)' min_clock:1685894270 now:1685982709
1988:20230605:183153.384 query [txnlev:0] [select eventid from events where clock<1685896309 and events.source=3 and events.object=0 and not exists(select null from problem where events.eventid=problem.eventid) and not exists(select null from problem where events.eventid=problem.r_eventid) order by eventid limit 100000]
1988:20230605:183153.755 query without transaction detected
1988:20230605:183153.755 query [txnlev:0] [delete from events where (eventid in (2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,....
1988:20230606:085603.910 query [txnlev:0] [select eventid from events where clock<1685896309 and events.source=3 and events.object=0 and not exists(select null from problem where events.eventid=problem.eventid) and not exists(select null from problem where events.eventid=problem.r_eventid) order by eventid limit 100000]
1988:20230606:085604.470 query without transaction detected
1988:20230606:085604.470 query [txnlev:0] [delete from events where (eventid in (103090,103091,103092,103093
deleting events takes a long time and then 15 hours later starts again with 100000 IDs.
Housekeeper runs since 5.6.23 18:30 on 100% through
DB query :
661726 | -08:10:36.971176 | zabbix | delete from events where (eventid in (103090,103091,103092,103093,.....
- duplicates
-
ZBX-22889 Performance problems due to missing problem.cause_eventid index on PostgreSQL
- Closed