[ZBX-12867] Too slow dashboard when big events table Created: 2017 Oct 12  Updated: 2024 Apr 10  Resolved: 2021 Mar 22

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Frontend (F)
Affects Version/s: 3.4.1, 3.4.2
Fix Version/s: 4.0.30rc1, 5.0.10rc1, 5.2.6rc1, 5.4.0beta2, 5.4 (plan)

Type: Problem report Priority: Trivial
Reporter: Denis Ryazanov Assignee: Andrejs Griščenko
Resolution: Fixed Votes: 13
Labels: performance
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

CentOS 7.3.1611, percona mysql 5.6.37, php 5.6.31


Attachments: PNG File Selection_434.png     PNG File image-2021-03-19-22-20-46-371.png     PNG File loader.png     PNG File tags-after-fix.png     PNG File tags-before-fix.png    
Issue Links:
Duplicate
duplicates ZBX-15176 Zabbix Dashboard load very slow on 4... Closed
is duplicated by ZBX-15079 Patch to boost up wiget problems Closed
is duplicated by ZBX-13789 Widget Problems : very bad request ex... Closed
is duplicated by ZBX-13417 Slow Query for Dashbord Problems Closed
Sub-task
Team: Team D
Sprint: Sprint 44, Sprint 45, Sprint 46, Nov 2018, Sprint 47, Dec 2018, Sprint 48, Jan 2019, Sprint 56 (Sep 2019), Sprint 55 (Aug 2019), Sprint 49 (Feb 2019), Sprint 50 (Mar 2019), Sprint 51 (Apr 2019), Sprint 52 (May 2019), Sprint 53 (Jun 2019), Sprint 54 (Jul 2019), Sprint 57 (Oct 2019), Sprint 58 (Nov 2019), Sprint 59 (Dec 2019), Sprint 60 (Jan 2020), Sprint 61 (Feb 2020), Sprint 62 (Mar 2020), Sprint 63 (Apr 2020), Sprint 64 (May 2020), Sprint 65 (Jun 2020), Sprint 66 (Jul 2020), Sprint 67 (Aug 2020), Sprint 68 (Sep 2020), Sprint 69 (Oct 2020), Sprint 70 (Nov 2020), Sprint 71 (Dec 2020), Sprint 72 (Jan 2021), Sprint 73 (Feb 2021), Sprint 74 (Mar 2021)
Story Points: 3

 Description   

Dashboard too slow after upgrade from 3.2.4 to 3.4.1. Mysql process utilized all CPUs.

In mysql slow log:

SELECT DISTINCT e.eventid,e.clock,e.ns,e.objectid,er1.r_eventid FROM events e LEFT JOIN event_recovery er1 ON er1.eventid=e.eventid WHERE e.source='0' AND e.object='0'
AND e.objectid='31590' AND e.eventid<='12599826' AND e.value='1' ORDER BY e.eventid DESC LIMIT 20;
# User@Host: zabbix[zabbix] @ localhost []  Id:   489
# Schema: zabbix2  Last_errno: 0  Killed: 0
# Query_time: 193.328440  Lock_time: 0.000280  Rows_sent: 20  Rows_examined: 7980737  Rows_affected: 0
# Bytes_sent: 1274

Execution plan show full table scan for getting only 20 rows from big events table.

Currently we are fix this behaviour by adding lower limit for eventid in next files.
include/classes/api/services/CEvent.php

diff --git a/CEvent.php b/CEvent.php
index b1c2053..d4ec83a 100644
--- a/CEvent.php
+++ b/CEvent.php
@@ -339,6 +339,8 @@ class CEvent extends CApiService {

                // eventid_till
                if ($options['eventid_till'] !== null) {
+                       $eventid_before =(int) $options['eventid_till'] - 10000;
+                       $sqlParts['where'][] = 'e.eventid>='.zbx_dbstr($eventid_before);
                        $sqlParts['where'][] = 'e.eventid<='.zbx_dbstr($options['eventid_till']);
                }

include/classes/api/services/CProblem.php

diff --git a/CProblem.php b/CProblem.php
index 81bcd8a..6c2cdb8 100644
--- a/CProblem.php
+++ b/CProblem.php
@@ -314,6 +314,8 @@ class CProblem extends CApiService {

                // eventid_till
                if ($options['eventid_till'] !== null) {
+                       $eventid_before =(int) $options['eventid_till'] - 10000;
+                       $sqlParts['where'][] = 'p.eventid>='.zbx_dbstr($eventid_before);
                        $sqlParts['where'][] = 'p.eventid<='.zbx_dbstr($options['eventid_till']);
                }


 Comments   
Comment by Stefan Priebe [ 2018 Oct 03 ]

As there's currently no fix for big event tables (i've 3,7 million rows) - is there any way to houskeep events table and delete the data in it? I haven't found a setting.

Comment by Stefan Priebe [ 2018 Oct 09 ]

I merged this branch (ZBX-12867) to my zabbix 3.4 but it does not solve ultra slow screens with host group issues widget. The problematic SQL is:

EXPLAIN select eventid FROM events WHERE objectid='482607':
 * 
 ** 
 *** row 1 ***
 table: events
 type: index
 possible_keys: NULL
 key: events_1
 key_len: 20
 ref: NULL
 rows: 1803874
 Extra: Using where; Using index

and this one:

EXPLAIN select distinct t.triggerid,t.description,t.expression,t.error,t.priority,t.type,t.value,t.state,t.lastchange,t.status,t.recovery_mode,t.recovery_expression,t.correlation_mode,t.correlation_tag from hosts h,items i,functions f,triggers t where h.hostid=i.hostid and i.itemid=f.itemid and f.triggerid=t.triggerid and h.status in (0,1) and t.flags<>2:

*** row 1 ***
 table: t
 type: ALL
 possible_keys: PRIMARY
 key: NULL
 key_len: NULL
 ref: NULL
 rows: 153219
 Extra: Using where; Using temporary
*** row 2 ***
 table: f
 type: ref
 possible_keys: functions_1,functions_2
 key: functions_1
 key_len: 8
 ref: zabbix.t.triggerid
 rows: 3
 Extra: Distinct
*** row 3 ***
 table: i
 type: eq_ref
 possible_keys: PRIMARY,items_1
 key: PRIMARY
 key_len: 8
 ref: zabbix.f.itemid
 rows: 1
 Extra: Distinct
*** row 4 ***
 table: h
 type: eq_ref
 possible_keys: PRIMARY,hosts_2
 key: PRIMARY
 key_len: 8
 ref: zabbix.i.hostid
 rows: 1
 Extra: Using where; Distinct
Comment by Miks Kronkalns [ 2018 Oct 10 ]

Hi spriebe and thank you for your feedback. Just want to ask you - is the problem in dashboard now solved and only screen remains slow?

Comment by Stefan Priebe [ 2018 Oct 10 ]

I'm really sorry i only use screens no dashboards.

Comment by Alexander Vladishev [ 2018 Oct 29 ]

A patch is provided in ZBX-15079.

zalex_ua Just a note that the patch just creates indexes in DB schema for new database schema.
So, the main idea is to add indexes, which should be done in upgrade patches too.

Comment by Stefan Priebe [ 2019 Jan 31 ]

yes but i manually created the indexes for the tables but it did NOT help for screens.

Comment by Andrejs Griščenko [ 2021 Mar 04 ]

Resolved in development branches:

Comment by Alexander Vladishev [ 2021 Mar 20 ]

Available in:

Generated at Wed Jun 17 03:34:09 EEST 2026 using Jira 10.3.18#10030018-sha1:5642e4ad348b6c2a83ebdba689d04763a2393cab.