[ZBX-8277] Templates are not removed Created: 2014 May 29  Updated: 2024 Apr 10  Resolved: 2017 Sep 29

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: API (A)
Affects Version/s: 2.2.2
Fix Version/s: 3.0.12rc1, 3.2.9rc1, 3.4.3rc1, 4.0.0alpha1, 4.0 (plan)

Type: Problem report Priority: Blocker
Reporter: Sergey Assignee: Alexander Vladishev
Resolution: Fixed Votes: 0
Labels: templates
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

In used: zabbix server 2.2.2 - ESXI4, Freebsd 9.1; web interface 2.2.2 - ESXI4, Freebsd9.1; Dedicated DB Server - Ubuntu Linux 13.04, MariaDB-10.0.11, CPU 2x8 Xeon, 4 HDDx10k - RAID10, 64Gb RAM, included partitioning, disk for temporary tables is located in memory and has size 10Gb. Number of templates / hosts / elements - 570/4900/ 451331, the number of active users ~ 24.


Team: Team B
Sprint: Sprint 17
Story Points: 1

 Description   

When I'm tried to remove template through the web interface I see slow query in mysql process list like:

SELECT DISTINCT t.triggerid,t.description,t.flags,t.expression,h.name as host FROM  triggers t,hosts h WHERE  EXISTS (SELECT ff.triggerid FROM functions ff,items ii WHERE ff.triggerid=t.templateid AND ii.itemid=ff.itemid AND ii.hostid='13856') AND t.flags IN ('0','1')

The process is handled about 20 min and tmp tables are growing so fast up to 10 Gb.

root@db:/ramdisk# ll
-rw-rw---- 1 mysql mysql  9837510656 	??? 29 11:48 #sql_985_0.MAD
-rw-rw---- 1 mysql mysql   902660096 	??? 29 11:48 #sql_985_0.MAI

When I am killing this process I saw Error message like that in my dashboard.

mysqli_query(): MySQL server has gone away [templates.php:393 ? CAPIObject->delete() ? CAPIObject->__call() ? czbxrpc::call() ? czbxrpc::callAPI() ? call_user_func() ? CTemplate->delete() ? CHostGeneral->unlink() ? DBselect() ? mysqli_query() in /nmc/httpd/zabbix/include/db.inc.php:379]
mysqli_query(): Error reading result set's header [templates.php:393 ? CAPIObject->delete() ? CAPIObject->__call() ? czbxrpc::call() ? czbxrpc::callAPI() ? call_user_func() ? CTemplate->delete() ? CHostGeneral->unlink() ? DBselect() ? mysqli_query() in /nmc/httpd/zabbix/include/db.inc.php:379]
Error in query [SELECT DISTINCT t.triggerid,t.description,t.flags,t.expression,h.name as host FROM  triggers t,hosts h WHERE  EXISTS (SELECT ff.triggerid FROM functions ff,items ii WHERE ff.triggerid=t.templateid AND ii.itemid=ff.itemid AND ii.hostid='13856') AND t.flags IN ('0','1')] [MySQL server has gone away]
Error in query [SELECT DISTINCT i1.itemid,i1.flags,i1.name,i1.hostid,h.name as host FROM  items i1,items i2,hosts h WHERE  i2.itemid=i1.templateid AND i2.hostid='13856' AND i1.flags IN ('0','1') AND h.hostid=i1.hostid] [MySQL server has gone away]
Error in query [SELECT DISTINCT g.graphid,g.name,g.flags,h.name as host FROM  graphs g,hosts h WHERE  EXISTS (SELECT ggi.graphid FROM graphs_items ggi,items ii WHERE ggi.graphid=g.templateid AND ii.itemid=ggi.itemid AND ii.hostid='13856') AND g.flags IN ('0','1')] [MySQL server has gone away]
Error in query [SELECT DISTINCT ht1.httptestid,ht1.name,h.name as host FROM httptest ht1 INNER JOIN httptest ht2 ON ht2.httptestid=ht1.templateid INNER JOIN hosts h ON h.hostid=ht1.hostid WHERE ht2.hostid='13856'] [MySQL server has gone away]
Error in query [SELECT at.application_templateid,at.applicationid,h.name,h.host,h.hostid FROM applications a1,application_template at,applications a2,hosts h WHERE a1.applicationid=at.applicationid AND at.templateid=a2.applicationid AND a2.hostid='13856' AND a1.hostid=h.hostid] [MySQL server has gone away]
Error in query [DELETE FROM hosts_templates WHERE templateid='13856'] [MySQL server has gone away]
SQL statement execution has failed "DELETE FROM hosts_templates WHERE templateid='13856'"
Error in query [ROLLBACK] [MySQL server has gone away]

Finally I can't remove template with ID=13856, and most of all I can't remove any template from the list of templates because query execution time exceeds reasonable limits.

P.S.

MariaDB [zabbix]> select count(*) from triggers;
329966

MariaDB [zabbix]> select count(*) from hosts;
5506 


 Comments   
Comment by Sergey [ 2014 May 30 ]

Some additional Information. With the same hardware configuration but fewer hosts / items / triggers = 3742/196618/105753 templats are removed in 10-15 minutes. All other elements are created and removed without any problems.

Comment by Andrei Gushchin (Inactive) [ 2014 Aug 13 ]

It seems it is simmilar to ZBX-6118.

Comment by Alexander Vladishev [ 2017 Sep 22 ]

Fixed in dev branch svn://svn.zabbix.com/branches/dev/ZBX-8277

Performance improvements (number of hosts: ~100000; number of triggers/items/functions: ~2500000):

 Action                                | Before fix                                                | After fix                                                 |
---------------------------------------+-----------------------------------------------------------+-----------------------------------------------------------+
 Delete template with two hosts linked | ******************** Script profiler ******************** | ******************** Script profiler ******************** |
                                       | Total time: 7.752717                                      | Total time: 0.857457                                      |
                                       | Total SQL time: 7.658996                                  | Total SQL time: 0.712773                                  |
                                       | SQL count: 456 (selects: 144 | executes: 312)             | SQL count: 452 (selects: 140 | executes: 312)             |
                                       | Peak memory usage: 62.503906M                             | Peak memory usage: 62.503906M                             |
                                       | Memory limit: 4G                                          | Memory limit: 4G                                          |
---------------------------------------+-----------------------------------------------------------+-----------------------------------------------------------+
 Delete and clear template with two    | ******************** Script profiler ******************** | ******************** Script profiler ******************** |
 hosts linked                          | Total time: 53.02138                                      | Total time: 1.507103                                      |
                                       | Total SQL time: 47.995504                                 | Total SQL time: 1.369812                                  |
                                       | SQL count: 1049 (selects: 208 | executes: 841)            | SQL count: 1047 (selects: 206 | executes: 841)            |
                                       | Peak memory usage: 1287.878906M                           | Peak memory usage: 58.503906M                             |
                                       | Memory limit: 4G                                          | Memory limit: 4G                                          |
Comment by Alexander Vladishev [ 2017 Sep 22 ]

(1) [F] No translation strings changed

iivs Looks like some strings were deleted:

  • Unlinked: Trigger "%1$s" on "%2$s".
  • Unlinked: Trigger prototype "%1$s" on "%2$s".

Please, confirm.

sasha Thanks! These strings really were removed.

CLOSED

Comment by Alexander Vladishev [ 2017 Sep 29 ]

Fixed in:

  • 3.0.12rc1 r73076
  • 3.2.9rc1 r73077
  • 3.4.3rc1 r73079
  • 4.0.0alpha1 r73080
Generated at Thu May 09 00:47:20 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.