Uploaded image for project: 'ZABBIX BUGS AND ISSUES'
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-14748

Deadlock found in server when deleting LLD discovery rules

    XMLWordPrintable

    Details

    • Type: Problem report
    • Status: Open
    • Priority: Trivial
    • Resolution: Unresolved
    • Affects Version/s: 3.0.20, 3.4.12, 4.0.0beta1
    • Fix Version/s: None
    • Component/s: Frontend (F), Server (S)
    • Labels:

      Description

      Configure frequent LLD discovery of VMware, when lots of hosts are discovered delete them all through frontend, notice that some times server will display error below in case of 3.0.20:

       18701:20180821:115004.900 [Z3005] query failed: [1213] Deadlock found when trying to get lock; try restarting transaction [insert into items (itemid,name,key_,hostid,type,value_type,delay,history,trends,status,trapper_hosts,units,formula,logtimefmt,valuemapid,params,ipmi_sensor,snmp_community,snmp_oid,port,snmpv3_securityname,snmpv3_securitylevel,snmpv3_authprotocol,snmpv3_authpassphrase,snmpv3_privprotocol,snmpv3_privpassphrase,authtype,username,password,publickey,privatekey,description,interfaceid,flags,snmpv3_contextname,jmx_endpoint,master_itemid) values (43543,'Average number of bytes read from the disk Hard disk 1','vmware.vm.vfs.dev.read[{$URL},{HOST.HOST},scsi0:0,bps]',10427,3,3,'1m','90d','365d',0,'','Bps','','',null,'','','','','','',0,0,'',0,'',0,'{$USERNAME}','{$PASSWORD}','','','',322,4,'','',null),(43544,'Average number of reads from the disk Hard disk 1','vmware.vm.vfs.dev.read[{$URL},{HOST.HOST},scsi0:0,ops]',10427,3,3,'1m','90d','365d',0,'','','','',null,'','','','','','',0,0,'',0,'',0,'{$USERNAME}','{$PASSWORD}','','','',322,4,'','',null),(43545,'Average number of bytes written to the disk Hard disk 1','vmware.vm.vfs.dev.write[{$URL},{HOST.HOST},scsi0:0,bps]',10427,3,3,'1m','90d','365d',0,'','Bps','','',null,'','','','','','',0,0,'',0,'',0,'{$USERNAME}','{$PASSWORD}','','','',322,4,'','',null),(43546,'Average number of writes to the disk Hard disk 1','vmware.vm.vfs.dev.write[{$URL},{HOST.HOST},scsi0:0,ops]',10427,3,3,'1m','90d','365d',0,'','','','',null,'','','','','','',0,0,'',0,'',0,'{$USERNAME}','{$PASSWORD}','','','',322,4,'','',null);
      ]
       18704:20180821:115004.957 [Z3005] query failed: [1213] Deadlock found when trying to get lock; try restarting transaction [update ids set nextid=nextid+4 where table_name='items' and field_name='itemid']
       18702:20180821:115004.964 [Z3005] query failed: [1213] Deadlock found when trying to get lock; try restarting transaction [insert into items (itemid,name,key_,hostid,type,value_type,delay,history,trends,status,trapper_hosts,units,formula,logtimefmt,valuemapid,params,ipmi_sensor,snmp_community,snmp_oid,port,snmpv3_securityname,snmpv3_securitylevel,snmpv3_authprotocol,snmpv3_authpassphrase,snmpv3_privprotocol,snmpv3_privpassphrase,authtype,username,password,publickey,privatekey,description,interfaceid,flags,snmpv3_contextname,jmx_endpoint,master_itemid) values (43543,'Average number of bytes read from the disk Hard disk 1','vmware.vm.vfs.dev.read[{$URL},{HOST.HOST},scsi0:0,bps]',10437,3,3,'1m','90d','365d',0,'','Bps','','',null,'','','','','','',0,0,'',0,'',0,'{$USERNAME}','{$PASSWORD}','','','',332,4,'','',null),(43544,'Average number of reads from the disk Hard disk 1','vmware.vm.vfs.dev.read[{$URL},{HOST.HOST},scsi0:0,ops]',10437,3,3,'1m','90d','365d',0,'','','','',null,'','','','','','',0,0,'',0,'',0,'{$USERNAME}','{$PASSWORD}','','','',332,4,'','',null),(43545,'Average number of bytes written to the disk Hard disk 1','vmware.vm.vfs.dev.write[{$URL},{HOST.HOST},scsi0:0,bps]',10437,3,3,'1m','90d','365d',0,'','Bps','','',null,'','','','','','',0,0,'',0,'',0,'{$USERNAME}','{$PASSWORD}','','','',332,4,'','',null),(43546,'Average number of writes to the disk Hard disk 1','vmware.vm.vfs.dev.write[{$URL},{HOST.HOST},scsi0:0,ops]',10437,3,3,'1m','90d','365d',0,'','','','',null,'','','','','','',0,0,'',0,'',0,'{$USERNAME}','{$PASSWORD}','','','',332,4,'','',null);
      ]
      zabbix_server [18704]: ERROR [file:db.c,line:700] Something impossible has just happened.
      

      Output of show engine innodb status; in MySQL shows that frontend is to blame:

      LATEST DETECTED DEADLOCK
      ------------------------
      2018-08-21 08:50:04 0x7f121870c700
      *** (1) TRANSACTION:
      TRANSACTION 323818, ACTIVE 0 sec inserting
      mysql tables in use 1, locked 1
      LOCK WAIT 6 lock struct(s), heap size 1136, 4 row lock(s), undo log entries 2
      MySQL thread id 12254, OS thread handle 139715702572800, query id 1641402 172.17.0.1 zabbix update
      insert into items (itemid,name,key_,hostid,type,value_type,delay,history,trends,status,trapper_hosts,units,formula,logtimefmt,valuemapid,params,ipmi_sensor,snmp_community,snmp_oid,port,snmpv3_securityname,snmpv3_securitylevel,snmpv3_authprotocol,snmpv3_authpassphrase,snmpv3_privprotocol,snmpv3_privpassphrase,authtype,username,password,publickey,privatekey,description,interfaceid,flags,snmpv3_contextname,jmx_endpoint,master_itemid) values (43543,'Average number of bytes read from the disk Hard disk 1','vmware.vm.vfs.dev.read[{$URL},{HOST.HOST},scsi0:0,bps]',10437,3,3,'1m','90d','365d',0,'','Bps','','',null,'','','','','','',0,0,'',0,'',0,'{$USERNAME}','{$PASSWORD}','','','',332,4,'','',null),(43544,'Average number of reads from the disk Hard disk 1','vmware.vm.vfs.dev.read[{$URL},{HOST.HOST},scsi0:0,ops]',10437,3,3,'1m','90d','365d',0,'','','','',null,'','','','','','',0,0,'',0,'',0,'{$USERNAME}','{$PASSWORD}','
      *** (1) WAITING FOR THIS LOCK TO BE GRANTED:
      RECORD LOCKS space id 3823 page no 264 n bits 88 index PRIMARY of table `zabbix`.`items` trx id 323818 lock_mode X insert intention waiting
      Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
       0: len 8; hex 73757072656d756d; asc supremum;;
      
      *** (2) TRANSACTION:
      TRANSACTION 323813, ACTIVE 0 sec fetching rows
      mysql tables in use 1, locked 1
      264 lock struct(s), heap size 41168, 22376 row lock(s), undo log entries 12357
      MySQL thread id 12292, OS thread handle 139715696183040, query id 1641377 172.17.0.1 zabbix updating
      DELETE FROM hosts WHERE (hostid IN ('10401','10402','10403','10404','10405','10406','10407','10408','10409','10410','10411','10412','10413','10414','10415','10416','10417','10418','10419','10420','10421','10422','10423','10424','10425','10426','10427','10428','10429','10430','10431','10432','10433','10434','10435','10436','10437','10438','10439','10440','10441','10442'))
      *** (2) HOLDS THE LOCK(S):
      RECORD LOCKS space id 3823 page no 264 n bits 88 index PRIMARY of table `zabbix`.`items` trx id 323813 lock_mode X
      Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
       0: len 8; hex 73757072656d756d; asc supremum;;
      

        Attachments

          Activity

            People

            Assignee:
            zabbix.dev Zabbix Development Team
            Reporter:
            vso Vladislavs Sokurenko
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated: