Details
-
Problem report
-
Status: Closed
-
Trivial
-
Resolution: Cannot Reproduce
-
None
-
None
-
None
-
None
-
CentOS 8, MariaDB 10.3, Zabbix 5.0.14
Description
I am using zabbix-server 5.0.14 on CentOS 8 with MariaDB.
In my environment, zabbix-server works with other systems and I made some tools to close problems or to change the severity.
I sometimes find that some problems are closing state for more than 1day. Please see attached image.
And the following logs are written in zabbix_server.log.
4502:20210827:163439.404 [Z3005] query failed: [1213] Deadlock found when trying to get lock; try restarting transaction [update problem set r_eventid=232854,r_clock=1630049678,r_ns=94954861,userid=1 where eventid=172849;] 4502:20210827:163439.724 [Z3005] query failed: [1213] Deadlock found when trying to get lock; try restarting transaction [insert into event_recovery (eventid,r_eventid,correlationid,c_eventid,userid) values (172850,232855,null,null,1);]
It seems that DB deadlock occurs and it causes problem close failure.
I attached the result of "show engine innodb status" in innodb.log. Please see "LATEST DETECTED DEADLOCK" section.
Steps to reproduce:
This is how I checked to reproduce the issue.
1. create many problems (~30k problems)
2. close all problems by API
3. change problem severity by API repeatedly
I used a python script for 2. and 3. Please see attached file close_problem.py.
Repeating this procedure several times caused the problem reproduction.
Result:
DB deadlock occurs and some problems remains in closing state.
Expected:
All problems are closed.