[ZBX-1401] Zabbix_server continues failed transactions Created: 2009 Dec 02  Updated: 2017 May 30  Resolved: 2012 Jun 18

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Server (S)
Affects Version/s: 1.6.7
Fix Version/s: 1.8.7, 1.9.6 (beta)

Type: Incident report Priority: Major
Reporter: Gergely Czuczy Assignee: Unassigned
Resolution: Fixed Votes: 4
Labels: database
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Postgresql 8.4, zabbix 1.6.7

Issue Links:
Duplicate
duplicates ZBX-4158 Within transaction, SQL statements af... Closed
duplicates ZBX-3889 MySQL InnoDB deadlocks are not handle... Closed
is duplicated by ZBX-1402 Zabbix_server creates deadlocks in tr... Closed
is duplicated by ZBX-2897 Zabbix eats up MySQL server connections Closed

 Description   

I've noticed some flooding in my postgresql server logs, caused by zabbix, on continuing a failed transaction. Here are some log details from the very beginning:
Dec 2 16:26:23 pgdb1 postgres[67695]: [2-1] ERROR: deadlock detected
Dec 2 16:26:23 pgdb1 postgres[67695]: [2-2] DETAIL: Process 67695 waits for ShareLock on transaction 586724714; blocked by process 67700.
Dec 2 16:26:23 pgdb1 postgres[67695]: [2-3] Process 67700 waits for ShareLock on transaction 586724679; blocked by process 67695.
Dec 2 16:26:23 pgdb1 postgres[67695]: [2-4] Process 67695: update ids set nextid=nextid+1 where nodeid=0 and table_name='events' and field_name='eventid'
Dec 2 16:26:23 pgdb1 postgres[67695]: [2-5] Process 67700: update triggers set value=0,lastchange=1259767486,error='' where triggerid=17086
Dec 2 16:26:23 pgdb1 postgres[67695]: [2-6] HINT: See server log for query details.
Dec 2 16:26:23 pgdb1 postgres[67695]: [2-7] STATEMENT: update ids set nextid=nextid+1 where nodeid=0 and table_name='events' and field_name='eventid'
Dec 2 16:26:23 pgdb1 postgres[67695]: [3-1] ERROR: current transaction is aborted, commands ignored until end of transaction block
Dec 2 16:26:23 pgdb1 postgres[67695]: [3-2] STATEMENT: select nextid from ids where nodeid=0 and table_name='events' and field_name='eventid'
Dec 2 16:26:23 pgdb1 postgres[67695]: [4-1] ERROR: current transaction is aborted, commands ignored until end of transaction block
Dec 2 16:26:23 pgdb1 postgres[67695]: [4-2] STATEMENT: select nextid from ids where nodeid=0 and table_name='events' and field_name='eventid'
Dec 2 16:26:23 pgdb1 postgres[67695]: [5-1] ERROR: current transaction is aborted, commands ignored until end of transaction block

And it just went on, and produced a 768MB log file, when I restarted zabbix.

Could you please add some checks whether current transaction is alive or not?

If the transaction is failed, a zabbix panic wouldn't prefered, later try-again would be better, especially in the case i've just pasted (deadlock)

 Comments   
Comment by Eric Renfro [ 2011 Apr 19 ]

I have just seen this happen with Zabbix 1.8.4 with PostgreSQL 9.0.3 as well. Same scenario, and this was a brand new install, sorta, of Zabbix.

I say sorta because I migrated from using MySQL to PostgreSQL just by exporting my templates and host definitions and re-importing them into the new postgresql housed database.

In my case, 225 minutes of this I had 25 GiB of logfiles of the transaction being aborted. I had to kill the pid of the aborted transaction to get it to stop.

Comment by Alexander Vladishev [ 2011 Aug 20 ]

Fixed in ZBX-3889 in version pre1.8.7, r21259.

Comment by dimir [ 2011 Oct 04 ]

Fixed in ZBX-4158 .

Comment by richlv [ 2011 Oct 13 ]

a problem, possibly connected to this issue, has appeared.
1.8 branch from yesterday evening

after pgsql went away and then reappeared, zabbix server started repeatedly writing error messages in the log :

snmp_build: unknown failure 17629:20111012:141030.165 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141059.191 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141100.192 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141100.192 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141100.192 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141100.193 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141100.193 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141100.193 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141100.194 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141108.210 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141118.246 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141119.246 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141119.247 [Z3005] query failed: [0] result is NULL [rollback;]
snmp_build: unknown failure 17629:20111012:141130.251 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141159.262 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141200.263 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141200.263 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141200.263 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141200.264 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141200.264 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141200.264 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141200.265 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141208.274 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141218.289 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141219.291 [Z3005] query failed: [0] result is NULL [rollback;]
17629:20111012:141219.291 [Z3005] query failed: [0] result is NULL [rollback;]

... and so on. only single poller (most likely) process seems to have this issue

<Sasha> it's an other issue. Please open a new ZBX, but there is not enough information for fixing it

Generated at Thu Apr 25 22:27:28 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.