[ZBX-25722] zabbix-proxy-sqlite3 failing to create database due to out of memory Created: 2024 Dec 10  Updated: 2025 Jan 17  Resolved: 2024 Dec 19

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Packages (C), Proxy (P)
Affects Version/s: 7.2.0
Fix Version/s: 7.2.1rc1, 7.4.0alpha1

Type: Problem report Priority: Blocker
Reporter: bunkzilla Assignee: Dmitrijs Goloscapovs
Resolution: Fixed Votes: 3
Labels: sqlite
Remaining Estimate: Not Specified
Time Spent: 4h
Original Estimate: Not Specified
Environment:

amazon linux 2023
instance type: t3.medium
over 2G memory free


Issue Links:
Causes
Duplicate
is duplicated by ZBX-25735 zabbix proxy 7.2.0 cannot create a SQ... Closed
is duplicated by ZBX-25757 Cannot create SQLite3 database in Zab... Closed
Related
related to ZBX-24053 Proxy package with sqlite3 backend by... Needs specification
Team: Team B
Sprint: S24-W50/51/52/1
Story Points: 1

 Description   

Steps to reproduce:

  1. proxy was running 7.0.6r1 without any issues
  2. did an rpm upgrade

Result:

seeing this in the logs:

 

 24463:20241210:192937.080 using configuration file: /etc/zabbix/zabbix_proxy.conf
 24463:20241210:192937.089 The proxy does not match Zabbix database. Current database version (mandatory/optional): 07000000/07000015. Required mandatory version: 07020000.
 24463:20241210:192937.089 Zabbix does not support SQLite3 database upgrade.
 24463:20241210:192937.089 removing database file: "/ebs/var/lib/sqlite/zabbix.db"
 24463:20241210:192937.090 === Backtrace: ===
 24463:20241210:192937.091 8: /usr/sbin/zabbix_proxy(zbx_backtrace+0x41) [0x5608c643e461]
 24463:20241210:192937.091 7: /usr/sbin/zabbix_proxy(zbx_db_close+0x4e) [0x5608c643342e]
 24463:20241210:192937.091 6: /usr/sbin/zabbix_proxy(+0x9b0c2) [0x5608c624d0c2]
 24463:20241210:192937.091 5: /usr/sbin/zabbix_proxy(MAIN_ZABBIX_ENTRY+0xb2a) [0x5608c624dd1a]
 24463:20241210:192937.091 4: /usr/sbin/zabbix_proxy(zbx_daemon_start+0x13f) [0x5608c644094f]
 24463:20241210:192937.091 3: /usr/sbin/zabbix_proxy(main+0x3d8) [0x5608c623f6d8]
 24463:20241210:192937.091 2: /lib64/libc.so.6(+0x3feb0) [0x7eff42a3feb0]
 24463:20241210:192937.091 1: /lib64/libc.so.6(__libc_start_main+0x80) [0x7eff42a3ff60]
 24463:20241210:192937.091 0: /usr/sbin/zabbix_proxy(_start+0x25) [0x5608c6246a75]
zabbix_proxy [24463]: ERROR [file and function: <dbconn_compat.c,zbx_db_close>, revision:626558708cc, line:64] Something unexpected has just happened.
 24463:20241210:192937.091 cannot open database file "/ebs/var/lib/sqlite/zabbix.db": [2] No such file or directory
 24463:20241210:192937.091 creating database ...
 24463:20241210:192937.091 [Z3001] connection to database '/ebs/var/lib/sqlite/zabbix.db' failed: [0] unable to open database file
 24463:20241210:192937.091 [Z3002] cannot create database '/ebs/var/lib/sqlite/zabbix.db': [0] out of memory
 24463:20241210:192937.091 cannot initialize database: cannot open database
 24526:20241210:192949.234 Starting Zabbix Proxy (active) [hostname_redactred]. Zabbix 7.2.0 (revision 626558708cc).
 24526:20241210:192949.234 **** Enabled features ****
 24526:20241210:192949.234 SNMP monitoring:       YES
 24526:20241210:192949.234 IPMI monitoring:        NO
 24526:20241210:192949.234 Web monitoring:        YES
 24526:20241210:192949.234 VMware monitoring:     YES
 24526:20241210:192949.234 ODBC:                  YES
 24526:20241210:192949.234 SSH support:           YES
 24526:20241210:192949.234 IPv6 support:          YES
 24526:20241210:192949.234 TLS support:           YES
 24526:20241210:192949.234 **************************
 24526:20241210:192949.234 using configuration file: /etc/zabbix/zabbix_proxy.conf
 24526:20241210:192949.235 cannot open database file "/ebs/var/lib/sqlite/zabbix.db": [2] No such file or directory
 24526:20241210:192949.235 creating database ...
 24526:20241210:192949.235 [Z3001] connection to database '/ebs/var/lib/sqlite/zabbix.db' failed: [0] unable to open database file
 24526:20241210:192949.235 [Z3002] cannot create database '/ebs/var/lib/sqlite/zabbix.db': [0] out of memory
 24526:20241210:192949.235 cannot initialize database: cannot open database

 

config file:

 

Server=server_redacted
Hostname=hostname_redacted
LogFile=/var/log/zabbix/zabbix_proxy.log
LogFileSize=0
PidFile=/var/run/zabbix/zabbix_proxy.pid
SocketDir=/var/run/zabbix
DBName=/ebs/var/lib/sqlite/zabbix.db
DBUser=
DBPassword=
StartPollers=12
StartIPMIPollers=0
StartPreprocessors=3
StartPollersUnreachable=10
StartTrappers=5
StartPingers=20
StartDiscoverers=1
StartHTTPPollers=12
JavaGateway=127.0.0.1
JavaGatewayPort=10052
StartJavaPollers=5
SNMPTrapperFile=/var/log/snmptrap/snmptrap.log
HistoryCacheSize=8M
Timeout=15
ExternalScripts=/usr/lib/zabbix/externalscripts
LogSlowQueries=3000
TLSConnect=unencrypted

 

 

See log file...
See memory dump...
Expected:
See screenshot....
See attached patch file...



 Comments   
Comment by Milan Brzobohatý [ 2024 Dec 11 ]

We have the same problem on OS Rocky Linux 9.5 (vm).
 

84595:20241211:104345.647 3: /usr/sbin/zabbix_proxy(main+0x3d8) [0x558daa3d4a38]
84595:20241211:104345.647 2: /lib64/libc.so.6(+0x295d0) [0x7f86ef2295d0]
84595:20241211:104345.647 1: /lib64/libc.so.6(__libc_start_main+0x80) [0x7f86ef229680]
84595:20241211:104345.647 0: /usr/sbin/zabbix_proxy(_start+0x25) [0x558daa3dbdd5]
zabbix_proxy [84595]: ERROR [file and function: <dbconn_compat.c,zbx_db_close>, revision:626558708cc, line:64] Something unexpected has just happened.
Comment by Brennan Wellman [ 2024 Dec 11 ]

I ran into this as well and opened an enterprise support ticket which support is currently working on.  They let me know about this ticket and I just want to share the workaround I came up with which got my proxies running.  This should work for anyone running a RHEL based operating system.  This downloads the zabbix-sql-scripts and sqlite packages and uses them to regenerate the database.

sudo dnf install -y sqlite zabbix-sql-scripts && cat /usr/share/zabbix/sql-scripts/sqlite3/proxy.sql | sudo sqlite3 /var/lib/zabbix/zabbix.db && sudo chown zabbix:zabbix /var/lib/zabbix/zabbix.db
Comment by bunkzilla [ 2024 Dec 11 ]

brennan.wellman Thank you for this work around.

Comment by Dmitrijs Goloscapovs [ 2024 Dec 17 ]

Available in versions:

Comment by Dmitrijs Goloscapovs [ 2024 Dec 17 ]

Clarification

This fix fixes 'Something unexpected has just happened' and '[0] out of memory'.  These errors happened due to incorrect handling of sqlite-related DB removal/creation failures when proxy has no rights to sqlite file. 

For rights issue please follow ZBX-24053.

Generated at Mon Apr 21 22:08:44 EEST 2025 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.