I'm experiencing this on two proxies. Happens roughly every 12 hours

$ lsb_release -a
Distributor ID: Ubuntu
Description: Ubuntu 20.04.2 LTS
Release: 20.04
Codename: focal


dpkg -l | grep zabbix 
ii zabbix-agent 1:5.4.0-1+ubuntu20.04 amd64 Zabbix network monitoring solution - agent 
ii zabbix-proxy-sqlite3 1:5.4.0-1+ubuntu20.04 amd64 Zabbix network monitoring solution - proxy (SQLite3) 
ii zabbix-sender 1:5.4.0-1+ubuntu20.04 amd64 Zabbix network monitoring solution - sender

I'm seeing the same issue I guess. I doesn't happen regularly, so I tried to avoid enabling the max debugging level all the time. Neither executing zabbix_proxy --runtime-control diaginfo while in this state doesn't produce any log output, nor does increasing the log level at runtime change anything. When I send a config_cache_reload it displays "configuration cache reloading is already in progress". It's on Debian 10, amd64, zabbix-proxy-sqlite 5.4.0. systemd can't restart it, unless if you first stop it with --signal=SIGKILL.
https://support.zabbix.com/secure/AddComment!default.jspa?id=82529

I am also facing this problem. After upgrading from version 5.0 to 5.4 my zabbix proxy is unable to send data to the zabbix server. I use Debian 10, with proxy, server and database installed on different servers. I tried to redo the bank of the proxy that uses sqlite but without success. When downloading the service or restarting the host there is a very long delay. I have 350 vps on average.

 zabbix-proxy-sqlite3 1:5.4.0-2+debian10

zabbix_server.conf

AlertScriptsPath=/usr/lib/zabbix/alertscripts
CacheSize=512M
DBHost=IP
DBName=LOGIN
DBPassword=SENHA
DBUser=USER
DebugLevel=5
ExternalScripts=/usr/lib/zabbix/externalscripts
Fping6Location=/usr/bin/fping6
FpingLocation=/usr/bin/fping
LogFile=/var/log/zabbix/zabbix_server.log
LogSlowQueries=3000
MaxHousekeeperDelete=0
PidFile=/var/run/zabbix/zabbix_server.pid
SNMPTrapperFile=/var/log/snmptrap/snmptrap.log
SocketDir=/var/run/zabbix
StartDiscoverers=20
StartVMwareCollectors=5
StatsAllowedIP=127.0.0.1
ValueCacheSize=64M

zabbix_proxy.conf

CacheSize=512M
ConfigFrequency=120
DBName=/opt/zabbix-proxy-db/zbpx.db
DebugLevel=5
ExternalScripts=/usr/lib/zabbix/externalscripts
Fping6Location=/usr/bin/fping6
FpingLocation=/usr/bin/fping
Hostname=host
LogFileSize=0
LogFile=/var/log/zabbix/zabbix_proxy.log
LogSlowQueries=3000
PidFile=/run/zabbix/zabbix_proxy.pid
ProxyOfflineBuffer=8
Server=SERVER
SNMPTrapperFile=/var/log/snmptrap/snmptrap.log
SocketDir=/run/zabbix
StartDiscoverers=20
StartVMwareCollectors=5
StatsAllowedIP=127.0.0.1
Timeout=4
TLSAccept=psk
TLSConnect=psk
TLSPSKFile=/home/zbpx.psk
TLSPSKIdentity=ZBpx001

Looks like I am seeing this issue also on Zabbix-proxy with sqlite3 on raspberry pi.  I have several of these proxies and most of them have experienced this problem.  The restart issue is kind of a big deal also as just a simple restart doesn't easily fix it.

Oh! its not happening only to me! It started to happen after upgrade 5.2 to 5.4 using zabbix proxy containered. I have 3 proxies at separated locations. All with the same symptoms. (200vps each)

I only have 20 resp. 40 values per second, so it does not seem to have to to with this. The 40 vps one locks up more often though. 

It seems to have been caused by ZBXNEXT-6292,
Backtrace provided by Kalimulin

0x00007f747f7a829c in __lll_lock_wait () from /lib/x86_64-linux-gnu/libpthread.so.0
(gdb) bt
#0  0x00007f747f7a829c in __lll_lock_wait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x00007f747f7a1714 in pthread_mutex_lock () from /lib/x86_64-linux-gnu/libpthread.so.0
#2  0x0000561e93929f4c in __zbx_mutex_lock ()
#3  0x0000561e9397667c in zbx_db_vselect ()
#4  0x0000561e9395e75a in DBselect ()
#5  0x0000561e938da887 in zbx_dbsync_compare_item_tags ()
#6  0x0000561e938b9727 in DCsync_configuration ()
#7  0x0000561e9396d27b in process_proxyconfig ()
#8  0x0000561e937d4fac in ?? ()
#9  0x0000561e937d5172 in proxyconfig_thread ()
#10 0x0000561e9392a274 in zbx_thread_start ()
#11 0x0000561e937cc793 in MAIN_ZABBIX_ENTRY ()
#12 0x0000561e93924c31 in daemon_start ()
#13 0x0000561e937cbda1 in main ()

As you see zbx_dbsync_compare_item_tags is called under configuration cache lock, meaning that configuration cache lock is locked for write lock, then for database queries sqlite mutex is locked.

Expected:
No database queries should be performed under configuration cache lock.

The other process is database syncer that locks mutex for sqlite3 and then locks mutex for configuration cache thus deadlock occur.

 0x00007f747f7a4025 in pthread_rwlock_wrlock () from /lib/x86_64-linux-gnu/libpthread.so.0
(gdb) #0  0x00007f747f7a4025 in pthread_rwlock_wrlock () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000561e93929d48 in __zbx_rwlock_wrlock ()
#2  0x0000561e938cb23d in DCconfig_items_apply_changes ()
#3  0x0000561e938a5379 in ?? ()
#4  0x0000561e938a7099 in ?? ()
#5  0x0000561e938a8276 in zbx_sync_history_cache ()
#6  0x0000561e937cd7f5 in dbsyncer_thread ()
#7  0x0000561e9392a274 in zbx_thread_start ()
#8  0x0000561e937cca12 in MAIN_ZABBIX_ENTRY ()
#9  0x0000561e93924c31 in daemon_start ()
#10 0x0000561e937cbda1 in main ()

Fixed in pull request feature/ZBX-19398-5.4

Releasing in 5.4.1rc2 today. 

Fixed in:

• pre-5.4.1rc2 9be2c307e3d
• pre-6.0.0alpha1 (master) f94bea4af76

In the headers of this Jira issue:

Status: Done
Resolution: Unresolved
Fix Version/s: [ ...long list... ]

Is it really fixed or unresolved?

constantin.oshmyan, it's fixed and available in the recently released 5.4.1.