Uploaded image for project: 'ZABBIX BUGS AND ISSUES'
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-17658

zabbix-server restarted with out of memory

XMLWordPrintable

    • Icon: Incident report Incident report
    • Resolution: Won't fix
    • Icon: Trivial Trivial
    • None
    • 4.4.7, 4.4.8
    • Server (S)
    • None

      On 4.4.7 some times I see

      5439:20200429:020007.410 file:dbconfig.c,line:94 __zbx_mem_malloc(): out of memory (requested 449 bytes)
      5439:20200429:020007.410 file:dbconfig.c,line:94 __zbx_mem_malloc(): please increase CacheSize configuration parameter
      5439:20200429:020007.410 === memory statistics for configuration cache ===
      5439:20200429:020007.410 free chunks of size 88 bytes: 1
      5439:20200429:020007.410 free chunks of size 136 bytes: 1
      5439:20200429:020007.410 free chunks of size 152 bytes: 1
      5439:20200429:020007.410 free chunks of size 176 bytes: 1
      5439:20200429:020007.410 free chunks of size 248 bytes: 1
      5439:20200429:020007.410 free chunks of size >= 256 bytes: 6
      5439:20200429:020007.410 min chunk size: 88 bytes
      5439:20200429:020007.410 max chunk size: 384 bytes
      5439:20200429:020007.410 memory of total size 8388232 bytes fragmented into 71464 chunks
      5439:20200429:020007.410 of those, 2736 bytes are in 11 free chunks
      5439:20200429:020007.410 of those, 7242088 bytes are in 71453 used chunks
      5439:20200429:020007.410 ================================
      5439:20200429:020007.410 === Backtrace: ===
      5439:20200429:020007.552 15: /usr/sbin/zabbix_server: history syncer #4 [processed 0 values, 1 triggers in 0.016211 sec, syncing history](zbx_backtrace+0x3f) [0x561d7228aaf7]
      5439:20200429:020007.552 14: /usr/sbin/zabbix_server: history syncer #4 [processed 0 values, 1 triggers in 0.016211 sec, syncing history](__zbx_mem_malloc+0x180) [0x561d72286009]
      5439:20200429:020007.552 13: /usr/sbin/zabbix_server: history syncer #4 [processed 0 values, 1 triggers in 0.016211 sec, syncing history](+0x1763c3) [0x561d7224f3c3]
      5439:20200429:020007.552 12: /usr/sbin/zabbix_server: history syncer #4 [processed 0 values, 1 triggers in 0.016211 sec, syncing history](zbx_hashset_insert_ext+0x12f) [0x561d72290044]
      5439:20200429:020007.552 11: /usr/sbin/zabbix_server: history syncer #4 [processed 0 values, 1 triggers in 0.016211 sec, syncing history](+0x176f6f) [0x561d7224ff6f]
      5439:20200429:020007.552 10: /usr/sbin/zabbix_server: history syncer #4 [processed 0 values, 1 triggers in 0.016211 sec, syncing history](DCstrpool_replace+0x55) [0x561d72250070]
      5439:20200429:020007.552 9: /usr/sbin/zabbix_server: history syncer #4 [processed 0 values, 1 triggers in 0.016211 sec, syncing history](DCconfig_items_apply_changes+0xfe) [0x561d7226bad0]
      5439:20200429:020007.552 8: /usr/sbin/zabbix_server: history syncer #4 [processed 0 values, 1 triggers in 0.016211 sec, syncing history](+0x1732f6) [0x561d7224c2f6]
      5439:20200429:020007.552 7: /usr/sbin/zabbix_server: history syncer #4 [processed 0 values, 1 triggers in 0.016211 sec, syncing history](zbx_sync_history_cache+0x87) [0x561d7224cdbc]
      5439:20200429:020007.552 6: /usr/sbin/zabbix_server: history syncer #4 [processed 0 values, 1 triggers in 0.016211 sec, syncing history](dbsyncer_thread+0x1ca) [0x561d721276cb]
      5439:20200429:020007.552 5: /usr/sbin/zabbix_server: history syncer #4 [processed 0 values, 1 triggers in 0.016211 sec, syncing history](zbx_thread_start+0x37) [0x561d722986b4]
      5439:20200429:020007.552 4: /usr/sbin/zabbix_server: history syncer #4 [processed 0 values, 1 triggers in 0.016211 sec, syncing history](MAIN_ZABBIX_ENTRY+0xc6f) [0x561d7211adc6]
      5439:20200429:020007.552 3: /usr/sbin/zabbix_server: history syncer #4 [processed 0 values, 1 triggers in 0.016211 sec, syncing history](daemon_start+0x2ff) [0x561d7228a2e2]
      5439:20200429:020007.552 2: /usr/sbin/zabbix_server: history syncer #4 [processed 0 values, 1 triggers in 0.016211 sec, syncing history](main+0x2f5) [0x561d7211a155]
      5439:20200429:020007.552 1: /lib64/libc.so.6(__libc_start_main+0xf3) [0x7fe8ca27f873]
      5439:20200429:020007.552 0: /usr/sbin/zabbix_server: history syncer #4 [processed 0 values, 1 triggers in 0.016211 sec, syncing history](_start+0x2e) [0x561d7211921e]
      3578:20200429:020007.689 One child process died (PID:5439,exitcode/signal:1). Exiting ...

      After update to 4.4.8 server stop working

      2284:20200429:200521.141 Starting Zabbix Server. Zabbix 4.4.8 (revision 3d7663fd7c).
      2284:20200429:200521.141 ****** Enabled features ******
      2284:20200429:200521.141 SNMP monitoring: YES
      2284:20200429:200521.141 IPMI monitoring: YES
      2284:20200429:200521.141 Web monitoring: YES
      2284:20200429:200521.141 VMware monitoring: YES
      2284:20200429:200521.141 SMTP authentication: YES
      2284:20200429:200521.141 ODBC: YES
      2284:20200429:200521.141 SSH support: YES
      2284:20200429:200521.141 IPv6 support: YES
      2284:20200429:200521.141 TLS support: YES
      2284:20200429:200521.141 ******************************
      2284:20200429:200521.141 using configuration file: /etc/zabbix/zabbix_server.conf
      2284:20200429:200521.158 current database version (mandatory/optional): 04040000/04040002
      2284:20200429:200521.158 required mandatory version: 04040000
      2284:20200429:200521.169 server #0 started [main process]
      2288:20200429:200521.169 server #1 started configuration syncer #1
      2288:20200429:200521.407 __mem_malloc: skipped 0 asked 24 skip_min 18446744073709551615 skip_max 0
      2288:20200429:200521.407 file:dbconfig.c,line:94 __zbx_mem_realloc(): out of memory (requested 16 bytes)
      2288:20200429:200521.407 file:dbconfig.c,line:94 __zbx_mem_realloc(): please increase CacheSize configuration parameter
      2288:20200429:200521.407 === memory statistics for configuration cache ===
      2288:20200429:200521.407 min chunk size: 18446744073709551615 bytes
      2288:20200429:200521.407 max chunk size: 0 bytes
      2288:20200429:200521.407 memory of total size 8388232 bytes fragmented into 74719 chunks
      2288:20200429:200521.407 of those, 0 bytes are in 0 free chunks
      2288:20200429:200521.407 of those, 7192744 bytes are in 74719 used chunks
      2288:20200429:200521.407 ================================
      2288:20200429:200521.407 === Backtrace: ===
      2288:20200429:200521.408 11: /usr/sbin/zabbix_server: configuration syncer [syncing configuration](zbx_backtrace+0x3f) [0x5638176160a7]
      2288:20200429:200521.408 10: /usr/sbin/zabbix_server: configuration syncer [syncing configuration](__zbx_mem_realloc+0x160) [0x56381761172d]
      2288:20200429:200521.408 9: /usr/sbin/zabbix_server: configuration syncer [syncing configuration](+0x1763a7) [0x5638175da3a7]
      2288:20200429:200521.408 8: /usr/sbin/zabbix_server: configuration syncer [syncing configuration](+0x1818c2) [0x5638175e58c2]
      2288:20200429:200521.408 7: /usr/sbin/zabbix_server: configuration syncer [syncing configuration](DCsync_configuration+0x1157) [0x5638175e6c1d]
      2288:20200429:200521.408 6: /usr/sbin/zabbix_server: configuration syncer [syncing configuration](dbconfig_thread+0x116) [0x5638174b2b07]
      2288:20200429:200521.408 5: /usr/sbin/zabbix_server: configuration syncer [syncing configuration](zbx_thread_start+0x37) [0x563817623c64]
      2288:20200429:200521.408 4: /usr/sbin/zabbix_server: configuration syncer [syncing configuration](MAIN_ZABBIX_ENTRY+0x97d) [0x5638174a5b04]
      2288:20200429:200521.408 3: /usr/sbin/zabbix_server: configuration syncer [syncing configuration](daemon_start+0x2ff) [0x563817615892]
      2288:20200429:200521.408 2: /usr/sbin/zabbix_server: configuration syncer [syncing configuration](main+0x2f5) [0x5638174a5185]
      2288:20200429:200521.408 1: /lib64/libc.so.6(__libc_start_main+0xf3) [0x7f98e13d6873]
      2288:20200429:200521.408 0: /usr/sbin/zabbix_server: configuration syncer [syncing configuration](_start+0x2e) [0x5638174a424e]
      2284:20200429:200521.410 One child process died (PID:2288,exitcode/signal:1). Exiting ...
      2284:20200429:200521.417 syncing trend data...
      2284:20200429:200521.418 syncing trend data done
      2284:20200429:200521.418 Zabbix Server stopped. Zabbix 4.4.8 (revision 3d7663fd7c).

      [root@dev-1 zabbix]# free -m
      total used free shared buff/cache available
      Mem: 32160 16690 968 1647 14501 13416
      Swap: 18419 1697 16722

            zabbix.support Zabbix Support Team
            rakhinskiy Vyacheslav
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Created:
              Updated:
              Resolved: